50 Commits

Author SHA1 Message Date
enricobuehler 75627c8afe feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients
apple / swift (push) Failing after 10s
release / apple (push) Failing after 7s
apple / screenshots (push) Has been skipped
audit / cargo-audit (push) Failing after 1m19s
windows-host / package (push) Failing after 2m44s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 39s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 45s
android / android (push) Successful in 5m17s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 45s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 56s
ci / rust (push) Successful in 9m19s
ci / bench (push) Successful in 4m40s
decky / build-publish (push) Successful in 26s
deb / build-publish (push) Successful in 2m57s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m56s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m35s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m20s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
flatpak / build-publish (push) Successful in 4m22s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m51s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m50s
Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client
(previously stereo-only):

- core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream
  mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome
  `audio_channels` negotiation via the trailing-byte back-compat pattern (old
  peers fall back to stereo); C-ABI `punktfunk_connect_ex6`,
  `punktfunk_connection_audio_channels`, and in-core multistream decode
  `punktfunk_connection_next_audio_pcm` for embedders without a multistream
  Opus decoder. Real-libopus channel-identity round-trip test.
- host: native audio thread captures + Opus-(multi)stream-encodes at the
  negotiated count (with a cross-session cached-capturer channel-mismatch fix);
  GameStream surround unified onto the safe `opus::MSEncoder`, dropping
  `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround;
  WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask.
- clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via
  `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM →
  AVAudioEngine with an explicit wire-order channel layout; each gains a
  Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless
  validator.

Verified on Linux: core/host/linux/probe test suites + the Android Rust
(cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple
builds, all on-glass checks, and the live native loopback are pending (CI / a
free box).

Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it
shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so
cannot be committed separately from the surround changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 21:11:05 +00:00
enricobuehler 6383e5f4fd feat(client/android): CI screenshot capture via Roborazzi
Play-listing/marketing screenshots of the Compose client rendered on the host JVM
by Roborazzi (Robolectric Native Graphics) — no emulator, GPU, KVM, host, or JNI
core. Five scenes render the REAL composables with embedded mock state under a
forced brand palette (Material You has no wallpaper to seed from on the JVM):
hosts grid, settings, TOFU + PIN dialogs, and the live stats HUD. Validated 5/5
locally.

- New JVM unit-test source set (app/src/test) + Roborazzi/Robolectric test deps;
  @Config(sdk=36) is mandatory (no android-all jar for compileSdk 37) and the
  animation clock is paused so a text-bearing scene reaches idle.
- kit: `-PskipRustBuild` skips the cargo-ndk native build so the JVM-only test job
  needs no Rust/NDK; normal APK/AAR builds are unchanged.
- Widen BrandDark / StatsOverlay to internal so the tests can use them.
- Standalone best-effort tag-gated workflow; PNGs upload as a 30-day artifact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:05:54 +00:00
enricobuehler 6a93d164a0 feat(client/linux): CI screenshot capture
Host-free UI screenshots of the GTK4/libadwaita client under a virtual X display
(clients/linux/tools/screenshots.sh) — Xvfb + software GL (llvmpipe) + a root-window
grab, one app launch per scene. PUNKTFUNK_SHOT_SCENE routes build_ui to render one
mock-populated REAL view (hosts grid / settings dialog / TOFU + PIN dialogs) and
print PF_SHOT_READY once it has settled; the saved-hosts grid is driven by a seeded
client-known-hosts.json. NON_UNIQUE in shot mode so back-to-back launches don't
collide. The stream scene is deferred — its page needs a live NativeClient.

Gated to stable release tags in a standalone best-effort workflow that builds the
client in the rust-ci image and captures under Xvfb; PNGs upload as a 30-day
artifact, not committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:05:38 +00:00
enricobuehler 9e98618e5f feat(web): CI screenshot capture for the mgmt console
Marketing/store screenshots of the console, captured from the built Storybook
with headless Chromium (web/tools/screenshots.mjs) — every Pages/* + Shell/*
story rendered at 1440x900@2x. The page stories render from fixtures, so no live
mgmt API, login, or GPU is needed (the web analogue of apple.yml's screenshots
job). Gated to stable release tags in a standalone best-effort workflow; PNGs
upload as a 30-day artifact, not committed.

- Add Stats + Pairing stories (the two pages that lacked them) with stats/pairing
  fixtures typed against the generated models.
- Extract a pure PairingView (index.tsx -> view.tsx), matching the
  Dashboard/Clients/Stats split, so the page renders host-free from mock state
  instead of racing its polling queries. Container wiring is behaviour-identical.
- Playwright driver + a chromium-capable tag-gated job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:05:27 +00:00
enricobuehler 1bd60ffb34 refactor(docs): use shared @unom/app-ui/footer component
apple / swift (push) Successful in 1m2s
android / android (push) Successful in 4m23s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 13s
ci / rust (push) Successful in 4m47s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 58s
apple / screenshots (push) Successful in 5m16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
ci / bench (push) Successful in 4m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m29s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m23s
docker / deploy-docs (push) Successful in 18s
The docs footer was a hand-maintained mirror of the marketing site's. Both now
render the same @unom/app-ui/footer component, so they stay in sync. The shared
view themes itself through @unom/style tokens (which the docs already map onto
their Fumadocs surfaces), and a resolveHref hook rebases root-relative links
onto the marketing-site origin. Footer types now come from the library too.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 14:34:45 +00:00
enricobuehler 30d0d36efe feat(decky): self-update without the store + Gaming-Mode launch polish, and ship the Steam Deck docs
apple / swift (push) Successful in 1m4s
apple / screenshots (push) Successful in 5m26s
android / android (push) Successful in 3m27s
ci / web (push) Successful in 1m7s
ci / docs-site (push) Successful in 1m16s
ci / rust (push) Successful in 4m21s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
ci / bench (push) Successful in 4m46s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m0s
flatpak / build-publish (push) Successful in 4m55s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m38s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m25s
Plugin self-update (no Decky store): CI publishes a per-channel manifest.json
({version, immutable per-version artifact, sha256}) beside the zip and bakes
update.json {channel, manifest} into the plugin. main.py `check_update` reads the
installed version from package.json (the value Decky reports — not plugin.json),
fetches the channel manifest, and the frontend shows an "Update to vX" button that
drives Decky Loader's own install RPC (root downloads + SHA-256-verifies + hot-reloads).
CI now stamps a plain-numeric semver (0.3.<run> canary / X.Y.Z stable) into
package.json — a -ciN suffix would mis-order under compare-versions.

Linux client: `--fullscreen` (plus SteamDeck/gamescope env fallback) enters GTK
fullscreen on stream start so Gaming-Mode chrome is hidden; native-mode resolution
falls back to the display's first monitor when the window isn't mapped yet (was
dropping to the 1080p floor — wrong on the Deck's 1280×800); add a confirmed
"Remove saved host" action (KnownHosts::remove_by_fp).

Docs: new docs/steam-deck.md (Decky install/pair/stream/self-update/troubleshooting),
wired into meta.json nav, and cross-linked from clients/install-client/channels. This
is the page docs.punktfunk.unom.io/docs/steam-deck — the website's download link
pointed at it before it existed; committing it makes that link resolve.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 13:03:44 +00:00
enricobuehler 3947d5b07a fix(host/audio): drive the Linux virtual mic with RT_PROCESS (was silent)
apple / swift (push) Successful in 1m1s
ci / rust (push) Successful in 4m36s
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 55s
apple / screenshots (push) Successful in 5m9s
ci / bench (push) Successful in 4m36s
windows-host / package (push) Successful in 7m8s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m19s
release / apple (push) Successful in 9m52s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m16s
android / android (push) Successful in 3m21s
decky / build-publish (push) Successful in 11s
deb / build-publish (push) Successful in 2m45s
flatpak / build-publish (push) Successful in 4m11s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m48s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m35s
docker / deploy-docs (push) Successful in 18s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
The punktfunk-mic PipeWire source connected without RT_PROCESS, so it ran as an
async/main-loop node. In the host's busy multi-stream graph (desktop audio + video
capture + the session) it never acquired a driver, stayed suspended, and its
process() callback never fired — every recorder reading the remote mic heard pure
silence (the long-standing "Linux host mic broken"). Connect the mic stream with
RT_PROCESS so it is a synchronous node that joins its consumer's driver group and
is actually driven.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:46:06 +00:00
enricobuehler 238501597e feat(host/gamestream): follow Desktop<->Game session switches
android / android (push) Successful in 4m49s
ci / web (push) Successful in 55s
apple / swift (push) Successful in 59s
ci / rust (push) Successful in 4m52s
ci / docs-site (push) Successful in 56s
apple / screenshots (push) Successful in 5m16s
windows-host / package (push) Successful in 7m1s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 42s
ci / bench (push) Successful in 4m41s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m59s
The GameStream/Moonlight video plane is a separate encode loop that lacked the
session-following the native punktfunk/1 plane has, so a mid-stream Desktop<->Game
switch killed the stream ("video stream failed") instead of following it.

* Normalize the session env like the native plane: extract open_gs_virtual_source,
  which detects the LIVE compositor + apply_session_env/apply_input_env (gamescope
  ATTACH default -> resize-on-attach to the box's own game-mode session at the
  client mode; KWin/Mutter retargeting). GameStream previously ran a bare detect()
  against raw process env, so in game mode it bare-spawned a COMPETING gamescope
  instead of attaching to the box's session.

* In-place capture-loss rebuild: replace the `?` that ended the stream with a
  bounded rebuild (re-detect the live compositor via the same factory, build the
  new source BEFORE dropping the old, reopen the encoder, force an IDR) — keeping
  the send thread + packetizer + socket + RTP clock. A same-resolution
  Desktop<->Game toggle is now FOLLOWED with no Moonlight reconnect.

Protocol limit (unchanged): a mid-stream RESOLUTION change is impossible on
GameStream (WxH locked at ANNOUNCE; no Reconfigure) — a session toggle keeps the
negotiated mode, so this isn't hit. The portal/synthetic source passes no rebuild
closure (propagates as before).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:22:12 +00:00
enricobuehler 04dd3e3a19 docs: refresh Windows host page for new users; drop stale Status/NVIDIA-only/SudoVDA
Rewrite the Windows host docs page for first-time setup, on par with the
other host guides: remove the standout "Status:" banner, restructure into
Requirements / Install (web console + pairing + configure) / How it works /
Notes & limits.

Bring the content up to date with the shipping host:
- encode is all-vendor (NVENC/AMF/QSV + software fallback), not NVIDIA-only
- virtual display is punktfunk's own pf-vdisplay IDD (SudoVDA removed)
- gamepads need no prerequisite — UMDF drivers bundled; ViGEmBus is gone
- add HDR10 + Vulkan-game HDR layer coverage

Fix the same stale claims where other pages cross-reference the Windows host
(requirements, running-as-a-service, install, roadmap, status).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:22:50 +00:00
enricobuehler 61aa1053e7 feat(host/gamescope): headless game mode that follows the box + matches the client
apple / swift (push) Successful in 1m2s
android / android (push) Successful in 4m43s
ci / rust (push) Successful in 4m53s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 5m6s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
windows-host / package (push) Successful in 9m2s
ci / bench (push) Successful in 4m41s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m6s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m43s
Make Steam game mode work on a display-less streaming host and stream it at the
client's resolution:

* Ship /etc/gamescope-session-plus/sessions.d/steam (packaging/bazzite/
  gamescope-headless-session, installed by the RPM + Arch PKGBUILD): fall back to
  gamescope's headless backend when no display is connected, so "Switch to Game
  Mode" boots offscreen instead of crashing on the missing panel (and 5-striking
  back to desktop). No-op on display-attached boxes; only sets unset values so
  the host's per-client mode still wins.

* Default Bazzite/SteamOS to ATTACH (PUNKTFUNK_GAMESCOPE_ATTACH=1 in host.env):
  the box owns its session (Desktop<->Game, persistent), the host follows +
  captures it and never tears it down — so switching is rock-solid and a
  disconnect leaves the box in its mode (reconnect returns there).

* Resize-on-attach (gamescope.rs): on connect, ensure the box's own game-mode
  session runs at the CLIENT's resolution — reuse it when already matching (fast
  path, no restart), else reconfigure + restart the box's own autologin
  gamescope-session-plus@<client> at the client mode (cooperative: no competing
  unit, so no autologin-respawn fight). Detect the live gamescope's -W/-H via
  argv[0] in /proc (its /proc/<pid>/exe is unreadable for that process).

Validated live on a headless bazzite-deck-nvidia box: game mode boots headless +
stable (0 strikes); the host attaches + streams video/audio/EIS input; a
5120x1440 client reuses the matching session and streams at 5120x1440.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:09:45 +00:00
enricobuehler 50e17b3508 fix(host/capture): hold the session through a slow compositor switch
apple / swift (push) Successful in 1m1s
ci / docs-site (push) Successful in 54s
apple / screenshots (push) Successful in 5m14s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 11s
android / android (push) Successful in 4m41s
ci / rust (push) Successful in 4m52s
ci / web (push) Successful in 49s
windows-host / package (push) Successful in 7m54s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m34s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m10s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m7s
A Bazzite/SteamOS Gaming↔Desktop switch tears the old compositor down and can
take 15s+ to bring the new one up — longer than the capture-loss rebuild's
~10s window, so the session failed mid-switch ("disconnect — session failed")
and forced the client to cold-reconnect. Retry the rebuild within a 40s budget
instead of giving up after one round, and re-detect the live compositor on
each attempt so the stream follows the box to whatever session comes up (a new
instance of the same compositor, or a different one — the kind-change case).
The QUIC keepalive runs on its own thread, so the client stays connected
(frozen on the last frame) and the stream resumes when the new output appears,
with no reconnect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 09:31:47 +00:00
enricobuehler 94c556f0e3 fix(host/capture): recover from compositor loss instead of freezing
apple / screenshots (push) Successful in 5m7s
apple / swift (push) Successful in 1m1s
windows-host / package (push) Successful in 7m26s
android / android (push) Successful in 4m50s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
decky / build-publish (push) Successful in 11s
ci / rust (push) Successful in 4m51s
deb / build-publish (push) Successful in 2m29s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m37s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m1s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m47s
When the compositor is torn down mid-stream (a Gaming↔Desktop switch removes
the virtual output), its PipeWire stream leaves Streaming for Paused rather
than disconnecting. try_latest treated that as Ok(None) ("static desktop —
repeat the last frame"), so the stream froze on the last frame forever and
neither recovery path fired: the capture-loss rebuild keys on Err, and the
session watcher keys on a session-KIND change (a desktop→desktop new KWin
instance is the same kind).

Track the PipeWire stream state via state_changed (a `streaming` flag) and,
in try_latest, surface a sustained non-Streaming state (1.5s grace for a
transient renegotiation blip) as a capture-loss Err — which the encode loop
already handles by rebuilding the pipeline in place. A static desktop stays
Streaming, so no false trigger. Complements the now-default session watcher.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 09:00:35 +00:00
enricobuehler 32c1929948 feat(host/session-watch): default Gaming↔Desktop follow on for Bazzite/SteamOS
apple / swift (push) Successful in 1m2s
android / android (push) Successful in 4m52s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 54s
decky / build-publish (push) Successful in 22s
windows-host / package (push) Successful in 9m7s
ci / bench (push) Successful in 4m40s
apple / screenshots (push) Successful in 5m20s
deb / build-publish (push) Successful in 2m31s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m40s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m39s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m24s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 47s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m19s
docker / deploy-docs (push) Successful in 22s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m29s
The mid-stream session watcher (rebuild the backend in place when the box
flips Gaming↔Desktop) was opt-in via PUNKTFUNK_SESSION_WATCH, so it never
ran on a stock Bazzite/SteamOS box — switching modes froze the stream on the
now-dead compositor. Default it ON when os-release ID/ID_LIKE is
bazzite/steamos (the platforms that flip sessions); still off on plain
desktops. Also parse the env properly so PUNKTFUNK_SESSION_WATCH=0 actually
disables it (was: any value, including "0", enabled it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 08:43:27 +00:00
enricobuehler 3915a82780 fix(host/input): route KWin auto-detect to the fake_input backend
apple / swift (push) Successful in 1m1s
apple / screenshots (push) Successful in 5m2s
windows-host / package (push) Successful in 6m56s
android / android (push) Successful in 4m42s
ci / rust (push) Successful in 4m52s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 56s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m4s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m0s
apply_input_env() hard-pinned PUNKTFUNK_INPUT_BACKEND=libei for KWin, and
default_backend() reads that env first — so the auto-detecting host (the
normal `serve` service) ignored the new KwinFakeInput backend and fell back
to the RemoteDesktop portal path that needs a user to approve. Route KWin to
"kwin" (org_kde_kwin_fake_input); GNOME/Mutter stay on libei (no fake_input
there).

Validated live on a Bazzite KDE box via the auto-detect path:
backend=KwinFakeInput, "KWin fake_input ready (no portal)", input events
forwarded with no errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:52:02 +00:00
enricobuehler a4833e4780 feat(android/touch): trackpad-relative cursor (default), with a direct-touch toggle
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 4m53s
ci / rust (push) Successful in 5m1s
ci / web (push) Successful in 58s
ci / docs-site (push) Successful in 55s
apple / screenshots (push) Successful in 5m28s
deb / build-publish (push) Successful in 2m30s
windows-host / package (push) Successful in 8m41s
decky / build-publish (push) Successful in 29s
ci / bench (push) Successful in 4m27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m43s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m35s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m25s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 48s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m46s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m1s
docker / deploy-docs (push) Successful in 24s
One-finger touch was absolute "direct pointing" — the host cursor jumped to the
finger and was recomputed from each touch-start, so you couldn't precisely reach a
target. Now a relative trackpad: the cursor stays put on touch-down and moves by the
finger delta (host MouseMove via nativeSendPointerMove, already supported — no
protocol change), with mild pointer acceleration and sub-pixel remainder
accumulation so slow precise moves aren't lost to Int truncation. Swipe, lift, and
re-swipe to walk it across; tap = left-click at the cursor's current position.
Two-finger scroll / right-click, three-finger HUD toggle, and tap-then-hold-drag are
preserved unchanged; finger-id re-anchoring keeps multi-touch transitions jump-free.

Added Settings → Pointer → "Trackpad mode" (default on); turning it off restores the
old direct-pointing path verbatim.

:app:compileDebugKotlin green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:34:03 +00:00
enricobuehler 4e79e6cdad fix(android/audio): kill the AAudio crackle (RT-safe ring + deeper buffer + XRun sizing)
The jitter ring was a port of the Linux client's, but Linux runs on PipeWire
(adaptive resampling masks host↔DAC drift + a shallow buffer); AAudio hands us a
raw realtime callback and we own the buffer, so the same code crackled only on
Android. Three converging causes, all fixed:

- Heap free on the realtime audio thread every quantum (Android's Scudo free() has
  unbounded tail latency → XRun → click). Decoded buffers are now recycled back to
  the producer via a free-list instead of freed on the audio thread; the ring is
  pre-reserved so extend() never reallocates there.
- The ring collapsed to ~15 ms on the tiny LowLatency burst and re-primed (a fresh
  silence) on every single empty callback. Now ~40 ms prime / ~150 ms hard cap,
  decoupled from the burst size, with de-prime hysteresis (re-prime only after a
  sustained drain).
- AAudio's anti-glitch knobs were unused: prime the HW buffer above its 2-burst
  default and grow it on getXRunCount(). The post-open log now reports
  perf/sharing/buffer so a fall to a resampled legacy path is visible.

Steady-state audio latency ~15 → ~40 ms (within lip-sync tolerance; matches the
Moonlight/Sunshine operating point). cargo-ndk build both ABIs + fmt + clippy green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:33:51 +00:00
enricobuehler f74bc4a3f1 feat(host/input): headless KDE input via org_kde_kwin_fake_input
Desktop-mode (KWin) streaming had no input: the path was libei via the
RemoteDesktop portal, which (a) isn't reachable from the host service env
and (b) requires a human to approve "Allow remote control?" — a
non-starter on a headless box. KWin's own headless RDP server (krdpserver)
solves this with org_kde_kwin_fake_input, authorized by the exact same
.desktop X-KDE-Wayland-Interfaces grant we already ship
(org_kde_kwin_fake_input is listed alongside zkde_screencast_unstable_v1).

Add a fake_input injector: vendor the protocol XML, bind the global as an
ordinary Wayland client, authenticate (auto-accepted for an
interface-authorized client — no dialog), and translate pointer (rel/abs),
button, scroll, keyboard (raw evdev keycodes resolved by KWin's own keymap)
and touch. Select it for KWin (compositor=="kwin" or XDG_CURRENT_DESKTOP
KDE); GNOME stays on libei (it has neither fake_input nor the wlr
protocols). PUNKTFUNK_INPUT_BACKEND=kwin forces it.

cargo check + clippy + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:26:04 +00:00
enricobuehler 8e18d01af5 fix(host/kwin): authorize Desktop-mode streaming via a shipped .desktop
Streaming the KDE *Desktop* (KWin) session failed on a real interactive
Plasma session with "KWin does not expose zkde_screencast_unstable_v1":
KWin treats the screencast/virtual-output and fake_input globals as
restricted and advertises them only to a client whose installed .desktop
lists them under X-KDE-Wayland-Interfaces (matched by /proc/<pid>/exe ->
Exec, and cached per-executable on first connect). The host shipped no
.desktop, so it was permanently denied; it only ever worked on the
headless dev box via KWIN_WAYLAND_NO_PERMISSION_CHECKS=1.

Ship packaging/linux/io.unom.Punktfunk.Host.desktop (least-privilege:
only the host, only zkde_screencast_unstable_v1 + org_kde_kwin_fake_input)
and install it from the RPM/.deb/Arch host packaging so it is present
before the host first connects. Drop the blunt session-wide
NO_PERMISSION_CHECKS hack from kde-desktop-setup.sh (it now only seeds the
RemoteDesktop input grant) and fix the now-misleading kwin.rs docs/errors.

Validated live on a Bazzite Kinoite box (KWin 6.6.4): probe-compositor +
spike --source kwin-virtual succeed against a KWin running WITHOUT the
permission bypass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:15:39 +00:00
enricobuehler 3477cbe7ce fix(audio/windows): stop the client mic echoing back through the loopback
The Windows virtual mic fakes a capture endpoint by writing the client's
uplinked PCM into a virtual device's *render* endpoint, while the
desktop-audio plane loopback-captures the *default render* endpoint — with
no mutual exclusion between the two. WASAPI loopback captures the mixed
output of an endpoint (everything any app renders to it, including our mic
writes), so when both resolve to the same device — VB-CABLE used for both,
or the auto-installed Steam Streaming Microphone being the default render on
a headless box — the injected mic is captured straight back into the
host->client audio stream: an infinite echo.

find_device() now resolves the loopback's endpoint id (default render) and
skips any candidate matching it, scanning on to the next non-loopback match,
so the mic can never land on the device the loopback reads. The auto-install
path now provisions the full Steam pair (Streaming Microphone + Streaming
Speakers) so a bare host gets two distinct devices instead of one shared
one. Errors distinguish "no device" from "only candidate is the loopback
device". Linux was already immune (its mic is a dedicated Audio/Source node,
structurally separate from the monitored sink).

Windows-only (#[cfg(windows)]); rustfmt-clean, compile-checked in
windows-host CI, needs on-glass validation on the RTX box. Does not force
the system default playback onto Steam Streaming Speakers (IPolicyConfig) —
not required to break the echo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 23:51:46 +00:00
enricobuehler 5a2e07e865 style(windows): rustfmt install.rs to unbreak cargo fmt --all --check
apple / swift (push) Successful in 1m3s
ci / rust (push) Successful in 4m52s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 59s
apple / screenshots (push) Successful in 5m12s
ci / bench (push) Successful in 4m40s
windows-host / package (push) Successful in 6m28s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m17s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m13s
release / apple (push) Successful in 10m9s
deb / build-publish (push) Successful in 2m44s
decky / build-publish (push) Successful in 11s
android / android (push) Successful in 3m33s
flatpak / build-publish (push) Successful in 4m9s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m5s
docker / deploy-docs (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m41s
The pnputil /add-driver call in windows/install.rs was committed unwrapped;
`cargo fmt --all --check` (which checks cfg(windows) files too) flagged it and
failed the `rust` CI job at the Format step, skipping clippy/build/test. Apply
rustfmt — no behavior change. Clears the way to cut the v0.2.0 release from
green main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 23:19:12 +00:00
enricobuehler 6e949b6748 fix(readme): make the logo readable on light + dark themes
apple / swift (push) Successful in 1m3s
apple / screenshots (push) Successful in 5m25s
ci / rust (push) Failing after 1m5s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 1m0s
android / android (push) Successful in 3m59s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
ci / bench (push) Successful in 4m35s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m55s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
docker / deploy-docs (push) Successful in 7s
The wordmark was light violet only — low-contrast on a light README
background. Swap to a single theme-adaptive SVG: an internal
`prefers-color-scheme` media query paints it deep violet (the brand-mark
palette) on light backgrounds and the original light violet on dark, so it
reads on both GitHub/Gitea themes with no markup change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:54:03 +00:00
enricobuehler 8ae161fe61 docs(windows): README - install via punktfunk-host.exe driver install / web setup (not .ps1)
apple / swift (push) Successful in 1m0s
windows-host / package (push) Successful in 6m20s
apple / screenshots (push) Successful in 5m26s
ci / rust (push) Failing after 26s
ci / web (push) Successful in 54s
deb / build-publish (push) Successful in 2m30s
ci / docs-site (push) Successful in 1m3s
android / android (push) Successful in 3m19s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m35s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m2s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m48s
docker / deploy-docs (push) Successful in 6s
Option A removed install-pf-vdisplay.ps1 / install-gamepad-drivers.ps1 / web-setup.ps1;
the installer now calls the exe subcommands. Drop the stale table rows + reword the
install-flow + 'thin installer' notes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:46:05 +00:00
enricobuehler 3a89ee8cd7 docs(readme): add logo banner + refresh Windows-host status
- Add the centered punktfunk wordmark banner at the top (assets/punktfunk-logo.svg,
  the same logo + layout the marketing site's README uses).
- Refresh the now-stale Windows-host facts: all-vendor (NVENC + AMF/QSV), its own
  all-Rust pf-vdisplay IddCx virtual display (was SudoVDA), bundled UMDF virtual-gamepad
  drivers (ViGEmBus gone), HDR incl. Vulkan-game HDR; x64-only, no longer NVIDIA-only.
- Note punktfunk-host covers Linux + Windows; point design/ at its new README index.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:45:29 +00:00
enricobuehler dac0fee4e3 docs(windows): reflect the install-via-exe (Option A) landing in the build/packaging doc
apple / swift (push) Successful in 1m3s
apple / screenshots (push) Successful in 5m31s
ci / web (push) Successful in 49s
decky / build-publish (push) Successful in 14s
ci / rust (push) Failing after 32s
ci / docs-site (push) Successful in 1m1s
android / android (push) Successful in 3m21s
deb / build-publish (push) Successful in 2m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
ci / bench (push) Successful in 4m49s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m32s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m47s
docker / deploy-docs (push) Successful in 6s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:44:47 +00:00
enricobuehler 125a51d81d feat(windows-installer): move driver + web install into the host exe (ASCII root fix)
apple / swift (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m16s
windows-host / package (push) Successful in 6m25s
ci / rust (push) Failing after 28s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m1s
android / android (push) Successful in 3m21s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m2s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m50s
docker / deploy-docs (push) Successful in 17s
Port the three install-time PowerShell *files* (install-pf-vdisplay.ps1,
install-gamepad-drivers.ps1, web-setup.ps1) into punktfunk-host.exe subcommands:
`driver install [--gamepad] --dir <stage>` and `web setup --app-dir <app>
[--password-file <f>]` (windows/install.rs).

Why: PowerShell 5.1 reads a BOM-less .ps1 FILE in the machine ANSI codepage, so a
stray non-ASCII byte mis-decodes and aborts on a non-English box - exactly how the
pf-vdisplay driver install silently failed. A compiled subcommand drives the same
external tools (certutil/pnputil/nefconc/schtasks/netsh/icacls) as fixed string
literals, with no file-codepage surface. (The .iss's INLINE -Command PowerShell is a
command-line string, not a file read, so it's unaffected and stays.)

- windows/install.rs: faithful port - cert trust, gated nefconc node create + pnputil
  for pf-vdisplay; pnputil per-inf for gamepads; web-password ACL, the PunktfunkWeb task
  (generated UTF-16 XML), firewall rule, start. Best-effort (a hiccup warns, never aborts).
- punktfunk-host.iss [Run]: call the exe instead of `powershell -File`; drop the
  web-setup.ps1 staging + WebSetup define; WebSetupParams emits --app-dir/--password-file.
- pack-host-installer.ps1: stop copying the three install scripts into the stages.
- delete the three .ps1 files.

The `mod install;` + dispatch arms in main.rs landed in the preceding docs commit
(swept up by a concurrent commit); this commit adds the module + installer wiring.
CI-compile-validated via windows-host; the install path is on-glass-validated on the
next canary install (the test box is offline).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:43:18 +00:00
enricobuehler 7b99b41ede docs(design): trim shipped plans, consolidate cluster, add index
Much of design/ described work that has since shipped. Trim each doc to
its durable rationale + still-open items (the code is the source of truth
for shipped detail; git history holds the full originals).

- Shipped plans -> status stubs: stats-capture, gamestream-host-plan,
  apple-stage2-presenter, windows-service.
- Trimmed completed-out / open-kept: implementation-plan, hdr-pipeline,
  host-latency, gpu-contention (fixed stale status table), game-library,
  linux-setup (fixed m0->spike + stale zero-copy claim),
  session-aware-host-followups, windows-client-bootstrap,
  windows-dualsense-{scoping,game-detection}, windows-virtual-display,
  security-review (per-finding status table; #12 still open),
  apollo-comparison (shipped backlog collapsed to one-liners).
- Windows-host cluster consolidated: windows-host.md -> redirect into
  windows-host-rewrite.md (whose stale scorecard is corrected -- goal1 is
  merged, M4 done); windows-secure-desktop.md archived (now a fallback
  behind IDD-push primary).
- Kept evergreen: ci.md, gamescope-multiuser.md, windows-build-and-packaging.md.
- New design/README.md: per-doc status table + consolidated open-items
  roll-up so nothing is tracked in only one buried doc.
- Repoint 5 code comments to the archived secure-desktop doc path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:39:06 +00:00
enricobuehler 9ea2c17419 docs(windows): add design/windows-build-and-packaging.md + refresh packaging README
apple / swift (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m19s
windows-host / package (push) Successful in 6m20s
android / android (push) Successful in 4m42s
ci / rust (push) Successful in 4m47s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 58s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
ci / bench (push) Successful in 4m40s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m16s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m3s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m0s
docker / deploy-docs (push) Successful in 22s
A single repo-internal source of truth for the Windows build/packaging: what ships, the
all-Rust driver workspace built FROM SOURCE in CI (+ the anti-stale rationale), the
toolchain (clang 22 + bindgen 0.72, no LLVM pin), the Inno installer, the web console
bundle, the CI workflows, signing, and the dev loop. (design/, not the docs-site.)

packaging/windows/README.md: drop the deleted vendored-driver dir + its "Vendored driver"
callout, add the build-* / install-gamepad / clear-force-integrity rows, point at the new
design doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:22:40 +00:00
enricobuehler a9cca82fb8 chore(windows): clean up build/packaging - drop vendored driver binaries + the LLVM-21 pin
windows-drivers-provision / provision (push) Successful in 13s
windows-drivers / probe-and-proto (push) Successful in 17s
android / android (push) Failing after 40s
apple / swift (push) Successful in 1m0s
ci / web (push) Successful in 58s
windows-drivers / driver-build (push) Successful in 1m9s
ci / docs-site (push) Successful in 1m18s
ci / rust (push) Successful in 4m25s
apple / screenshots (push) Successful in 5m24s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
deb / build-publish (push) Successful in 2m29s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
ci / bench (push) Successful in 4m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
windows-host / package (push) Successful in 6m38s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m31s
docker / deploy-docs (push) Successful in 18s
Now that the drivers build from source in CI, remove the dead checked-in binaries and
the toolchain cruft they left behind:

- Delete packaging/windows/{pf-vdisplay,gamepad-drivers}/ (the prebuilt .dll/.inf/.cat/.cer).
  pack-host-installer.ps1 builds + signs all three drivers from the drivers/ workspace and
  nothing reads the vendored dirs anymore; stage-pf-vdisplay.ps1's -VendorDir is now a
  mandatory build-output path, not a vendored default.
- Drop the LLVM-21 pin. The vendored bindgen 0.71->0.72 bump (the shipping pack already
  builds green on the runner-default clang 22) retired the bindgen-0.71 layout-test overflow
  that needed LLVM 21.1.2, so windows-drivers.yml + provision-windows-wdk.ps1 no longer
  install/point at C:\llvm-21 (~898 MB off a fresh provision) - both driver builds now use one
  toolchain (clang 22 + bindgen 0.72).
- pack -SkipBuild on the gamepad build (build-pf-vdisplay.ps1 already builds the whole
  workspace), build-web.ps1 reaps a stale node too, deploy-dev.ps1 nefconc path + comments.
- Reword the vendored-driver references (build scripts, .iss, READMEs, the vite web-bundle
  comment) to the build-from-source reality.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:16:46 +00:00
enricobuehler 7ab0661ddc fix(windows-installer): escape the brace in the [UninstallRun] PowerShell so ISCC compiles
windows-drivers / probe-and-proto (push) Successful in 21s
apple / swift (push) Successful in 1m4s
windows-drivers / driver-build (push) Successful in 1m9s
android / android (push) Successful in 4m25s
ci / web (push) Successful in 53s
apple / screenshots (push) Successful in 5m32s
ci / rust (push) Successful in 4m45s
ci / docs-site (push) Successful in 52s
windows-host / package (push) Successful in 6m47s
deb / build-publish (push) Successful in 2m28s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m45s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m55s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
The Bug C [UninstallRun] one-liner had `ForEach-Object { Stop-Process ... }`; Inno
Setup parses `{...}` as a constant in [Run]/[UninstallRun] sections, so ISCC aborted
with "Unknown constant" and the windows-host pack failed at the ISCC step (the host
build, clippy, driver build + web smoke-boot all passed). Escape `{` as `{{`. The
same one-liner in the [Code] StopWebConsole proc is inside a Pascal string literal,
so its brace is literal and must NOT be escaped. Validated: ISCC now parses past
[UninstallRun] + [Code] (fails only later on the absent dummy payload).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 15:15:07 +00:00
enricobuehler 92e68024f1 fix(windows-installer): build the gamepad drivers from source in CI too
Fold the pf-dualsense (DualSense / DualShock 4) and pf-xusb (Xbox 360 / XInput)
UMDF drivers into the in-tree drivers workspace (their source had stale
../../crates/wdk-* path-deps from before the wdk vendoring reorg and could no
longer build at all) and build them from source per release, exactly like
pf-vdisplay - same anti-stale reasoning. One `cargo build --release` now builds
all three drivers against the vendored wdk-sys (incl. the bindgen 0.72 pin), and
build-gamepad-drivers.ps1 signs pf_dualsense + pf_xusb (clear FORCE_INTEGRITY ->
sign dll -> stampinf -> Inf2Cat -> sign cat) with one shared cert + .cer,
matching the layout install-gamepad-drivers.ps1 expects. pack-host-installer.ps1
builds + stages them instead of the retired checked-in binaries.

Validated on the runner: the whole workspace (pf-vdisplay + pf-dualsense +
pf-xusb) builds with CARGO_TARGET_DIR=C:\t set, and build-gamepad-drivers.ps1
produces signed pf_dualsense.{dll,inf,cat} + pf_xusb.{dll,inf,cat} + the .cer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 15:08:40 +00:00
enricobuehler 64abce6daa fix(windows-installer): pf-vdisplay CI build - default target dir + non-fatal cat guard
apple / swift (push) Successful in 59s
android / android (push) Successful in 4m23s
ci / rust (push) Successful in 4m43s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
windows-host / package (push) Failing after 5m39s
apple / screenshots (push) Successful in 5m15s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m6s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m52s
The CI driver build panicked in wdk-sys's build script - "a Cargo.lock file should
exist in the same directory as the top-level Cargo.toml". wdk-build's
find_top_level_cargo_manifest() walks UP from OUT_DIR for the first ancestor holding a
Cargo.lock and explicitly does NOT support non-default target dirs - but
build-pf-vdisplay.ps1 pointed CARGO_TARGET_DIR at an out-of-tree dir (to isolate from
CI's shared C:\t), so no ancestor of OUT_DIR had a Cargo.lock. Build into the driver
workspace's DEFAULT target dir instead (its ancestors include the driver Cargo.lock);
the driver's own [workspace] already isolates it and it has no CMake deps needing C:\t.
Also make the Test-FileCatalog coverage guard non-fatal (it can't open a catalog
signed by a not-yet-trusted cert). Validated on the runner with CARGO_TARGET_DIR=C:\t.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 14:58:20 +00:00
enricobuehler bdfab8e0d5 fix(windows-installer): build pf-vdisplay from source in CI; ASCII scripts; upgrade-safe web console
windows-drivers / probe-and-proto (push) Successful in 24s
apple / swift (push) Successful in 1m4s
windows-drivers / driver-build (push) Successful in 1m8s
android / android (push) Successful in 4m4s
ci / rust (push) Successful in 4m39s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 53s
apple / screenshots (push) Successful in 5m10s
windows-host / package (push) Failing after 5m35s
deb / build-publish (push) Successful in 2m29s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
ci / bench (push) Successful in 4m42s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m57s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
The pf-vdisplay virtual-display driver shipped as a checked-in PREBUILT binary
that went stale - two field failures on a fresh install (live-repro'd on a
German-locale Dell laptop):

  * Bug A (every box): a repo-wide rename edited the vendored pf_vdisplay.inf
    but never re-signed pf_vdisplay.cat, so the catalog stopped covering the INF
    -> `pnputil /add-driver` fails SPAPI_E_FILE_HASH_NOT_IN_CATALOG -> driver
    never installs -> every session dies "pf-vdisplay driver interface not
    found".
  * the prebuilt binary also predated IOCTL_SET_RENDER_ADAPTER (added to the
    driver source after the vendor freeze) that the host needs to pin the IDD
    render GPU on hybrid/Optimus boxes.

Fix: build the driver FROM SOURCE every release (build-pf-vdisplay.ps1, wired
into pack-host-installer.ps1) so .dll/.inf/.cat are always in lockstep and
current driver features ship. The runner's clang 22 made the driver's pinned
bindgen 0.71 emit opaque structs (157 layout-assert errors), so bump the
vendored wdk-sys/wdk-build bindgen 0.71 -> 0.72 (+ lock). The build self-signs
the driver per build (installer trusts the bundled .cer); a stable
DRIVER_CERT_PFX_B64 secret can override.

  * Bug B (non-English boxes): the installer runs install-pf-vdisplay.ps1 etc.
    via powershell.exe (5.1), which reads a BOM-less script in the ANSI codepage
    - an em-dash's trailing 0x94 byte becomes a curly quote on German
    Windows-1252 and the script aborts "unterminated string", so the driver
    never installed (the gamepad script survived only because it was already
    ASCII). Scrub every installer-run .ps1/.cmd to ASCII + add a CI gate that
    fails on any non-ASCII so it can't regress.

  * Bug C (upgrades): nothing stopped the OLD web console before re-registering
    its task, so a stale server kept :3000 (the new one restart-looped on
    EADDRINUSE) and served a broken old bundle (500 on /login). Stop + reap it
    (runtime-agnostic, by the :3000 listener owner) in web-setup.ps1 and in the
    .iss before the file copy + on uninstall.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 14:33:34 +00:00
enricobuehler 8e87e617df fix(windows-host): force EXTEND topology so a new IddCx display isn't cloned
A freshly-added IddCx virtual display lands in CLONE/duplicate mode when a
physical display is already active (a laptop panel, an attached monitor): the
cloned output shares that display's source, so the OS never commits a distinct
path for it, never calls ASSIGN_SWAPCHAIN, and capture sees no frames - the
session fails "not an active display path / needs a WDDM GPU to activate" and
tears down with 0 frames (seen live on an Intel-iGPU + NVIDIA-Optimus laptop).

force_extend_topology() applies the EXTEND preset (the programmatic Win+P
"Extend") right after ADD so the IDD comes up as its own active path; the
existing resolve_gdi_name -> set_active_mode -> isolate_displays_ccd bring-up
then proceeds. Idempotent / no-op on a sole-display (headless single-GPU) box,
so it's safe on the path that already worked.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 14:33:15 +00:00
enricobuehler 5bf787eb2b feat(host): web-console performance capture — record stream stats, graph them
apple / swift (push) Successful in 1m1s
android / android (push) Successful in 4m13s
ci / rust (push) Successful in 4m42s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 53s
windows-host / package (push) Successful in 5m51s
apple / screenshots (push) Successful in 5m1s
deb / build-publish (push) Successful in 2m29s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
ci / bench (push) Successful in 4m35s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m9s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m10s
Arm streaming-perf-stats capture from the web console, play, stop, and review the
run as graphs; finished captures are saved to disk as browsable/exportable
recordings. Covers both the native punktfunk/1 path and GameStream.

- stats_recorder.rs: one shared Arc<StatsRecorder> ring (created in gamestream::serve,
  shared with the mgmt API + both streaming loops, mirroring NativePairing). The
  hot-path gate is a runtime AtomicBool that replaces the startup-only PUNKTFUNK_PERF
  for *recording* (PERF stdout logging unchanged); bounded ring (~3 h); atomic
  temp+rename writes to ~/.config/punktfunk/captures/*.json; path-traversal-safe ids;
  poison-resilient locks.
- native (punktfunk1.rs) + GameStream (stream.rs) emit a StatsSample at their existing
  ~2 s / ~1 s aggregation boundary — per-stage latency p50/p99, fps new/repeat, goodput,
  loss/FEC deltas — with no new per-frame work beyond the cheap atomic check.
  FrameMsg.was_measured keeps pre-arm in-flight frames out of the first window's
  percentiles (without zeroing the Windows-relay path's fps/encode).
- mgmt.rs: 7 bearer-only /api/v1/stats/* endpoints (capture start/stop/status/live;
  recordings list/get/delete); api/openapi.json regenerated, in sync.
- web: new "Performance" page (recharts, rendered SSR-safe) — capture control, live
  graphs while armed, recordings table (view / download-JSON / delete), and a detail
  view with the latency stacked-area bottleneck breakdown (p50/p99 toggle) + throughput
  + health. Charts adapt to either path's stage set.

Design: design/stats-capture-plan.md. Built and adversarially reviewed via a multi-agent
workflow; workspace build/clippy(-D warnings)/fmt/tests green, OpenAPI no-drift. Not yet
on-glass validated against a live session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 13:59:39 +00:00
enricobuehler 0a6c9d8852 docs: point Android install at Discord for beta access + add community links
apple / swift (push) Successful in 1m32s
apple / screenshots (push) Successful in 3m26s
android / android (push) Successful in 4m7s
ci / rust (push) Successful in 4m36s
ci / web (push) Successful in 44s
ci / docs-site (push) Successful in 53s
deb / build-publish (push) Successful in 2m18s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 42s
ci / bench (push) Successful in 4m42s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m12s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m8s
docker / deploy-docs (push) Successful in 6s
The Android app is in Google Play Internal Testing, so the public Play Store URL
doesn't resolve for non-testers. Lead the Android install instructions with a
"request a tester invite on Discord" CTA (the Play listing unlocks once a Google
account is added to the test track), and surface the Discord + r/Punktfunk
community links in the README, the docs intro, and the docs-site nav.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 11:59:25 +00:00
enricobuehler 0eedfb3c1f docs: first-class Linux + Windows positioning + IDD-push differentiator
apple / swift (push) Failing after 0s
apple / screenshots (push) Has been skipped
windows-drivers-provision / provision (push) Successful in 13s
windows-drivers / probe-and-proto (push) Successful in 17s
windows-drivers / driver-build (push) Successful in 1m10s
android / android (push) Successful in 3m19s
ci / web (push) Successful in 39s
ci / docs-site (push) Successful in 53s
windows-host / package (push) Successful in 6m6s
ci / rust (push) Successful in 11m12s
decky / build-publish (push) Successful in 11s
ci / bench (push) Successful in 5m9s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 21s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
deb / build-publish (push) Successful in 7m31s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m14s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m12s
release / apple (push) Failing after 1s
docker / deploy-docs (push) Successful in 19s
flatpak / build-publish (push) Successful in 4m43s
Drop the "Linux-first" framing across the README and docs site in favor of
first-class Linux AND Windows hosts, and surface the Windows IDD-push
virtual-display path as a distinct differentiator (punktfunk's own indirect
display driver the host pushes frames into — a real virtual display, no physical
monitor or dummy plug, even on the secure desktop).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 11:53:02 +00:00
enricobuehler f6490f4c28 fix: complete the docs/→design/ and openapi→api/ rename references
The file moves (docs/ → design/, docs/api/openapi.json → api/openapi.json) landed
in d01a8fd, but the matching reference updates did not — so mgmt.rs's drift-test
`include_str!("../../../docs/api/openapi.json")` pointed at a path that no longer
exists and the host failed to build. This restores it and updates every reference:

  - mgmt.rs include_str! → ../../../api/openapi.json (fixes the build)
  - web/orval.config.ts codegen target, web/Dockerfile, .dockerignore
  - deb/rpm/Arch packaging install paths
  - CLAUDE.md, the .gitea CI workflows, code doc-comments, design-doc cross-links

docs-site route URLs (/docs/...) untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 11:53:02 +00:00
enricobuehler d01a8fd17a feat(host): HDR Vulkan layer so Vulkan games get HDR on the virtual display
ci / web (push) Failing after 22s
windows-host / package (push) Failing after 4m16s
ci / rust (push) Failing after 4m56s
ci / docs-site (push) Successful in 1m7s
android / android (push) Successful in 9m19s
ci / bench (push) Successful in 4m47s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 3s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / deploy-docs (push) Has been skipped
deb / build-publish (push) Failing after 6m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 7m4s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 7m17s
apple / swift (push) Successful in 1m13s
apple / screenshots (push) Successful in 5m27s
NVIDIA/AMD Vulkan ICDs refuse to *advertise* an HDR color space for a surface on an
IddCx indirect/virtual display, so Vulkan games (Doom: The Dark Ages, id Tech, Indiana
Jones, …) report "device does not support HDR" — even though Windows HDR, DWM compose,
and the client PQ stream all work, and the ICD happily *accepts + presents* a forced HDR
swapchain there. The whole gap is enumeration; the community (Apollo/Sunshine/VDD) wrote
this off as kernel-side / unfixable.

Add VK_LAYER_PUNKTFUNK_hdr_inject (packaging/windows/pf-vkhdr-layer/): a standalone
cdylib Vulkan implicit layer that appends {A2B10G10R10, HDR10_ST2084} + {RGBA16F, scRGB}
to vkGetPhysicalDeviceSurfaceFormats[2]KHR (no need to hook vkCreateSwapchainKHR — the
ICD doesn't validate the color space there). Self-gated on the surface monitor's actual
advanced-color state (DisplayConfig GET_ADVANCED_COLOR_INFO), so it is a complete no-op
on SDR sessions and real monitors (dedup). Always-on (registry-discovered) so it works
regardless of how a game is launched — env-scoping silently fails for already-running
Steam. Escape hatches: DISABLE_PF_VKHDR, PF_VKHDR_EXCLUDE, and a built-in kernel-anti-
cheat denylist.

The installer builds/signs/stages it and registers it under
HKLM64\SOFTWARE\Khronos\Vulkan\ImplicitLayers (opt-out "Install the HDR Vulkan layer"
task); windows-host CI fmt+clippy-gates it (msvc-only FFI).

Live-validated on the RTX box: Doom: The Dark Ages enables HDR over the pf-vdisplay
virtual display.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 11:33:20 +00:00
enricobuehler 3e7c9bd059 fix(host): remove unsound unsafe impl Sync for HelperRelay
apple / screenshots (push) Has been skipped
windows-drivers / probe-and-proto (push) Successful in 29s
audit / cargo-audit (push) Failing after 1m20s
windows-drivers / driver-build (push) Successful in 1m14s
ci / web (push) Successful in 46s
ci / docs-site (push) Successful in 1m3s
windows-host / package (push) Successful in 6m46s
apple / swift (push) Failing after 0s
release / apple (push) Failing after 0s
android / android (push) Failing after 2m5s
ci / bench (push) Successful in 4m34s
decky / build-publish (push) Successful in 22s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m25s
ci / rust (push) Successful in 8m36s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m11s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 59s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m37s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m3s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
deb / build-publish (push) Successful in 7m50s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m52s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m33s
flatpak / build-publish (push) Successful in 3m56s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m46s
docker / deploy-docs (push) Successful in 22s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m26s
The one genuine soundness defect the unsafe-proof program surfaced (flagged
SUSPECT in program 3/N). `HelperRelay` holds an `rx: Receiver<RelayAu>`, which is
`!Sync` (std mpsc is single-consumer), so asserting `Sync` claimed more than the
fields support — an `Arc<HelperRelay>` recv'd from two threads would compile and
be UB.

It was never live-exploited, and it turns out `Sync` is also unnecessary: the
relay is a single-owner `mut relay` local in the punktfunk1 two-process mux loop
(recv_timeout/try_recv/request_keyframe all called on the owning thread; no `Arc`,
no `thread::spawn` capturing it). So the fix is simply to delete the impl — the
struct keeps its sound `unsafe impl Send` (needed for the raw `HANDLE` fields),
which is all the code uses.

Box-verified: cargo clippy -p punktfunk-host --features nvenc --target
x86_64-pc-windows-msvc -- -D warnings stays green without the Sync impl.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 10:00:40 +00:00
enricobuehler 7aa787a789 docs(host): prove the last 3 files + crate-root deny (unsafe-proof program 4/N, final)
Completes the unsafe-proof program now that the parallel WIP has landed:

- idd_push.rs (25 sites), nvenc.rs (7), punktfunk1.rs (21): a SAFETY proof on
  every unsafe block — D3D11/DXGI COM (same-device textures, immediate-context
  single-thread, keyed-mutex-held convert), the NVENC SDK table (versioned POD,
  register/map/lock-bitstream pairing), cross-process shm reads (atomic
  magic/generation handshake), and the C-ABI harness (each call cross-checked
  against its abi.rs `# Safety` doc). No SUSPECT (UB) blocks.
- capture.rs / encode.rs: the parent-module deny is restored (their WIP children
  are now proven), and main.rs gains a crate-root
  #![deny(clippy::undocumented_unsafe_blocks)] — the permanent catch-all gate so
  no future unsafe block anywhere in the crate can land without a proof.
- Fixed 4 blocks the agents missed: unsafe blocks nested inside `assert_eq!(...)`
  macro args (the comment-above-statement didn't associate) — hoisted to a `let`.
- rustfmt-canonicalized the Windows files (the agents' SAFETY comments + some
  pre-existing 1.9.0 drift) so `cargo fmt --all --check` is clean.

Verified: cargo clippy -p punktfunk-host --all-targets -- -D warnings AND
cargo fmt -p punktfunk-host --check both green with the crate-root deny active.
Windows cfg(windows) re-verified on the box next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 09:57:00 +00:00
enricobuehler 3514702d8c feat(windows-host): IDD-push encodes native NV12/P010 (skip NVENC's SM-side CSC)
GPU-contention work (host-latency plan §5.A): the IDD-push output ring now hands
NVENC native YUV instead of RGB, so NVENC skips its internal RGB→YUV colour
conversion on the SM/3D engine the running game saturates.

- idd_push.rs: out_ring is now NV12 (SDR, BT.709 limited) via a D3D11 VIDEO-engine
  BGRA→NV12 VideoConverter (keeps the CSC off the contended 3D/compute engine), or
  P010 (HDR, BT.2020 PQ limited) via the FP16→P010 shader (NVIDIA's VideoProcessor
  can't do RGB→P010). The ring drops its per-slot RTV (textures only), matching the
  WGC YUV ring; converters rebuild on a size/HDR flip.
- nvenc.rs: NV12 input forces bit_depth=8 so an HDR→SDR toggle (or a 10-bit-
  negotiated client on an SDR display) re-inits the session at the matching depth —
  NV12 can't feed a 10-bit session (register_resource rejects it).
- punktfunk1.rs: per-stage latency instrumentation under PUNKTFUNK_PERF
  (cap=try_latest, submit=encode_picture, wait=lock_bitstream µs p50/p99/max) to
  pinpoint where capture→encoded latency goes under GPU saturation.
2026-06-26 09:35:23 +00:00
enricobuehler 327a5fa828 docs(host): prove unsafe blocks in the Windows + cross-platform files + gate them (unsafe-proof program 3/N)
Continues the unsafe-proof program across the Windows/cross-platform host files
(~75 blocks, 21 files), each with a SAFETY proof of the real invariant and a
per-file #![deny(clippy::undocumented_unsafe_blocks)] gate:

  capture/windows: dxgi.rs, wgc_relay.rs, wgc.rs, desktop_watch.rs, composed_flip.rs
                   (windows-rs COM: interface validity, same-D3D11-device textures,
                    immediate-context single-thread, borrowed args outlive the call)
  windows: service.rs (SCM/token/CreateProcessAsUserW/event handles — OwnedHandle
           liveness, no double-close/signal race), win_display, wgc_helper, interactive
  vdisplay/windows: manager.rs, pf_vdisplay.rs (SwDeviceCreate/IddCx/ioctl handle
                    liveness via the OnceLock VDM singleton + OwnedHandle)
  encode/windows: ffmpeg_win.rs (full AVBufferRef refcount audit — balanced, NO leaks,
                  unlike the vaapi sibling), sw.rs
  cross-platform: gamestream/audio.rs (libopus), gamestream/stream.rs (sendmmsg),
                  inject/windows/sendinput.rs, audio/windows/wasapi_mic.rs,
                  session_tuning.rs, vdisplay.rs

Two findings (handled separately):
- wgc_relay.rs `unsafe impl Sync for HelperRelay` is UNSOUND (its mpsc Receiver is
  !Sync) though not live-exploited — marked SUSPECT inline; fix pending box check
  (it touches the in-flight punktfunk1.rs).
- capture.rs / encode.rs (PARENT modules of the WIP idd_push.rs / nvenc.rs) do NOT
  get the file deny yet — it would propagate the lint into the undocumented WIP
  children. The deny lands there once those are documented (after the WIP commits).

Linux-visible parts verified green (cargo clippy -p punktfunk-host --all-targets
-- -D warnings). The cfg(windows) deny gates are box-verified next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 09:23:25 +00:00
enricobuehler 9777ed7fb3 fix(host/vaapi): plug two AVBufferRef leaks in DmabufInner::open
Surfaced while writing the unsafe-soundness proofs (2/N): both are refcount
leaks (sound — never dangling/double-free — so the SAFETY proofs held, but real
bugs on the persistent punktfunk1-host listener that opens a fresh encoder per
session).

1. Per-session leak: `par->hw_frames_ctx = av_buffer_ref(drm_frames)` created a
   second owned ref. `av_buffersrc_parameters_set` takes its OWN ref of
   `par->hw_frames_ctx`, and `av_free(par)` frees only the struct, not the ref —
   so the extra ref leaked every session, pinning the DRM frames ctx + device.
   Fix: assign `drm_frames` borrowed (the standard ffmpeg pattern); our single
   owned ref lives in DmabufInner and is unref'd in Drop.

2. Error-path leak: the final `open_vaapi_encoder(...)?` returned without the
   unref ladder every other error path runs, leaking graph/drm_frames/
   vaapi_device/drm_device on encoder-open failure. Fix: match + clean up before
   returning (nv12_ctx is borrowed from the sink → freed by graph teardown).

cargo clippy -p punktfunk-host --all-targets -- -D warnings clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 09:02:54 +00:00
enricobuehler ba68a98873 docs(host): prove every unsafe block in the Linux FFI files + gate them (unsafe-proof program 2/N)
Continues the structural unsafe-proof program (every unsafe carries a documented
proof of soundness; the file gains #![deny(clippy::undocumented_unsafe_blocks)]
so it stays proven). This batch covers all 10 remaining pure-Linux files
(104 blocks), each proof stating the REAL invariant — not boilerplate:

  zerocopy/cuda.rs (26)   leaked process-lifetime libcuda fn-ptr table; opaque
                          CUcontext never dereferenced; free-exactly-once via the
                          Arc<Mutex<PoolInner>> ownership graph; dmabuf fd take/close split
  zerocopy/egl.rs (18)    eglGetProcAddress'd procs with the GL context current;
                          EGLImage liveness; the two-call modifier-query bounds
  zerocopy/vulkan.rs (4)  copy-bounds arithmetic (src_size>=span); Send = thread
                          confinement to the punktfunk-pipewire thread
  dmabuf_fence.rs (4)     poll/ioctl/close fd liveness + ownership
  capture/linux/mod.rs (16)  spa_data repr(transparent) cast; null-checked spa
                          derefs; single-loop-thread buffer ownership until requeue
  inject/linux/gamepad.rs (10)  uinput ioctl request-number ↔ struct-size match
                          (static-asserted); InputEventRaw no-padding for the byte cast
  encode/linux/vaapi.rs (15) + encode/linux/mod.rs (9)  ffmpeg object ownership/
                          free ladders; VAAPI/DRM graph; Send = single-thread transfer
  inject/linux/wlr.rs (2), vdisplay/linux/kwin.rs (1)

No memory-unsafety SUSPECT blocks were found — the unsafe is sound. The vaapi
agent did flag two real AVBufferRef *leaks* (not UB) in DmabufInner::open; marked
inline with NOTE(leak) and addressed in a follow-up.

Verified: cargo clippy -p punktfunk-host --all-targets -- -D warnings is clean
(each file's deny gate hard-errors on any undocumented block).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 09:00:30 +00:00
enricobuehler 22359f5dc8 docs(host): prove every unsafe block in drm_sync.rs + gate it (unsafe-proof program 1/N)
Start of the structural unsafe-proof program (per the "every unsafe needs a
documented proof of soundness" goal): each `unsafe` block gets an accurate
`// SAFETY:` proof of WHY it is sound, and the file gains
`#![deny(clippy::undocumented_unsafe_blocks)]` so the proof requirement is
permanently enforced (a future undocumented unsafe in this file fails CI).

drm_sync.rs (10 blocks: libc open/ioctl/clock_gettime/close + 3 in tests): each
proof states the real invariant — fd liveness/ownership, the ioctl request number
encoding the matching struct size, the `&mut req` being a live correctly-sized
`#[repr(C)]` struct, and (for the timeline ioctls) the `handles`/`points` arrays
outliving the synchronous call with `count_handles` matching their length.

The gate grows file-by-file (CI stays green; undone files don't carry the lint
yet); it promotes to a crate-root deny once every file is done. ~122 Linux blocks
+ the Windows files remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:35:32 +00:00
enricobuehler 7e9023faad feat(gamestream): launch apps on Windows + Linux non-gamescope hosts
GameStream's apps.json `cmd` is delivered via set_launch_command, which ONLY the Linux
gamescope backend nests. On Windows (no gamescope) and Linux kwin/mutter/wlroots (which
stream the existing desktop) the command was silently dropped. Now, after capture is live,
stream.rs spawns it via library::launch_gamestream_command for those backends — Windows:
into the interactive USER session (spawn_in_active_session, since the host is SYSTEM);
Linux: a plain `sh -c` spawn into the host's own graphical session so the app lands on the
streamed (primary) output. Linux gamescope keeps nesting via set_launch_command and is
skipped here to avoid a double launch. The command is operator-typed apps.json (trusted),
never client-set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:12:53 +00:00
enricobuehler 5acc12d9e9 feat(library): shared cover-art warmer + cache (GOG + Xbox art)
A disk-backed art cache (library-art-cache.json in the canonical host config dir) is the
source of truth read by all_games(), so the library list + launch-resolve never block on
the network. A host-lifetime background warmer (start_art_warmer, started in serve())
fetches uncached art OFF the hot path: GOG via the public no-auth api.gog.com product API,
Xbox via the unofficial no-auth displaycatalog (keyed by StoreId). Both best-effort
(protocol-relative URLs normalized to https; results cached even when empty so they aren't
re-fetched). The GOG + Xbox providers now read cached_art() (title-only until warmed).

Cross-platform (ureq blocking HTTP — no tokio on this path) so the fetch/parse code is
compiled + checked everywhere; a host whose stores all self-provide art (Steam CDN /
Heroic CDN / Lutris data: URLs) does no fetching. Dep: ureq (webpki roots, no system certs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:00:31 +00:00
enricobuehler aed0bf0c2a feat(library): Windows Xbox / Game Pass store provider
XboxProvider scans each fixed drive's <drive>:\XboxGames for GDK games (presence of
Content\MicrosoftGame.config marks a game vs. an ordinary UWP app), parsing title /
Identity name / Executable Id / StoreId via roxmltree. The PackageFamilyName is READ
from the AppRepository\Packages\<PackageFullName> dir name (reduced to Name_Hash) —
never computed from the publisher. Launch via the AUMID (shell:AppsFolder\<PFN>!<AppId>)
through explorer in the interactive user session (UWP activation needs the user token,
which spawn_in_active_session already provides). Cover art (displaycatalog) is deferred
→ title-only. Known v1 gaps: custom .GamingRoot install folders + non-GDK pure-UWP Store
games (under the ACL-locked WindowsApps) aren't enumerated.

New windows_launch_for `aumid` arm; XboxProvider wired into all_games() under cfg(windows).
Dep: roxmltree (Windows). Windows unit tests cover MicrosoftGame.config parsing (incl. the
ms-resource title fallback), the PackageFullName→PFN reduction, and the aumid launch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:49:03 +00:00
enricobuehler b65745284e feat(library): Windows Epic + GOG store providers
EpicProvider reads the launcher's local .item manifests under %ProgramData% (no auth,
launcher need not run) with Playnite's exclusion filter (skip UE_* components +
non-launchable addons + dead install dirs); cover art from the base64 catcache.bin
(public Epic CDN, best-effort). Launch via the com.epicgames.launcher:// URI opened
through explorer.exe — the namespace:catalogItemId:appName triple, with a bare-appName
fallback so a launch is never dropped.

GogProvider enumerates HKLM\SOFTWARE\WOW6432Node\GOG.com\Games (winreg) + each
goggame-<id>.info primary FileTask into a direct-exe spawn (no Galaxy, dodges its
cold-start/anti-cheat). GOG cover art (public api.gog.com) is deferred — it needs an
HTTP fetch + cache off the hot all_games() path — so GOG is title-only for now.

windows_launch_for gains epic/gog arms; both providers wired into all_games() under
cfg(windows). Deps: base64 moved to the cross-platform table (Epic catcache decode +
Lutris art encode both need it); winreg added on the Windows target. Windows unit tests
cover the Epic exclusion filter + URI builder and the GOG spawn + play-task parsing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:37:30 +00:00
enricobuehler 8ca695eb4c docs(windows-host): SCM event redesign done + runtime-validated (D2 complete)
The service.rs STOP/SESSION events are now OnceLock<OwnedHandle> (61c02e6) — the
last host-side raw-handle smuggle retired. Runtime-validated on the RTX box: swap
in, sc start -> RUNNING, sc stop -> clean STOPPED in ~1s, original restored. D2
(OwnedHandle/RAII rollout) is complete; only the deferred host P0 lints remain in
Goal 3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:28:29 +00:00
262 changed files with 15581 additions and 5681 deletions
+2 -2
View File
@@ -1,9 +1,9 @@
# Root build context is used only by web/Dockerfile, which needs web/ and
# docs/api/openapi.json. Allowlist those; keep everything else (target/, .git, crates)
# api/openapi.json. Allowlist those; keep everything else (target/, .git, crates)
# out of the context upload.
*
!web
!docs/api/openapi.json
!api/openapi.json
web/node_modules
web/.output
web/dist
+57
View File
@@ -0,0 +1,57 @@
# Android client screenshots for the Play listing / marketing. Roborazzi renders the real Compose
# UI with mock state on the host JVM via Robolectric — NO emulator, GPU, KVM, host, or JNI core
# (`-PskipRustBuild` skips the cargo-ndk native build). The Android analogue of apple.yml's
# `screenshots` job, gated to STABLE RELEASE tags only. Standalone + best-effort: a failure here
# reds nothing else. PNGs land as a 30-day artifact; not committed or published.
name: android-screenshots
on:
push:
tags: ["v*"]
workflow_dispatch:
jobs:
screenshots:
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- name: JDK 21 (AGP 9.2 + Robolectric's SDK-36 android-all jar both want 1721)
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "21"
- name: Android SDK
uses: android-actions/setup-android@v3
# No NDK/CMake — the screenshot unit tests are pure JVM. compileSdk 37 auto-downloads via AGP
# if the platform channel lacks it (same note as android.yml).
- name: platform-tools + platform 36 + build-tools
run: sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0"
- name: Cache (gradle)
uses: actions/cache@v4
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: android-screenshots-${{ hashFiles('clients/android/**/*.gradle.kts') }}
restore-keys: android-screenshots-
# Roborazzi renders Compose on the JVM (Robolectric Native Graphics). `-PskipRustBuild` keeps
# the cargo-ndk native build out of the graph — the tests never load libpunktfunk_android.so.
- name: Capture screenshots (Roborazzi)
working-directory: clients/android
run: ./gradlew :app:testDebugUnitTest -PskipRustBuild --stacktrace
- name: Upload screenshots
if: always()
# v3: Gitea's API rejects upload-artifact@v4 (see apple.yml). Download is a zip.
uses: actions/upload-artifact@v3
with:
name: punktfunk-android-screenshots
path: clients/android/app/build/outputs/roborazzi
retention-days: 30
+44 -13
View File
@@ -11,12 +11,18 @@
# punktfunk.zip
# punktfunk/ <- single top-level dir == plugin.json "name"
# plugin.json [required]
# package.json [required]
# package.json [required; CI stamps "version" — Decky reads the installed version here]
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
#
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
# plugin polls it (main.py check_update), and the frontend drives Decky's own install RPC to
# apply a newer build. See clients/decky/README.md "Updating".
#
# REGISTRY_TOKEN: repo Actions secret, a PAT with write:package scope (shared with deb/rpm/docker).
name: decky
@@ -56,20 +62,26 @@ jobs:
pnpm install --frozen-lockfile
pnpm run build # rollup -> clients/decky/dist/index.js
- name: Version + channel
# Tag vX.Y.Z -> X.Y.Z (stable `latest/` alias + Gitea Release); main push -> 0.3.0-ciN.g<sha>
# (`canary/` alias). Used for the registry version path + the zip name (the plugin.json
# version is the source of truth Decky reads after install — bump it in the release commit).
- name: Version + channel + stamp
# Tag vX.Y.Z -> X.Y.Z (stable `latest/` alias + Gitea Release); main push -> 0.3.<run>
# (`canary/` alias). Decky reads a plugin's INSTALLED version from package.json (NOT
# plugin.json), and the plugin's own update check (clients/decky/main.py check_update)
# compares against it — so the build version is STAMPED into package.json here (mirrored
# into plugin.json for store parity). Canary is a PLAIN numeric semver, never a
# `-ci<N>` prerelease: compare-versions orders prerelease identifiers lexically
# (ci10 < ci9), which would break update detection; the run number is monotonic.
working-directory: ${{ gitea.workspace }}
run: |
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; ALIAS=latest ;;
*) V="0.3.0-ci${GITHUB_RUN_NUMBER}.g${SHORT}"; ALIAS=canary ;;
*) V="0.3.${GITHUB_RUN_NUMBER}"; ALIAS=canary ;;
esac
BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE"
echo "VERSION=$V" >> "$GITHUB_ENV"
echo "ALIAS=$ALIAS" >> "$GITHUB_ENV"
echo "BASE=$BASE" >> "$GITHUB_ENV"
echo "decky version $V -> alias '$ALIAS'"
VERSION="$V" node -e 'const fs=require("fs");for(const f of ["clients/decky/package.json","clients/decky/plugin.json"]){const j=JSON.parse(fs.readFileSync(f,"utf8"));j.version=process.env.VERSION;fs.writeFileSync(f,JSON.stringify(j,null,2)+"\n");}'
- name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }}
@@ -89,9 +101,20 @@ jobs:
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip"
# The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias).
SHA=$(sha256sum "$RUNNER_TEMP/punktfunk.zip" | cut -d' ' -f1)
printf '{"version":"%s","artifact":"%s/%s/punktfunk.zip","sha256":"%s"}\n' \
"$VERSION" "$BASE" "$VERSION" "$SHA" > "$RUNNER_TEMP/manifest.json"
cat "$RUNNER_TEMP/manifest.json"
- name: Publish to the Gitea generic registry
working-directory: ${{ gitea.workspace }}
@@ -99,18 +122,26 @@ jobs:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE"
# 1) Immutable, versioned URL.
# 1) Immutable, versioned URL + its update manifest (the manifest's `artifact` points
# here, so the published sha256 keeps matching what Decky later downloads).
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/punktfunk.zip" \
"$BASE/$VERSION/punktfunk.zip"
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/manifest.json" \
"$BASE/$VERSION/manifest.json"
echo "published $BASE/$VERSION/punktfunk.zip"
# 2) Channel alias (stable release -> latest/, canary main build -> canary/) — the link
# to paste into Decky's "install from URL". The generic registry rejects re-uploading
# an existing version/file (409), so delete the prior alias first (ignore 404 on run #1).
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
"$BASE/$ALIAS/punktfunk.zip" || true
# 2) Channel alias (stable release -> latest/, canary main build -> canary/) — the
# zip is the "install from URL" link; manifest.json is what the installed plugin
# polls for updates. The generic registry rejects re-uploading an existing
# version/file (409), so delete the prior alias copies first (ignore 404 on run #1).
for f in punktfunk.zip manifest.json; do
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE "$BASE/$ALIAS/$f" || true
done
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/punktfunk.zip" \
"$BASE/$ALIAS/punktfunk.zip"
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/manifest.json" \
"$BASE/$ALIAS/manifest.json"
echo "install-from-URL link: $BASE/$ALIAS/punktfunk.zip"
echo "update manifest: $BASE/$ALIAS/manifest.json"
- name: Attach zip to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v')
+1 -1
View File
@@ -24,7 +24,7 @@ on:
push:
branches: [main]
# The flatpak is the CLIENT — only rebuild when the client/core/manifest change, not on every
# docs/host push (this is a heavy flatpak-builder run). Tags (v*, the client release) build too.
# design/host push (this is a heavy flatpak-builder run). Tags (v*, the client release) build too.
paths:
- 'clients/linux/**'
- 'crates/punktfunk-core/**'
@@ -0,0 +1,67 @@
# Native Linux client screenshots for the app/marketing listings. The client renders
# host-free mock scenes (PUNKTFUNK_SHOT_SCENE) under a virtual X display; the driver
# (clients/linux/tools/screenshots.sh) grabs each one — no host, GPU, or Wayland. The
# Linux analogue of apple.yml's `screenshots` job, gated to STABLE RELEASE tags only.
# Standalone + best-effort: a failure here reds nothing else. PNGs land as a 30-day
# artifact; they are not committed or published.
name: linux-client-screenshots
on:
push:
tags: ["v*"]
workflow_dispatch:
jobs:
screenshots:
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
# Same image as ci.yml/deb.yml — already carries the Rust toolchain + GTK/SDL build deps.
container:
image: git.unom.io/unom/punktfunk-rust-ci:latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
# Client link deps (baked into the image; kept here so the job is green across image
# rebuilds — a no-op once present) PLUS the headless-render extras: a virtual X server,
# software GL+Vulkan (llvmpipe/lavapipe), the icon theme + fonts the UI draws with, and a
# root-window grab tool.
- name: Client link + headless-render deps
run: |
apt-get update
apt-get install -y --no-install-recommends \
libgtk-4-dev libadwaita-1-dev libsdl3-dev \
xvfb x11-utils imagemagick scrot \
libgl1-mesa-dri mesa-vulkan-drivers \
adwaita-icon-theme fonts-cantarell fonts-dejavu-core
# Reuse the workspace cargo caches (same keys as ci.yml/deb.yml).
- name: Cache keys
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
- uses: actions/cache@v4
with:
path: |
/usr/local/cargo/registry
/usr/local/cargo/git
key: cargo-home-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-home-
- uses: actions/cache@v4
with:
path: target
key: cargo-target-v3-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-target-v3-${{ env.rustc }}-
- name: Build client
run: cargo build --release -p punktfunk-client-linux --locked
- name: Capture screenshots
run: bash clients/linux/tools/screenshots.sh
- name: Upload screenshots
if: always()
# v3: Gitea's API rejects upload-artifact@v4 (see apple.yml). Download is a zip.
uses: actions/upload-artifact@v3
with:
name: punktfunk-linux-client-screenshots
path: clients/linux/screenshots
retention-days: 30
+53
View File
@@ -0,0 +1,53 @@
# Management-console screenshots for the app/marketing listings. Captured from the
# built Storybook with headless Chromium (web/tools/screenshots.mjs) — the page
# stories render from fixtures, so no live mgmt API, login, or GPU is needed. This
# is the web analogue of apple.yml's `screenshots` job, but gated to STABLE RELEASE
# tags only (the console has no release workflow of its own — it ships inside the
# host packaging). Best-effort: a standalone workflow, so a failure here reds
# nothing else. PNGs land as a 30-day artifact; they are not committed or published.
name: web-screenshots
on:
push:
tags: ["v*"]
workflow_dispatch:
jobs:
screenshots:
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
container:
image: oven/bun:1
timeout-minutes: 30
defaults:
run:
working-directory: web
steps:
# oven/bun ships neither git nor a real node (the driver runs under node), and
# the slim Debian base lacks a CA bundle — without it actions/checkout's HTTPS
# fetch dies with "Problem with the SSL CA cert" (same as ci.yml's web job).
- name: Install git + node + CA certs
working-directory: /
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates git nodejs
- uses: actions/checkout@v4
# --ignore-scripts skips the prepare→codegen hook (mirrors ci.yml); run codegen
# explicitly since build-storybook has no prebuild hook of its own.
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
- name: Generate API client + i18n messages
run: bun run codegen
# Pulls the matching Chromium build + the apt libs it needs (root in-container).
- name: Install Chromium
run: bunx playwright install --with-deps chromium
- name: Build Storybook
run: bun run build-storybook
- name: Capture screenshots
run: bun run screenshots
- name: Upload screenshots
if: always()
# v3: Gitea's API rejects upload-artifact@v4 (see apple.yml). Download is a zip.
uses: actions/upload-artifact@v3
with:
name: punktfunk-web-console-screenshots
path: web/screenshots
retention-days: 30
@@ -1,5 +1,5 @@
# One-shot provisioning of the WDK + cargo-wdk onto the persistent self-hosted windows-amd64 runner, so
# the all-Rust UMDF drivers can build there (docs/windows-host-rewrite.md, M0). The runner has the base
# the all-Rust UMDF drivers can build there (design/windows-host-rewrite.md, M0). The runner has the base
# Windows SDK + MSVC + LLVM + Rust but NOT the WDK (no km/wdf/iddcx headers) or cargo-wdk.
#
# Dispatch manually (workflow_dispatch). Idempotent: re-running is a near no-op once provisioned. The
+6 -6
View File
@@ -1,5 +1,5 @@
# Windows driver workspace CI — runs on the self-hosted Windows runner (home-windows-1, host mode;
# label windows-amd64). Part of the Windows-host rewrite (docs/windows-host-rewrite.md, M0).
# label windows-amd64). Part of the Windows-host rewrite (design/windows-host-rewrite.md, M0).
#
# Stage 1 (this file): PROBE the runner's driver toolchain (WDK / EWDK / cargo-make / LLVM / the
# inf2cat/stampinf/devgen/signtool tools) so we know what's provisioned BEFORE writing driver code,
@@ -76,7 +76,7 @@ jobs:
head "EWDK"
Write-Host ("EWDKROOT = " + ($env:EWDKROOT ?? '<unset>'))
head "LLVM / clang (README pins 21.1.2 for wdk-sys bindgen)"
head "LLVM / clang (bindgen 0.72 builds on the runner default clang)"
Write-Host ("LIBCLANG_PATH = " + ($env:LIBCLANG_PATH ?? '<unset>'))
$clang = Get-Command clang -ErrorAction SilentlyContinue
if ($clang) { & clang --version } else { Write-Host "clang: NOT on PATH" }
@@ -119,12 +119,12 @@ jobs:
env:
# wdk-build otherwise picks 10.0.28000.0 (no km/crt) and bindgen fails — pin the WDK SDK version.
Version_Number: '10.0.26100.0'
# wdk-sys bindgen layout tests overflow (E0080) on the runner's default LLVM (ToT/22-dev); point at
# the pinned LLVM 21.1.2 that windows-drivers-rs builds clean against (provisioned to C:\llvm-21).
LIBCLANG_PATH: 'C:\llvm-21\bin'
# No LIBCLANG_PATH pin: the vendored bindgen 0.72 builds clean on the runner's default clang 22
# (the shipping pack proves it). A 0.71-era layout-test overflow once needed LLVM 21; the 0.72 bump
# retired that — see design/windows-build-and-packaging.md.
steps:
- uses: actions/checkout@v4
- name: Ensure WDK + cargo-wdk + LLVM 21.1.2 (idempotent self-provision)
- name: Ensure WDK + cargo-wdk (idempotent self-provision)
# Run the provisioning script here too so driver-build is self-sufficient and never races a
# separate provision run on the single runner. Path is relative to the job working-directory
# (packaging/windows/drivers). Near-noop once the toolchain is present.
+28
View File
@@ -56,6 +56,22 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Locale-safety gate (installer-run scripts must be ASCII)
shell: pwsh
# The installer runs these via powershell.exe (Windows PowerShell 5.1) and cmd.exe on the END
# USER's box. PS 5.1 reads a BOM-less script in the active ANSI codepage, so on a non-UTF-8 locale
# (e.g. German Windows-1252) a stray em-dash mis-decodes into a curly quote and the script aborts
# with "unterminated string" - exactly how the pf-vdisplay driver install silently failed in the
# field. Keep every installer-run script pure ASCII (matches install-gamepad-drivers.ps1).
run: |
$bad = Get-ChildItem packaging/windows/*.ps1, scripts/windows/*.ps1, scripts/windows/*.cmd -ErrorAction SilentlyContinue |
Where-Object { [IO.File]::ReadAllText($_.FullName) -match '[^\x00-\x7F]' }
if ($bad) {
$bad.FullName | ForEach-Object { Write-Output "::error::non-ASCII in installer-run script: $_" }
throw "installer-run scripts must be pure ASCII (PS 5.1 mis-parses them on non-UTF-8 locales)"
}
Write-Output "installer-run scripts are ASCII-clean"
- name: Configure + version
shell: pwsh
run: |
@@ -96,6 +112,18 @@ jobs:
# First-ever Windows lint coverage for the host (Linux CI never lints the windows-cfg code).
run: cargo clippy -p punktfunk-host --features nvenc,amf-qsv -- -D warnings
- name: Build + lint the HDR Vulkan layer (pf-vkhdr-layer)
shell: pwsh
# Standalone cdylib (own [workspace]) the installer bundles + registers (it lets Vulkan games
# like Doom use HDR on the virtual display). Lint here so a regression fails CI instead of
# silently shipping the host without the layer (pack-host-installer.ps1 builds it non-fatally).
# Windows-only FFI (user32 + the vk_layer loader glue) → can't be linted on the Linux CI.
run: |
Push-Location packaging/windows/pf-vkhdr-layer
cargo fmt --check; if ($LASTEXITCODE) { throw "pf-vkhdr-layer rustfmt" }
cargo clippy --release -- -D warnings; if ($LASTEXITCODE) { throw "pf-vkhdr-layer clippy" }
Pop-Location
- name: Ensure Inno Setup
shell: pwsh
run: |
+1
View File
@@ -13,6 +13,7 @@ clients/apple/PunktfunkCore.xcframework/
clients/apple/.swiftpm/
# Generated App Store screenshots (tools/screenshots.sh output; uploaded as a CI artifact)
clients/apple/screenshots/
clients/linux/screenshots/
# Xcode per-user state
xcuserdata/
+41 -10
View File
@@ -2,7 +2,7 @@
Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protocol core
(`punktfunk-core`) exposed over a C ABI and native clients per platform. Full design:
[`docs/implementation-plan.md`](docs/implementation-plan.md). Status table: `README.md`.
[`design/implementation-plan.md`](design/implementation-plan.md). Status table: `README.md`.
## Where the work stands
@@ -27,7 +27,15 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
Input: mouse/keyboard (libei via RemoteDesktop portal on KWin/GNOME, gamescope's own EIS
socket, wlr protocols on Sway) and **gamepads** (uinput X-Box-360 pads + rumble
back-channel; validated live — pad created/destroyed with the session). Management REST API +
checked-in OpenAPI doc (`mgmt.rs`).
checked-in OpenAPI doc (`mgmt.rs`). **Web-console performance capture** (`stats_recorder.rs`,
design: [`design/stats-capture-plan.md`](design/stats-capture-plan.md)): the operator arms stats
recording from the web console, plays, stops, and reviews the run as graphs (per-stage latency
breakdown · fps new/repeat · goodput · loss/FEC). A shared `Arc<StatsRecorder>` ring (the hot-path
gate is a runtime `AtomicBool`, replacing the startup-only `PUNKTFUNK_PERF`) is fed by **both** the
native `virtual_stream` and the GameStream encode loop at their existing ~2 s/~1 s aggregation
boundary, and finished captures are saved as on-disk recordings
(`~/.config/punktfunk/captures/*.json`) browsable/exportable from the console's **Performance** page
(recharts). Endpoints `/api/v1/stats/*` (bearer-only). *Implemented; not yet on-glass validated.*
- **Native protocol (`punktfunk/1`): full session planes, validated live.** QUIC
control plane (`punktfunk-core` `quic` feature: Hello{mode}/Welcome{full Config}/Start), data
plane = the hardened core `Session` over raw UDP with **GF(2¹⁶) Leopard FEC + AES-GCM**
@@ -104,9 +112,16 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
captures the HDR desktop as FP16/Rgb10a2 (DDA FP16 for the secure desktop), the encoder forces HEVC
Main10 + BT.2020 PQ (NVENC ABGR10/P010; AMF/QSV P010 + a swscale Rgb10a2→P010 fallback), the client
auto-detects PQ from the HEVC VUI — gated by `PUNKTFUNK_10BIT` + client `VIDEO_CAP_10BIT`; **Windows
host only** (the Linux host stays 8-bit, blocked upstream). **AMF/QSV is CI-green but not yet
on-glass validated** (no AMD/Intel Windows box in the lab); NVENC is live-validated. Newer/less
battle-tested than the Linux host. Packaging: `packaging/windows/`.
host only** (the Linux host stays 8-bit, blocked upstream). **Vulkan-game HDR over the virtual
display**: NVIDIA/AMD Vulkan ICDs refuse to *advertise* an HDR color space for a surface on an IddCx
indirect display (so Vulkan games — Doom: The Dark Ages, id Tech, etc. — say "device does not support
HDR"), even though the ICD happily *accepts + presents* a forced HDR swapchain there. A tiny always-on
Vulkan **implicit layer** (`packaging/windows/pf-vkhdr-layer/`, `VK_LAYER_PUNKTFUNK_hdr_inject`)
injects the `HDR10_ST2084`/scRGB surface formats into `vkGetPhysicalDeviceSurfaceFormats[2]KHR`,
self-gated on the display's actual advanced-color state (no-op on SDR / real monitors); bundled +
HKLM-registered by the installer. **Live-validated: Doom: The Dark Ages enables HDR over the virtual
display.** **AMF/QSV is CI-green but not yet on-glass validated** (no AMD/Intel Windows box in the
lab); NVENC is live-validated. Newer/less battle-tested than the Linux host. Packaging: `packaging/windows/`.
## What's left
@@ -245,8 +260,8 @@ bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip
```
Generated artifacts are **checked in** and CI fails on drift: `include/punktfunk_core.h`
(cbindgen from `punktfunk-core/src/abi.rs`) and `docs/api/openapi.json` (regenerate with
`cargo run -p punktfunk-host -- openapi > docs/api/openapi.json`; spec lives in `mgmt.rs`).
(cbindgen from `punktfunk-core/src/abi.rs`) and `api/openapi.json` (regenerate with
`cargo run -p punktfunk-host -- openapi > api/openapi.json`; spec lives in `mgmt.rs`).
CI is Gitea Actions (`.gitea/workflows/`, guide: docs-site `ci.md`): `ci.yml` runs the
workspace checks inside the `git.unom.io/unom/punktfunk-rust-ci` image plus web/docs-site
@@ -268,7 +283,7 @@ crates/punktfunk-host/
zerocopy/{egl,cuda,vulkan}.rs dmabuf → CUDA → NVENC (tiled via EGL/GL, LINEAR via Vulkan)
inject/{libei,wlr,gamepad,dualsense}.rs input backends (uinput xpad + UHID DualSense)
encode/{nvenc,linux,vaapi,ffmpeg_win,sw}.rs per-GPU encoders (NVENC · Linux NVENC/CUDA · VAAPI · AMF/QSV · openh264)
capture.rs · encode.rs · audio.rs · spike.rs · punktfunk1.rs · mgmt.rs · native_pairing.rs
capture.rs · encode.rs · audio.rs · spike.rs · punktfunk1.rs · mgmt.rs · native_pairing.rs · stats_recorder.rs
clients/probe/ punktfunk/1 reference/probe client (headless test/measurement tool)
clients/linux/ native Linux client (GTK4/libadwaita · FFmpeg · PipeWire · SDL3)
clients/windows/ native Windows client (WinUI 3 via windows-reactor · D3D11 · WASAPI · SDL3)
@@ -276,7 +291,7 @@ clients/apple/ native macOS/iOS/tvOS client (Swift · VideoToolbox · GameCon
clients/android/ native Android client (Kotlin app + native/ Rust JNI core over punktfunk-core)
clients/decky/ Steam Deck Decky plugin
crates/punktfunk-host/src/{capture/dxgi,vdisplay/sudovda,encode/ffmpeg_win,inject/gamepad_windows,audio/wasapi_*,service}.rs Windows host backends
web/ TanStack web console over the mgmt API (status · devices · pairing)
web/ TanStack web console over the mgmt API (status · devices · pairing · performance graphs)
packaging/ apt(deb) · RPM/COPR · Arch/sysext · Flatpak · Bazzite bootc · Windows host installer (per-dir READMEs)
tools/{loss-harness,latency-probe}/ measurement (plan §10)
scripts/ 60-punktfunk.rules · punktfunk-host.service · host.env.example · headless/
@@ -331,7 +346,23 @@ FFI also link-needs `libGL`/`libgbm`/`libcuda` at build time). Env knobs: `PUNKT
`PUNKTFUNK_COMPOSITOR=kwin|gamescope|mutter`, `PUNKTFUNK_ZEROCOPY=1`, `PUNKTFUNK_GAMESCOPE_APP=...`,
`PUNKTFUNK_INPUT_BACKEND=...`, `PUNKTFUNK_PERF=1` (per-stage timing), `PUNKTFUNK_VIDEO_DROP=N` (FEC
test), `PUNKTFUNK_FEC_PCT=N`, `PUNKTFUNK_DSCP=1` (opt-in DSCP/SO_PRIORITY media QoS on the data +
GameStream video/audio sockets; no-op on the wire on Windows without a qWAVE policy).
GameStream video/audio sockets; no-op on the wire on Windows without a qWAVE policy),
`PUNKTFUNK_444=1` (full-chroma HEVC 4:4:4, see below).
**HEVC 4:4:4 (full chroma, Range Extensions)**: opt-in via `PUNKTFUNK_444`, negotiated like 10-bit —
the host emits 4:4:4 only when the client advertised `VIDEO_CAP_444` (wire bit `0x04` + ABI
`PUNKTFUNK_VIDEO_CAP_444`), the codec is HEVC, the session is single-process, **and** a GPU probe
(`encode::can_encode_444`, run before the Welcome) confirms support — else it resolves to 4:2:0 and
`Welcome::chroma_format` reflects the real value (honest downgrade; the client reads it via
`punktfunk_connection_chroma_format`). **punktfunk/1-native only** — GameStream/Moonlight stays 4:2:0
(stock clients can't decode 4:4:4). **NVENC is the implemented path**: Linux `hevc_nvenc` feeds a
swscale'd `yuv444p` (RGB-in is always 4:2:0 — verified on the RTX 5070 Ti — so the session forces CPU
RGB capture for 4:4:4); Windows NVENC keeps ARGB input + FREXT profile + `chromaFormatIDC=3` and the
DDA capturer delivers RGB. VAAPI / AMF / QSV **decline** (probe returns false — no validated 4:4:4
hardware in the lab; they'd produce 4:2:0). Software (openh264) is 4:2:0-only. Test with
`PUNKTFUNK_CLIENT_444=1 punktfunk-probe --out x.h265` then `ffprobe x.h265` (expect `pix_fmt yuv444p`).
*Linux NVENC mechanism validated on the RTX 5070 Ti (ffmpeg CLI); Windows NVENC + 10-bit-4:4:4 not yet
on-glass validated.*
## Conventions
Generated
+333 -1
View File
@@ -2,6 +2,12 @@
# It is not intended for manual editing.
version = 3
[[package]]
name = "adler2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
[[package]]
name = "aead"
version = "0.5.2"
@@ -735,6 +741,15 @@ dependencies = [
"libc",
]
[[package]]
name = "crc32fast"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511"
dependencies = [
"cfg-if",
]
[[package]]
name = "criterion"
version = "0.5.1"
@@ -1100,6 +1115,16 @@ version = "0.5.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d674e81391d1e1ab681a28d99df07927c6d4aa5b027d7da16ba32d1d21ecd99"
[[package]]
name = "flate2"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"crc32fast",
"miniz_oxide",
]
[[package]]
name = "flume"
version = "0.11.1"
@@ -1751,12 +1776,115 @@ dependencies = [
"tower-service",
]
[[package]]
name = "icu_collections"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2984d1cd16c883d7935b9e07e44071dca8d917fd52ecc02c04d5fa0b5a3f191c"
dependencies = [
"displaydoc",
"potential_utf",
"utf8_iter",
"yoke",
"zerofrom",
"zerovec",
]
[[package]]
name = "icu_locale_core"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92219b62b3e2b4d88ac5119f8904c10f8f61bf7e95b640d25ba3075e6cac2c29"
dependencies = [
"displaydoc",
"litemap",
"tinystr",
"writeable",
"zerovec",
]
[[package]]
name = "icu_normalizer"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c56e5ee99d6e3d33bd91c5d85458b6005a22140021cc324cea84dd0e72cff3b4"
dependencies = [
"icu_collections",
"icu_normalizer_data",
"icu_properties",
"icu_provider",
"smallvec",
"zerovec",
]
[[package]]
name = "icu_normalizer_data"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da3be0ae77ea334f4da67c12f149704f19f81d1adf7c51cf482943e84a2bad38"
[[package]]
name = "icu_properties"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bee3b67d0ea5c2cca5003417989af8996f8604e34fb9ddf96208a033901e70de"
dependencies = [
"icu_collections",
"icu_locale_core",
"icu_properties_data",
"icu_provider",
"zerotrie",
"zerovec",
]
[[package]]
name = "icu_properties_data"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e2bbb201e0c04f7b4b3e14382af113e17ba4f63e2c9d2ee626b720cbce54a14"
[[package]]
name = "icu_provider"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "139c4cf31c8b5f33d7e199446eff9c1e02decfc2f0eec2c8d71f65befa45b421"
dependencies = [
"displaydoc",
"icu_locale_core",
"writeable",
"yoke",
"zerofrom",
"zerotrie",
"zerovec",
]
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "idna"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de"
dependencies = [
"idna_adapter",
"smallvec",
"utf8_iter",
]
[[package]]
name = "idna_adapter"
version = "1.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb68373c0d6620ef8105e855e7745e18b0d00d3bdb07fb532e434244cdb9a714"
dependencies = [
"icu_normalizer",
"icu_properties",
]
[[package]]
name = "if-addrs"
version = "0.15.0"
@@ -2022,6 +2150,12 @@ version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "litemap"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0"
[[package]]
name = "lock_api"
version = "0.4.14"
@@ -2116,6 +2250,16 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
[[package]]
name = "miniz_oxide"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "mio"
version = "1.2.1"
@@ -2548,6 +2692,15 @@ dependencies = [
"universal-hash",
]
[[package]]
name = "potential_utf"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0103b1cef7ec0cf76490e969665504990193874ea05c85ff9bab8b911d0a0564"
dependencies = [
"zerovec",
]
[[package]]
name = "powerfmt"
version = "0.2.0"
@@ -2675,6 +2828,7 @@ dependencies = [
"fec-rs",
"hmac",
"libc",
"opus",
"proptest",
"quinn",
"rand 0.9.4",
@@ -2702,7 +2856,6 @@ dependencies = [
"anyhow",
"ash",
"ashpd",
"audiopus_sys",
"axum",
"axum-server",
"base64",
@@ -2728,6 +2881,7 @@ dependencies = [
"rand 0.8.6",
"rcgen",
"reis",
"roxmltree",
"rsa",
"rusqlite",
"rustls",
@@ -2741,6 +2895,7 @@ dependencies = [
"tower",
"tracing",
"tracing-subscriber",
"ureq",
"utoipa",
"utoipa-axum",
"utoipa-scalar",
@@ -2752,6 +2907,7 @@ dependencies = [
"wayland-scanner",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"windows-service",
"winreg",
"x509-parser",
"xkbcommon",
]
@@ -3054,6 +3210,15 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "roxmltree"
version = "0.21.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1964b10c76125c36f8afe190065a4bf9a87bf324842c05701330bba9f1cacbb"
dependencies = [
"memchr",
]
[[package]]
name = "rpkg-config"
version = "0.1.2"
@@ -3555,6 +3720,12 @@ dependencies = [
"rand_core 0.6.4",
]
[[package]]
name = "simd-adler32"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214"
[[package]]
name = "siphasher"
version = "1.0.3"
@@ -3637,6 +3808,12 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "stable_deref_trait"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
[[package]]
name = "strsim"
version = "0.11.1"
@@ -3789,6 +3966,16 @@ dependencies = [
"time-core",
]
[[package]]
name = "tinystr"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8323304221c2a851516f22236c5722a72eaa19749016521d6dff0824447d96d"
dependencies = [
"displaydoc",
"zerovec",
]
[[package]]
name = "tinytemplate"
version = "1.2.1"
@@ -4094,6 +4281,40 @@ version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1"
[[package]]
name = "ureq"
version = "2.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02d1a66277ed75f640d608235660df48c8e3c19f3b4edb6a263315626cc3c01d"
dependencies = [
"base64",
"flate2",
"log",
"once_cell",
"rustls",
"rustls-pki-types",
"url",
"webpki-roots 0.26.11",
]
[[package]]
name = "url"
version = "2.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff67a8a4397373c3ef660812acab3268222035010ab8680ec4215f38ba3d0eed"
dependencies = [
"form_urlencoded",
"idna",
"percent-encoding",
"serde",
]
[[package]]
name = "utf8_iter"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
[[package]]
name = "utf8parse"
version = "0.2.2"
@@ -4421,6 +4642,24 @@ dependencies = [
"rustls-pki-types",
]
[[package]]
name = "webpki-roots"
version = "0.26.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521bc38abb08001b01866da9f51eb7c5d647a19260e00054a8c7fd5f9e57f7a9"
dependencies = [
"webpki-roots 1.0.8",
]
[[package]]
name = "webpki-roots"
version = "1.0.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf85cb06032201fa7c6f829d7db5a7e5aa45bcc0655327713065f6f0576731bf"
dependencies = [
"rustls-pki-types",
]
[[package]]
name = "wide"
version = "0.7.33"
@@ -4946,6 +5185,16 @@ dependencies = [
"memchr",
]
[[package]]
name = "winreg"
version = "0.56.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d6f32a0ff4a9f6f01231eb2059cc85479330739333e0e58cadf03b6af2cca10"
dependencies = [
"cfg-if",
"windows-sys 0.61.2",
]
[[package]]
name = "wit-bindgen"
version = "0.51.0"
@@ -5040,6 +5289,12 @@ dependencies = [
"wasmparser",
]
[[package]]
name = "writeable"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
[[package]]
name = "x509-parser"
version = "0.16.0"
@@ -5083,6 +5338,29 @@ dependencies = [
"time",
]
[[package]]
name = "yoke"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "709fe23a0424b6a435d82152b1bd3fdfb0833487d5fa90d05d42762a9891fef5"
dependencies = [
"stable_deref_trait",
"yoke-derive",
"zerofrom",
]
[[package]]
name = "yoke-derive"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e"
dependencies = [
"proc-macro2",
"quote",
"syn",
"synstructure",
]
[[package]]
name = "zbus"
version = "5.16.0"
@@ -5159,12 +5437,66 @@ dependencies = [
"syn",
]
[[package]]
name = "zerofrom"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ec05a11813ea801ff6d75110ad09cd0824ddba17dfe17128ea0d5f68e6c5272"
dependencies = [
"zerofrom-derive",
]
[[package]]
name = "zerofrom-derive"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1"
dependencies = [
"proc-macro2",
"quote",
"syn",
"synstructure",
]
[[package]]
name = "zeroize"
version = "1.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0"
[[package]]
name = "zerotrie"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0f9152d31db0792fa83f70fb2f83148effb5c1f5b8c7686c3459e361d9bc20bf"
dependencies = [
"displaydoc",
"yoke",
"zerofrom",
]
[[package]]
name = "zerovec"
version = "0.11.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90f911cbc359ab6af17377d242225f4d75119aec87ea711a880987b18cd7b239"
dependencies = [
"yoke",
"zerofrom",
"zerovec-derive",
]
[[package]]
name = "zerovec-derive"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "625dc425cab0dca6dc3c3319506e6593dcb08a9f387ea3b284dbd52a92c40555"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "zmij"
version = "1.0.21"
+21 -9
View File
@@ -1,13 +1,20 @@
# punktfunk
<p align="center">
<img src="assets/punktfunk-logo.svg" alt="punktfunk" width="320" />
</p>
**Low-latency desktop and game streaming, Linux-first.** Run the host on a Linux machine — or a
Windows PC — with an NVIDIA GPU, connect from a Mac, PC, phone, tablet, or TV, and stream your desktop
or games — each device at its **own native resolution and refresh rate**, over your local network.
<p align="center"><b>Low-latency desktop and game streaming with first-class Linux and Windows hosts.</b></p>
Run the host on a Linux machine or a Windows PC, connect from a Mac, PC, phone, tablet, or TV, and
stream your desktop or games — each device at its **own native resolution and refresh rate**, over
your local network.
📖 **Documentation: [docs.punktfunk.unom.io](https://docs.punktfunk.unom.io)** — start with
[How It Works](https://docs.punktfunk.unom.io/docs/how-it-works) or the
[Quick Start](https://docs.punktfunk.unom.io/docs/quickstart).
💬 **Community: [Discord](https://discord.gg/kaPNvzMuGU)** — chat, support, and **Android beta
access** · **[r/Punktfunk](https://www.reddit.com/r/Punktfunk/)**.
punktfunk pairs a **virtual-display streaming host** with native clients on every platform. It speaks
the existing **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/) client works
day one — and adds its own faster **`punktfunk/1`** protocol that breaks the ~1 Gbps FEC wall with a
@@ -19,6 +26,11 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
- **Your device's exact mode.** For each client that connects, the host spins up a virtual display
sized to that device — 1080p60 to a laptop, 1440p120 to a desktop, 4K to a TV, all at once. No
letterboxing, no scaling, no rearranging your real monitors.
- **A real virtual display on Windows, too.** On Linux the host uses per-compositor virtual outputs;
on Windows you get the same on-the-fly virtual display — at the client's exact mode, no physical
monitor or dummy HDMI plug, even on the secure desktop (UAC / lock screen). It also has **its own
indirect display driver (IDD)** the host pushes finished frames straight into, rather than scraping
a screen — tight, push-based integration that's unusual for a Windows streaming host.
- **Low latency, GPU end to end.** Frames go straight from the compositor to the NVENC encoder with
zero CPU copies (dmabuf → CUDA/Vulkan → NVENC), over a transport tuned for responsiveness rather
than throughput. Stable 240 fps at 5120×1440; sub-millisecond capture-to-reassembly on a LAN.
@@ -35,7 +47,7 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
| **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened |
| **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
| **Native protocol**`punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
| **Windows host** (NVIDIA, x64) | 🟡 Implemented & shipping as a signed installer (DXGI capture · SudoVDA virtual display · NVENC · WASAPI · ViGEm); NVIDIA-only, newer than the Linux host |
| **Windows host** (x64) | 🟡 Implemented & shipping as a signed installer: DXGI/WGC capture · its own all-Rust IddCx **virtual display** (secure-desktop capable) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test |
| **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch |
| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, Oboe audio, controllers, discovery, pairing |
@@ -61,14 +73,14 @@ roadmap: **[/docs/roadmap](https://docs.punktfunk.unom.io/docs/roadmap)**.
Pick your platform and install from its package registry — the per-platform guide covers adding the
repo, first run, and the web console. The Linux host is the primary, most battle-tested path; a
Windows host (NVIDIA-only) also ships as a signed installer.
Windows host also ships as a signed installer (all-vendor: NVIDIA, AMD, Intel).
| Platform | Install | Guide |
|--------|---------|-------|
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [Ubuntu — GNOME](https://docs.punktfunk.unom.io/docs/ubuntu-gnome) · [KDE](https://docs.punktfunk.unom.io/docs/ubuntu-kde) |
| **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(or the bootc image)* | [Fedora — KDE](https://docs.punktfunk.unom.io/docs/fedora-kde) · [Bazzite](https://docs.punktfunk.unom.io/docs/bazzite) |
| **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS)* | [packaging/arch](packaging/arch/README.md) |
| **Windows** (NVIDIA, x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) |
| **Windows** (x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) |
`punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status).
After install, run `punktfunk-host serve` inside your desktop session (the secure native default;
@@ -113,7 +125,7 @@ and the [docs site](https://docs.punktfunk.unom.io).
```
crates/
punktfunk-core/ protocol · FEC · pacing · crypto · QUIC control plane — the C ABI (lib + cdylib + staticlib)
punktfunk-host/ Linux host: virtual displays · capture · encode · input · GameStream · punktfunk/1 · mgmt
punktfunk-host/ the host (Linux + Windows): virtual displays · capture · encode · input · GameStream · punktfunk/1 · mgmt
clients/
apple/ macOS / iOS / tvOS app (Swift · VideoToolbox · Metal · GameController)
linux/ Linux desktop app (Rust · GTK4/libadwaita · FFmpeg/VAAPI · PipeWire · SDL3)
@@ -124,7 +136,7 @@ clients/
web/ web console (TanStack) over the management API — status · devices · pairing
packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite bootc image
docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io
docs/ design notes & deep-dive plans
design/ design notes & deep-dive plans (index: design/README.md)
include/punktfunk_core.h cbindgen-generated C header (checked in)
tools/ latency-probe · loss-harness (measurement)
```
+528
View File
@@ -978,6 +978,309 @@
}
}
},
"/api/v1/stats/capture/live": {
"get": {
"tags": [
"stats"
],
"summary": "Live in-progress capture",
"description": "The full sample time-series of the capture currently recording, for live graphing. `404` when\nnothing is armed.",
"operationId": "statsCaptureLive",
"responses": {
"200": {
"description": "The in-progress capture (meta + samples so far)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Capture"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No capture is currently recording",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/capture/start": {
"post": {
"tags": [
"stats"
],
"summary": "Start a stats capture",
"description": "Arms a new performance-stats capture. Idempotent: if a capture is already running this returns\nthe current status unchanged. While armed, the streaming loops emit aggregated samples (~ every\n12 s) into the in-progress capture, readable live via `GET /stats/capture/live`.",
"operationId": "statsCaptureStart",
"responses": {
"200": {
"description": "Capture armed (or already running)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/StatsStatus"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/capture/status": {
"get": {
"tags": [
"stats"
],
"summary": "Stats capture status",
"description": "Whether a capture is armed, its sample count, and start time. Poll this (e.g. every 2 s) to\ndrive the capture-control UI.",
"operationId": "statsCaptureStatus",
"responses": {
"200": {
"description": "In-progress capture status (idle when not armed)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/StatsStatus"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/capture/stop": {
"post": {
"tags": [
"stats"
],
"summary": "Stop the stats capture",
"description": "Disarms the in-progress capture and writes it to disk atomically, returning its summary. If\nnothing was recording, returns `204 No Content`.",
"operationId": "statsCaptureStop",
"responses": {
"200": {
"description": "Capture stopped and saved",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CaptureMeta"
}
}
}
},
"204": {
"description": "Nothing was recording"
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not write the recording to disk",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/recordings": {
"get": {
"tags": [
"stats"
],
"summary": "List saved recordings",
"description": "Every saved capture's summary (the `meta` head only — not the sample body), newest first.",
"operationId": "statsRecordingsList",
"responses": {
"200": {
"description": "Saved capture summaries, newest first",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/CaptureMeta"
}
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/recordings/{id}": {
"get": {
"tags": [
"stats"
],
"summary": "Get a saved recording",
"description": "The full capture (meta + samples) for `id`, for graphing or download.",
"operationId": "statsRecordingGet",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The recording id (its filename stem)",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "The full capture",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Capture"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No recording with that id",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "The recording file is unreadable",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
},
"delete": {
"tags": [
"stats"
],
"summary": "Delete a saved recording",
"description": "Removes the recording `id` from disk. `404` if there is no such recording.",
"operationId": "statsRecordingDelete",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The recording id (its filename stem)",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"204": {
"description": "Recording deleted"
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No recording with that id",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not delete the recording",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/status": {
"get": {
"tags": [
@@ -1125,6 +1428,89 @@
}
}
},
"Capture": {
"type": "object",
"description": "A full capture: summary + the sample time-series. The wire + on-disk shape.",
"required": [
"meta",
"samples"
],
"properties": {
"meta": {
"$ref": "#/components/schemas/CaptureMeta"
},
"samples": {
"type": "array",
"items": {
"$ref": "#/components/schemas/StatsSample"
}
}
}
},
"CaptureMeta": {
"type": "object",
"description": "Capture summary — the filename stem plus the negotiated mode/codec/client. Stored at the head\nof each on-disk recording and listed standalone (without the sample body) by\n[`StatsRecorder::list`].",
"required": [
"id",
"started_unix_ms",
"duration_ms",
"kind",
"width",
"height",
"fps",
"codec",
"client",
"sample_count"
],
"properties": {
"client": {
"type": "string",
"description": "Short label / fingerprint prefix, or `\"\"` if unknown."
},
"codec": {
"type": "string",
"description": "`\"h264\" | \"hevc\" | \"av1\"`."
},
"duration_ms": {
"type": "integer",
"format": "int64",
"minimum": 0
},
"fps": {
"type": "integer",
"format": "int32",
"minimum": 0
},
"height": {
"type": "integer",
"format": "int32",
"minimum": 0
},
"id": {
"type": "string",
"description": "e.g. `\"2026-06-26T20-14-03Z_5120x1440\"` — also the filename stem."
},
"kind": {
"type": "string",
"description": "`\"native\" | \"gamestream\"`."
},
"sample_count": {
"type": "integer",
"format": "int32",
"minimum": 0
},
"started_unix_ms": {
"type": "integer",
"format": "int64",
"minimum": 0
},
"width": {
"type": "integer",
"format": "int32",
"minimum": 0
}
}
},
"CustomEntry": {
"type": "object",
"description": "A user-added title, persisted in `~/.config/punktfunk/library.json`. Same shape the API\nreturns and the web console edits.",
@@ -1595,6 +1981,144 @@
}
}
},
"StageTiming": {
"type": "object",
"description": "One pipeline stage's latency in an aggregation window (microseconds).",
"required": [
"name",
"p50_us",
"p99_us"
],
"properties": {
"name": {
"type": "string",
"description": "`\"capture\" | \"submit\" | \"encode\" | \"packetize\" | \"send\"` (path-dependent)."
},
"p50_us": {
"type": "number",
"format": "float"
},
"p99_us": {
"type": "number",
"format": "float"
}
}
},
"StatsSample": {
"type": "object",
"description": "One aggregated sample (~ every 2 s native, ~ every 1 s GameStream).",
"required": [
"t_ms",
"session_id",
"stages",
"fps",
"repeat_fps",
"mbps",
"bitrate_kbps",
"frames_dropped",
"packets_dropped",
"send_dropped",
"fec_recovered"
],
"properties": {
"bitrate_kbps": {
"type": "integer",
"format": "int32",
"description": "Configured target bitrate.",
"minimum": 0
},
"fec_recovered": {
"type": "integer",
"format": "int32",
"description": "FEC shards recovered this window (delta).",
"minimum": 0
},
"fps": {
"type": "number",
"format": "float",
"description": "Genuine NEW frames/s from the source."
},
"frames_dropped": {
"type": "integer",
"format": "int32",
"description": "Frames dropped this window (delta).",
"minimum": 0
},
"mbps": {
"type": "number",
"format": "float",
"description": "Transmit goodput (Mb/s)."
},
"packets_dropped": {
"type": "integer",
"format": "int32",
"description": "Packets dropped this window (receiver-side / reassembler, where known).",
"minimum": 0
},
"repeat_fps": {
"type": "number",
"format": "float",
"description": "Re-encoded holds/s (source-starvation indicator)."
},
"send_dropped": {
"type": "integer",
"format": "int32",
"description": "Host send-buffer overflow / EAGAIN this window (delta).",
"minimum": 0
},
"session_id": {
"type": "integer",
"format": "int32",
"description": "Disambiguates concurrent sessions (usually constant).",
"minimum": 0
},
"stages": {
"type": "array",
"items": {
"$ref": "#/components/schemas/StageTiming"
},
"description": "Ordered pipeline stages for this path."
},
"t_ms": {
"type": "integer",
"format": "int64",
"description": "Milliseconds since capture start (monotonic; stamped by [`StatsRecorder::push_sample`]).",
"minimum": 0
}
}
},
"StatsStatus": {
"type": "object",
"description": "Snapshot of the in-progress capture for the management API.",
"required": [
"armed",
"sample_count",
"started_unix_ms",
"kind"
],
"properties": {
"armed": {
"type": "boolean",
"description": "Capture currently running."
},
"kind": {
"type": "string",
"description": "Path of the in-progress capture (`\"\"` if idle)."
},
"sample_count": {
"type": "integer",
"format": "int32",
"description": "Samples in the in-progress capture.",
"minimum": 0
},
"started_unix_ms": {
"type": "integer",
"format": "int64",
"description": "Unix start time of the in-progress capture (`0` if idle).",
"minimum": 0
}
}
},
"StreamInfo": {
"type": "object",
"description": "RTSP-negotiated stream parameters.",
@@ -1696,6 +2220,10 @@
{
"name": "library",
"description": "Game library: installed-store titles (Steam) plus user-curated custom entries"
},
{
"name": "stats",
"description": "Streaming performance-stats capture: arm/stop a recording, read the live + saved time-series for graphing"
}
]
}
+33
View File
@@ -0,0 +1,33 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg width="100%" height="100%" viewBox="0 0 579 298" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:2;">
<style>
/* Theme-adaptive so the logo stays readable on both light and dark README
backgrounds: deep violet (the brand-mark palette) on light, the original
light violet on dark. Evaluated by the viewer's color scheme. */
.pf-wm { fill: #6c5bf3; }
.pf-back { fill: #a79ff8; }
.pf-deep { fill: #6c5bf3; }
@media (prefers-color-scheme: dark) {
.pf-wm { fill: #cec9fb; }
.pf-back { fill: #f2f1fe; }
.pf-deep { fill: #8c7ef5; }
}
</style>
<g>
<g>
<path class="pf-wm" style="fill-rule:nonzero;" d="M21.144,176.635l0,102.687l31.253,0l0,-35.563l73.436,0l0,-23.555l-73.436,0l0,-19.398l77.285,0l0,-24.171l-108.537,0Z"/>
<path class="pf-wm" style="fill-rule:nonzero;" d="M136.148,176.635l0,47.264c0.154,16.627 0.154,16.627 0.308,20.014c0.77,15.087 2.463,21.4 7.544,26.634c7.698,8.16 20.014,10.315 59.272,10.315c23.863,0 34.178,-0.616 43.415,-2.463c11.7,-2.463 19.552,-10.623 21.246,-22.323c0.924,-7.236 1.078,-8.929 1.54,-32.176l0,-47.264l-31.253,0l0,47.264c0,2.155 -0.154,7.082 -0.308,10.623c-0.462,9.699 -1.232,12.47 -3.695,15.087c-3.387,3.695 -9.853,4.619 -31.407,4.619c-26.634,0 -32.638,-1.693 -34.332,-9.853c-0.77,-4.157 -0.77,-4.311 -1.078,-20.476l0,-47.264l-31.253,0Z"/>
<path class="pf-wm" style="fill-rule:nonzero;" d="M275.938,176.527l0,102.687l31.868,0l-0.77,-76.669l3.387,0l54.038,76.669l54.346,0l0,-102.687l-31.868,0l0.77,76.515l-3.233,0l-53.73,-76.515l-54.808,0Z"/>
<path class="pf-wm" style="fill-rule:nonzero;" d="M425.273,176.527l0,102.687l31.253,0l0,-39.258l17.089,0l46.032,39.258l47.418,0l-64.353,-52.344l59.426,-50.959l-47.88,0l-40.644,37.873l-17.089,0l0,-37.257l-31.253,0Z"/>
</g>
<path class="pf-back" style="fill-rule:nonzero;" d="M65.442,150.143c24.514,0 44.298,-19.784 44.298,-44.298c0,-24.514 -19.784,-44.298 -44.298,-44.298c-24.514,0 -44.298,19.784 -44.298,44.298c0,24.514 19.784,44.298 44.298,44.298Z"/>
<path class="pf-deep" style="fill-rule:nonzero;" d="M141.063,92.871c17.334,-17.334 17.334,-45.312 0,-62.647c-17.334,-17.334 -45.312,-17.334 -62.647,-0c-17.334,17.334 -17.334,45.312 0,62.647c17.334,17.334 45.312,17.334 62.647,-0Z"/>
<path style="fill:url(#_Linear1);" d="M121.228,104.359c-14.777,3.965 -31.187,0.136 -42.811,-11.488c-11.624,-11.624 -15.453,-28.034 -11.488,-42.811c14.777,-3.965 31.187,-0.136 42.811,11.488c11.624,11.624 15.453,28.034 11.488,42.811Z"/>
</g>
<defs>
<linearGradient id="_Linear1" x1="0" y1="0" x2="1" y2="0" gradientUnits="userSpaceOnUse" gradientTransform="matrix(31.323323,-31.323323,31.323323,31.323323,78.416832,92.870811)">
<stop offset="0" style="stop-color:#cec9fb;stop-opacity:0"/>
<stop offset="1" style="stop-color:#fcfcff;stop-opacity:1"/>
</linearGradient>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 3.0 KiB

+21
View File
@@ -62,6 +62,10 @@ android {
buildFeatures { compose = true }
// Roborazzi/Robolectric render Compose on the host JVM (the CI screenshot harness) and need the
// merged Android resources + the app's manifest/theme available to the unit tests.
testOptions { unitTests { isIncludeAndroidResources = true } }
compileOptions {
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
@@ -99,4 +103,21 @@ dependencies {
// Android TV components (we target phone + TV) land in the TV-UI milestone:
// implementation("androidx.tv:tv-material:1.1.0")
// The manifest already declares leanback so the scaffold installs on TV.
// --- CI screenshot harness (Roborazzi on the JVM via Robolectric — no emulator/GPU). The
// screenshot tests render the real Compose UI with mock state; never load the JNI core, so the
// job runs `:app:testDebugUnitTest -PskipRustBuild` (see kit/build.gradle.kts). ---
testImplementation(composeBom)
testImplementation("androidx.compose.ui:ui-test-junit4")
debugImplementation("androidx.compose.ui:ui-test-manifest") // the ComponentActivity test host
testImplementation("junit:junit:4.13.2")
testImplementation("org.robolectric:robolectric:4.16.1")
testImplementation("io.github.takahirom.roborazzi:roborazzi:1.64.0")
testImplementation("io.github.takahirom.roborazzi:roborazzi-compose:1.64.0")
}
// Record (write) the screenshots when the unit tests run. These tests exist to GENERATE marketing
// images, not to diff goldens, so always capture rather than verify.
tasks.withType<Test>().configureEach {
systemProperty("roborazzi.test.record", "true")
}
@@ -163,7 +163,7 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
targetHost, targetPort, w, h, hz,
id.certPem, id.privateKeyPem, pinHex ?: "",
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled,
hdrEnabled, settings.audioChannels,
)
}
connecting = false
@@ -16,9 +16,18 @@ data class Settings(
val bitrateKbps: Int = 0,
val compositor: Int = 0,
val gamepad: Int = 0,
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
* can capture; the resolved count drives the decoder + AAudio layout. */
val audioChannels: Int = 2,
val micEnabled: Boolean = false,
/** Show the live stats overlay (FPS / throughput / latency) during a stream. */
val statsHudEnabled: Boolean = true,
/**
* Touch input model. `true` (default) = trackpad: the cursor stays put on touch-down and moves
* by the finger's relative delta (swipe to nudge, lift and re-swipe to walk it across), tap to
* click where it is. `false` = direct pointing: the cursor jumps to the finger (the old behaviour).
*/
val trackpadMode: Boolean = true,
)
/** Loads/saves [Settings] in the app-private `punktfunk_settings` prefs. */
@@ -33,8 +42,10 @@ class SettingsStore(context: Context) {
bitrateKbps = prefs.getInt(K_BITRATE, 0),
compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0),
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
micEnabled = prefs.getBoolean(K_MIC, false),
statsHudEnabled = prefs.getBoolean(K_HUD, true),
trackpadMode = prefs.getBoolean(K_TRACKPAD, true),
)
fun save(s: Settings) {
@@ -45,8 +56,10 @@ class SettingsStore(context: Context) {
.putInt(K_BITRATE, s.bitrateKbps)
.putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad)
.putInt(K_AUDIO_CH, s.audioChannels)
.putBoolean(K_MIC, s.micEnabled)
.putBoolean(K_HUD, s.statsHudEnabled)
.putBoolean(K_TRACKPAD, s.trackpadMode)
.apply()
}
@@ -57,8 +70,10 @@ class SettingsStore(context: Context) {
const val K_BITRATE = "bitrate_kbps"
const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad"
const val K_AUDIO_CH = "audio_channels"
const val K_MIC = "mic_enabled"
const val K_HUD = "stats_hud_enabled"
const val K_TRACKPAD = "trackpad_mode"
}
}
@@ -124,6 +139,13 @@ val REFRESH_OPTIONS = listOf(
240 to "240 Hz",
)
/** (channel count, label). 2 = stereo (default), 6 = 5.1, 8 = 7.1. */
val AUDIO_CHANNEL_OPTIONS = listOf(
2 to "Stereo",
6 to "5.1 Surround",
8 to "7.1 Surround",
)
/** (kbps, label). `0` = host default. */
val BITRATE_OPTIONS = listOf(
0 to "Automatic",
@@ -104,6 +104,12 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
}
SettingsGroup("Audio") {
SettingDropdown(
label = "Audio channels",
options = AUDIO_CHANNEL_OPTIONS,
selected = s.audioChannels,
) { ch -> update(s.copy(audioChannels = ch)) }
ToggleRow(
title = "Microphone",
subtitle = "Send your mic to the host's virtual microphone",
@@ -119,6 +125,16 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
)
}
SettingsGroup("Pointer") {
ToggleRow(
title = "Trackpad mode",
subtitle = "Relative cursor like a laptop touchpad — swipe to nudge, tap to click. " +
"Off = the cursor jumps to your finger.",
checked = s.trackpadMode,
onCheckedChange = { on -> update(s.copy(trackpadMode = on)) },
)
}
SettingsGroup("Overlay") {
ToggleRow(
title = "Stats overlay",
@@ -41,6 +41,7 @@ import io.unom.punktfunk.kit.NativeBridge
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.delay
import kotlin.math.abs
import kotlin.math.hypot
import kotlin.math.roundToInt
// Touch-gesture tuning (px / ms). TAP_SLOP: movement under this still counts as a tap, not a drag.
@@ -50,6 +51,15 @@ private const val TAP_SLOP = 12f
private const val TAP_DRAG_MS = 250L
private const val SCROLL_DIV = 4f
// Trackpad-mode pointer ballistics (relative one-finger motion). POINTER_SENS: base finger-px →
// host-px gain (~1:1, never twitchy). The rest is mild acceleration so a flick crosses the screen
// while a slow drag stays precise: above ACCEL_SPEED_FLOOR px/ms the gain ramps by ACCEL_GAIN per
// px/ms, capped at ACCEL_MAX (so a fast swipe can't fling the cursor uncontrollably).
private const val POINTER_SENS = 1.3f
private const val ACCEL_GAIN = 0.6f
private const val ACCEL_SPEED_FLOOR = 0.3f
private const val ACCEL_MAX = 3.0f
@Composable
fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val context = LocalContext.current
@@ -68,8 +78,11 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Live decode stats for the HUD. Poll once a second for the whole stream (cheap, and each call
// drains+resets the native window so it never grows unbounded even while the overlay is hidden);
// `showStats` only gates rendering. A 3-finger tap toggles it live; the default comes from Settings.
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var showStats by remember { mutableStateOf(SettingsStore(context).load().statsHudEnabled) }
var showStats by remember { mutableStateOf(initialSettings.statsHudEnabled) }
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
val trackpad = initialSettings.trackpadMode
LaunchedEffect(handle) {
while (true) {
delay(1000)
@@ -145,13 +158,18 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (showStats) {
stats?.let { StatsOverlay(it, Modifier.align(Alignment.TopStart).padding(12.dp)) }
}
// Touch → mouse, absolute "direct pointing" like the Apple client: the host cursor follows
// your finger (MouseMoveAbs, host-normalized against the overlay size — which fills the video,
// so finger position maps straight onto the remote screen). Gestures: tap = left click;
// two-finger tap = right click; two-finger drag = scroll; tap-then-press-and-drag = left-drag
// (text selection / moving windows); three-finger tap = toggle the stats HUD.
// Touch → mouse. Two models, chosen by the Trackpad-mode setting:
// • trackpad (default): the cursor STAYS where it is on touch-down and moves by the finger's
// relative delta (MouseMove) with mild pointer acceleration — swipe to nudge, lift and
// re-swipe to walk it across, tap to click where it is. This is what makes the cursor
// reachable on a small screen.
// • direct (opt-out): the cursor jumps to the finger and follows it (MouseMoveAbs,
// host-normalized against the overlay size), the old "direct pointing" behaviour.
// Both share the same gesture vocabulary: tap = left click; two-finger tap = right click;
// two-finger drag = scroll; tap-then-press-and-drag = left-drag (text selection / moving
// windows); three-finger tap = toggle the stats HUD.
Box(
Modifier.fillMaxSize().pointerInput(handle) {
Modifier.fillMaxSize().pointerInput(handle, trackpad) {
var lastTapUp = 0L
var lastTapX = 0f
var lastTapY = 0f
@@ -176,7 +194,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val isDrag = down.uptimeMillis - lastTapUp < TAP_DRAG_MS &&
abs(startX - lastTapX) < TAP_SLOP && abs(startY - lastTapY) < TAP_SLOP
lastTapUp = 0L // consume the arming either way
moveAbs(startX, startY) // cursor jumps to the finger immediately
// Direct mode jumps the cursor to the finger; trackpad mode leaves it put (the
// whole point — you nudge it with swipes instead).
if (!trackpad) moveAbs(startX, startY)
if (isDrag) NativeBridge.nativeSendPointerButton(handle, 1, true)
var moved = false
@@ -185,6 +205,14 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
var prevCx = startX
var prevCy = startY
var upTime = down.uptimeMillis
// Trackpad relative-motion state: the tracked finger, its last position/time, and
// the sub-pixel remainder so a slow drag isn't lost to Int truncation.
var trackId = down.id
var prevX = startX
var prevY = startY
var prevT = down.uptimeMillis
var accX = 0f
var accY = 0f
while (true) {
val ev = awaitPointerEvent()
@@ -217,15 +245,46 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
moved = true
}
} else if (!scrolling) {
// One finger → the cursor follows it (skipped once a gesture turned into
// a scroll, so dropping back to one finger doesn't jerk the cursor).
// One finger (skipped once a gesture turned into a scroll, so dropping
// back to one finger doesn't jerk the cursor).
val p = pressed.firstOrNull { it.id == down.id } ?: pressed.first()
if (abs(p.position.x - startX) > TAP_SLOP ||
abs(p.position.y - startY) > TAP_SLOP
) {
moved = true
}
moveAbs(p.position.x, p.position.y)
if (trackpad) {
// Relative: move by the finger delta × (sensitivity × acceleration),
// carrying the sub-pixel remainder. Re-anchor (zero delta this frame)
// if the tracked finger changed, so lifting one of several fingers
// never jumps the cursor.
if (p.id != trackId) {
trackId = p.id
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
}
val dx = p.position.x - prevX
val dy = p.position.y - prevY
val dt = (p.uptimeMillis - prevT).coerceAtLeast(1L)
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
val speed = hypot(dx, dy) / dt // finger px per ms
val accel = (1f + ACCEL_GAIN * (speed - ACCEL_SPEED_FLOOR).coerceAtLeast(0f))
.coerceAtMost(ACCEL_MAX)
accX += dx * POINTER_SENS * accel
accY += dy * POINTER_SENS * accel
val outX = accX.toInt() // truncates toward zero → remainder kept w/ sign
val outY = accY.toInt()
if (outX != 0 || outY != 0) {
NativeBridge.nativeSendPointerMove(handle, outX, outY)
accX -= outX
accY -= outY
}
} else {
moveAbs(p.position.x, p.position.y) // direct: cursor follows the finger
}
}
ev.changes.forEach { it.consume() }
}
@@ -239,7 +298,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
NativeBridge.nativeSendPointerButton(handle, 3, true)
NativeBridge.nativeSendPointerButton(handle, 3, false)
}
else -> { // tap → left click, and arm tap-and-drag
else -> { // tap → left click (at the cursor's current spot), arm tap-drag
NativeBridge.nativeSendPointerButton(handle, 1, true)
NativeBridge.nativeSendPointerButton(handle, 1, false)
lastTapUp = upTime
@@ -260,7 +319,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped]`.
*/
@Composable
private fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
if (s.size < 10) return
val w = s[6].toInt()
val h = s[7].toInt()
@@ -10,7 +10,9 @@ import androidx.compose.ui.platform.LocalContext
// punktfunk brand violets (from the app icon: #6C5BF3 / #A79FF8 / #D2C9FB on a #16132A indigo).
// Used as the fallback dark scheme on pre-Android-12 devices; on 12+ we defer to Material You.
private val BrandDark = darkColorScheme(
// `internal` (not private) so the CI screenshot tests can force the deterministic brand palette —
// Material You dynamic colour has no wallpaper to seed from under the Robolectric JVM renderer.
internal val BrandDark = darkColorScheme(
primary = Color(0xFFA79FF8),
onPrimary = Color(0xFF1B1442),
primaryContainer = Color(0xFF4C3FB3),
@@ -0,0 +1,74 @@
package io.unom.punktfunk.screenshots
import androidx.activity.ComponentActivity
import androidx.compose.ui.test.junit4.createAndroidComposeRule
import androidx.compose.ui.test.onRoot
import com.github.takahirom.roborazzi.captureRoboImage
import com.github.takahirom.roborazzi.captureScreenRoboImage
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
/**
* App-store / marketing screenshots of the native Android client, rendered on the JVM by Roborazzi
* (Robolectric Native Graphics) — no emulator, GPU, host, or JNI core. The scenes (ShotScenes.kt)
* render the REAL Compose UI with mock state.
*
* `sdk = [36]` is mandatory: Robolectric ships android-all jars only up to API 36 (Android 16), and
* the app's compileSdk is 37. PNGs land in build/outputs/roborazzi/.
*/
@RunWith(RobolectricTestRunner::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(sdk = [36], qualifiers = "w360dp-h800dp-xxhdpi")
class ScreenshotTest {
@get:Rule
val compose = createAndroidComposeRule<ComponentActivity>()
private val out = "build/outputs/roborazzi"
// Pausing the animation clock before composing (then advancing once past the entrance animation
// and freezing) is what makes a text-field-bearing scene capturable: a focused field blinks its
// cursor via an infinite animation that otherwise keeps Compose perpetually "busy", so
// setContent's wait-for-idle never returns. Frozen, the capture is also deterministic.
/** Full-screen content scenes: the compose root fills the device, so a root capture is the shot. */
private fun shootRoot(name: String, content: @androidx.compose.runtime.Composable () -> Unit) {
compose.mainClock.autoAdvance = false
compose.setContent { ShotTheme(content) }
compose.mainClock.advanceTimeBy(800)
compose.onRoot().captureRoboImage("$out/phone-$name.png")
}
/** Dialog scenes: the AlertDialog is a separate window, so capture the whole screen (all windows). */
private fun shootScreen(name: String, content: @androidx.compose.runtime.Composable () -> Unit) {
compose.mainClock.autoAdvance = false
compose.setContent { ShotTheme(content) }
compose.mainClock.advanceTimeBy(800)
captureScreenRoboImage("$out/phone-$name.png")
}
@Test
fun hosts() = shootRoot("hosts") { HostsScene() }
@Test
fun settings() = shootRoot("settings") { SettingsScene() }
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi") // landscape — the stream is immersive
fun stream() = shootRoot("stream") { StreamScene() }
@Test
fun trust() = shootScreen("trust") {
HostsScene()
TrustDialog()
}
@Test
fun pair() = shootScreen("pair") {
HostsScene()
PairDialog()
}
}
@@ -0,0 +1,195 @@
package io.unom.punktfunk.screenshots
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.grid.GridCells
import androidx.compose.foundation.lazy.grid.GridItemSpan
import androidx.compose.foundation.lazy.grid.LazyVerticalGrid
import androidx.compose.foundation.lazy.grid.items
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import io.unom.punktfunk.BrandDark
import io.unom.punktfunk.Settings
import io.unom.punktfunk.SettingsScreen
import io.unom.punktfunk.StatsOverlay
import io.unom.punktfunk.components.HostCard
import io.unom.punktfunk.components.SectionLabel
import io.unom.punktfunk.models.HostStatus
// The CI screenshot scenes: the REAL app composables, fed embedded mock state, under the forced
// brand palette (Material You has no wallpaper to seed from on the JVM). The stream-video surface
// and ConnectScreen/App are intentionally absent — they require the live JNI core / a session.
/** Forces the deterministic punktfunk brand scheme (see Theme.kt) instead of dynamic colour. */
@Composable
internal fun ShotTheme(content: @Composable () -> Unit) {
MaterialTheme(colorScheme = BrandDark, content = content)
}
private data class MockHost(val name: String, val address: String, val status: HostStatus)
private val SAVED = listOf(
MockHost("Living Room PC", "192.168.1.42:9777", HostStatus.PAIRED),
MockHost("Office", "192.168.1.50:9777", HostStatus.TOFU),
)
private val DISCOVERED = listOf(
MockHost("studio-deck", "192.168.1.61:9777", HostStatus.PAIRING),
MockHost("HTPC", "192.168.1.70:9777", HostStatus.TOFU),
)
/** The connect screen's host grid, reconstructed from the real HostCard/SectionLabel components. */
@Composable
internal fun HostsScene() {
Surface(Modifier.fillMaxSize(), color = MaterialTheme.colorScheme.background) {
LazyVerticalGrid(
columns = GridCells.Adaptive(minSize = 160.dp),
modifier = Modifier.fillMaxSize(),
contentPadding = androidx.compose.foundation.layout.PaddingValues(16.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
item(span = { GridItemSpan(maxLineSpan) }) {
Column(
horizontalAlignment = Alignment.CenterHorizontally,
modifier = Modifier.fillMaxWidth(),
) {
Spacer(Modifier.height(8.dp))
Text("Punktfunk", style = MaterialTheme.typography.headlineLarge)
Text(
"stream a remote desktop",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.height(24.dp))
}
}
item(span = { GridItemSpan(maxLineSpan) }) { SectionLabel("Saved hosts") }
items(SAVED) { h ->
HostCard(h.name, h.address, h.status, enabled = true, onConnect = {}, onForget = {}, onRename = {})
}
item(span = { GridItemSpan(maxLineSpan) }) {
Spacer(Modifier.height(12.dp))
SectionLabel("Discovered on the network")
}
items(DISCOVERED) { h ->
HostCard(h.name, h.address, h.status, enabled = true, onConnect = {}, onForget = null)
}
}
}
}
/** The real SettingsScreen, fed a representative non-default Settings. */
@Composable
internal fun SettingsScene() {
Surface(Modifier.fillMaxSize(), color = MaterialTheme.colorScheme.background) {
SettingsScreen(
initial = Settings(
width = 1920,
height = 1080,
hz = 120,
bitrateKbps = 50_000,
compositor = 1,
gamepad = 2,
micEnabled = true,
statsHudEnabled = true,
trackpadMode = true,
),
onChange = {},
onBack = {},
)
}
}
/** The real TOFU AlertDialog (mirrors ConnectScreen's PendingTrust.Kind.TRUST_NEW), shown over the host grid. */
@Composable
internal fun TrustDialog() {
AlertDialog(
onDismissRequest = {},
title = { Text("Trust this host?") },
text = {
Column {
Text("First connection to 192.168.1.61:9777.")
Text("Fingerprint 9f8e7d6c5b4a3928…")
Text(
"This host allows trust-on-first-use, but that can't tell an impostor " +
"from the real host. Pairing with a PIN is stronger — it proves both sides.",
)
}
},
confirmButton = { TextButton({}) { Text("Trust (TOFU)") } },
dismissButton = { TextButton({}) { Text("Pair with PIN…") } },
)
}
/** The PIN-pairing AlertDialog (mirrors ConnectScreen's PendingTrust.Kind.PAIR). The live screen
* uses OutlinedTextFields, but a TextField inside a Dialog window never reaches idle under
* Robolectric (its focus/cursor machinery animates forever) — so the PIN is shown as a static
* display here, which also reads better in a marketing shot. */
@Composable
internal fun PairDialog() {
AlertDialog(
onDismissRequest = {},
title = { Text("Pair with PIN") },
text = {
Column {
Text("Enter the 4-digit PIN shown on the host.")
Spacer(Modifier.height(16.dp))
Surface(
color = MaterialTheme.colorScheme.surfaceVariant,
shape = MaterialTheme.shapes.medium,
modifier = Modifier.fillMaxWidth(),
) {
Text(
"4 8 2 7",
style = MaterialTheme.typography.headlineMedium,
textAlign = TextAlign.Center,
modifier = Modifier.fillMaxWidth().padding(vertical = 16.dp),
)
}
Spacer(Modifier.height(12.dp))
Text(
"This device: Pixel 9 Pro",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
},
confirmButton = { TextButton({}) { Text("Pair") } },
dismissButton = { TextButton({}) { Text("Cancel") } },
)
}
/** The live stats HUD (the real StatsOverlay) over a synthetic "streamed frame" gradient. */
@Composable
internal fun StreamScene() {
Box(
Modifier
.fillMaxSize()
.background(
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
),
) {
// [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped]
StatsOverlay(
doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0),
Modifier.align(Alignment.TopStart).padding(12.dp),
)
}
}
+8 -2
View File
@@ -99,6 +99,12 @@ val cargoNdkDebug = registerCargoNdk("cargoNdkDebug", release = false)
val cargoNdkRelease = registerCargoNdk("cargoNdkRelease", release = true)
afterEvaluate {
tasks.named("preDebugBuild").configure { dependsOn(cargoNdkDebug) }
tasks.named("preReleaseBuild").configure { dependsOn(cargoNdkRelease) }
// `-PskipRustBuild` skips the cargo-ndk native build — for JVM-only tasks (the Roborazzi
// screenshot unit tests render Compose on the JVM and never load libpunktfunk_android.so), so
// CI/local screenshot runs don't need the Rust toolchain or NDK. The native build stays wired
// for every normal APK/AAR build.
if (!project.hasProperty("skipRustBuild")) {
tasks.named("preDebugBuild").configure { dependsOn(cargoNdkDebug) }
tasks.named("preReleaseBuild").configure { dependsOn(cargoNdkRelease) }
}
}
@@ -45,6 +45,7 @@ object NativeBridge {
compositorPref: Int,
gamepadPref: Int,
hdrEnabled: Boolean,
audioChannels: Int,
): Long
/** 64-hex SHA-256 of the cert the host presented on [handle]; valid after a successful connect. */
+189 -27
View File
@@ -1,8 +1,21 @@
//! Android audio playback (android-only): pull Opus packets from the connector, decode to
//! interleaved f32 stereo, and feed AAudio (LowLatency) via its realtime data callback through a
//! jitter ring. Mirrors [`crate::decode`]: one thread we own (the Opus decode producer) plus a
//! shutdown flag; the realtime callback thread is owned by AAudio. Ring logic ported from
//! `punktfunk-client-linux/src/audio.rs` (prime ~3 quanta, drop-oldest cap, re-prime on drain).
//! interleaved f32 (stereo or 5.1/7.1 surround), and feed AAudio (LowLatency) via its realtime data
//! callback through a jitter ring. Mirrors [`crate::decode`]: one thread we own (the Opus decode
//! producer) plus a shutdown flag; the realtime callback thread is owned by AAudio.
//!
//! The layout is the host-RESOLVED channel count (`NativeClient::audio_channels`, negotiated at
//! connect), so an older/clamping host that can only capture stereo is decoded + played as stereo.
//! 2 = stereo / 6 = 5.1 / 8 = 7.1, in the canonical wire order FL FR FC LFE RL RR SL SR.
//!
//! The ring started as a port of `punktfunk-client-linux/src/audio.rs`, but AAudio — unlike
//! PipeWire, which adaptively rate-matches the stream and absorbs a shallow buffer — hands us a raw
//! realtime callback and makes us own the buffer. So this client diverges deliberately to stop the
//! Android-only crackle: (1) the callback is allocation/free-free — decoded buffers are recycled to
//! the producer via a free-list instead of being freed on the audio thread (Android's Scudo `free`
//! has unbounded tail latency); (2) the jitter ring is deeper (~40 ms prime / ~150 ms hard cap) and
//! decoupled from the tiny LowLatency burst size, with de-prime hysteresis so a transient drain
//! doesn't manufacture a silence; (3) the AAudio HW buffer is primed above its 2-burst default and
//! grown on XRuns (Google's anti-glitch technique).
use ndk::audio::{
AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode,
@@ -13,16 +26,75 @@ use punktfunk_core::error::PunktfunkError;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::mpsc::{sync_channel, SyncSender, TrySendError};
use std::sync::mpsc::{sync_channel, Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
const CHANNELS: usize = 2;
const SAMPLE_RATE: i32 = 48_000;
/// Decoded-chunk hand-off depth: 64 × 5 ms = 320 ms slack (matches the core's AUDIO_QUEUE).
const RING_CHUNKS: usize = 64;
/// Opus decode scratch: worst-case 120 ms stereo frame (5760 samples/ch × 2 ch).
const PCM_SCRATCH: usize = 5760 * CHANNELS;
// --- Jitter-ring depths, in MILLISECONDS (scaled to interleaved-f32 samples at runtime). --------
// The channel count is negotiated, not a compile-time const, so these are kept in ms and multiplied
// by `ms` (interleaved-f32 samples per millisecond at the resolved layout) inside `start`.
// Unlike the Linux client (PipeWire adaptively rate-matches the stream to the graph clock, masking
// host↔DAC drift + a shallow ring), AAudio hands us a raw callback and we own the buffer: drift and
// WiFi power-save bunching land as underruns/overflows = crackle. So Android runs a deliberately
// deeper, smoothly-managed ring than Linux — keep the two clients' depths intentionally divergent.
/// Prime/target floor: fill to ~40 ms before playing (and after a sustained drain). Deep enough to
/// ride out WiFi arrival jitter + clock drift; the dominant Android-only anti-crackle lever.
const PRIME_FLOOR_MS: usize = 40;
/// Ceiling for the burst-scaled target (so a large quantum can't push the prime depth too high).
const PRIME_CEIL_MS: usize = 80;
/// Drop-oldest headroom above the target before trimming — a ~80 ms band swallows an arrival burst
/// without overflowing.
const JITTER_HEADROOM_MS: usize = 80;
/// Hard latency bound: never let the ring exceed ~150 ms (the only thing that caps added latency).
const HARD_CAP_MS: usize = 150;
/// Re-prime (go silent to refill) only after this many CONSECUTIVE empty callbacks, so one transient
/// drain doesn't manufacture a fresh 40 ms silence (the old `if ring.is_empty()` re-primed instantly).
const DEPRIME_AFTER_CALLBACKS: u32 = 5;
/// Throttle the AAudio XRun-driven HW-buffer grow check (cheap, but no need to poll every quantum).
const XRUN_CHECK_EVERY: u32 = 128;
/// Opus decoder for the audio plane: a plain stereo decoder (the validated path) or a multistream
/// decoder for 5.1/7.1, both behind one `decode_float`. Built from the host-RESOLVED channel count
/// via the shared layout table. Mirrors the Linux client's `AudioDec`.
enum AudioDec {
Stereo(opus::Decoder),
Surround(opus::MSDecoder),
}
impl AudioDec {
fn new(channels: u8) -> Result<AudioDec, opus::Error> {
if channels == 2 {
Ok(AudioDec::Stereo(opus::Decoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
)?))
} else {
let l = punktfunk_core::audio::layout_for(channels, false);
Ok(AudioDec::Surround(opus::MSDecoder::new(
SAMPLE_RATE as u32,
l.streams,
l.coupled,
l.mapping,
)?))
}
}
fn decode_float(
&mut self,
input: &[u8],
out: &mut [f32],
fec: bool,
) -> Result<usize, opus::Error> {
match self {
AudioDec::Stereo(d) => d.decode_float(input, out, fec),
AudioDec::Surround(d) => d.decode_float(input, out, fec),
}
}
}
/// Diagnostics — written by the decode thread + the realtime callback, logged periodically. The
/// audio analogue of the video `fed`/`rendered` counters (we can't "screenshot" sound).
@@ -42,27 +114,57 @@ pub struct AudioPlayback {
}
impl AudioPlayback {
/// Open AAudio (LowLatency, 48 kHz/stereo/f32) with a realtime callback draining a jitter ring,
/// then spawn the Opus decode thread. `None` on failure (the caller leaves video streaming).
/// Open AAudio (LowLatency, 48 kHz/f32, the host-resolved channel layout) with a realtime
/// callback draining a jitter ring, then spawn the Opus decode thread. `None` on failure (the
/// caller leaves video streaming).
pub fn start(client: Arc<NativeClient>) -> Option<AudioPlayback> {
// Build playback from the host-RESOLVED channel count (never the request): 2 = stereo /
// 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = punktfunk_core::audio::normalize_channels(client.audio_channels) as usize;
// Interleaved f32 samples per millisecond at this layout (48 kHz × channels); the ms-
// denominated jitter-ring depths scale by it.
let ms = (SAMPLE_RATE as usize / 1000) * channels;
let prime_floor = PRIME_FLOOR_MS * ms;
let prime_ceil = PRIME_CEIL_MS * ms;
let jitter_headroom = JITTER_HEADROOM_MS * ms;
let hard_cap_max = HARD_CAP_MS * ms;
let counters = Arc::new(Counters::default());
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list: drained PCM buffers go BACK to the decode thread to be refilled, so the
// realtime callback never frees heap (Android's Scudo allocator has unbounded free() tail
// latency — a free on the audio thread is an XRun = a click) and the decode thread rarely
// allocates. Same depth as the data channel.
let (free_tx, free_rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Realtime consumer state, owned by the callback (FnMut) — no lock: AAudio calls it from a
// single high-priority thread, and the decode thread only touches `tx`.
// single high-priority thread, and the decode thread only touches `tx`/`free_rx`.
let cb_counters = counters.clone();
let mut ring: VecDeque<f32> = VecDeque::with_capacity(PCM_SCRATCH);
// Pre-reserve the ring so `extend` never reallocates on the realtime thread. Worst transient
// before the trim below = the hard cap plus one full channel of 5 ms (480-f32) frames — the
// punktfunk protocol always sends 5 ms Opus frames (host `audio_thread`); a larger frame
// would force a one-time realloc, asserted (not silently corrupted) in `decode_loop`.
let mut ring: VecDeque<f32> = VecDeque::with_capacity(hard_cap_max + RING_CHUNKS * 5 * ms);
let mut primed = false;
let callback = move |_s: &AudioStream, data: *mut c_void, num_frames: i32| {
let want = num_frames as usize * CHANNELS;
let mut empties: u32 = 0; // consecutive empty callbacks (de-prime hysteresis)
let mut cb_count: u32 = 0; // callbacks since open (throttles the XRun grow check)
let mut last_xrun: i32 = 0; // last AAudio XRun count we grew the buffer for
let callback = move |s: &AudioStream, data: *mut c_void, num_frames: i32| {
let want = num_frames as usize * channels;
// SAFETY: AAudio provides `num_frames * channel_count` F32 slots at `data`.
let out = unsafe { std::slice::from_raw_parts_mut(data as *mut f32, want) };
while let Ok(chunk) = rx.try_recv() {
ring.extend(chunk);
// Drain decoded chunks into the ring WITHOUT freeing on the RT thread: `drain(..)` empties
// each Vec but keeps its capacity, then the empty buffer is handed back for reuse. The
// only RT-thread free is the rare case where the recycle channel is momentarily full.
while let Ok(mut chunk) = rx.try_recv() {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
// Prime to ~3 quanta (15 ms; floor 15 ms / ceiling 200 ms); drop OLDEST above the cap.
let target = (3 * want).clamp(720 * CHANNELS, 9600 * CHANNELS);
while ring.len() > target.max(want) + want {
// Jitter buffer: prime to ~40 ms (prime_floor) before playing and after a sustained drain;
// drop-oldest only above a wide ~120 ms band. Decoupled from the AAudio burst `want` (tiny
// on the LowLatency MMAP path) so the depth doesn't collapse to a single quantum.
let target = (3 * want).clamp(prime_floor, prime_ceil);
let hard_cap = (target + jitter_headroom).min(hard_cap_max);
while ring.len() > hard_cap {
ring.pop_front();
}
if !primed && ring.len() >= target {
@@ -79,12 +181,34 @@ impl AudioPlayback {
out.fill(0.0);
cb_counters.underruns.fetch_add(1, Ordering::Relaxed);
}
// Re-prime only after a RUN of empty callbacks, not a single transient one — otherwise
// every momentary drain costs a fresh 40 ms silence (the old behaviour, self-inflicted
// crackle on any jitter spike).
if ring.is_empty() {
primed = false; // re-prime after a genuine drain (avoids sustained crackle on loss)
empties += 1;
if empties >= DEPRIME_AFTER_CALLBACKS {
primed = false;
}
} else {
empties = 0;
}
cb_counters
.ring_depth
.store(ring.len() as u64, Ordering::Relaxed);
// Google's AAudio anti-glitch technique: when the device reports new XRuns, grow the HW
// buffer by one burst (up to capacity). getXRunCount + setBufferSizeInFrames are both
// callback-safe / non-blocking, and set clamps to capacity so it self-limits. Throttled.
cb_count = cb_count.wrapping_add(1);
if cb_count % XRUN_CHECK_EVERY == 0 {
let xr = s.x_run_count();
if xr > last_xrun {
last_xrun = xr;
let burst = s.frames_per_burst().max(1);
let grown =
(s.buffer_size_in_frames() + burst).min(s.buffer_capacity_in_frames());
let _ = s.set_buffer_size_in_frames(grown);
}
}
AudioCallbackResult::Continue
};
@@ -93,7 +217,11 @@ impl AudioPlayback {
.ok()?
.direction(AudioDirection::Output)
.sample_rate(SAMPLE_RATE)
.channel_count(CHANNELS as i32)
// The wire order (FL FR FC LFE RL RR SL SR) is the standard AAudio/Android channel
// order, so this is an IDENTITY mapping — no permute. AAudio infers the 5.1/7.1 mask
// from `channel_count` (the ndk crate's builder exposes no setChannelMask); the host
// captures + Opus-encodes in exactly this order.
.channel_count(channels as i32)
.format(AudioFormat::PCM_Float)
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(AudioSharingMode::Shared)
@@ -109,19 +237,31 @@ impl AudioPlayback {
log::error!("audio: request_start: {e}");
return None;
}
// Lift the AAudio HW buffer off its brittle ~2-burst LowLatency default so a single late
// callback doesn't immediately underrun; the in-callback XRun loop grows it further if the
// device still glitches. set_buffer_size_in_frames clamps to capacity.
let burst = stream.frames_per_burst().max(1);
let _ =
stream.set_buffer_size_in_frames((burst * 3).min(stream.buffer_capacity_in_frames()));
// perf != LowLatency or rate != 48000 means AAudio silently fell to a resampled legacy path
// (different burst behaviour) — surface it so the field can tell that apart from plain jitter.
log::info!(
"audio: AAudio started rate={} ch={} fmt={:?} burst={}",
"audio: AAudio started rate={} ch={} fmt={:?} perf={:?} share={:?} burst={} buf={}/{}",
stream.sample_rate(),
stream.channel_count(),
stream.format(),
stream.performance_mode(),
stream.sharing_mode(),
stream.frames_per_burst(),
stream.buffer_size_in_frames(),
stream.buffer_capacity_in_frames(),
);
let shutdown = Arc::new(AtomicBool::new(false));
let sd = shutdown.clone();
let join = std::thread::Builder::new()
.name("pf-audio".into())
.spawn(move || decode_loop(client, tx, sd, counters))
.spawn(move || decode_loop(client, tx, free_rx, sd, counters, channels))
.ok();
Some(AudioPlayback {
@@ -143,31 +283,53 @@ impl Drop for AudioPlayback {
}
/// Producer: `next_audio` → Opus `decode_float` → push interleaved f32 into the ring channel.
/// Buffers come from (and return to) the realtime callback's recycle free-list so the steady state
/// is allocation-free on both threads.
fn decode_loop(
client: Arc<NativeClient>,
tx: SyncSender<Vec<f32>>,
free_rx: Receiver<Vec<f32>>,
shutdown: Arc<AtomicBool>,
counters: Arc<Counters>,
channels: usize,
) {
let mut dec = match opus::Decoder::new(SAMPLE_RATE as u32, opus::Channels::Stereo) {
// Interleaved f32 samples per millisecond at this layout — the ring's 5 ms reserve check below.
let ms = (SAMPLE_RATE as usize / 1000) * channels;
// Opus decode scratch: worst-case 120 ms frame (5760 samples/ch) × channels.
let pcm_scratch = 5760 * channels;
let mut dec = match AudioDec::new(channels as u8) {
Ok(d) => d,
Err(e) => {
log::error!("audio: opus decoder init: {e} — audio disabled");
return;
}
};
let mut pcm = vec![0f32; PCM_SCRATCH];
let mut pcm = vec![0f32; pcm_scratch];
let mut window_peak = 0f32; // loudest |sample| since the last log — tells a tone from silence
while !shutdown.load(Ordering::Relaxed) {
match client.next_audio(Duration::from_millis(5)) {
Ok(pkt) => match dec.decode_float(&pkt.data, &mut pcm, false) {
Ok(samples) => {
let n = samples * CHANNELS;
let n = samples * channels;
for &s in &pcm[..n] {
window_peak = window_peak.max(s.abs());
}
// The ring's pre-reservation in `start` assumes the protocol's 5 ms (≤480-f32/ch)
// frames; a larger frame would force a one-time realloc on the RT thread. Catch a
// future host frame-size change here in debug, not as a silent audio glitch.
debug_assert!(
n <= 5 * ms,
"audio frame {n} f32 exceeds the 5 ms ring reserve"
);
let count = counters.opus_decoded.fetch_add(1, Ordering::Relaxed) + 1;
match tx.try_send(pcm[..n].to_vec()) {
// Reuse a recycled buffer if the callback handed one back; only allocate when the
// free-list is momentarily empty (startup / after a backpressure drop).
let mut buf = free_rx
.try_recv()
.unwrap_or_else(|_| Vec::with_capacity(pcm_scratch));
buf.clear();
buf.extend_from_slice(&pcm[..n]);
match tx.try_send(buf) {
Ok(()) | Err(TrySendError::Full(_)) => {} // drop-newest under backpressure
Err(TrySendError::Disconnected(_)) => break,
}
+12 -4
View File
@@ -140,10 +140,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeGenerateIde
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref): Long`. `certPem`/`keyPem` empty = anonymous, else presented as the
/// persistent identity. `pinHex` empty = TOFU (read `nativeHostFingerprint` after), else 64-hex
/// SHA-256 to pin the host (mismatch → 0). `bitrateKbps` 0 = host default. `compositorPref`/
/// `gamepadPref` are `CompositorPref`/`GamepadPref` wire bytes (0 = Auto; unknown → Auto).
/// compositorPref, gamepadPref, hdrEnabled, audioChannels): Long`. `certPem`/`keyPem` empty =
/// anonymous, else presented as the persistent identity. `pinHex` empty = TOFU (read
/// `nativeHostFingerprint` after), else 64-hex SHA-256 to pin the host (mismatch → 0). `bitrateKbps`
/// 0 = host default. `compositorPref`/`gamepadPref` are `CompositorPref`/`GamepadPref` wire bytes
/// (0 = Auto; unknown → Auto). `audioChannels` is the requested surround layout (2/6/8; normalized,
/// anything else → stereo) — the host clamps it and the resolved count drives playback.
/// Returns an opaque handle, or 0 on failure (logged).
#[no_mangle]
#[allow(clippy::too_many_arguments)]
@@ -162,6 +164,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
compositor_pref: jint,
gamepad_pref: jint,
hdr_enabled: jboolean,
audio_channels: jint,
) -> jlong {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
@@ -213,6 +216,11 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
} else {
0
},
// Requested surround layout (2 = stereo / 6 = 5.1 / 8 = 7.1). The host clamps to what it can
// capture and echoes the resolved count in `connector.audio_channels`, which drives the
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
// normalizes to stereo here.
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
None, // launch: default app
pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous)
+1 -1
View File
@@ -361,4 +361,4 @@ ever switched to a logged-in GUI session, re-adding macOS to the job's capture s
- Mid-stream renegotiation (resolution change without reconnect) is designed-for but not
implemented (the Welcome is one-shot today).
- Host-side gamepad injection needs `/dev/uinput` access on the box (udev rule from
`docs/linux-setup.md`).
`design/linux-setup.md`).
@@ -25,6 +25,7 @@ struct ContentView: View {
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@@ -252,6 +253,7 @@ struct ContentView: View {
setting: PunktfunkConnection.GamepadType(
rawValue: UInt32(clamping: gamepadType)) ?? .auto),
bitrateKbps: UInt32(clamping: bitrateKbps),
audioChannels: UInt8(clamping: audioChannels),
launchID: launchID,
allowTofu: host.pinnedSHA256 == nil)
}
@@ -351,6 +353,7 @@ struct ContentView: View {
compositor: pref,
gamepad: pad,
bitrateKbps: bitrate,
audioChannels: UInt8(clamping: audioChannels),
autoTrust: true)
}
}
@@ -99,6 +99,7 @@ final class SessionModel: ObservableObject {
compositor: PunktfunkConnection.Compositor = .auto,
gamepad: PunktfunkConnection.GamepadType = .auto,
bitrateKbps: UInt32 = 0,
audioChannels: UInt8 = 2,
hdrEnabled: Bool = true,
launchID: String? = nil,
allowTofu: Bool = false,
@@ -137,7 +138,7 @@ final class SessionModel: ObservableObject {
width: width, height: height, refreshHz: hz,
pinSHA256: pin, identity: identity, compositor: compositor,
gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps,
launchID: launchID) }
audioChannels: audioChannels, launchID: launchID) }
await MainActor.run { [weak self] in
guard let self else { return }
// The user may have abandoned this attempt (window closed, another host
@@ -25,6 +25,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@ObservedObject private var gamepads = GamepadManager.shared
@@ -173,6 +174,10 @@ struct SettingsView: View {
TVSelectionRow(title: "Stream mode", options: options, selection: modeTag)
TVSelectionRow(
title: "Bitrate", options: bitrateOptions, selection: $bitrateKbps)
TVSelectionRow(
title: "Audio channels",
options: [("Stereo", 2), ("5.1 Surround", 6), ("7.1 Surround", 8)],
selection: $audioChannels)
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.caption)
@@ -271,6 +276,11 @@ struct SettingsView: View {
@ViewBuilder private var audioSection: some View {
Section {
Picker("Audio channels", selection: $audioChannels) {
Text("Stereo").tag(2)
Text("5.1 Surround").tag(6)
Text("7.1 Surround").tag(8)
}
#if os(macOS)
Picker("Speaker", selection: $speakerUID) {
Text("System default").tag("")
@@ -15,6 +15,9 @@ public enum DefaultsKey {
public static let gamepadType = "punktfunk.gamepadType"
public static let gamepadID = "punktfunk.gamepadID"
public static let bitrateKbps = "punktfunk.bitrateKbps"
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the in-core decode + AVAudioEngine layout.
public static let audioChannels = "punktfunk.audioChannels"
public static let micEnabled = "punktfunk.micEnabled"
public static let speakerUID = "punktfunk.speakerUID"
public static let micUID = "punktfunk.micUID"
@@ -235,6 +235,12 @@ public final class PunktfunkConnection {
/// drain `nextHdrMeta`.
public var isHDR: Bool { colorTransfer == 16 || colorTransfer == 18 }
/// The audio channel count the host resolved for this session (the Welcome's echo of the
/// requested `audioChannels`, clamped to what the host can capture): `2` (stereo), `6` (5.1)
/// or `8` (7.1). Build the playback layout from THIS, never the request. `2` for an older host.
/// PCM from `nextAudioPcm` is interleaved in the canonical wire order FL FR FC LFE RL RR SL SR.
public private(set) var resolvedAudioChannels: UInt8 = 2
/// Connect and start a session at the requested mode (the host creates a native virtual
/// output at exactly this size/refresh). Blocks up to `timeoutMs`.
///
@@ -264,6 +270,7 @@ public final class PunktfunkConnection {
gamepad: GamepadType = .auto,
bitrateKbps: UInt32 = 0,
videoCaps: UInt8 = 0,
audioChannels: UInt8 = 2,
launchID: String? = nil,
timeoutMs: UInt32 = 10_000
) throws {
@@ -279,16 +286,16 @@ public final class PunktfunkConnection {
withOptionalCString(launchID) { launch in
if let pin = pinSHA256 {
return pin.withUnsafeBytes { p in
punktfunk_connect_ex5(
punktfunk_connect_ex6(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, launch,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, launch,
p.bindMemory(to: UInt8.self).baseAddress, &observed,
cert, key, timeoutMs)
}
}
return punktfunk_connect_ex5(
return punktfunk_connect_ex6(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, launch,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, launch,
nil, &observed, cert, key, timeoutMs)
}
}
@@ -320,6 +327,9 @@ public final class PunktfunkConnection {
colorMatrix = mtx
colorFullRange = fullRange != 0
bitDepth = depth
var ac: UInt8 = 2
_ = punktfunk_connection_audio_channels(handle, &ac)
resolvedAudioChannels = ac
}
/// A bandwidth speed-test measurement (see `startSpeedTest`). Partial until `done`.
@@ -468,6 +478,50 @@ public final class PunktfunkConnection {
}
}
/// One decoded audio frame from `nextAudioPcm`: interleaved 32-bit float at 48 kHz, in the
/// canonical wire channel order FL FR FC LFE RL RR SL SR (the first `channels`).
public struct AudioPCM: Sendable {
/// Interleaved f32 samples (`frameCount * channels` long), wire channel order.
public let samples: [Float]
/// Samples per channel.
public let frameCount: Int
/// Channel count (2/6/8) `resolvedAudioChannels`.
public let channels: Int
public let ptsNs: UInt64
public let seq: UInt32
}
/// Pull the next audio frame, **decoded in-core** to interleaved f32 PCM Apple's AudioToolbox
/// Opus path is stereo-only, so surround (and, for uniformity, stereo too) is decoded by the
/// Rust core (libopus multistream) and handed back as PCM. nil on timeout, throws `.closed` once
/// the session ended. Drain from a dedicated audio thread (do NOT also call `nextAudio` they
/// share the underlying queue). The returned `samples` are copied out, so the buffer is owned.
public func nextAudioPcm(timeoutMs: UInt32 = 100) throws -> AudioPCM? {
audioLock.lock()
defer { audioLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkAudioPcm()
let rc = punktfunk_connection_next_audio_pcm(h, &out, timeoutMs)
switch rc {
case statusOK:
let channels = Int(out.channels)
let total = Int(out.frame_count) * channels
guard let base = out.samples, total > 0 else { return nil }
// Copy: the pointer borrows connection memory only until the next PCM call.
let samples = Array(UnsafeBufferPointer(start: base, count: total))
return AudioPCM(
samples: samples, frameCount: Int(out.frame_count),
channels: channels, ptsNs: out.pts_ns, seq: out.seq)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Pull the next force-feedback update for the GCController haptics engine:
/// `(pad, lowFrequency, highFrequency)` with 0...0xFFFF amplitudes, (0, 0) = stop.
/// Drain from the (single) feedback thread, alongside `nextHidOutput`.
@@ -19,13 +19,13 @@ import os
private let log = Logger(subsystem: "io.unom.punktfunk", category: "audio")
/// SPSC-ish jitter ring (interleaved stereo float), drain thread render callback.
/// The unfair lock is held for microseconds; fine at render-callback rates. Priming:
/// SPSC-ish jitter ring (interleaved float, `channels` per frame), drain thread render
/// callback. The unfair lock is held for microseconds; fine at render-callback rates. Priming:
/// reads return silence until enough is buffered (at least `prefill`, and at least one
/// packet more than the device's render quantum large-buffer devices would otherwise
/// chronically out-demand the prefill and oscillate prime dropout re-prime), and an
/// underrun re-primes, concealing jitter as one short dip instead of sustained crackle.
/// All counts stay even (whole stereo frames), so L/R interleave can never flip.
/// All counts stay whole frames (multiples of `channels`), so the interleave can never slip.
final class AudioRing: @unchecked Sendable {
private var buf: [Float]
private var readIdx = 0
@@ -34,12 +34,14 @@ final class AudioRing: @unchecked Sendable {
private var renderQuantum = 0
private let prefill: Int
private let highWater: Int
private let channels: Int
private let lock = OSAllocatedUnfairLock()
/// `capacity`/`prefill` in samples (interleaved 2 per frame, both must be even).
init(capacity: Int, prefill: Int) {
/// `capacity`/`prefill` in samples (interleaved `channels` per frame, both whole frames).
init(capacity: Int, prefill: Int, channels: Int) {
buf = [Float](repeating: 0, count: capacity)
self.prefill = prefill
self.channels = channels
highWater = prefill * 4
}
@@ -74,8 +76,8 @@ final class AudioRing: @unchecked Sendable {
renderQuantum = max(renderQuantum, count)
let available = writeIdx - readIdx
if !primed {
// 480 samples = one 5 ms host packet of slack beyond the device's demand.
if available >= max(prefill, renderQuantum + 480) {
// One 5 ms host packet (240 frames × channels) of slack beyond the device's demand.
if available >= max(prefill, renderQuantum + 240 * channels) {
primed = true
} else {
for i in 0..<count { out[i] = 0 }
@@ -113,10 +115,55 @@ private final class StopFlag: @unchecked Sendable {
/// Render-block-owned scratch storage: freed exactly when the closure (and thus the
/// last possible render call) is released never racing CoreAudio.
private final class ScratchBuffer {
let ptr = UnsafeMutablePointer<Float>.allocate(capacity: 8192 * 2)
// 8192 frames × up to 8 channels (7.1) the render block caps `frames` at 8192.
let ptr = UnsafeMutablePointer<Float>.allocate(capacity: 8192 * 8)
deinit { ptr.deallocate() }
}
/// CoreAudio channel layout for the canonical wire order FL FR FC LFE RL RR [SL SR]. nil for
/// stereo (the standard layout is correct). For 5.1/7.1 we list explicit channel labels via
/// `kAudioChannelLayoutTag_UseChannelDescriptions` preset tags (DTS_5_1 etc.) don't reliably
/// match Moonlight's order. NB the 7.1 mapping (verified against the WASAPI 0x63F + SPA orderings):
/// wire idx 4-5 = RL/RR = the WAVE *back* pair LeftSurround/RightSurround; idx 6-7 = SL/SR = the
/// WAVE *side* pair LeftSurroundDirect/RightSurroundDirect. (Using RearSurround* for 6-7 would
/// swap side/back vs the Windows/Linux clients.)
private func wireChannelLayout(channels: Int) -> AVAudioChannelLayout? {
let labels: [AudioChannelLabel]
switch channels {
case 6:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen, kAudioChannelLabel_LeftSurround,
kAudioChannelLabel_RightSurround,
]
case 8:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen,
kAudioChannelLabel_LeftSurround, kAudioChannelLabel_RightSurround, // wire RL/RR (back)
kAudioChannelLabel_LeftSurroundDirect, kAudioChannelLabel_RightSurroundDirect, // wire SL/SR (side)
]
default:
return nil
}
let size = MemoryLayout<AudioChannelLayout>.size
+ (labels.count - 1) * MemoryLayout<AudioChannelDescription>.stride
let raw = UnsafeMutableRawPointer.allocate(byteCount: size, alignment: 16)
defer { raw.deallocate() }
let layout = raw.bindMemory(to: AudioChannelLayout.self, capacity: 1)
layout.pointee.mChannelLayoutTag = kAudioChannelLayoutTag_UseChannelDescriptions
layout.pointee.mChannelBitmap = AudioChannelBitmap(rawValue: 0)
layout.pointee.mNumberChannelDescriptions = UInt32(labels.count)
let descs = UnsafeMutableBufferPointer(
start: &layout.pointee.mChannelDescriptions, count: labels.count)
for (i, lbl) in labels.enumerated() {
descs[i] = AudioChannelDescription(
mChannelLabel: lbl, mChannelFlags: AudioChannelFlags(rawValue: 0),
mCoordinates: (0, 0, 0))
}
return AVAudioChannelLayout(layout: layout)
}
public final class SessionAudio {
private let connection: PunktfunkConnection
private let flag = StopFlag()
@@ -229,9 +276,13 @@ public final class SessionAudio {
// MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) {
// 1 s of interleaved stereo capacity, ~20 ms prefill: four 5 ms host packets of
// jitter absorption before the first sample plays.
let ring = AudioRing(capacity: 96_000, prefill: 1920)
// Build the playback layout from the host-RESOLVED channel count (never the request):
// 2 = stereo / 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = Int(connection.resolvedAudioChannels)
// 1 s interleaved capacity, ~20 ms prefill (four 5 ms host packets of jitter absorption
// before the first sample plays), both scaled by the channel count.
let ring = AudioRing(
capacity: 48_000 * channels, prefill: 960 * channels, channels: channels)
let engine = AVAudioEngine()
#if os(macOS)
@@ -247,21 +298,32 @@ public final class SessionAudio {
}
#endif
// Engine-native deinterleaved float; the render block deinterleaves from the ring.
guard let format = AVAudioFormat(standardFormatWithSampleRate: 48_000, channels: 2)
else { return }
// Engine-native deinterleaved float; the render block deinterleaves from the ring. Surround
// uses an explicit wire-order channel layout; the mixer downmixes to the output device when
// it has fewer speakers (e.g. an iPhone's stereo built-ins). (Explicit if/else rather than
// map/flatMap so it's correct whether the channelLayout initializer is failable or not.)
var format: AVAudioFormat?
if channels == 2 {
format = AVAudioFormat(standardFormatWithSampleRate: 48_000, channels: 2)
} else if let layout = wireChannelLayout(channels: channels) {
format = AVAudioFormat(standardFormatWithSampleRate: 48_000, channelLayout: layout)
}
guard let format else {
log.error("could not build \(channels)-channel audio format — audio disabled")
return
}
let scratch = ScratchBuffer() // block-owned; freed with the closure
let source = AVAudioSourceNode(format: format) { _, _, frameCount, abl -> OSStatus in
let frames = Int(frameCount)
guard frames <= 8192 else { return kAudioUnitErr_TooManyFramesToProcess }
ring.read(into: scratch.ptr, count: frames * 2)
ring.read(into: scratch.ptr, count: frames * channels)
let buffers = UnsafeMutableAudioBufferListPointer(abl)
if buffers.count >= 2,
let left = buffers[0].mData?.assumingMemoryBound(to: Float.self),
let right = buffers[1].mData?.assumingMemoryBound(to: Float.self) {
for f in 0..<frames {
left[f] = scratch.ptr[f * 2]
right[f] = scratch.ptr[f * 2 + 1]
// Deinterleave the wire-order interleaved ring into the engine's per-channel buses.
if buffers.count >= channels {
for ch in 0..<channels {
if let dst = buffers[ch].mData?.assumingMemoryBound(to: Float.self) {
for f in 0..<frames { dst[f] = scratch.ptr[f * channels + ch] }
}
}
}
return noErr
@@ -292,29 +354,20 @@ public final class SessionAudio {
stateLock.unlock()
let thread = Thread { [connection, flag, drainDone] in
defer { drainDone.signal() }
guard let decoder = try? OpusDecoder(framesPerPacket: 240),
let pcm = AVAudioPCMBuffer(
pcmFormat: decoder.pcmFormat, frameCapacity: 5760)
else {
log.error("Opus decoder unavailable — audio playback disabled")
return
}
// Decode happens IN-CORE (libopus multistream) AudioToolbox's Opus path is
// stereo-only and is handed back as interleaved f32 PCM in wire channel order.
while !flag.isStopped {
let packet: AudioPacket?
let pcm: PunktfunkConnection.AudioPCM?
do {
packet = try connection.nextAudio(timeoutMs: 100)
pcm = try connection.nextAudioPcm(timeoutMs: 100)
} catch {
break // session closed
}
guard let packet else { continue }
do {
let frames = try decoder.decode(packet.data, into: pcm)
if frames > 0, let p = pcm.floatChannelData?[0] {
ring.write(p, count: Int(frames) * 2)
guard let pcm, pcm.frameCount > 0 else { continue }
pcm.samples.withUnsafeBufferPointer { p in
if let base = p.baseAddress {
ring.write(base, count: pcm.frameCount * pcm.channels)
}
} catch {
// One corrupt packet a dead stream; skip it.
log.warning("audio decode failed: \(error.localizedDescription)")
}
}
}
+36 -1
View File
@@ -45,8 +45,9 @@ Gaming Mode automatically.
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. |
| `src/backend.ts` | Typed `callable` bridges to `main.py`. |
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut targets (so the window is focusable). |
| `main.py` | Backend: `discover` / `pair` / `runner_info` / `get_settings` / `set_settings` / `kill_stream`. |
| `main.py` | Backend: `discover` / `pair` / `runner_info` / `get_settings` / `set_settings` / `kill_stream` / `check_update`. |
| `plugin.json` | Decky plugin manifest. |
| `update.json` | CI-baked `{channel, manifest}` — where `check_update()` polls (absent on dev builds). |
| `decky.pyi` | Type stub for the injected `decky` module (vendored from the template). |
### Discovery (`discover()`)
@@ -140,6 +141,40 @@ shows up in the Quick Access Menu.
> [`../../packaging/flatpak/README.md`](../../packaging/flatpak/README.md)) — install that on
> the Deck too, or the panel's Connect surfaces a `client-not-found` error.
## Updating (self-update, no store)
The plugin updates itself without the official Decky store. CI (`decky.yml`) publishes a tiny
per-channel `manifest.json` next to the zip in the Gitea registry:
```json
{"version":"0.3.123","artifact":".../punktfunk-decky/0.3.123/punktfunk.zip","sha256":"…"}
```
and bakes an `update.json` (`{channel, manifest}`) into the plugin so it knows which channel it was
installed from. The backend `check_update()` reads the **installed** version from `package.json`
the value Decky itself reports (it does **not** read `plugin.json`) — fetches the channel manifest,
and compares. When a newer build exists the frontend shows an **Update to vX** button that drives
Decky Loader's own install RPC:
```ts
window.DeckyBackend.callable("utilities/install_plugin")(artifact, "punktfunk", version, hash, /*UPDATE=*/2)
```
The loader (root) downloads the immutable per-version zip, **SHA-256-verifies** it against `hash`,
replaces `~/homebrew/plugins/punktfunk`, and hot-reloads — the unprivileged backend never writes the
root-owned plugins dir itself. `window.DeckyBackend` / `utilities/install_plugin` are loader
internals (not `@decky/api`), so every access is guarded; missing them, the button falls back to a
toast pointing at **Install Plugin from URL**.
> CI stamps a **plain numeric** semver per channel (`0.3.<run>` canary, `X.Y.Z` stable) into
> `package.json`. Decky's `compare-versions` orders pre-release identifiers lexically (so `ci10 < ci9`)
> — a `-ciN` suffix would mis-detect updates.
**Optional — native Updates tab:** Decky's store is single-source (a custom store URL *replaces* the
official catalog), so punktfunk doesn't ship one by default. A user who wants the native update badge
can point Decky → Settings → **Custom store** at a punktfunk-only store JSON — not recommended if you
use other plugins, since it hides the official catalog.
## Limitations / next steps
- **Needs on-Deck validation in Gaming Mode**: the Steam-shortcut launch (`AddShortcut` /
+3 -1
View File
@@ -31,4 +31,6 @@ fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
# exec so the flatpak client IS the game process — when it exits, Steam ends the "game" and
# Gaming Mode reclaims focus automatically (no manual refocus needed).
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST"
# --fullscreen: present the stream chrome-less and fullscreen (the client also auto-detects the
# Deck/gamescope env, and ignores the flag harmlessly on older builds that predate it).
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen
+141 -4
View File
@@ -17,6 +17,8 @@ The backend's jobs are the things Steam can't do:
* **get_settings() / set_settings()** — read/write the flatpak client's stream settings JSON
(resolution / bitrate / gamepad), so the Deck UI configures the stream the client reads.
* **kill_stream()** — force-stop a wedged stream (``flatpak kill``).
* **check_update()** — poll the registry's per-channel ``manifest.json`` and report whether a
newer build is available (the frontend then drives Decky's own install RPC to apply it).
The TXT-record keys parsed (``proto`` / ``fp`` / ``pair`` / ``id``) are defined by the host
advert in ``crates/punktfunk-host/src/discovery.rs``.
@@ -26,7 +28,10 @@ import asyncio
import json
import os
import shutil
import ssl
import stat
import time
import urllib.request
from pathlib import Path
import decky
@@ -37,22 +42,99 @@ APP_ID = "io.unom.Punktfunk"
# Service type advertised by punktfunk/1 hosts (matches NATIVE_SERVICE in the Rust host).
SERVICE_TYPE = "_punktfunk._udp"
# The flatpak client persists identity / known-hosts / settings under HOME/.config/punktfunk;
# inside the flatpak sandbox HOME is ~/.var/app/<APP_ID>, so the real on-disk location is this.
# The backend writes settings here so the (sandboxed) client reads them.
# The flatpak client persists identity / known-hosts / settings under HOME/.config/punktfunk.
# The sandbox HOME resolves to the REAL user home (== DECKY_USER_HOME), NOT the per-app
# ~/.var/app/<APP_ID> dir — verified on-device (`flatpak run … sh -c 'echo $HOME'` prints
# /home/deck, and the manifest's `--filesystem=~/.config/punktfunk` grants exactly that path;
# we also pass HOME=DECKY_USER_HOME into `flatpak run`, see _flatpak_env). Pointing here is what
# lets plugin settings actually reach the client AND lets us read the client's known-hosts to
# tell whether THIS device is already paired with a given host.
def _client_config_dir() -> Path:
return Path(decky.DECKY_USER_HOME) / ".var" / "app" / APP_ID / ".config" / "punktfunk"
return Path(decky.DECKY_USER_HOME) / ".config" / "punktfunk"
def _settings_path() -> Path:
return _client_config_dir() / "client-gtk-settings.json"
def _paired_fingerprints() -> set[str]:
"""Host cert fingerprints (lowercase hex) this client has PIN-paired, from the client's
known-hosts store. Keyed by fingerprint so it survives a host changing IP address."""
try:
data = json.loads((_client_config_dir() / "client-known-hosts.json").read_text())
except (OSError, json.JSONDecodeError):
return set()
hosts = data.get("hosts", []) if isinstance(data, dict) else []
return {
h["fp_hex"].lower()
for h in hosts
if isinstance(h, dict) and h.get("paired") and isinstance(h.get("fp_hex"), str)
}
def _runner_path() -> str:
"""Absolute path to the launch wrapper shipped with the plugin (bin/punktfunkrun.sh)."""
return str(Path(decky.DECKY_PLUGIN_DIR) / "bin" / "punktfunkrun.sh")
# ----------------------------------------------------------------------------------------
# Self-update check (no Decky store). The plugin is distributed via "Install Plugin from
# URL" pointing at our Gitea generic registry, so the official store never sees it and
# can't offer updates. Instead the backend polls a tiny per-channel ``manifest.json`` the
# CI publishes next to the zip, compares it to the installed version, and the frontend
# offers a one-tap update that drives Decky's own (root, privileged) install RPC. The
# channel + manifest URL are baked into ``update.json`` by CI (.gitea/workflows/decky.yml);
# a dev/sideload build has no ``update.json`` and update checks are simply disabled.
_UPDATE_TTL_S = 1800.0 # cache a successful check for 30 min (the QAM remounts often)
_update_cache: dict = {"at": 0.0, "data": None}
def _update_config() -> dict:
"""The CI-baked ``{channel, manifest}`` next to the plugin (absent on dev builds)."""
try:
return json.loads((Path(decky.DECKY_PLUGIN_DIR) / "update.json").read_text())
except (OSError, json.JSONDecodeError):
return {}
def _installed_version() -> str:
"""The version Decky itself reports for this plugin — it reads ``package.json`` (NOT
plugin.json), so the CI stamps the build version there."""
try:
pkg = json.loads((Path(decky.DECKY_PLUGIN_DIR) / "package.json").read_text())
return str(pkg.get("version", "0.0.0"))
except (OSError, json.JSONDecodeError):
return "0.0.0"
def _semver_tuple(v: str) -> tuple[int, int, int]:
"""A tolerant (major, minor, patch) tuple for ``>`` comparison. We control the version
format (plain numeric ``X.Y.Z`` on both channels), so leading-int-per-component is
enough; any pre-release suffix is dropped before comparing."""
parts: list[int] = []
for comp in str(v).split("-", 1)[0].split(".")[:3]:
digits = ""
for ch in comp:
if ch.isdigit():
digits += ch
else:
break
parts.append(int(digits) if digits else 0)
while len(parts) < 3:
parts.append(0)
return (parts[0], parts[1], parts[2])
def _fetch_json(url: str, timeout: float = 8.0) -> dict:
"""Blocking HTTPS GET of a small JSON document (run in an executor)."""
req = urllib.request.Request(
url, headers={"Accept": "application/json", "User-Agent": "punktfunk-decky"}
)
ctx = ssl.create_default_context()
with urllib.request.urlopen(req, timeout=timeout, context=ctx) as resp:
return json.loads(resp.read().decode("utf-8", errors="replace"))
def _flatpak() -> str | None:
return shutil.which("flatpak") or (
"/usr/bin/flatpak" if Path("/usr/bin/flatpak").exists() else None
@@ -179,6 +261,13 @@ class Plugin:
if stderr:
decky.logger.debug("avahi-browse stderr: %s", stderr.decode(errors="replace"))
hosts = _parse_avahi_browse(stdout.decode(errors="replace"))
# Mark which hosts THIS device has already paired (by cert fingerprint), so the UI can
# show "Stream" instead of "Pair" — the mDNS `pair` field is the host's policy, not our
# per-device pairing state.
paired = _paired_fingerprints()
for h in hosts:
fp = h.get("fp") or ""
h["paired"] = bool(fp) and fp.lower() in paired
decky.logger.info("discovered %d punktfunk host(s)", len(hosts))
return hosts
@@ -279,6 +368,54 @@ class Plugin:
return {"ok": False}
return {"ok": True}
async def check_update(self, force: bool = False) -> dict:
"""Is a newer build available in our registry? Compares the installed version
(``package.json``) against the per-channel ``manifest.json`` the CI publishes, and
returns everything the frontend needs to drive Decky's install RPC. Non-fatal: any
failure (no channel baked in, network down) returns ``update_available: False``.
"""
current = _installed_version()
cfg = _update_config()
result = {
"current": current,
"latest": current,
"artifact": "",
"hash": "",
"channel": str(cfg.get("channel", "")),
"update_available": False,
}
manifest_url = cfg.get("manifest")
if not manifest_url:
result["error"] = "update-channel-unknown" # dev / sideloaded build
return result
now = time.monotonic()
cached = _update_cache["data"]
if not force and cached and (now - _update_cache["at"]) < _UPDATE_TTL_S:
return cached
try:
loop = asyncio.get_running_loop()
manifest = await loop.run_in_executor(None, _fetch_json, manifest_url)
except Exception as exc: # noqa: BLE001
decky.logger.warning("update check failed: %s", exc)
result["error"] = "fetch-failed"
return result # transient — don't cache, retry next open
latest = str(manifest.get("version", current))
result["latest"] = latest
result["artifact"] = str(manifest.get("artifact", ""))
result["hash"] = str(manifest.get("sha256", ""))
result["update_available"] = bool(result["artifact"]) and (
_semver_tuple(latest) > _semver_tuple(current)
)
if result["update_available"]:
decky.logger.info("update available: %s -> %s (%s)", current, latest, result["channel"])
_update_cache["at"] = now
_update_cache["data"] = result
return result
# ---- Decky lifecycle ----
async def _main(self):
+13 -1
View File
@@ -5,8 +5,9 @@ export interface Host {
name: string;
host: string;
port: number;
pair: string; // "required" | "optional"
pair: string; // "required" | "optional" — the HOST's policy
fp: string;
paired: boolean; // whether THIS device has already PIN-paired this host (by fingerprint)
}
export interface PairResult {
@@ -32,6 +33,16 @@ export interface StreamSettings {
mic_enabled: boolean;
}
export interface UpdateInfo {
current: string; // installed version (package.json)
latest: string; // newest version in our registry for this channel
artifact: string; // immutable zip URL Decky should install
hash: string; // sha256 of that zip (Decky verifies it)
channel: string; // "latest" (stable) | "canary"
update_available: boolean;
error?: string; // "update-channel-unknown" (dev build) | "fetch-failed"
}
export const discover = callable<[], Host[]>("discover");
export const pair = callable<
[host: string, port: number, pin: string, name: string],
@@ -43,3 +54,4 @@ export const setSettings = callable<[settings: StreamSettings], { ok: boolean }>
"set_settings",
);
export const killStream = callable<[], { ok: boolean }>("kill_stream");
export const checkUpdate = callable<[force: boolean], UpdateInfo>("check_update");
+269 -38
View File
@@ -10,12 +10,22 @@ import {
PanelSectionRow,
SliderField,
Spinner,
Tabs,
ToggleField,
showModal,
staticClasses,
} from "@decky/ui";
import { definePlugin, routerHook, toaster } from "@decky/api";
import { FC, useCallback, useEffect, useState } from "react";
import {
Component,
CSSProperties,
ErrorInfo,
FC,
ReactNode,
useCallback,
useEffect,
useState,
} from "react";
import {
FaTv,
FaSyncAlt,
@@ -23,19 +33,130 @@ import {
FaLockOpen,
FaPlay,
FaArrowLeft,
FaDownload,
} from "react-icons/fa";
import {
discover,
getSettings,
pair,
setSettings,
checkUpdate,
Host,
StreamSettings,
UpdateInfo,
} from "./backend";
import { launchStream } from "./steam";
const ROUTE = "/punktfunk";
// Decky Loader exposes its already-authenticated WSRouter as a global. This is NOT part of
// @decky/api (it's a loader internal), so we treat it as optional and guard every use — on a
// loader without it we fall back to manual "Install Plugin from URL". We use it to drive
// Decky's own privileged install path (the root loader does the download + SHA-256 verify +
// extract + hot-reload), which is the only way a plugin can update itself: ~/homebrew/plugins
// is root-owned, so our unprivileged backend can't swap its own files.
declare global {
interface Window {
DeckyBackend?: {
callable: (route: string) => (...args: unknown[]) => Promise<unknown>;
};
}
}
// PluginInstallType.UPDATE in decky-loader's browser.py (INSTALL=0/REINSTALL=1/UPDATE=2/…).
const INSTALL_TYPE_UPDATE = 2;
// ----------------------------------------------------------------------------------------
// Error boundary — contains ANY render failure in our UI so a single bad render can never take
// down the whole Quick Access "Decky" section (Decky's tab-level boundary shows the generic
// "Something went wrong while displaying this content" for the entire tab when one plugin
// throws). The realistic trigger is a future Steam client update that makes a @decky/ui
// component resolve to `undefined` (React then throws "Element type is invalid"). The fallback
// is built from ONLY plain DOM elements + inline styles, so it cannot itself depend on a
// (possibly broken) Steam-internal component — it is guaranteed to render.
// ----------------------------------------------------------------------------------------
class PluginErrorBoundary extends Component<
{ children: ReactNode },
{ error: Error | null }
> {
state: { error: Error | null } = { error: null };
static getDerivedStateFromError(error: Error) {
return { error };
}
componentDidCatch(error: Error, info: ErrorInfo) {
// Surface it for diagnosis, but never rethrow — containment is the whole point.
// eslint-disable-next-line no-console
console.error("[punktfunk] contained UI render error:", error, info?.componentStack);
}
render() {
const { error } = this.state;
if (!error) return this.props.children;
return (
<div style={{ padding: "1em", lineHeight: 1.45 }}>
<div style={{ fontWeight: "bold", marginBottom: "0.4em" }}>
punktfunk couldnt draw this view
</div>
<div style={{ opacity: 0.8, marginBottom: "0.6em" }}>
The plugin hit a display error your Steam Deck is fine. Reload punktfunk from
Decky&apos;s plugin list, or update the plugin.
</div>
<div
style={{
opacity: 0.55,
fontFamily: "monospace",
fontSize: "0.8em",
wordBreak: "break-word",
}}
>
{String(error?.message ?? error)}
</div>
</div>
);
}
}
// Checks our registry for a newer build on mount (the backend caches + is non-fatal offline).
function useUpdate() {
const [info, setInfo] = useState<UpdateInfo | null>(null);
useEffect(() => {
void checkUpdate(false)
.then(setInfo)
.catch(() => {});
}, []);
return info;
}
async function applyUpdate(info: UpdateInfo) {
try {
const backend = window.DeckyBackend;
if (backend?.callable) {
// Fire-and-forget: the loader reinstalls + reloads THIS plugin, tearing the panel down
// before any result could arrive — so never await it. Decky shows its own confirm prompt.
void backend.callable("utilities/install_plugin")(
info.artifact,
"punktfunk",
info.latest,
info.hash,
INSTALL_TYPE_UPDATE,
);
toaster.toast({
title: "punktfunk",
body: `Updating to v${info.latest}… confirm the Decky prompt.`,
});
return;
}
} catch {
// fall through to the manual path
}
toaster.toast({
title: "punktfunk",
body: "Update from Decky → Developer → Install Plugin from URL.",
});
}
// ----------------------------------------------------------------------------------------
// Discovery hook — shared by the QAM panel and the full page.
// ----------------------------------------------------------------------------------------
@@ -255,20 +376,24 @@ const SettingsSection: FC = () => {
// One host row on the full page.
// ----------------------------------------------------------------------------------------
const HostRow: FC<{ host: Host }> = ({ host }) => {
const pairRequired = host.pair === "required";
// The host's policy is `pair=required`, but if THIS device is already paired we don't need to
// pair again — show it as trusted and go straight to Stream.
const needsPair = host.pair === "required" && !host.paired;
return (
<Field
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{pairRequired ? <FaLock /> : <FaLockOpen />}
{needsPair ? <FaLock /> : <FaLockOpen />}
{host.name}
</span>
}
description={`${host.host}:${host.port}${pairRequired ? " · pairing required" : ""}`}
description={`${host.host}:${host.port}${
needsPair ? " · pairing required" : host.paired ? " · paired" : ""
}`}
childrenContainerWidth="max"
>
<Focusable style={{ display: "flex", gap: "0.5em" }}>
{pairRequired && (
{needsPair && (
<DialogButton
style={{ minWidth: "5em" }}
onClick={() =>
@@ -288,52 +413,129 @@ const HostRow: FC<{ host: Host }> = ({ host }) => {
};
// ----------------------------------------------------------------------------------------
// The fullscreen page (registered as the /punktfunk route).
// The fullscreen page (registered as the /punktfunk route) — a tabbed Hosts / Settings view.
// ----------------------------------------------------------------------------------------
// Bottom inset so the last control clears Gaming Mode's footer hint bar. Routed pages render
// *under* that bar otherwise — that's why the last Stream-settings row was getting hidden. The
// value is generous on purpose (and harmless where the tab area already insets); tune to taste.
const SAFE_BOTTOM = "80px";
// Each tab is its own scroll area so long content is always reachable above the footer.
const tabScroll: CSSProperties = {
height: "100%",
overflowY: "auto",
padding: "0.5em 2.5em",
paddingBottom: SAFE_BOTTOM,
boxSizing: "border-box",
};
const HostsTab: FC<{
hosts: Host[];
scanning: boolean;
refresh: () => void;
}> = ({ hosts, scanning, refresh }) => (
<div style={tabScroll}>
<Field
label="Discover"
description={
scanning
? "Scanning the LAN…"
: `${hosts.length} host${hosts.length === 1 ? "" : "s"} on your network`
}
childrenContainerWidth="max"
bottomSeparator={hosts.length ? "standard" : "none"}
>
<DialogButton style={{ minWidth: "8em" }} disabled={scanning} onClick={refresh}>
{scanning ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
) : (
<FaSyncAlt style={{ marginRight: "0.5em" }} />
)}
{scanning ? "Scanning…" : "Refresh"}
</DialogButton>
</Field>
{hosts.length === 0 && !scanning && (
<Field
focusable={false}
description="No punktfunk hosts found. Make sure a host is running on the same network."
>
No hosts found
</Field>
)}
{hosts.map((h) => (
<HostRow key={h.fp || `${h.host}:${h.port}`} host={h} />
))}
</div>
);
const SettingsTab: FC = () => (
<div style={tabScroll}>
<SettingsSection />
</div>
);
const PunktfunkPage: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const update = useUpdate();
const [tab, setTab] = useState("hosts");
return (
<div
style={{
marginTop: "40px",
height: "calc(100% - 40px)",
overflowY: "auto",
padding: "0 2.5em 2.5em",
display: "flex",
flexDirection: "column",
}}
>
<Focusable style={{ display: "flex", alignItems: "center", gap: "1em", marginBottom: "1em" }}>
<Focusable
style={{
display: "flex",
alignItems: "center",
gap: "1em",
padding: "0 2.5em",
marginBottom: "0.4em",
flexShrink: 0,
}}
>
<DialogButton
style={{ width: "3em", minWidth: "3em" }}
style={{ width: "3em", minWidth: "3em", padding: 0 }}
onClick={() => Navigation.NavigateBack()}
>
<FaArrowLeft />
</DialogButton>
<div className={staticClasses.Title} style={{ flex: 1 }}>
<div className={staticClasses?.Title} style={{ flex: 1, margin: 0 }}>
punktfunk
</div>
<DialogButton style={{ width: "10em" }} disabled={scanning} onClick={refresh}>
{scanning ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
) : (
<FaSyncAlt style={{ marginRight: "0.5em" }} />
)}
{scanning ? "Scanning…" : "Refresh"}
</DialogButton>
{update?.update_available && (
<DialogButton style={{ minWidth: "9em" }} onClick={() => applyUpdate(update)}>
<FaDownload style={{ marginRight: "0.4em" }} />
Update v{update.latest}
</DialogButton>
)}
</Focusable>
<div style={{ fontSize: "1.1em", fontWeight: "bold", margin: "0.5em 0" }}>Hosts</div>
{hosts.length === 0 && !scanning && (
<Field focusable={false}>No hosts discovered on the LAN.</Field>
)}
{hosts.map((h) => (
<HostRow key={h.fp || `${h.host}:${h.port}`} host={h} />
))}
<div style={{ fontSize: "1.1em", fontWeight: "bold", margin: "1.5em 0 0.5em" }}>
Stream settings
<div style={{ flex: 1, minHeight: 0 }}>
<Tabs
activeTab={tab}
onShowTab={(id: string) => setTab(id)}
autoFocusContents
tabs={[
{
id: "hosts",
title: "Hosts",
content: <HostsTab hosts={hosts} scanning={scanning} refresh={refresh} />,
},
{
id: "settings",
title: "Settings",
content: <SettingsTab />,
},
]}
/>
</div>
<SettingsSection />
</div>
);
};
@@ -343,9 +545,25 @@ const PunktfunkPage: FC = () => {
// ----------------------------------------------------------------------------------------
const QamPanel: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const update = useUpdate();
return (
<>
{update?.update_available && (
<PanelSection title="Update">
<PanelSectionRow>
<ButtonItem
layout="below"
onClick={() => applyUpdate(update)}
label={`v${update.current} → v${update.latest}`}
>
<FaDownload style={{ marginRight: "0.5em" }} />
Update punktfunk
</ButtonItem>
</PanelSectionRow>
</PanelSection>
)}
<PanelSection title="punktfunk">
<PanelSectionRow>
<ButtonItem
@@ -378,25 +596,25 @@ const QamPanel: FC = () => {
</PanelSectionRow>
)}
{hosts.map((h) => {
const pairRequired = h.pair === "required";
const needsPair = h.pair === "required" && !h.paired;
return (
<PanelSectionRow key={h.fp || `${h.host}:${h.port}`}>
<ButtonItem
layout="below"
onClick={() =>
pairRequired
needsPair
? showModal(<PairModal host={h} onPaired={() => startStream(h)} />)
: startStream(h)
}
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{pairRequired ? <FaLock /> : <FaLockOpen />}
{needsPair ? <FaLock /> : <FaLockOpen />}
{h.name}
</span>
}
description={`${h.host}:${h.port}`}
description={`${h.host}:${h.port}${h.paired ? " · paired" : ""}`}
>
{pairRequired ? "Pair & Stream" : "Stream"}
{needsPair ? "Pair & Stream" : "Stream"}
</ButtonItem>
</PanelSectionRow>
);
@@ -406,12 +624,25 @@ const QamPanel: FC = () => {
);
};
// Full page behind the boundary — registered as the /punktfunk route.
const PunktfunkRoute: FC = () => (
<PluginErrorBoundary>
<PunktfunkPage />
</PluginErrorBoundary>
);
export default definePlugin(() => {
routerHook.addRoute(ROUTE, PunktfunkPage, { exact: true });
routerHook.addRoute(ROUTE, PunktfunkRoute, { exact: true });
return {
name: "punktfunk",
titleView: <div className={staticClasses.Title}>punktfunk</div>,
content: <QamPanel />,
// `staticClasses?.Title` is guarded so a future client that drops the export can't throw
// at plugin-load time (an error boundary only catches render-time, not load-time, errors).
titleView: <div className={staticClasses?.Title}>punktfunk</div>,
content: (
<PluginErrorBoundary>
<QamPanel />
</PluginErrorBoundary>
),
icon: <FaTv />,
onDismount() {
routerHook.removeRoute(ROUTE);
+22 -2
View File
@@ -24,12 +24,31 @@ declare const SteamClient: {
SetShortcutExe(appId: number, exe: string): void;
SetShortcutStartDir(appId: number, dir: string): void;
SetAppLaunchOptions(appId: number, options: string): void;
SetAppHidden(appId: number, hidden: boolean): void;
RunGame(gameId: string, _unused: string, _i: number, _j: number): void;
TerminateApp(gameId: string, _b: boolean): void;
};
};
// Steam removed `SteamClient.Apps.SetAppHidden`. Hiding a non-Steam shortcut now goes through
// `collectionStore.SetAppsAsHidden([appId], true)` — but that looks the app up in appStore, which
// only registers a freshly-created shortcut a moment later (calling it immediately throws on a
// null overview). So hiding is BEST-EFFORT + DEFERRED and must NEVER block the launch.
declare const collectionStore:
| { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void }
| undefined;
function hideShortcut(appId: number): void {
const attempt = () => {
try {
collectionStore?.SetAppsAsHidden?.([appId], true);
} catch {
/* overview not registered yet, or the API changed — cosmetic, ignore */
}
};
attempt(); // succeeds immediately for an already-registered (reused) shortcut
setTimeout(attempt, 2500); // fresh shortcut: retry once its app overview lands
}
const SHORTCUT_NAME = "punktfunk";
// The 64-bit "gameid" RunGame wants, derived from a 32-bit non-Steam shortcut appId: the
@@ -88,7 +107,8 @@ async function ensureShortcut(): Promise<number> {
);
SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME);
// Hide it from the library — it's an implementation detail, launched programmatically.
SteamClient.Apps.SetAppHidden(appId, true);
// Best-effort + deferred (see hideShortcut); never let it block the launch.
hideShortcut(appId);
rememberAppId(appId);
return appId;
}
+95 -2
View File
@@ -22,6 +22,8 @@ struct App {
gamepad: crate::gamepad::GamepadService,
/// One session at a time — ignore connects while one is starting/running.
busy: std::cell::Cell<bool>,
/// Steam Deck / Gaming-Mode launch: fullscreen the window (chrome-less) when a stream starts.
fullscreen: bool,
}
impl App {
@@ -41,7 +43,13 @@ pub fn run() -> glib::ExitCode {
if let Some(pin) = arg_value("--pair") {
return headless_pair(&pin);
}
let app = adw::Application::builder().application_id(APP_ID).build();
let mut builder = adw::Application::builder().application_id(APP_ID);
// Screenshot mode launches the app once per scene back-to-back; NON_UNIQUE keeps each
// launch its own primary instance instead of forwarding to a still-registered name.
if shot_scene().is_some() {
builder = builder.flags(gtk::gio::ApplicationFlags::NON_UNIQUE);
}
let app = builder.build();
app.connect_activate(build_ui);
// GTK doesn't see our argv (`--connect` is handled in `build_ui`); an empty argv also
// keeps GApplication from rejecting unknown options.
@@ -56,6 +64,20 @@ fn arg_value(flag: &str) -> Option<String> {
.filter(|v| !v.starts_with("--"))
}
/// True if argv contains `flag` (a valueless switch).
fn arg_flag(flag: &str) -> bool {
std::env::args().any(|a| a == flag)
}
/// Run the stream fullscreen with no window chrome — the Steam Deck / Gaming-Mode launch path.
/// The Decky wrapper passes `--fullscreen`; we also honor the Deck/gamescope env as a fallback
/// so a manual launch under Gaming Mode does the right thing too.
fn fullscreen_mode() -> bool {
arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
|| std::env::var_os("GAMESCOPE_WAYLAND_DISPLAY").is_some()
}
/// Run the SPAKE2 PIN ceremony without a GTK window and persist the verified host to the
/// known-hosts store as paired, so a later `--connect` connects silently. Same identity
/// store the streaming path uses (same binary), so pairing here makes the stream work.
@@ -161,6 +183,7 @@ fn build_ui(gtk_app: &adw::Application) {
identity,
gamepad: crate::gamepad::GamepadService::start(),
busy: std::cell::Cell::new(false),
fullscreen: fullscreen_mode(),
});
let hosts_page = crate::ui_hosts::new(
@@ -182,11 +205,65 @@ fn build_ui(gtk_app: &adw::Application) {
nav.add(&hosts_page);
window.present();
// CI screenshot mode: render one scripted, host-free scene and signal readiness
// (clients/linux/tools/screenshots.sh). Mutually exclusive with a real connect.
if let Some(scene) = shot_scene() {
run_shot(app, &scene);
return;
}
if let Some(req) = cli_connect_request() {
initiate_connect(app, req);
}
}
/// `PUNKTFUNK_SHOT_SCENE`, when set, selects a scripted host-free scene for CI screenshots.
fn shot_scene() -> Option<String> {
std::env::var("PUNKTFUNK_SHOT_SCENE")
.ok()
.filter(|s| !s.is_empty())
}
/// Render one mock-populated, host-free scene over the already-presented window, then print
/// `PF_SHOT_READY` once it has had a moment to map + settle so the driver knows when to capture.
/// No `NativeClient` or session is created. The stream scene is deliberately absent — its page
/// requires a live connector (`ui_stream::new` takes an `Arc<NativeClient>`).
fn run_shot(app: Rc<App>, scene: &str) {
// A plausible host for the trust/pair dialogs (fp_hex is 64 hex chars, like a real SHA-256).
let mock_req = || ConnectRequest {
name: "Living Room PC".to_string(),
addr: "192.168.1.42".to_string(),
port: 9777,
fp_hex: Some(
"9f8e7d6c5b4a39281706f5e4d3c2b1a0998877665544332211ffeeddccbbaa00".to_string(),
),
pair_optional: true,
};
match scene {
// The saved-hosts grid reads ~/.config/punktfunk/client-known-hosts.json, which the
// driver seeds — so the already-shown hosts page is the scene; nothing to do here.
"hosts" | "02-hosts" => {}
"settings" | "03-settings" => {
crate::ui_settings::show(&app.window, app.settings.clone(), &app.gamepad);
}
"trust" | "04-trust" => tofu_dialog(app.clone(), mock_req()),
"pair" | "05-pair" => pin_dialog(app.clone(), mock_req()),
other => tracing::warn!("unknown PUNKTFUNK_SHOT_SCENE={other:?}; showing hosts only"),
}
let settle_ms = std::env::var("PUNKTFUNK_SHOT_SETTLE_MS")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(900);
let scene = scene.to_string();
glib::timeout_add_local_once(std::time::Duration::from_millis(settle_ms), move || {
use std::io::Write as _;
println!("PF_SHOT_READY scene={scene}");
let _ = std::io::stdout().flush();
});
}
/// The trust gate in front of every connect. The host is the policy authority (it
/// advertises `pair=optional` only when it accepts unpaired clients); the client renders
/// its trust UI from that:
@@ -375,6 +452,7 @@ fn speed_test(app: Rc<App>, req: ConnectRequest) {
GamepadPref::Auto,
0, // bitrate_kbps (host default)
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
2, // audio_channels: speed-test probe, stereo
None, // launch: speed-test probe connect, no game
pin,
Some(identity),
@@ -443,11 +521,19 @@ fn resolve_mode(app: &App) -> punktfunk_core::config::Mode {
refresh_hz: s.refresh_hz,
};
if mode.width == 0 || mode.refresh_hz == 0 {
// Prefer the monitor the window is on; fall back to the display's first monitor. On a
// `--connect` launch the window may not be mapped yet when this runs, and without the
// fallback we'd drop to the 1920×1080 floor below — wrong on the Deck (1280×800).
let monitor = app
.window
.surface()
.zip(gdk::Display::default())
.and_then(|(surf, d)| d.monitor_at_surface(&surf));
.and_then(|(surf, d)| d.monitor_at_surface(&surf))
.or_else(|| {
gdk::Display::default()
.and_then(|d| d.monitors().item(0))
.and_then(|o| o.downcast::<gdk::Monitor>().ok())
});
if let Some(m) = monitor {
let geo = m.geometry();
let scale = m.scale_factor().max(1);
@@ -488,6 +574,7 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
},
bitrate_kbps: s.bitrate_kbps,
mic_enabled: s.mic_enabled,
audio_channels: s.audio_channels,
pin,
identity: app.identity.clone(),
};
@@ -540,6 +627,12 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
&title,
);
app.nav.push(&p.page);
// Steam Deck / Gaming Mode: gamescope fullscreens the window but GTK doesn't
// know it, so its header bar stays drawn. Enter GTK fullscreen explicitly —
// the stream page's `connect_fullscreened_notify` then hides all chrome.
if app.fullscreen {
app.window.fullscreen();
}
page = Some(p);
}
SessionEvent::Stats(s) => {
+21 -10
View File
@@ -27,16 +27,17 @@ pub struct AudioPlayer {
}
impl AudioPlayer {
/// Spawn the PipeWire playback thread. Failure (no PipeWire in the session) is
/// survivable — the caller streams video-only.
pub fn spawn() -> Result<AudioPlayer> {
/// Spawn the PipeWire playback thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no PipeWire in the session) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u32) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the PipeWire loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = pw_thread(pcm_rx, quit_rx) {
if let Err(e) = pw_thread(pcm_rx, quit_rx, channels as usize) {
tracing::warn!(error = %e, "audio playback thread ended");
}
})
@@ -48,8 +49,8 @@ impl AudioPlayer {
})
}
/// Queue one interleaved-stereo f32 chunk. Drops the chunk if the PipeWire side is
/// wedged (the renderer conceals the gap; never block the session pump).
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// PipeWire side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
@@ -71,11 +72,14 @@ struct PlayerData {
rx: Receiver<Vec<f32>>,
ring: VecDeque<f32>,
primed: bool,
/// Interleaved channel count this stream was opened with (2/6/8).
channels: usize,
}
fn pw_thread(
pcm_rx: Receiver<Vec<f32>>,
quit_rx: pipewire::channel::Receiver<Terminate>,
channels: usize,
) -> Result<()> {
use pipewire as pw;
use pw::{properties::properties, spa};
@@ -115,6 +119,7 @@ fn pw_thread(
rx: pcm_rx,
ring: VecDeque::new(),
primed: false,
channels,
};
let _listener = stream
@@ -130,19 +135,19 @@ fn pw_thread(
while let Ok(chunk) = ud.rx.try_recv() {
ud.ring.extend(chunk);
}
let stride = 4 * CHANNELS; // F32LE interleaved
let stride = 4 * ud.channels; // F32LE interleaved
let datas = buffer.datas_mut();
if datas.is_empty() {
return;
}
let data = &mut datas[0];
let want_frames = data.data().map(|s| s.len() / stride).unwrap_or(0);
let want = want_frames * CHANNELS;
let want = want_frames * ud.channels;
// Adaptive jitter buffer (same shape as the host's virtual mic): prime to
// ~3 quanta, cap at ~1 quantum of slack beyond that, re-prime after a
// genuine drain.
let target = (3 * want).clamp(720 * CHANNELS, 9600 * CHANNELS);
let target = (3 * want).clamp(720 * ud.channels, 9600 * ud.channels);
while ud.ring.len() > target.max(want) + want {
ud.ring.pop_front();
}
@@ -182,7 +187,13 @@ fn pw_thread(
let mut info = AudioInfoRaw::new();
info.set_format(AudioFormat::F32LE);
info.set_rate(SAMPLE_RATE);
info.set_channels(CHANNELS as u32);
info.set_channels(channels as u32);
// Channel positions in canonical wire order (FL FR FC LFE RL RR SL SR) so PipeWire routes each
// slot to the matching speaker (and downmixes when the sink has fewer). Identity, no permute.
let order = punktfunk_core::audio::spa_positions(channels as u8);
let mut positions = [0u32; 64];
positions[..order.len()].copy_from_slice(order);
info.set_position(positions);
let obj = pw::spa::pod::Object {
type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(),
id: pw::spa::param::ParamType::EnumFormat.as_raw(),
+50 -7
View File
@@ -20,6 +20,8 @@ pub struct SessionParams {
pub compositor: CompositorPref,
pub gamepad: GamepadPref,
pub bitrate_kbps: u32,
/// Requested audio channel count (2/6/8); the host echoes the resolved value.
pub audio_channels: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Pinned host fingerprint; `None` = trust on first use (caller persists the observed one).
@@ -83,6 +85,42 @@ fn now_ns() -> u64 {
.unwrap_or(0)
}
/// Opus decoder for the audio plane: a plain stereo decoder (the validated path) or a multistream
/// decoder for 5.1/7.1, both behind one `decode_float`. Built from the host-RESOLVED channel count
/// via the shared layout table.
enum AudioDec {
Stereo(opus::Decoder),
Surround(opus::MSDecoder),
}
impl AudioDec {
fn new(channels: u8) -> Result<AudioDec, opus::Error> {
if channels == 2 {
Ok(AudioDec::Stereo(opus::Decoder::new(
48_000,
opus::Channels::Stereo,
)?))
} else {
let l = punktfunk_core::audio::layout_for(channels, false);
Ok(AudioDec::Surround(opus::MSDecoder::new(
48_000, l.streams, l.coupled, l.mapping,
)?))
}
}
fn decode_float(
&mut self,
input: &[u8],
out: &mut [f32],
fec: bool,
) -> Result<usize, opus::Error> {
match self {
AudioDec::Stereo(d) => d.decode_float(input, out, fec),
AudioDec::Surround(d) => d.decode_float(input, out, fec),
}
}
}
fn pump(
params: SessionParams,
ev_tx: async_channel::Sender<SessionEvent>,
@@ -96,7 +134,8 @@ fn pump(
params.compositor,
params.gamepad,
params.bitrate_kbps,
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
params.audio_channels,
None, // launch: the Linux client has no library picker yet
params.pin,
Some(params.identity),
@@ -134,11 +173,14 @@ fn pump(
}
};
// Audio is best-effort: a session without it still streams. Gamepads are the
// app-lifetime service's job (the UI attaches it on Connected).
let player = audio::AudioPlayer::spawn()
// app-lifetime service's job (the UI attaches it on Connected). Build the decoder + playback
// from the host-RESOLVED channel count (never the request), so an older/clamping host that
// resolves stereo is decoded as stereo.
let channels = connector.audio_channels;
let player = audio::AudioPlayer::spawn(channels as u32)
.map_err(|e| tracing::warn!(error = %e, "audio disabled"))
.ok();
let mut opus_dec = opus::Decoder::new(48_000, opus::Channels::Stereo)
let mut opus_dec = AudioDec::new(channels)
.map_err(|e| tracing::warn!(error = %e, "opus decoder failed — audio disabled"))
.ok();
let _mic = params
@@ -157,8 +199,8 @@ fn pump(
let mut bytes_n = 0u64;
let mut decode_us_sum = 0u64;
let mut lat_us: Vec<u64> = Vec::with_capacity(256);
let mut pcm = vec![0f32; 5760 * 2]; // decode scratch: max Opus frame (120 ms stereo)
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut pcm = vec![0f32; 5760 * channels as usize]; // scratch: max Opus frame (120 ms) × channels
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
@@ -221,7 +263,8 @@ fn pump(
while let Ok(pkt) = connector.next_audio(Duration::ZERO) {
if let (Some(player), Some(dec)) = (&player, opus_dec.as_mut()) {
match dec.decode_float(&pkt.data, &mut pcm, false) {
Ok(samples) => player.push(pcm[..samples * 2].to_vec()),
// `samples` is per-channel; the interleaved frame is `samples * channels`.
Ok(samples) => player.push(pcm[..samples * channels as usize].to_vec()),
Err(e) => tracing::debug!(error = %e, "opus decode"),
}
}
+12
View File
@@ -90,6 +90,14 @@ impl KnownHosts {
self.hosts.iter().find(|h| h.addr == addr && h.port == port)
}
/// Forget the entry with this fingerprint. Returns true if one was removed (the user
/// will have to pair/trust again to reconnect).
pub fn remove_by_fp(&mut self, fp_hex: &str) -> bool {
let before = self.hosts.len();
self.hosts.retain(|h| h.fp_hex != fp_hex);
self.hosts.len() != before
}
/// Insert or refresh an entry, keyed by fingerprint. `paired` only ever upgrades
/// (a later TOFU connect must not demote a PIN-paired host).
pub fn upsert(&mut self, entry: KnownHost) {
@@ -124,6 +132,9 @@ pub struct Settings {
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + playback layout.
pub audio_channels: u8,
}
impl Default for Settings {
@@ -137,6 +148,7 @@ impl Default for Settings {
compositor: "auto".into(),
inhibit_shortcuts: true,
mic_enabled: false,
audio_channels: 2,
}
}
}
+46
View File
@@ -181,6 +181,52 @@ pub fn new(
// pinned connect; TOFU eligibility is irrelevant.
pair_optional: false,
};
// Forget this host (drops the pinned fingerprint — a later connect re-pairs).
// Confirmed first, since it's destructive and a misclick on the Deck is easy.
let remove_btn = gtk::Button::from_icon_name("user-trash-symbolic");
remove_btn.set_tooltip_text(Some("Remove saved host"));
remove_btn.set_valign(gtk::Align::Center);
remove_btn.add_css_class("flat");
{
let fp = k.fp_hex.clone();
let name = k.name.clone();
let saved_list = saved_list.clone();
let saved_label = saved_label.clone();
let row = row.clone();
remove_btn.connect_clicked(move |_| {
let dialog = adw::AlertDialog::new(
Some("Remove saved host?"),
Some(&format!(
"Forget “{name}”? You'll need to pair (or trust) it again to reconnect."
)),
);
dialog.add_responses(&[("cancel", "Cancel"), ("remove", "Remove")]);
dialog.set_response_appearance(
"remove",
adw::ResponseAppearance::Destructive,
);
dialog.set_default_response(Some("cancel"));
dialog.set_close_response("cancel");
{
// Scoped clones for the response handler so `row` survives for present().
let fp = fp.clone();
let saved_list = saved_list.clone();
let saved_label = saved_label.clone();
let row = row.clone();
dialog.connect_response(Some("remove"), move |_, _| {
let mut known = KnownHosts::load();
known.remove_by_fp(&fp);
let _ = known.save();
saved_list.remove(&row);
let empty = known.hosts.is_empty();
saved_list.set_visible(!empty);
saved_label.set_visible(!empty);
});
}
dialog.present(Some(&row));
});
}
row.add_suffix(&remove_btn);
let speed_btn = gtk::Button::from_icon_name("network-transmit-receive-symbolic");
speed_btn.set_tooltip_text(Some("Test network speed"));
speed_btn.set_valign(gtk::Align::Center);
+20
View File
@@ -140,6 +140,16 @@ pub fn show(
input.add(&inhibit_row);
let audio = adw::PreferencesGroup::builder().title("Audio").build();
let surround_row = adw::ComboRow::builder()
.title("Audio channels")
.subtitle("Request stereo or surround (the host downmixes if its output has fewer)")
.model(&gtk::StringList::new(&[
"Stereo",
"5.1 Surround",
"7.1 Surround",
]))
.build();
audio.add(&surround_row);
let mic_row = adw::SwitchRow::builder()
.title("Stream microphone")
.subtitle("Send the default input device to the host's virtual microphone")
@@ -170,6 +180,11 @@ pub fn show(
compositor_row.set_selected(comp_i as u32);
inhibit_row.set_active(s.inhibit_shortcuts);
mic_row.set_active(s.mic_enabled);
surround_row.set_selected(match s.audio_channels {
6 => 1,
8 => 2,
_ => 0,
});
}
let dialog = adw::PreferencesDialog::new();
@@ -186,6 +201,11 @@ pub fn show(
.to_string();
s.inhibit_shortcuts = inhibit_row.is_active();
s.mic_enabled = mic_row.is_active();
s.audio_channels = match surround_row.selected() {
1 => 6,
2 => 8,
_ => 2,
};
s.save();
});
dialog.present(Some(parent));
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env bash
# Capture host-free UI screenshots of the native Linux client under a virtual X
# display. Mirrors the iOS harness (clients/apple/tools/screenshots.sh): one app
# launch per scene (PUNKTFUNK_SHOT_SCENE), the app renders a mock-populated REAL
# view and prints `PF_SHOT_READY`, then we grab the X root window. No host, GPU, or
# live stream — only the chrome scenes (the stream page needs a live connector).
#
# cargo build --release -p punktfunk-client-linux
# bash clients/linux/tools/screenshots.sh # → clients/linux/screenshots/<scene>.png
# bash clients/linux/tools/screenshots.sh hosts pair # a subset
#
# Env knobs: BIN (client binary), OUT (output dir), GEOMETRY (Xvfb WxHxDepth),
# SETTLE (extra seconds after PF_SHOT_READY), SHOT_DISPLAY (X display), GSK_RENDERER
# (gl|ngl|cairo — gl/llvmpipe by default for full libadwaita fidelity).
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" # clients/linux
BIN="${BIN:-$here/../../target/release/punktfunk-client}"
OUT="${OUT:-$here/screenshots}"
# The client window maps at its 1100x720 default; with no WM under Xvfb it lands at the
# top-left, so keep the root just larger so the full window (incl. its CSD shadow) is
# captured by a root grab with only a thin margin to crop.
GEOMETRY="${GEOMETRY:-1280x800x24}"
SETTLE="${SETTLE:-1.2}"
SHOT_DISPLAY="${SHOT_DISPLAY:-:99}"
if [ "$#" -gt 0 ]; then SCENES=("$@"); else SCENES=(hosts settings trust pair); fi
[ -x "$BIN" ] || {
echo "client binary not found: $BIN (build it first: cargo build --release -p punktfunk-client-linux)" >&2
exit 1
}
# Isolated scratch HOME: the client generates its identity here on first run, and the
# saved-hosts grid is read from client-known-hosts.json, so seed mock hosts for the
# `hosts` scene (the dialogs/settings build their own mock state in-app).
WORK="$(mktemp -d)"
export HOME="$WORK"
mkdir -p "$HOME/.config/punktfunk"
cat >"$HOME/.config/punktfunk/client-known-hosts.json" <<'JSON'
{
"hosts": [
{ "name": "Living Room PC", "addr": "192.168.1.42", "port": 9777,
"fp_hex": "9f8e7d6c5b4a39281706f5e4d3c2b1a0998877665544332211ffeeddccbbaa00",
"paired": true },
{ "name": "Office", "addr": "192.168.1.50", "port": 9777,
"fp_hex": "a1b2c3d4e5f60718293a4b5c6d7e8f90112233445566778899aabbccddeeff00",
"paired": false }
]
}
JSON
# Software-rendered X session — no GPU/Wayland. GL/llvmpipe runs the real NGL renderer
# (cairo is documented-incomplete for 3D-transformed content / libadwaita transitions).
unset WAYLAND_DISPLAY
export DISPLAY="$SHOT_DISPLAY"
export GDK_BACKEND=x11
export LIBGL_ALWAYS_SOFTWARE=1
export GALLIUM_DRIVER="${GALLIUM_DRIVER:-llvmpipe}"
export GSK_RENDERER="${GSK_RENDERER:-gl}"
Xvfb "$SHOT_DISPLAY" -screen 0 "$GEOMETRY" -nolisten tcp >"$WORK/xvfb.log" 2>&1 &
XVFB_PID=$!
cleanup() {
kill "$XVFB_PID" 2>/dev/null || true
rm -rf "$WORK"
}
trap cleanup EXIT
# Wait for the display to accept connections.
for _ in $(seq 1 50); do
if command -v xdpyinfo >/dev/null 2>&1; then
xdpyinfo -display "$SHOT_DISPLAY" >/dev/null 2>&1 && break
else
[ -e "/tmp/.X11-unix/X${SHOT_DISPLAY#:}" ] && break
fi
sleep 0.1
done
capture() {
local out="$1"
if command -v import >/dev/null 2>&1; then
import -silent -window root "$out"
elif command -v scrot >/dev/null 2>&1; then
scrot -o "$out"
else
echo "no screenshot tool — install imagemagick or scrot" >&2
return 1
fi
}
mkdir -p "$OUT"
rc=0
for scene in "${SCENES[@]}"; do
: >"$WORK/log"
PUNKTFUNK_SHOT_SCENE="$scene" "$BIN" >"$WORK/log" 2>&1 &
pid=$!
ready=0
for _ in $(seq 1 200); do # up to ~20s
if grep -q "PF_SHOT_READY" "$WORK/log"; then
ready=1
break
fi
if ! kill -0 "$pid" 2>/dev/null; then break; fi
sleep 0.1
done
if [ "$ready" = 1 ]; then
sleep "$SETTLE"
if capture "$OUT/$scene.png"; then
echo "$scene$OUT/$scene.png"
else
rc=1
fi
else
echo "$scene: client never signalled PF_SHOT_READY" >&2
sed 's/^/ /' "$WORK/log" >&2 || true
rc=1
fi
kill "$pid" 2>/dev/null || true
wait "$pid" 2>/dev/null || true
done
exit "$rc"
+3 -4
View File
@@ -18,8 +18,7 @@ tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# LAN host discovery (`--discover`): browse the native `_punktfunk._udp` mDNS service the host
# advertises (same crate/version the host advertises with).
mdns-sd = "0.20"
# Linux-only: --mic-test's Opus encoder (libopus). The mic UPLINK itself is portable —
# only this synthetic-tone test rig needs the encoder.
[target.'cfg(target_os = "linux")'.dependencies]
# Opus: multistream DECODE of the host's audio plane (the surround validator) + `--mic-test`'s
# encoder. libopus is already in the graph via `punktfunk-core`'s quic feature; this exposes the
# name directly. Cross-platform (cmake-vendored), so the probe builds + validates everywhere.
opus = "0.3"
+51 -6
View File
@@ -78,6 +78,10 @@ struct Args {
gamepad: GamepadPref,
/// `--bitrate KBPS` — request this encoder bitrate (kilobits/s); 0 = host default.
bitrate_kbps: u32,
/// `--audio-channels N` — request stereo (2), 5.1 (6) or 7.1 (8) audio; default 2. The probe
/// multistream-decodes the host's frames and asserts the per-channel sample count, so it's the
/// headless validator for the surround encode path.
audio_channels: u8,
/// `--launch ID` — ask the host to launch a library title in this session (a store-qualified
/// id from the host's `GET /api/v1/library`, e.g. `steam:570`). Host resolves it; `None` = none.
launch: Option<String>,
@@ -201,6 +205,11 @@ fn parse_args() -> Args {
compositor,
gamepad,
bitrate_kbps: get("--bitrate").and_then(|s| s.parse().ok()).unwrap_or(0),
audio_channels: punktfunk_core::audio::normalize_channels(
get("--audio-channels")
.and_then(|s| s.parse().ok())
.unwrap_or(2),
),
launch: get("--launch").map(str::to_string),
speed_test: get("--speed-test").and_then(|s| {
let (kbps, ms) = s.split_once(':')?;
@@ -385,13 +394,23 @@ async fn session(args: Args) -> Result<()> {
// `--launch ID` — host resolves it against its own library and runs it this session.
launch: args.launch.clone(),
// This headless tool just dumps the bitstream (no decode), so it can always claim
// 10-bit support. Gated by env so latency runs stay on the 8-bit baseline:
// PUNKTFUNK_CLIENT_10BIT=1 advertises VIDEO_CAP_10BIT to exercise the host Main10 path.
video_caps: if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() {
punktfunk_core::quic::VIDEO_CAP_10BIT
} else {
0
// 10-bit / 4:4:4 support. Gated by env so latency runs stay on the 8-bit 4:2:0 baseline:
// PUNKTFUNK_CLIENT_10BIT=1 advertises VIDEO_CAP_10BIT (host Main10 path);
// PUNKTFUNK_CLIENT_444=1 advertises VIDEO_CAP_444 (host HEVC 4:4:4 path) — verify the
// resulting chroma with `ffprobe` on the `--out` .h265.
video_caps: {
let mut caps = 0u8;
if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_10BIT;
}
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444;
}
caps
},
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the
// host's frames to exercise the surround encode path headlessly.
audio_channels: args.audio_channels,
}
.encode(),
)
@@ -408,6 +427,8 @@ async fn session(args: Args) -> Result<()> {
bit_depth = welcome.bit_depth,
color = ?welcome.color,
hdr = welcome.color.is_hdr(),
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format,
"session offer"
);
@@ -830,13 +851,37 @@ async fn session(args: Args) -> Result<()> {
hidout_pkts.clone(),
);
let conn2 = conn.clone();
// Build a multistream decoder for the host-RESOLVED layout so the probe actually decodes
// the surround stream (not just counts bytes) — the headless validator for the encode path.
let audio_channels = welcome.audio_channels;
tokio::spawn(async move {
use std::sync::atomic::Ordering::Relaxed;
let mut hdr_logged = false;
let layout = punktfunk_core::audio::layout_for(audio_channels, false);
let mut audio_dec =
opus::MSDecoder::new(48_000, layout.streams, layout.coupled, layout.mapping).ok();
let mut pcm = vec![0f32; 5760 * audio_channels as usize];
let mut audio_decoded_logged = false;
while let Ok(d) = conn2.read_datagram().await {
if let Some((_, _, opus)) = punktfunk_core::quic::decode_audio_datagram(&d) {
a.fetch_add(1, Relaxed);
ab.fetch_add(opus.len() as u64, Relaxed);
// Decode + validate: the per-channel sample count must be a legal Opus frame
// size; log the first success so a loopback test can assert surround decoded.
if let Some(dec) = audio_dec.as_mut() {
match dec.decode_float(opus, &mut pcm, false) {
Ok(samples) if !audio_decoded_logged => {
audio_decoded_logged = true;
tracing::info!(
channels = audio_channels,
samples_per_channel = samples,
"audio decoded (Opus multistream)"
);
}
Ok(_) => {}
Err(e) => tracing::debug!(error = %e, "probe audio decode"),
}
}
} else if punktfunk_core::quic::decode_rumble_datagram(&d).is_some() {
r.fetch_add(1, Relaxed);
} else if let Some(meta) = punktfunk_core::quic::decode_hdr_meta_datagram(&d) {
+32 -2
View File
@@ -39,6 +39,9 @@ const DECODERS: &[(&str, &str)] = &[
];
/// Bitrate presets in Mb/s; `0` = host default.
const BITRATES_MBPS: &[u32] = &[0, 10, 20, 30, 50, 80, 150];
/// Audio channel presets: `(channel count, display label)`. The host clamps to what it can
/// capture; the resolved count drives the decoder + WASAPI render layout.
const AUDIO_CHANNELS: &[(u8, &str)] = &[(2, "Stereo"), (6, "5.1 Surround"), (8, "7.1 Surround")];
#[derive(Clone, PartialEq)]
enum Screen {
@@ -598,6 +601,7 @@ fn connect(
compositor: CompositorPref::Auto,
gamepad: gamepad_pref,
bitrate_kbps: s.bitrate_kbps,
audio_channels: s.audio_channels,
mic_enabled: s.mic_enabled,
hdr_enabled: s.hdr_enabled,
decoder: DecoderPref::from_name(&s.decoder),
@@ -886,6 +890,23 @@ fn settings_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Eleme
s.save();
})
};
let ac_i = AUDIO_CHANNELS
.iter()
.position(|&(v, _)| v == s.audio_channels)
.unwrap_or(0) as i32;
let ac_names: Vec<String> = AUDIO_CHANNELS.iter().map(|&(_, l)| l.to_string()).collect();
let channels_combo = {
let ctx = ctx.clone();
ComboBox::new(ac_names)
.header("Audio channels")
.selected_index(ac_i)
.on_selection_changed(move |i: i32| {
let (v, _) = AUDIO_CHANNELS[(i.max(0) as usize).min(AUDIO_CHANNELS.len() - 1)];
let mut s = ctx.settings.lock().unwrap();
s.audio_channels = v;
s.save();
})
};
let header = grid((
text_block("Settings")
@@ -934,8 +955,17 @@ fn settings_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Eleme
.spacing(10.0),
);
let audio_card =
card(vstack((text_block("Audio").font_size(15.0).semibold(), mic_toggle)).spacing(10.0));
let audio_card = card(
vstack((
text_block("Audio").font_size(15.0).semibold(),
text_block("Request stereo or surround — the host downmixes if its output has fewer.")
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
channels_combo,
mic_toggle,
))
.spacing(10.0),
);
page(vec![
header.into(),
+28 -12
View File
@@ -21,9 +21,9 @@ use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
const SAMPLE_RATE: usize = 48_000;
/// The microphone uplink stays stereo (the host's virtual mic is stereo). The render path is
/// multichannel — its channel count + block align are runtime, driven by the host-resolved layout.
const CHANNELS: usize = 2;
/// 48 kHz stereo f32: 2 channels * 4 bytes = 8 bytes per frame.
const BLOCK_ALIGN: usize = CHANNELS * 4;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
@@ -34,9 +34,10 @@ pub struct AudioPlayer {
}
impl AudioPlayer {
/// Spawn the WASAPI render thread. Failure (no render endpoint on this box) is
/// survivable — the caller streams video-only.
pub fn spawn() -> Result<AudioPlayer> {
/// Spawn the WASAPI render thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no render endpoint on this box) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u8) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the WASAPI loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
@@ -45,14 +46,14 @@ impl AudioPlayer {
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, stop_t, ready_tx) {
if let Err(e) = render_thread(pcm_rx, stop_t, ready_tx, channels) {
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!("WASAPI render: 48 kHz stereo f32 (default endpoint)");
tracing::info!(channels, "WASAPI render: 48 kHz f32 (default endpoint)");
Ok(AudioPlayer {
pcm_tx,
stop,
@@ -66,8 +67,8 @@ impl AudioPlayer {
}
}
/// Queue one interleaved-stereo f32 chunk. Drops the chunk if the WASAPI side is wedged
/// (the renderer conceals the gap; never block the session pump).
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// WASAPI side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
@@ -88,6 +89,7 @@ fn render_thread(
pcm_rx: Receiver<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
channels: u8,
) -> Result<()> {
if let Err(e) = wasapi::initialize_mta()
.ok()
@@ -97,12 +99,26 @@ fn render_thread(
return Ok(());
}
let res = (|| -> Result<()> {
// F32LE interleaved: channels × 4 bytes/sample. Stereo (channels == 2) is byte-identical
// to the old fixed path (mask 0x3, block align 8).
let block_align = channels as usize * 4;
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
// The explicit dwChannelMask is the wire order (FL FR FC LFE RL RR SL SR); 5.1 = 0x3F,
// 7.1 = 0x63F. WASAPI delivers channels in ascending mask-bit order, which equals the wire
// order, so the render mapping is the identity — no permute. `autoconvert` (below) lets the
// audio engine downmix when the endpoint has fewer speakers.
let desired = WaveFormat::new(
32,
32,
&SampleType::Float,
SAMPLE_RATE,
channels as usize,
Some(punktfunk_core::audio::wasapi_channel_mask(channels)),
);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
@@ -139,10 +155,10 @@ fn render_thread(
if avail_frames == 0 {
continue;
}
let want_bytes = avail_frames * BLOCK_ALIGN;
let want_bytes = avail_frames * block_align;
// Prime to ~3 quanta; cap at ~1 quantum of slack beyond that; re-prime on drain.
let target = (3 * want_bytes).clamp(720 * BLOCK_ALIGN, 9600 * BLOCK_ALIGN);
let target = (3 * want_bytes).clamp(720 * block_align, 9600 * block_align);
while ring.len() > target.max(want_bytes) + want_bytes {
ring.pop_front();
}
+49 -6
View File
@@ -23,6 +23,8 @@ pub struct SessionParams {
pub compositor: CompositorPref,
pub gamepad: GamepadPref,
pub bitrate_kbps: u32,
/// Requested audio channel count (2/6/8); the host echoes the resolved value.
pub audio_channels: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Advertise 10-bit + HDR10 so the host may upgrade HDR content to a Main10/PQ stream.
@@ -94,6 +96,42 @@ fn now_ns() -> u64 {
.unwrap_or(0)
}
/// Opus decoder for the audio plane: a plain stereo decoder (the validated path) or a multistream
/// decoder for 5.1/7.1, both behind one `decode_float`. Built from the host-RESOLVED channel count
/// via the shared layout table.
enum AudioDec {
Stereo(opus::Decoder),
Surround(opus::MSDecoder),
}
impl AudioDec {
fn new(channels: u8) -> Result<AudioDec, opus::Error> {
if channels == 2 {
Ok(AudioDec::Stereo(opus::Decoder::new(
48_000,
opus::Channels::Stereo,
)?))
} else {
let l = punktfunk_core::audio::layout_for(channels, false);
Ok(AudioDec::Surround(opus::MSDecoder::new(
48_000, l.streams, l.coupled, l.mapping,
)?))
}
}
fn decode_float(
&mut self,
input: &[u8],
out: &mut [f32],
fec: bool,
) -> Result<usize, opus::Error> {
match self {
AudioDec::Stereo(d) => d.decode_float(input, out, fec),
AudioDec::Surround(d) => d.decode_float(input, out, fec),
}
}
}
fn pump(
params: SessionParams,
ev_tx: async_channel::Sender<SessionEvent>,
@@ -122,6 +160,7 @@ fn pump(
}
0
},
params.audio_channels,
None, // launch: the Windows client has no library picker yet
params.pin,
Some(params.identity),
@@ -161,11 +200,14 @@ fn pump(
let mut hardware = decoder.is_hardware();
let mut hdr = false;
// Audio is best-effort: a session without it still streams. Gamepads are the
// app-lifetime service's job (the UI attaches it on Connected).
let player = audio::AudioPlayer::spawn()
// app-lifetime service's job (the UI attaches it on Connected). Build the decoder + playback
// from the host-RESOLVED channel count (never the request), so an older/clamping host that
// resolves stereo is decoded as stereo.
let channels = connector.audio_channels;
let player = audio::AudioPlayer::spawn(channels)
.map_err(|e| tracing::warn!(error = %e, "audio disabled"))
.ok();
let mut opus_dec = opus::Decoder::new(48_000, opus::Channels::Stereo)
let mut opus_dec = AudioDec::new(channels)
.map_err(|e| tracing::warn!(error = %e, "opus decoder failed — audio disabled"))
.ok();
let _mic = params
@@ -184,8 +226,8 @@ fn pump(
let mut bytes_n = 0u64;
let mut decode_us_sum = 0u64;
let mut lat_us: Vec<u64> = Vec::with_capacity(256);
let mut pcm = vec![0f32; 5760 * 2]; // decode scratch: max Opus frame (120 ms stereo)
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut pcm = vec![0f32; 5760 * channels as usize]; // scratch: max Opus frame (120 ms) × channels
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
@@ -253,7 +295,8 @@ fn pump(
while let Ok(pkt) = connector.next_audio(Duration::ZERO) {
if let (Some(player), Some(dec)) = (&player, opus_dec.as_mut()) {
match dec.decode_float(&pkt.data, &mut pcm, false) {
Ok(samples) => player.push(pcm[..samples * 2].to_vec()),
// `samples` is per-channel; the interleaved frame is `samples * channels`.
Ok(samples) => player.push(pcm[..samples * channels as usize].to_vec()),
Err(e) => tracing::debug!(error = %e, "opus decode"),
}
}
+4
View File
@@ -130,6 +130,9 @@ pub struct Settings {
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + WASAPI render layout.
pub audio_channels: u8,
/// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream (the client
/// presents it on a 10-bit ST.2084 swapchain). No effect on SDR content.
pub hdr_enabled: bool,
@@ -148,6 +151,7 @@ impl Default for Settings {
compositor: "auto".into(),
inhibit_shortcuts: true,
mic_enabled: false,
audio_channels: 2,
hdr_enabled: true,
decoder: "auto".into(),
}
+1 -1
View File
@@ -276,7 +276,7 @@ pub mod frame {
/// These were hand-duplicated as `OFF_*`/`SHM_*` constants in `inject/{gamepad,dualsense}_windows.rs`
/// and (as bare literals — `*view.add(140)`) in the standalone `xusb-driver`/`dualsense-driver`
/// workspaces, guarded only by "must match" comments — the top ABI-drift hazard the audit flagged
/// (`docs/windows-host-rewrite.md` §2.7). Owning them here with `Pod` derives + `offset_of!`
/// (`design/windows-host-rewrite.md` §2.7). Owning them here with `Pod` derives + `offset_of!`
/// asserts makes a one-sided edit a compile error.
///
/// The host creates the section (privileged, permissive DACL so the restricted WUDFHost token can
+7 -1
View File
@@ -19,7 +19,7 @@ crate-type = ["lib", "cdylib", "staticlib"]
default = []
# Control-plane QUIC (pairing, config, reverse audio). tokio is permitted ONLY here,
# never on the per-frame hot path. Off by default so the core stays runtime-free.
quic = ["dep:quinn", "dep:tokio", "dep:rustls", "dep:rcgen", "dep:rustls-pki-types", "dep:sha2", "dep:hmac", "dep:spake2"]
quic = ["dep:quinn", "dep:tokio", "dep:rustls", "dep:rcgen", "dep:rustls-pki-types", "dep:sha2", "dep:hmac", "dep:spake2", "dep:opus"]
[dependencies]
reed-solomon-simd = "3.1" # GF(2^16) Leopard-RS, SIMD, O(n log n) — the wall-breaker (P2)
@@ -51,6 +51,12 @@ sha2 = { version = "0.10", optional = true }
hmac = { version = "0.12", optional = true }
spake2 = { version = "0.4", optional = true }
tokio = { version = "1", optional = true, features = ["rt-multi-thread", "net", "sync", "macros"] }
# In-core Opus (multistream) DECODE for the C-ABI `punktfunk_connection_next_audio_pcm` path —
# used by embedders without a multistream-capable Opus decoder (Apple's AudioToolbox is
# stereo-only). The Rust clients link `opus` themselves and decode the raw `next_audio` frames,
# so this only matters when the connection API (quic) is built. Same libopus the host vendors;
# cargo unifies the build. Multistream API: `opus::MSDecoder` (lib.rs:1187).
opus = { version = "0.3", optional = true }
# `libc` for batched UDP syscalls: `sendmmsg`/`recvmmsg` on Linux (the 1 Gbps+ lever) and the
# `recv(MSG_DONTWAIT)` drain on the other unix (Apple/BSD) targets, which have no `recvmmsg`
+219
View File
@@ -467,6 +467,23 @@ pub struct PunktfunkConnection {
last: std::sync::Mutex<Option<crate::session::Frame>>,
/// Same, for `punktfunk_connection_next_audio` (independent of the video slot).
last_audio: std::sync::Mutex<Option<crate::client::AudioPacket>>,
/// Decode-in-core state for `punktfunk_connection_next_audio_pcm` (Apple / any embedder
/// without a multistream Opus decoder). The decoder is built lazily from the negotiated
/// `inner.audio_channels`; `pcm` is a fixed-capacity reusable buffer the returned pointer
/// borrows until the next PCM call (same contract as `last_audio`).
audio_pcm: std::sync::Mutex<AudioPcmState>,
}
/// Lazily-initialized in-core Opus decode state. A coupled-1-stream multistream decoder is
/// equivalent to a plain stereo decoder, so one [`opus::MSDecoder`] handles 2/6/8 channels.
#[cfg(feature = "quic")]
#[derive(Default)]
struct AudioPcmState {
decoder: Option<opus::MSDecoder>,
/// Interleaved f32 PCM, wire channel order. Pre-sized to the largest legal Opus frame
/// (120 ms @ 48 kHz = 5760 samples/ch) × 8 channels so decode never reallocates (which would
/// dangle the pointer handed to the embedder).
pcm: Vec<f32>,
}
/// `PunktfunkHidOutput::kind` — lightbar RGB (`r`/`g`/`b` valid).
@@ -708,12 +725,18 @@ pub const PUNKTFUNK_VIDEO_CAP_10BIT: u8 = 0x01;
/// Video-capability bit for [`punktfunk_connect_ex5`] (`video_caps`): the client can present
/// BT.2020 PQ HDR10 (implies 10-bit). (Mirrors `quic::VIDEO_CAP_HDR`.)
pub const PUNKTFUNK_VIDEO_CAP_HDR: u8 = 0x02;
/// Video-capability bit for [`punktfunk_connect_ex5`] (`video_caps`): the client can decode a
/// full-chroma 4:4:4 HEVC stream (Range Extensions). The host emits 4:4:4 only when this is set,
/// the host opted in, the codec is HEVC, and the GPU supports it — else the stream stays 4:2:0 and
/// [`punktfunk_connection_chroma_format`] reports the real value. (Mirrors `quic::VIDEO_CAP_444`.)
pub const PUNKTFUNK_VIDEO_CAP_444: u8 = 0x04;
// Keep the ABI cap bits in lockstep with the wire constants (compile-time guard against drift).
#[cfg(feature = "quic")]
const _: () = {
assert!(PUNKTFUNK_VIDEO_CAP_10BIT == crate::quic::VIDEO_CAP_10BIT);
assert!(PUNKTFUNK_VIDEO_CAP_HDR == crate::quic::VIDEO_CAP_HDR);
assert!(PUNKTFUNK_VIDEO_CAP_444 == crate::quic::VIDEO_CAP_444);
};
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
@@ -980,6 +1003,58 @@ pub unsafe extern "C" fn punktfunk_connect_ex5(
client_cert_pem: *const std::os::raw::c_char,
client_key_pem: *const std::os::raw::c_char,
timeout_ms: u32,
) -> *mut PunktfunkConnection {
// Delegate to the surround-aware variant requesting stereo (the pre-surround behaviour).
unsafe {
punktfunk_connect_ex6(
host,
port,
width,
height,
refresh_hz,
compositor,
gamepad,
bitrate_kbps,
video_caps,
2, // audio_channels = stereo
launch_id,
pin_sha256,
observed_sha256_out,
client_cert_pem,
client_key_pem,
timeout_ms,
)
}
}
/// Like [`punktfunk_connect_ex5`], but additionally requests the audio channel count:
/// `2` (stereo, the default behaviour of every earlier variant), `6` (5.1) or `8` (7.1). The host
/// clamps the request to what it can actually capture and echoes the resolved count via
/// [`punktfunk_connection_audio_channels`]; the `0xC9` audio frames are Opus-(multi)stream encoded
/// for that layout. A client that wants surround calls this; everything else inherits stereo.
///
/// # Safety
/// Same as [`punktfunk_connect`].
#[cfg(feature = "quic")]
#[no_mangle]
#[allow(clippy::too_many_arguments)]
pub unsafe extern "C" fn punktfunk_connect_ex6(
host: *const std::os::raw::c_char,
port: u16,
width: u32,
height: u32,
refresh_hz: u32,
compositor: u32,
gamepad: u32,
bitrate_kbps: u32,
video_caps: u8,
audio_channels: u8,
launch_id: *const std::os::raw::c_char,
pin_sha256: *const u8,
observed_sha256_out: *mut u8,
client_cert_pem: *const std::os::raw::c_char,
client_key_pem: *const std::os::raw::c_char,
timeout_ms: u32,
) -> *mut PunktfunkConnection {
let r = std::panic::catch_unwind(AssertUnwindSafe(|| {
if host.is_null() {
@@ -1029,6 +1104,7 @@ pub unsafe extern "C" fn punktfunk_connect_ex5(
gamepad,
bitrate_kbps,
video_caps,
crate::audio::normalize_channels(audio_channels),
launch,
pin,
identity,
@@ -1045,6 +1121,7 @@ pub unsafe extern "C" fn punktfunk_connect_ex5(
inner: c,
last: std::sync::Mutex::new(None),
last_audio: std::sync::Mutex::new(None),
audio_pcm: std::sync::Mutex::new(AudioPcmState::default()),
}))
}
Err(_) => std::ptr::null_mut(),
@@ -1250,6 +1327,121 @@ pub unsafe extern "C" fn punktfunk_connection_next_audio(
})
}
/// Read the audio channel count the host resolved for this session (from its Welcome): `2`
/// (stereo), `6` (5.1) or `8` (7.1). `*out` is filled when non-NULL. The `0xC9` Opus frames are
/// (multistream-)encoded for this layout; an embedder decoding raw frames itself must build its
/// decoder from THIS value (see [`crate::audio::layout_for`]) — or use
/// [`punktfunk_connection_next_audio_pcm`], which decodes in-core. Available immediately after a
/// successful connect (it doesn't change without a reconfigure).
///
/// # Safety
/// `c` is a valid connection handle; `out` is NULL or writable for one `u8`.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_audio_channels(
c: *mut PunktfunkConnection,
out: *mut u8,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if !out.is_null() {
// SAFETY: `out` is non-null and the caller guarantees it is writable for one `u8`.
unsafe { *out = c.inner.audio_channels };
}
PunktfunkStatus::Ok
})
}
/// One decoded audio frame from [`punktfunk_connection_next_audio_pcm`]: interleaved 32-bit
/// float PCM at 48 kHz, in the canonical wire channel order `FL FR FC LFE RL RR SL SR` (the
/// first `channels` of it). `samples` points at `frame_count * channels` floats and borrows
/// connection memory **until the next PCM call** on this handle.
#[cfg(feature = "quic")]
#[repr(C)]
pub struct PunktfunkAudioPcm {
/// Interleaved f32 samples (wire channel order), `frame_count * channels` long.
pub samples: *const f32,
/// Samples per channel in this frame.
pub frame_count: u32,
/// Channel count (2/6/8) — the negotiated [`punktfunk_connection_audio_channels`].
pub channels: u8,
/// Source packet sequence number.
pub seq: u32,
/// Capture presentation timestamp (ns).
pub pts_ns: u64,
}
/// Pull the next audio frame and **decode it in-core** to interleaved f32 PCM — for embedders
/// without a multistream-capable Opus decoder (e.g. Apple, whose AudioToolbox Opus path is
/// stereo-only). The decoder is built once from the negotiated channel count and handles 2/6/8
/// channels (a 1-coupled-stream multistream decoder is exactly a stereo decoder). Same
/// timeout/closed semantics as [`punktfunk_connection_next_audio`]; `out->samples` borrows
/// connection memory until the next PCM call on this handle. Use EITHER this or
/// [`punktfunk_connection_next_audio`] on a given connection, from one dedicated audio thread —
/// not both (they share the underlying queue).
///
/// # Safety
/// `c` is a valid connection handle; `out` is writable. At most one thread pulls audio.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_next_audio_pcm(
c: *mut PunktfunkConnection,
out: *mut PunktfunkAudioPcm,
timeout_ms: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if out.is_null() {
return PunktfunkStatus::NullPointer;
}
let channels = crate::audio::normalize_channels(c.inner.audio_channels);
let pkt = match c
.inner
.next_audio(std::time::Duration::from_millis(timeout_ms as u64))
{
Ok(pkt) => pkt,
Err(e) => return e.status(),
};
let mut state = c.audio_pcm.lock().unwrap();
if state.decoder.is_none() {
let layout = crate::audio::layout_for(channels, false);
match opus::MSDecoder::new(48_000, layout.streams, layout.coupled, layout.mapping) {
Ok(d) => {
// Largest legal Opus frame is 120 ms = 5760 samples/ch.
state.pcm = vec![0f32; 5760 * channels as usize];
state.decoder = Some(d);
}
Err(_) => return PunktfunkStatus::Unsupported,
}
}
let AudioPcmState { decoder, pcm } = &mut *state;
let dec = decoder.as_mut().unwrap();
// `decode_float` divides the output buffer length by the channel count to get the
// per-channel capacity; an empty payload requests packet-loss concealment.
match dec.decode_float(&pkt.data, pcm, false) {
Ok(frame_count) => {
unsafe {
*out = PunktfunkAudioPcm {
samples: pcm.as_ptr(),
frame_count: frame_count as u32,
channels,
seq: pkt.seq,
pts_ns: pkt.pts_ns,
};
}
PunktfunkStatus::Ok
}
Err(_) => PunktfunkStatus::BadPacket,
}
})
}
/// Pull the next rumble (force-feedback) update, waiting up to `timeout_ms`. Amplitudes
/// are 0..0xFFFF (`low` = low-frequency motor, `high` = high-frequency), `(0, 0)` = stop.
/// Same timeout/closed semantics as [`punktfunk_connection_next_audio`].
@@ -1414,6 +1606,33 @@ pub unsafe extern "C" fn punktfunk_connection_color_info(
})
}
/// Read the session's resolved chroma subsampling (from the host's Welcome) as the HEVC
/// `chroma_format_idc`: `1` = 4:2:0 (the default every pre-4:4:4 host produced), `3` = full-chroma
/// 4:4:4. `*out` is filled when non-NULL. The in-band SPS is authoritative; this lets the embedder
/// pre-size its decoder / pick a 4:4:4 pixel format up front. Available immediately after a
/// successful connect (it doesn't change without a reconfigure).
///
/// # Safety
/// `c` is a valid connection handle; `out` is NULL or writable for one `u8`.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_chroma_format(
c: *mut PunktfunkConnection,
out: *mut u8,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if !out.is_null() {
// SAFETY: `out` is non-null and the caller guarantees it is writable for one `u8`.
unsafe { *out = c.inner.chroma_format };
}
PunktfunkStatus::Ok
})
}
/// Send one input event to the host as a QUIC datagram (non-blocking enqueue).
///
/// # Safety
+298
View File
@@ -0,0 +1,298 @@
//! Shared audio layout: the single source of truth for Opus (multi)stream surround across the
//! host, the GameStream compatibility path, and every client decoder.
//!
//! **Canonical wire channel order** is `FL FR FC LFE RL RR SL SR` (the GameStream/Moonlight
//! order, and the PipeWire/PulseAudio default map for 6/8 channels). Every host capturer
//! delivers PCM in this order and every client decodes into it, so the Opus multistream
//! `mapping` is the **identity** (`[0, 1, …, channels-1]`) on both ends — punktfunk owns the
//! encoder and every decoder, so the GFE-style pre-rotation Moonlight needs over SDP
//! (`gamestream::audio::surround_params`) is a GameStream-only concern and never touches the
//! native `punktfunk/1` path.
//!
//! Channel counts the protocol negotiates: `2` (stereo), `6` (5.1) and `8` (7.1). Anything
//! else clamps to stereo ([`normalize_channels`]).
/// Canonical wire channel positions; the index is the channel's slot in the interleaved PCM
/// frame. A count of N uses positions `0..N` (always a prefix of this 8-channel order).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
#[repr(u8)]
pub enum WirePos {
FrontLeft = 0,
FrontRight = 1,
FrontCenter = 2,
Lfe = 3,
RearLeft = 4,
RearRight = 5,
SideLeft = 6,
SideRight = 7,
}
/// The full 8-channel wire order; the N-channel order is its first N entries.
pub const WIRE_ORDER_8: [WirePos; 8] = {
use WirePos::*;
[
FrontLeft,
FrontRight,
FrontCenter,
Lfe,
RearLeft,
RearRight,
SideLeft,
SideRight,
]
};
/// One Opus (multi)stream layout. `mapping` is the libopus multistream mapping we encode AND
/// decode with — identity, since punktfunk owns both ends. `streams`/`coupled` give the
/// normal-quality coupling (FL,FR)+(FC,LFE) [+(RL,RR) on 7.1] with the remaining channels as
/// mono streams; high quality is one mono stream per channel. Bitrates match Sunshine's
/// per-config values (stereo keeps punktfunk's live-validated 128 kbps).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct OpusLayout {
/// Interleaved channel count (2, 6 or 8).
pub channels: u8,
/// Number of Opus streams in the multistream packet.
pub streams: u8,
/// How many of those streams are coupled (stereo) pairs.
pub coupled: u8,
/// libopus multistream channel mapping — identity `[0, 1, …, channels-1]`.
pub mapping: &'static [u8],
/// Target Opus bitrate in bits/sec (hard CBR; constant packet size, which GameStream's
/// audio FEC relies on).
pub bitrate: i32,
}
/// Stereo: a plain coupled pair. The 128 kbps live-validated config.
pub const LAYOUT_STEREO: OpusLayout = OpusLayout {
channels: 2,
streams: 1,
coupled: 1,
mapping: &[0, 1],
bitrate: 128_000,
};
/// 5.1 normal quality: (FL,FR)+(FC,LFE) coupled, RL+RR mono.
pub const LAYOUT_51: OpusLayout = OpusLayout {
channels: 6,
streams: 4,
coupled: 2,
mapping: &[0, 1, 2, 3, 4, 5],
bitrate: 256_000,
};
/// 5.1 high quality: one mono stream per channel.
pub const LAYOUT_51_HQ: OpusLayout = OpusLayout {
channels: 6,
streams: 6,
coupled: 0,
mapping: &[0, 1, 2, 3, 4, 5],
bitrate: 1_536_000,
};
/// 7.1 normal quality: (FL,FR)+(FC,LFE)+(RL,RR) coupled, SL+SR mono.
pub const LAYOUT_71: OpusLayout = OpusLayout {
channels: 8,
streams: 5,
coupled: 3,
mapping: &[0, 1, 2, 3, 4, 5, 6, 7],
bitrate: 450_000,
};
/// 7.1 high quality: one mono stream per channel.
pub const LAYOUT_71_HQ: OpusLayout = OpusLayout {
channels: 8,
streams: 8,
coupled: 0,
mapping: &[0, 1, 2, 3, 4, 5, 6, 7],
bitrate: 2_048_000,
};
/// Pick the layout for a negotiated channel count. Unknown counts fall back to stereo (clients
/// only ever request 2/6/8). `high_quality` selects the uncoupled high-bitrate config.
pub fn layout_for(channels: u8, high_quality: bool) -> &'static OpusLayout {
match (channels, high_quality) {
(6, false) => &LAYOUT_51,
(6, true) => &LAYOUT_51_HQ,
(8, false) => &LAYOUT_71,
(8, true) => &LAYOUT_71_HQ,
_ => &LAYOUT_STEREO,
}
}
/// Clamp an arbitrary (wire / requested) channel count to one the protocol negotiates. `0`,
/// absent, or any unsupported value becomes stereo.
pub fn normalize_channels(requested: u8) -> u8 {
match requested {
6 => 6,
8 => 8,
_ => 2,
}
}
// ---- per-platform channel-layout helpers (pure data; no platform deps) --------------------
/// Windows `WAVEFORMATEXTENSIBLE.dwChannelMask` for the wire layout.
///
/// NB 7.1 == `0x63F` (FL FR FC LFE **BL BR SL SR**), NOT `0xFF` — `0xFF` selects the
/// front-of-center pair FLC/FRC, the wrong speakers. WASAPI delivers channels in ascending
/// mask-bit order, which equals the wire order, so the decoded PCM needs no permutation.
pub const fn wasapi_channel_mask(channels: u8) -> u32 {
const FL: u32 = 0x1;
const FR: u32 = 0x2;
const FC: u32 = 0x4;
const LFE: u32 = 0x8;
const BL: u32 = 0x10; // back left (wire RL)
const BR: u32 = 0x20; // back right (wire RR)
const SL: u32 = 0x200; // side left
const SR: u32 = 0x400; // side right
match channels {
6 => FL | FR | FC | LFE | BL | BR, // 0x3F
8 => FL | FR | FC | LFE | BL | BR | SL | SR, // 0x63F
_ => FL | FR, // 0x3 (stereo)
}
}
/// PipeWire / SPA `enum spa_audio_channel` positions in wire order — identical to the host
/// capture side (`punktfunk-host` `audio::linux::spa_positions`): FL=3 FR=4 FC=5 LFE=6 SL=7
/// SR=8 RL=12 RR=13. Identity routing: the client sets these on its playback node so PipeWire
/// maps each wire slot to the matching speaker (and downmixes when the sink has fewer).
pub fn spa_positions(channels: u8) -> &'static [u32] {
const STEREO: [u32; 2] = [3, 4]; // FL FR
const C51: [u32; 6] = [3, 4, 5, 6, 12, 13]; // FL FR FC LFE RL RR
const C71: [u32; 8] = [3, 4, 5, 6, 12, 13, 7, 8]; // FL FR FC LFE RL RR SL SR
match channels {
6 => &C51,
8 => &C71,
_ => &STEREO,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn layout_table_is_consistent() {
for l in [
&LAYOUT_STEREO,
&LAYOUT_51,
&LAYOUT_51_HQ,
&LAYOUT_71,
&LAYOUT_71_HQ,
] {
// Mapping is identity and exactly `channels` entries long.
assert_eq!(l.mapping.len(), l.channels as usize);
for (i, &m) in l.mapping.iter().enumerate() {
assert_eq!(m as usize, i, "mapping must be identity for {l:?}");
}
// libopus invariant: total channels == coupled*2 + (streams - coupled).
assert_eq!(
l.coupled * 2 + (l.streams - l.coupled),
l.channels,
"stream/coupled accounting for {l:?}"
);
assert!(l.coupled <= l.streams);
assert!(l.bitrate > 0);
}
}
#[test]
fn layout_for_picks_expected() {
assert_eq!(layout_for(2, false), &LAYOUT_STEREO);
assert_eq!(layout_for(6, false), &LAYOUT_51);
assert_eq!(layout_for(6, true), &LAYOUT_51_HQ);
assert_eq!(layout_for(8, false), &LAYOUT_71);
assert_eq!(layout_for(8, true), &LAYOUT_71_HQ);
// Unknown / 0 → stereo.
assert_eq!(layout_for(0, false), &LAYOUT_STEREO);
assert_eq!(layout_for(3, false), &LAYOUT_STEREO);
assert_eq!(layout_for(7, true), &LAYOUT_STEREO);
}
#[test]
fn normalize_clamps_to_negotiable() {
assert_eq!(normalize_channels(2), 2);
assert_eq!(normalize_channels(6), 6);
assert_eq!(normalize_channels(8), 8);
for bad in [0u8, 1, 3, 4, 5, 7, 9, 255] {
assert_eq!(normalize_channels(bad), 2, "{bad} must clamp to stereo");
}
}
#[test]
fn wasapi_masks_are_correct() {
assert_eq!(wasapi_channel_mask(2), 0x3);
assert_eq!(wasapi_channel_mask(6), 0x3F);
assert_eq!(wasapi_channel_mask(8), 0x63F); // NOT 0xFF
// Bit count must equal the channel count.
assert_eq!(wasapi_channel_mask(2).count_ones(), 2);
assert_eq!(wasapi_channel_mask(6).count_ones(), 6);
assert_eq!(wasapi_channel_mask(8).count_ones(), 8);
}
#[test]
fn spa_positions_match_wire_order() {
assert_eq!(spa_positions(2), &[3, 4]);
assert_eq!(spa_positions(6), &[3, 4, 5, 6, 12, 13]);
assert_eq!(spa_positions(8), &[3, 4, 5, 6, 12, 13, 7, 8]);
assert_eq!(spa_positions(2).len(), 2);
assert_eq!(spa_positions(6).len(), 6);
assert_eq!(spa_positions(8).len(), 8);
}
/// Real-libopus proof that the shared layout round-trips with channel identity: a tone fed
/// into wire channel N (host `opus::MSEncoder`) comes back out on channel N (client
/// `opus::MSDecoder`), for stereo / 5.1 / 7.1. This is the single guarantee the whole
/// feature rests on — encoder layout == decoder layout == identity mapping — so if a layout
/// constant is ever wrong, this fails. Gated on `quic` (where `opus` is a dependency).
#[cfg(feature = "quic")]
#[test]
fn multistream_layout_roundtrips_with_channel_identity() {
const SR: u32 = 48_000;
const SAMPLES: usize = 240; // 5 ms @ 48 kHz
for &channels in &[2u8, 6, 8] {
let l = layout_for(channels, false);
let ch = l.channels as usize;
let mut enc = opus::MSEncoder::new(
SR,
l.streams,
l.coupled,
l.mapping,
opus::Application::LowDelay,
)
.expect("MSEncoder");
enc.set_bitrate(opus::Bitrate::Bits(l.bitrate)).unwrap();
enc.set_vbr(false).unwrap();
let mut dec =
opus::MSDecoder::new(SR, l.streams, l.coupled, l.mapping).expect("MSDecoder");
for tone_ch in 0..ch {
let mut out = vec![0u8; 4000];
let mut energy = vec![0f64; ch];
// A few frames to clear the codec startup transient before measuring.
for f in 0..8 {
let mut frame = vec![0f32; SAMPLES * ch];
for t in 0..SAMPLES {
let phase = (f * SAMPLES + t) as f32 * 440.0 * 2.0 * std::f32::consts::PI
/ SR as f32;
frame[t * ch + tone_ch] = 0.5 * phase.sin();
}
let n = enc.encode_float(&frame, &mut out).unwrap();
let mut decoded = vec![0f32; SAMPLES * ch];
let got = dec.decode_float(&out[..n], &mut decoded, false).unwrap();
assert_eq!(got, SAMPLES, "{channels}ch frame size");
if f >= 4 {
for t in 0..SAMPLES {
for (c, e) in energy.iter_mut().enumerate() {
*e += (decoded[t * ch + c] as f64).powi(2);
}
}
}
}
let loudest = (0..ch)
.max_by(|&a, &b| energy[a].total_cmp(&energy[b]))
.unwrap();
assert_eq!(
loudest, tone_ch,
"{channels}ch: tone in channel {tone_ch} must come out on {tone_ch} (energies {energy:?})"
);
}
}
}
}
+34 -2
View File
@@ -40,8 +40,9 @@ enum CtrlRequest {
/// mode, the host-resolved compositor backend, the host-resolved gamepad backend, the host's
/// certificate fingerprint, the resolved encoder bitrate (kbps), and the host↔client clock offset
/// (ns, host minus client; 0 = no skew correction / an old host that didn't answer the handshake).
/// The trailing `u8` is the resolved encode bit depth (8/10) and [`ColorInfo`] the resolved colour
/// signalling, both from the [`Welcome`].
/// The trailing `u8`s are the resolved encode bit depth (8/10), the chroma `chroma_format_idc`
/// (1 = 4:2:0, 3 = 4:4:4), and the resolved audio channel count (2/6/8), with [`ColorInfo`] the
/// resolved colour signalling — all from the [`Welcome`].
type Negotiated = (
Mode,
CompositorPref,
@@ -51,6 +52,8 @@ type Negotiated = (
i64,
u8,
ColorInfo,
u8,
u8,
);
/// Accumulated state of an in-flight / finished speed test. The data-plane pump mirrors the
@@ -202,6 +205,17 @@ pub struct NativeClient {
/// decoder/presenter from this. [`ColorInfo::SDR_BT709`] for an older host. The static HDR
/// mastering metadata (when [`ColorInfo::is_hdr`]) arrives via [`NativeClient::next_hdr_meta`].
pub color: ColorInfo,
/// The chroma subsampling the host resolved for this session ([`Welcome::chroma_format`]), as the
/// HEVC `chroma_format_idc`: [`quic::CHROMA_IDC_420`] (4:2:0, the default / older host) or
/// [`quic::CHROMA_IDC_444`] (full-chroma 4:4:4). The in-band SPS is authoritative; this lets the
/// client pre-size its decoder. `CHROMA_IDC_420` for an older host that didn't report it.
pub chroma_format: u8,
/// The audio channel count the host resolved for this session ([`Welcome::audio_channels`]):
/// `2` (stereo), `6` (5.1) or `8` (7.1). The client MUST build its Opus (multistream) decoder
/// from this value (via [`crate::audio::layout_for`]) — never from its own request — so an older
/// host that omits it (→ `2`) yields working stereo. The `0xC9` audio frames are encoded with the
/// matching layout.
pub audio_channels: u8,
}
/// Pin the calling thread to the user-interactive QoS class on Apple targets.
@@ -246,6 +260,9 @@ impl NativeClient {
// VIDEO_CAP_HDR) — the host upgrades to a 10-bit / HDR encode only when the matching bit is
// set. 0 = the 8-bit BT.709 stream every client understands.
video_caps: u8,
// Requested audio channel count (2 = stereo / 6 = 5.1 / 8 = 7.1); the host clamps to what it
// can capture and echoes the result in [`NativeClient::audio_channels`].
audio_channels: u8,
launch: Option<String>,
pin: Option<[u8; 32]>,
identity: Option<(String, String)>,
@@ -298,6 +315,7 @@ impl NativeClient {
gamepad,
bitrate_kbps,
video_caps,
audio_channels,
launch,
pin,
identity,
@@ -329,6 +347,8 @@ impl NativeClient {
clock_offset_ns,
bit_depth,
color,
chroma_format,
audio_channels,
) = match ready_rx.recv_timeout(timeout) {
Ok(Ok(t)) => t,
Ok(Err(e)) => return Err(e),
@@ -360,6 +380,8 @@ impl NativeClient {
clock_offset_ns,
bit_depth,
color,
chroma_format,
audio_channels,
})
}
@@ -666,6 +688,7 @@ struct WorkerArgs {
gamepad: GamepadPref,
bitrate_kbps: u32,
video_caps: u8,
audio_channels: u8,
launch: Option<String>,
pin: Option<[u8; 32]>,
identity: Option<(String, String)>,
@@ -697,6 +720,7 @@ async fn worker_main(args: WorkerArgs) {
gamepad,
bitrate_kbps,
video_caps,
audio_channels,
launch,
pin,
identity,
@@ -763,6 +787,8 @@ async fn worker_main(args: WorkerArgs) {
// VIDEO_CAP_10BIT | VIDEO_CAP_HDR). The host only upgrades to a 10-bit / HDR encode
// when the matching bit is set, so `0` stays an 8-bit BT.709 stream.
video_caps,
// Requested surround channel count; the host echoes the resolved value in Welcome.
audio_channels,
}
.encode(),
)
@@ -834,6 +860,8 @@ async fn worker_main(args: WorkerArgs) {
clock_offset_ns,
welcome.bit_depth,
welcome.color,
welcome.chroma_format,
welcome.audio_channels,
))
};
@@ -850,6 +878,8 @@ async fn worker_main(args: WorkerArgs) {
clock_offset_ns,
bit_depth,
color,
chroma_format,
audio_channels,
) = match setup.await {
Ok(t) => t,
Err(e) => {
@@ -866,6 +896,8 @@ async fn worker_main(args: WorkerArgs) {
clock_offset_ns,
bit_depth,
color,
chroma_format,
audio_channels,
)));
// Input task: embedder events → QUIC datagrams.
+1
View File
@@ -25,6 +25,7 @@
#![forbid(unsafe_op_in_unsafe_fn)]
pub mod abi;
pub mod audio;
#[cfg(feature = "quic")]
pub mod client;
pub mod config;
+98 -7
View File
@@ -78,12 +78,33 @@ pub struct Hello {
/// zero-length name/launch placeholder precedes it when those are absent so the offset stays
/// deterministic. Omitted by older clients (decodes to `0`).
pub video_caps: u8,
/// Requested audio channel count: `2` (stereo, default), `6` (5.1) or `8` (7.1). The host
/// resolves it against what it can capture and echoes the final count in
/// [`Welcome::audio_channels`], which is what both ends build their Opus (multistream)
/// codec from. Appended after `video_caps` as a single trailing byte; when it differs from
/// the stereo default the name/launch/video_caps placeholders are forced (0) so it lands at a
/// deterministic offset. Omitted by older clients / when `2` (decodes to `2`, i.e. stereo) so
/// the stereo wire form stays byte-identical to the pre-surround build.
pub audio_channels: u8,
}
/// [`Hello::video_caps`] bit: the client can decode a 10-bit (Main10) HEVC stream.
pub const VIDEO_CAP_10BIT: u8 = 0x01;
/// [`Hello::video_caps`] bit: the client can present BT.2020 PQ HDR10 (implies 10-bit).
pub const VIDEO_CAP_HDR: u8 = 0x02;
/// [`Hello::video_caps`] bit: the client can decode a full-chroma **4:4:4** HEVC stream (HEVC
/// Range Extensions / Rec.ITU-T H.265 `chroma_format_idc = 3`). The host emits 4:4:4 ONLY when this
/// bit is set, the host opted in (`PUNKTFUNK_444`), the codec is HEVC, **and** the GPU/driver
/// actually supports a 4:4:4 encode (probed) — otherwise the session stays 4:2:0 and
/// [`Welcome::chroma_format`] reflects the real resolved value. Independent of 10-bit/HDR (4:4:4 is a
/// chroma decision, bit depth is a depth decision; the two may combine where the hardware allows).
pub const VIDEO_CAP_444: u8 = 0x04;
/// HEVC `chroma_format_idc` for 4:2:0 — what every pre-4:4:4 build produced and the back-compat
/// default when a peer omits [`Welcome::chroma_format`].
pub const CHROMA_IDC_420: u8 = 1;
/// HEVC `chroma_format_idc` for full-chroma 4:4:4 (Range Extensions).
pub const CHROMA_IDC_444: u8 = 3;
/// Per-session colour signalling (CICP / ITU-T H.273 code points) the host resolved for the
/// encoded video, carried on [`Welcome`]. A client configures its decoder/presenter from these
@@ -198,6 +219,22 @@ pub struct Welcome {
/// [`ColorInfo::SDR_BT709`]. The client configures its decoder/presenter from this instead of
/// guessing from the bitstream; the mastering metadata arrives separately on [`HDR_META_MAGIC`].
pub color: ColorInfo,
/// The chroma subsampling the host actually encodes at, as the HEVC `chroma_format_idc`:
/// [`CHROMA_IDC_420`] (4:2:0, default / older host) or [`CHROMA_IDC_444`] (full-chroma 4:4:4,
/// enabled only when the client advertised [`VIDEO_CAP_444`] *and* the host could open a real
/// 4:4:4 encode). The client sizes its decoder/surface pool from this; the in-band SPS carries
/// the authoritative value, so this is a hint (and the honest-downgrade channel — if the host
/// requested 4:4:4 but the GPU declined, this reads `CHROMA_IDC_420`). Appended after the colour
/// bytes as a single trailing byte; an older host that omits it decodes to [`CHROMA_IDC_420`].
pub chroma_format: u8,
/// The audio channel count the host actually resolved and **will** send on the `0xC9` plane:
/// `2` (stereo, default), `6` (5.1) or `8` (7.1). Echoes [`Hello::audio_channels`] clamped to
/// what the host can capture (Linux PipeWire always synthesizes the count; Windows WASAPI
/// loopback is clamped to the render endpoint's mix-format channels). The client builds its Opus
/// (multistream) decoder from THIS value via [`crate::audio::layout_for`] — never from its own
/// request — so an older host that omits the byte (→ `2`) always yields working stereo. Appended
/// after `chroma_format` as a single trailing byte.
pub audio_channels: u8,
}
/// `client → host`: data plane is bound, begin streaming.
@@ -630,10 +667,11 @@ impl Hello {
// so a Hello with neither name nor launch stays byte-identical to the bitrate-era form
// (26 bytes). When `launch` is present we must still emit name's length byte (0 for None)
// so `launch` lands at a deterministic offset.
// `video_caps` is the last trailing field, after `launch`; when it's present (non-zero)
// the name/launch length bytes must still be emitted (0 for absent) so it lands at a
// `video_caps`/`audio_channels` are the trailing fields, after `launch`; when either is
// present (video_caps non-zero / audio_channels not stereo) the name/launch length bytes
// AND the video_caps byte must still be emitted (0 / 0) so the later byte lands at a
// deterministic offset — the same discipline `launch` already imposes on `name`.
let need_placeholders = self.video_caps != 0;
let need_placeholders = self.video_caps != 0 || self.audio_channels != 2;
match (&self.name, &self.launch) {
(None, None) if !need_placeholders => {}
(name, _) => {
@@ -648,10 +686,15 @@ impl Hello {
b.push(l.len() as u8);
b.extend_from_slice(l.as_bytes());
}
// video_caps: single trailing byte. Last field.
if self.video_caps != 0 {
// video_caps: single trailing byte. Emitted when non-zero OR when audio_channels follows
// (so audio_channels lands at a deterministic offset right after it).
if self.video_caps != 0 || self.audio_channels != 2 {
b.push(self.video_caps);
}
// audio_channels: single trailing byte. Last field; omitted when stereo (default).
if self.audio_channels != 2 {
b.push(self.audio_channels);
}
b
}
@@ -714,6 +757,15 @@ impl Hello {
let launch_len = b.get(launch_off).copied().unwrap_or(0) as usize;
b.get(launch_off + 1 + launch_len).copied().unwrap_or(0)
},
// Optional trailing audio-channel byte, one past video_caps. Absent on an older client
// → stereo. Normalized so a corrupt/unsupported value can't build a bad decoder.
audio_channels: {
let name_len = b.get(26).copied().unwrap_or(0) as usize;
let launch_off = 27 + name_len;
let launch_len = b.get(launch_off).copied().unwrap_or(0) as usize;
let video_caps_off = launch_off + 1 + launch_len;
crate::audio::normalize_channels(b.get(video_caps_off + 1).copied().unwrap_or(2))
},
})
}
}
@@ -747,6 +799,10 @@ impl Welcome {
b.push(self.color.transfer);
b.push(self.color.matrix);
b.push(self.color.full_range);
// Chroma subsampling at offset 64 — older clients stop before this → 4:2:0 (CHROMA_IDC_420).
b.push(self.chroma_format);
// Audio channel count at offset 65 — older clients stop before this → stereo (2).
b.push(self.audio_channels);
b
}
@@ -755,7 +811,8 @@ impl Welcome {
// scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45]
// salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59]
// bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63]
// (everything from compositor on is an optional trailing byte; an older host stops earlier).
// chroma_format[64] audio_channels[65] (everything from compositor on is an optional
// trailing byte; an older host stops earlier).
if b.len() < 53 || &b[0..4] != MAGIC {
return Err(PunktfunkError::InvalidArg("bad Welcome"));
}
@@ -812,6 +869,15 @@ impl Welcome {
matrix: b.get(62).copied().unwrap_or(ColorInfo::MC_BT709),
full_range: b.get(63).copied().unwrap_or(0),
},
// Optional trailing chroma byte — absent on an older host (or an explicit 0 / unknown
// value) → 4:2:0. Only `CHROMA_IDC_444` flips the client to a 4:4:4 decode.
chroma_format: match b.get(64).copied() {
Some(CHROMA_IDC_444) => CHROMA_IDC_444,
_ => CHROMA_IDC_420,
},
// Optional trailing audio-channel byte — absent on an older host → stereo. Any
// non-{6,8} value normalizes to stereo so a corrupt byte never builds a bad decoder.
audio_channels: crate::audio::normalize_channels(b.get(65).copied().unwrap_or(2)),
})
}
@@ -1809,6 +1875,8 @@ mod tests {
bitrate_kbps: 50_000,
bit_depth: 10,
color: ColorInfo::HDR10_BT2020_PQ,
chroma_format: CHROMA_IDC_444,
audio_channels: 2,
};
assert_eq!(Welcome::decode(&w.encode()).unwrap(), w);
}
@@ -1851,6 +1919,7 @@ mod tests {
name: Some("Test Device".into()),
launch: Some("steam:570".into()),
video_caps: VIDEO_CAP_10BIT,
audio_channels: 2,
};
assert_eq!(Hello::decode(&h.encode()).unwrap(), h);
let s = Start {
@@ -1930,6 +1999,7 @@ mod tests {
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
};
let enc = h.encode();
assert_eq!(enc.len(), 26);
@@ -1969,9 +2039,11 @@ mod tests {
bitrate_kbps: 120_000,
bit_depth: 10,
color: ColorInfo::HDR10_BT2020_PQ,
chroma_format: CHROMA_IDC_444,
audio_channels: 6, // 5.1 — exercises the non-default trailing byte
};
let wenc = w.encode();
assert_eq!(wenc.len(), 64); // 60 base + 4 colour bytes
assert_eq!(wenc.len(), 66); // 60 base + 4 colour + 1 chroma + 1 audio-channels byte
let legacy_w = Welcome::decode(&wenc[..53]).unwrap();
assert_eq!(legacy_w.compositor, CompositorPref::Auto);
assert_eq!(legacy_w.gamepad, GamepadPref::Auto);
@@ -1991,13 +2063,29 @@ mod tests {
let pre_color_w = Welcome::decode(&wenc[..60]).unwrap();
assert_eq!(pre_color_w.bit_depth, 10);
assert_eq!(pre_color_w.color, ColorInfo::SDR_BT709);
assert_eq!(pre_color_w.chroma_format, CHROMA_IDC_420); // pre-chroma host → 4:2:0
assert_eq!(legacy_w.color, ColorInfo::SDR_BT709);
assert_eq!(legacy_w.chroma_format, CHROMA_IDC_420);
// A pre-chroma (64-byte) Welcome carries colour but no chroma/audio bytes → 4:2:0 + stereo.
let pre_chroma_w = Welcome::decode(&wenc[..64]).unwrap();
assert_eq!(pre_chroma_w.color, ColorInfo::HDR10_BT2020_PQ);
assert_eq!(pre_chroma_w.chroma_format, CHROMA_IDC_420);
assert_eq!(pre_chroma_w.audio_channels, 2); // audio byte (offset 65) absent → stereo
// A pre-audio (65-byte) Welcome carries chroma but no audio byte → 4:4:4 + stereo.
let pre_audio_w = Welcome::decode(&wenc[..65]).unwrap();
assert_eq!(pre_audio_w.chroma_format, CHROMA_IDC_444);
assert_eq!(pre_audio_w.audio_channels, 2);
assert_eq!(Welcome::decode(&wenc).unwrap().bitrate_kbps, 120_000);
assert_eq!(Welcome::decode(&wenc).unwrap().bit_depth, 10); // full form carries it
assert_eq!(
Welcome::decode(&wenc).unwrap().color,
ColorInfo::HDR10_BT2020_PQ
);
assert_eq!(
Welcome::decode(&wenc).unwrap().chroma_format,
CHROMA_IDC_444
); // full form carries 4:4:4
assert_eq!(Welcome::decode(&wenc).unwrap().audio_channels, 6); // ...and 5.1
}
#[test]
@@ -2015,6 +2103,7 @@ mod tests {
name: Some("Enrico's MacBook".into()),
launch: None,
video_caps: 0,
audio_channels: 2,
};
let enc = base.encode();
assert_eq!(
@@ -2062,6 +2151,7 @@ mod tests {
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
};
// launch alone (no name): a zero-length name placeholder keeps the offset deterministic.
let with_launch = Hello {
@@ -2268,6 +2358,7 @@ mod tests {
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
}
.encode();
assert!(PairRequest::decode(&h).is_err(), "abi {abi} parsed as pair");
+14 -2
View File
@@ -13,8 +13,10 @@ use std::process::Command;
fn native_libs() -> &'static [&'static str] {
if cfg!(target_os = "macos") {
// The workspace build unifies features into the staticlib, and `quic` pulls
// rustls's platform verifier → Security/CoreFoundation.
// rustls's platform verifier → Security/CoreFoundation, plus libopus (the in-core
// `next_audio_pcm` decode path) which the `abi.rs` object references.
&[
"-lopus",
"-liconv",
"-lm",
"-framework",
@@ -23,7 +25,17 @@ fn native_libs() -> &'static [&'static str] {
"CoreFoundation",
]
} else if cfg!(target_os = "linux") {
&["-lgcc_s", "-lutil", "-lrt", "-lpthread", "-lm", "-ldl"]
// `-lopus`: the `quic` feature pulls in-core Opus decode (`next_audio_pcm`), whose
// symbols the linked `abi.rs` object references. Before `-lm` (opus needs libm).
&[
"-lopus",
"-lgcc_s",
"-lutil",
"-lrt",
"-lpthread",
"-lm",
"-ldl",
]
} else {
&[]
}
+18 -10
View File
@@ -25,6 +25,14 @@ aes-gcm = "0.10"
cbc = { version = "0.1", features = ["alloc"] }
rand = "0.8"
hex = "0.4"
# Cover-art delivery in the game library: encode Lutris's local JPEGs into `data:` URLs and decode
# the Epic launcher's base64 `catcache.bin`. Cross-platform (Linux Lutris art + Windows Epic art).
base64 = "0.22"
# Blocking HTTP for the library cover-art warmer (no-auth GOG api.gog.com + Xbox displaycatalog),
# run on a background thread off the hot path. `ureq` is small + sync (no tokio here) and bundles
# webpki roots (no system cert dependency). Cross-platform so the fetch/parse code is compiled +
# checked everywhere even though only the Windows GOG/Xbox providers need it today.
ureq = "2"
rcgen = { version = "0.13", default-features = false, features = ["aws_lc_rs", "pem"] }
x509-parser = "0.16"
axum-server = { version = "0.7", features = ["tls-rustls"] }
@@ -53,9 +61,10 @@ utoipa-scalar = { version = "0.3", features = ["axum"] }
tower = { version = "0.5", features = ["util"] }
http-body-util = "0.1"
# Opus stereo encode for the host->client audio plane. The `opus` crate vendors libopus via
# `audiopus_sys` (cmake-built from source — no system lib, no vcpkg), so it builds on Windows MSVC
# too (needs CMake + NASM, both on the box). Both platforms that have an audio-capture backend.
# Opus encode for the host->client audio plane — stereo (`opus::Encoder`) AND 5.1/7.1 surround
# (`opus::MSEncoder`, the safe multistream API the crate exposes; no `audiopus_sys` needed). The
# crate vendors libopus (cmake-built from source — no system lib, no vcpkg), so it builds on Windows
# MSVC too (needs CMake + NASM, both on the box). Both platforms that have an audio-capture backend.
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
opus = "0.3"
@@ -89,15 +98,8 @@ serde_json = "1"
# SQLite (cc, already needed for ffmpeg/opus) so there's no system libsqlite3 runtime dependency —
# clean for the deb/rpm/flatpak packaging. Opened read-only/immutable (Lutris may hold it open).
rusqlite = { version = "0.40", features = ["bundled"] }
# Inline Lutris's local cover-art JPEGs as `data:` URLs in the library (Lutris has no public CDN
# keyed by a stable id, unlike Steam/Heroic; a `data:` URL is self-contained — no host-served endpoint).
base64 = "0.22"
# Builds/validates the xkb keymap uploaded to the virtual keyboard + tracks modifier state.
xkbcommon = "0.8"
# The safe `opus` crate is stereo-only; surround (5.1/7.1) needs the libopus *multistream*
# encoder (`opus_multistream_encoder_*`). `audiopus_sys` is the sys layer `opus` already
# vendors (same libopus link), so this adds bindings, not a second copy of the library.
audiopus_sys = "0.2"
# libei (EI sender) for the portable input path on KWin/GNOME (RemoteDesktop portal).
# The `tokio` feature wires reis's event stream into tokio's reactor.
reis = { version = "0.6.1", features = ["tokio"] }
@@ -176,6 +178,12 @@ windows = { version = "0.62", features = [
# handler / ServiceManager install). Wraps the Win32 service API; the supervision loop itself uses
# the `windows` crate above.
windows-service = "0.7"
# Read the GOG.com install registry (HKLM\SOFTWARE\WOW6432Node\GOG.com\Games) for the GOG store
# provider — ergonomic + correct-by-construction vs. hand-rolled Reg* FFI for subkey enumeration.
winreg = "0.56"
# Parse each Xbox/Game-Pass game's MicrosoftGame.config (GDK manifest XML) for the Xbox store
# provider — a small read-only DOM is all we need (Identity/Executable/ShellVisuals/StoreId).
roxmltree = "0.21"
# Software H.264 encoder (GPU-less path + NVENC fallback). The default `source` feature statically
# compiles OpenH264 (BSD-2) — no system lib, builds on MSVC; nasm on PATH adds the SIMD fast path.
openh264 = "0.9"
@@ -0,0 +1,73 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="fake_input">
<copyright>
SPDX-FileCopyrightText: 2015 Martin Gräßlin
SPDX-License-Identifier: LGPL-2.1-or-later
</copyright>
<interface name="org_kde_kwin_fake_input" version="4">
<description summary="Fake input manager">
This interface allows other processes to provide fake input events.
Purpose is on the one hand side to provide testing facilities like XTest
on X11, but also to support use cases like remote control (a remote
desktop server). The compositor gates the interface: it is only exposed
to clients authorized through their .desktop X-KDE-Wayland-Interfaces, so
binding it is the authorization — no per-event confirmation dialog.
</description>
<request name="authenticate">
<description summary="Information about the application requesting fake input">
A FakeInput is required to authenticate itself by providing the
application name and the reason for fake input. The compositor may use
this information to decide whether to allow or deny the request.
</description>
<arg name="application" type="string" summary="user visible name of the application requesting fake input"/>
<arg name="reason" type="string" summary="reason of why fake input is requested"/>
</request>
<request name="pointer_motion">
<description summary="pointer motion event"/>
<arg name="delta_x" type="fixed" summary="X delta of the relative pointer motion"/>
<arg name="delta_y" type="fixed" summary="Y delta of the relative pointer motion"/>
</request>
<request name="button">
<description summary="pointer button event"/>
<arg name="button" type="uint" summary="evdev button code"/>
<arg name="state" type="uint" summary="button state, 0 released, 1 pressed"/>
</request>
<request name="axis">
<description summary="pointer axis (scroll) event"/>
<arg name="axis" type="uint" summary="wl_pointer.axis (0 vertical, 1 horizontal)"/>
<arg name="value" type="fixed" summary="axis value"/>
</request>
<request name="touch_down" since="2">
<description summary="touch down event"/>
<arg name="id" type="uint" summary="unique id of this touch point; must not be reused until up"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="touch_motion" since="2">
<description summary="touch motion event"/>
<arg name="id" type="uint" summary="unique id of an existing touch point"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="touch_up" since="2">
<description summary="touch up event"/>
<arg name="id" type="uint" summary="unique id of an existing touch point"/>
</request>
<request name="touch_cancel" since="2">
<description summary="cancel all current touch points"/>
</request>
<request name="touch_frame" since="2">
<description summary="end a set of touch events (atomic frame)"/>
</request>
<request name="pointer_motion_absolute" since="3">
<description summary="absolute pointer motion event"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="keyboard_key" since="4">
<description summary="keyboard key event"/>
<arg name="button" type="uint" summary="evdev key code"/>
<arg name="state" type="uint" summary="key state, 0 released, 1 pressed"/>
</request>
</interface>
</protocol>
+8 -1
View File
@@ -320,11 +320,18 @@ fn mic_pw_thread(
.into_inner();
let mut params = [Pod::from_bytes(&values).context("mic pod from bytes")?];
// RT_PROCESS: run the producer callback on PipeWire's realtime data loop, so the source is a
// *synchronous* graph node that joins its consumer's driver group and is actually driven. Without
// it the node is async/main-loop and, in the host's busy multi-stream graph (desktop-audio +
// video capture + the session), never acquires a driver — it stays suspended and its process()
// never fires, so every recorder hears pure silence (the long-standing "Linux host mic broken").
stream
.connect(
spa::utils::Direction::Output, // we PRODUCE samples (a source)
None,
pw::stream::StreamFlags::AUTOCONNECT | pw::stream::StreamFlags::MAP_BUFFERS,
pw::stream::StreamFlags::AUTOCONNECT
| pw::stream::StreamFlags::MAP_BUFFERS
| pw::stream::StreamFlags::RT_PROCESS,
&mut params,
)
.context("pw mic stream connect")?;
@@ -1,7 +1,9 @@
//! WASAPI loopback capture of the default render endpoint (system output) — the Windows analogue
//! of the PipeWire sink-monitor backend. Delivers interleaved f32 PCM at 48 kHz stereo, ready for
//! the existing Opus path with NO resampling (WASAPI shared-mode autoconvert does any SRC). WASAPI
//! objects are COM-apartment-bound and not `Send`, so they live on a dedicated thread (mirrors
//! of the PipeWire sink-monitor backend. Delivers interleaved f32 PCM at 48 kHz in the requested
//! channel count (stereo / 5.1 / 7.1, canonical wire order FL FR FC LFE RL RR SL SR via the
//! explicit `dwChannelMask`), ready for the Opus path with NO resampling (WASAPI shared-mode
//! autoconvert does any SRC + up/downmix to the requested layout). WASAPI objects are
//! COM-apartment-bound and not `Send`, so they live on a dedicated thread (mirrors
//! `linux::PwAudioCapturer`); only the channel + stop flag + join handle are in the struct.
use super::{AudioCapturer, SAMPLE_RATE};
@@ -14,9 +16,6 @@ use std::thread::{self, JoinHandle};
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
// 48 kHz stereo 32-bit float: 2 channels * 4 bytes = 8 bytes per frame.
const BLOCK_ALIGN: usize = 2 * 4;
pub struct WasapiLoopbackCapturer {
chunks: Receiver<Vec<f32>>,
channels: u32,
@@ -27,8 +26,8 @@ pub struct WasapiLoopbackCapturer {
impl WasapiLoopbackCapturer {
pub fn open(channels: u32) -> Result<WasapiLoopbackCapturer> {
anyhow::ensure!(
channels == 2,
"WASAPI loopback backend is stereo-only (got {channels})"
matches!(channels, 2 | 6 | 8),
"WASAPI loopback backend supports 2/6/8 channels (got {channels})"
);
let (tx, rx) = sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
@@ -39,7 +38,7 @@ impl WasapiLoopbackCapturer {
let join = thread::Builder::new()
.name("punktfunk-wasapi-audio".into())
.spawn(move || {
if let Err(e) = capture_thread(tx, stop_t, ready_tx) {
if let Err(e) = capture_thread(tx, stop_t, ready_tx, channels) {
tracing::error!(error = format!("{e:#}"), "wasapi loopback thread failed");
}
})
@@ -47,7 +46,8 @@ impl WasapiLoopbackCapturer {
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!(
"WASAPI loopback capture: 48 kHz stereo f32 (default render endpoint)"
channels,
"WASAPI loopback capture: 48 kHz f32 (default render endpoint)"
);
Ok(WasapiLoopbackCapturer {
chunks: rx,
@@ -95,7 +95,10 @@ fn capture_thread(
tx: SyncSender<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
channels: u32,
) -> Result<()> {
// Interleaved f32: channels * 4 bytes per frame.
let block_align = channels as usize * 4;
// COM must be initialized on THIS thread (MTA), before any device call.
if let Err(e) = wasapi::initialize_mta()
.ok()
@@ -106,16 +109,29 @@ fn capture_thread(
}
let res = (|| -> Result<()> {
// Loopback = capture the RENDER endpoint: get the default render device, but open a CAPTURE
// client with loopback=true over it.
// client with loopback=true over it. NOTE: the virtual mic (`super::wasapi_mic`) is guarded
// to NEVER target this same endpoint — otherwise the client's injected mic would be captured
// here and streamed back to the client (infinite echo). Keep that guard in sync if this
// device selection ever changes.
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint (loopback needs a render device)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// 48 kHz stereo f32 interleaved; autoconvert lets WASAPI's shared-mode SRC match the engine
// mix format to ours, so we never resample in Rust. Loopback is implied by capturing a
// RENDER device with Direction::Capture in shared mode (wasapi sets STREAMFLAGS_LOOPBACK).
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE as usize, 2, None);
// 48 kHz f32 interleaved in the requested channel layout; autoconvert lets WASAPI's
// shared-mode SRC match the engine mix format to ours (incl. up/downmix to the requested
// channel count), so we never resample/remix in Rust. The explicit dwChannelMask pins the
// wire order (FL FR FC LFE RL RR SL SR; 7.1 = 0x63F, not 0xFF). Loopback is implied by
// capturing a RENDER device with Direction::Capture in shared mode (STREAMFLAGS_LOOPBACK).
let mask = punktfunk_core::audio::wasapi_channel_mask(channels as u8);
let desired = WaveFormat::new(
32,
32,
&SampleType::Float,
SAMPLE_RATE as usize,
channels as usize,
Some(mask),
);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
@@ -151,7 +167,7 @@ fn capture_thread(
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / BLOCK_ALIGN) * BLOCK_ALIGN;
let whole = (bytes.len() / block_align) * block_align;
if whole == 0 {
continue;
}
@@ -5,14 +5,27 @@
//!
//! Target device, by friendly-name substring (first match wins; override with `PUNKTFUNK_MIC_DEVICE`):
//! "Steam Streaming Microphone" (ships with Steam Remote Play — exactly this purpose), VB-Audio
//! "CABLE Input", VoiceMeeter, or anything with "virtual" in the name. If none is present we return an
//! error with install guidance and the host runs without mic passthrough.
//! "CABLE Input", VoiceMeeter, or anything with "virtual" in the name. If none is present we
//! auto-install the Steam Streaming audio pair (see [`install_steam_audio_pair`]); failing that we
//! return an error with install guidance and the host runs without mic passthrough.
//!
//! **Anti-echo guard (the whole point of this being non-trivial).** The desktop-audio plane
//! ([`super::wasapi_cap`]) loopback-captures the **default render endpoint**. WASAPI loopback
//! captures the *mixed* output of an endpoint — i.e. everything any app renders to it, including
//! what THIS module writes. So if the virtual-mic target is the same device the loopback captures,
//! the client's uplinked mic is captured straight back into the host→client audio stream: an
//! infinite echo. [`find_device`] therefore **excludes the default render endpoint** from the
//! candidates — the mic is guaranteed to land on a different device. (Linux gets this for free: its
//! mic is a dedicated `Audio/Source` node, structurally separate from the monitored sink.)
//!
//! `push` enqueues decoded interleaved-f32 PCM into a bounded ring (drop-oldest beyond ~80 ms so mic
//! latency stays bounded); a dedicated COM-apartment thread renders it event-driven, filling silence
//! when the client isn't talking. WASAPI objects are `!Send`, so they live entirely on that thread
//! (mirrors `WasapiLoopbackCapturer`).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{VirtualMic, SAMPLE_RATE};
use anyhow::{anyhow, Context, Result};
use std::collections::VecDeque;
@@ -110,8 +123,23 @@ impl VirtualMic for WasapiVirtualMic {
}
}
/// Resolve the virtual-mic target among render endpoints by friendly-name. Logs all candidates so a
/// missing device is diagnosable.
/// The endpoint ID of the device the desktop-audio loopback records (the **default render
/// endpoint**, see [`super::wasapi_cap`]). The virtual mic must never target this device — injecting
/// there echoes the client's mic back into the host→client audio stream. `None` if it can't be
/// resolved (then [`find_device`] can't prove a candidate is safe and falls back to name-only
/// matching — no worse than before the guard existed).
fn default_render_id() -> Option<String> {
wasapi::DeviceEnumerator::new()
.ok()?
.get_default_device(&Direction::Render)
.ok()?
.get_id()
.ok()
}
/// Resolve the virtual-mic target among render endpoints by friendly-name, **excluding the endpoint
/// the loopback captures** (the [`default_render_id`] anti-echo guard). Logs all candidates so a
/// missing/skipped device is diagnosable.
fn find_device() -> Result<wasapi::Device> {
let enumerator = wasapi::DeviceEnumerator::new().context("DeviceEnumerator")?;
let collection = enumerator
@@ -121,8 +149,11 @@ fn find_device() -> Result<wasapi::Device> {
let want = std::env::var("PUNKTFUNK_MIC_DEVICE")
.ok()
.map(|s| s.to_lowercase());
// The device the loopback captures — a name match on it is rejected below (would echo).
let loopback_id = default_render_id();
let mut names = Vec::new();
let mut found = None;
let mut skipped_loopback = false;
for i in 0..n {
let Ok(dev) = collection.get_device_at_index(i) else {
continue;
@@ -134,16 +165,37 @@ fn find_device() -> Result<wasapi::Device> {
None => CANDIDATES.iter().any(|c| lname.contains(c)),
};
if hit && found.is_none() {
found = Some(dev);
// Anti-echo guard: never inject into the endpoint the loopback captures.
let is_loopback = match (dev.get_id().ok(), loopback_id.as_deref()) {
(Some(id), Some(lb)) => id == lb,
_ => false,
};
if is_loopback {
skipped_loopback = true;
tracing::warn!(device = %name,
"virtual-mic candidate is the loopback (default render) endpoint — skipping; \
injecting there would echo the client's mic into the desktop-audio stream");
} else {
found = Some(dev);
}
}
names.push(name);
}
found.ok_or_else(|| {
anyhow!(
"no virtual-mic device among render endpoints {names:?}. Install VB-Audio Virtual Cable \
or enable Steam Remote Play's microphone (Steam Streaming Microphone), or set \
PUNKTFUNK_MIC_DEVICE=<friendly-name substring>."
)
if skipped_loopback {
anyhow!(
"the only virtual-mic candidate among render endpoints {names:?} is the default \
playback device the host loopback-captures — injecting there would echo the mic \
back to the client. Add a SEPARATE virtual audio device for the mic (e.g. the Steam \
Streaming Microphone) or set a different default playback device, then reconnect."
)
} else {
anyhow!(
"no virtual-mic device among render endpoints {names:?}. Install VB-Audio Virtual \
Cable or enable Steam Remote Play's microphone (Steam Streaming Microphone), or set \
PUNKTFUNK_MIC_DEVICE=<friendly-name substring>."
)
}
})
}
@@ -153,8 +205,15 @@ fn find_or_install_device() -> Result<wasapi::Device> {
match find_device() {
Ok(d) => Ok(d),
Err(e) => {
tracing::info!("no virtual mic device present — attempting auto-install");
if unsafe { try_install_virtual_mic() } {
tracing::info!("no usable virtual mic device present — attempting auto-install");
// SAFETY: `install_steam_audio_pair` is `unsafe` only because it `LoadLibraryExW`s
// `newdev.dll` and calls `DiInstallDriverW` through a `transmute`d function pointer;
// calling it imposes no extra precondition here (it takes no args and aliases nothing).
// Its internal contract holds: the `DiInstall` type matches the documented
// `BOOL DiInstallDriverW(HWND, PCWSTR, DWORD, PBOOL)` ABI, and it passes a
// NUL-terminated UTF-16 INF path with null/zero optional args. Invoked once on the
// dedicated mic thread.
if unsafe { install_steam_audio_pair() } {
find_device()
} else {
Err(e)
@@ -163,13 +222,26 @@ fn find_or_install_device() -> Result<wasapi::Device> {
}
}
/// Best-effort: install a virtual mic device so one exists without the user installing anything.
/// Mirrors Apollo's Steam Streaming Speakers install — Steam Remote Play ships
/// `SteamStreamingMicrophone.inf` next to the speakers INF, so install it via `DiInstallDriverW`
/// (loaded from `newdev.dll`, like Apollo, to avoid an extra windows-crate feature). Needs admin (the
/// host runs as SYSTEM). Returns true on success; false (no-op) if Steam isn't installed (INF absent),
/// the install is denied, or `PUNKTFUNK_NO_MIC_INSTALL` is set.
unsafe fn try_install_virtual_mic() -> bool {
/// Best-effort: install BOTH Steam Streaming audio devices (the "Steam pair") so mic passthrough
/// works out of the box and the host has a desktop-audio sink distinct from the mic. Steam Remote
/// Play ships `SteamStreamingMicrophone.inf` + `SteamStreamingSpeakers.inf`: the microphone gives the
/// virtual mic a target whose **capture** endpoint apps record from, and the speakers give a
/// **render** endpoint a headless box can loopback-capture that is NOT the mic — so the loopback and
/// the mic land on different devices and never echo (see [`find_device`]). Returns true if either
/// installed. No-op when Steam isn't installed (INFs absent), the install is denied (needs admin —
/// the host runs as SYSTEM), or `PUNKTFUNK_NO_MIC_INSTALL` is set.
unsafe fn install_steam_audio_pair() -> bool {
// Microphone first (the mic's actual target); speakers second (the distinct desktop-audio sink).
let mic = try_install_steam_audio("SteamStreamingMicrophone.inf");
let spk = try_install_steam_audio("SteamStreamingSpeakers.inf");
mic || spk
}
/// Install one Steam Streaming driver INF by filename via `DiInstallDriverW` (loaded from
/// `newdev.dll`, like Apollo, to avoid an extra windows-crate feature). See
/// [`install_steam_audio_pair`] for the contract; `inf_name` is a bare filename under Steam's
/// per-arch `drivers\Windows10\{arch}\` directory.
unsafe fn try_install_steam_audio(inf_name: &str) -> bool {
use windows::core::{s, w, PCWSTR};
use windows::Win32::Foundation::HWND;
use windows::Win32::System::Environment::ExpandEnvironmentStringsW;
@@ -187,12 +259,11 @@ unsafe fn try_install_virtual_mic() -> bool {
let subdir = "arm64";
#[cfg(not(any(target_arch = "x86_64", target_arch = "aarch64")))]
let subdir = "x86";
let template: Vec<u16> = format!(
"%CommonProgramFiles(x86)%\\Steam\\drivers\\Windows10\\{subdir}\\SteamStreamingMicrophone.inf"
)
.encode_utf16()
.chain(std::iter::once(0))
.collect();
let template: Vec<u16> =
format!("%CommonProgramFiles(x86)%\\Steam\\drivers\\Windows10\\{subdir}\\{inf_name}")
.encode_utf16()
.chain(std::iter::once(0))
.collect();
let mut path = vec![0u16; 1024];
let n = ExpandEnvironmentStringsW(PCWSTR(template.as_ptr()), Some(path.as_mut_slice()));
if n == 0 || n as usize > path.len() {
@@ -200,7 +271,7 @@ unsafe fn try_install_virtual_mic() -> bool {
}
let Ok(newdev) = LoadLibraryExW(w!("newdev.dll"), None, LOAD_LIBRARY_SEARCH_SYSTEM32) else {
tracing::warn!("could not load newdev.dll — virtual-mic auto-install unavailable");
tracing::warn!("could not load newdev.dll — Steam-audio auto-install unavailable");
return false;
};
let Some(addr) = GetProcAddress(newdev, s!("DiInstallDriverW")) else {
@@ -216,13 +287,17 @@ unsafe fn try_install_virtual_mic() -> bool {
std::ptr::null_mut(),
) != 0;
if ok {
tracing::info!("installed the Steam Streaming Microphone virtual device");
tracing::info!(
inf = inf_name,
"installed a Steam Streaming virtual audio device"
);
std::thread::sleep(Duration::from_secs(5)); // let the audio subsystem register the endpoint
} else {
let err = windows::Win32::Foundation::GetLastError();
tracing::info!(
inf = inf_name,
?err,
"no virtual mic auto-installed (Steam absent / not admin) — see manual-install guidance"
"Steam-audio device not auto-installed (Steam absent / not admin) — see install guidance"
);
}
ok
+44 -10
View File
@@ -2,6 +2,10 @@
//! CPU-copy fallback (the portal delivers a CPU buffer; the encoder uploads it to the GPU
//! internally). Zero-copy dmabuf→NVENC import is deferred (plan §9 risk).
// Every unsafe block in this module tree carries a `// SAFETY:` proof; enforce it (unsafe-proof
// program). As a parent module this also covers the child modules (capture::windows/linux::*).
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::Result;
/// Packed pixel layout of a [`CapturedFrame`]. The ScreenCast portal negotiates the
@@ -58,6 +62,11 @@ pub struct OutputFormat {
/// HDR: the capturer converts to 10-bit (IDD-push FP16 → `Rgb10a2`; the DDA secure-desktop HDR hint).
/// `false` = 8-bit SDR.
pub hdr: bool,
/// Full-chroma 4:4:4 session: the capturer must keep full chroma — deliver packed **RGB**
/// (`Bgra` / `Rgb10a2`), NOT the subsampled `Nv12`/`P010` the Windows video-engine path produces by
/// default — because 4:4:4 can only be recovered from a full-chroma source. NVENC then does the
/// RGB→YUV444 CSC at encode (chroma_format_idc=3). `false` on every 4:2:0 session.
pub chroma_444: bool,
}
impl OutputFormat {
@@ -69,6 +78,8 @@ impl OutputFormat {
OutputFormat {
gpu: gpu_encode(),
hdr,
// The GameStream + spike paths are always 4:2:0 (4:4:4 is punktfunk/1-native only).
chroma_444: false,
}
}
}
@@ -357,13 +368,16 @@ pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
#[cfg(target_os = "linux")]
pub fn capture_virtual_output(
vout: crate::vdisplay::VirtualOutput,
_want: OutputFormat,
want: OutputFormat,
_capture: crate::session_plan::CaptureBackend,
) -> Result<Box<dyn Capturer>> {
// The Linux host stays 8-bit (HDR is blocked upstream) and the portal negotiates its own format, so
// the `OutputFormat` is unused here; the capture backend is always the portal (the `CaptureBackend`
// arg is a Windows-only dispatch — ignored here).
linux::PortalCapturer::from_virtual_output(vout).map(|c| Box::new(c) as Box<dyn Capturer>)
// The Linux host stays 8-bit (HDR is blocked upstream) and the portal negotiates its own pixel
// format, so only `want.gpu` is honored here: it gates GPU zero-copy capture (the capture backend
// is always the portal — the `CaptureBackend` arg is a Windows-only dispatch). `gpu = false`
// (a 4:4:4 NVENC session) forces the CPU mmap path so the encoder gets CPU-resident RGB to swscale
// into YUV444P — otherwise it would receive CUDA frames and bail.
linux::PortalCapturer::from_virtual_output(vout, want.gpu)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// `PUNKTFUNK_NO_WGC=1` forces the pure single-process DDA (Desktop Duplication) path everywhere: it
@@ -390,6 +404,14 @@ pub fn capture_virtual_output(
})?;
let pref = vout.preferred_mode;
let keep = vout.keepalive;
// Full-chroma 4:4:4 needs a full-chroma RGB source. The IDD-push and WGC paths emit subsampled
// NV12/P010 by default, which can't reconstruct 4:4:4; route a 4:4:4 session to DDA, which delivers
// RGB (Bgra) when its `chroma_444` flag is set. (IDD-push/WGC 4:4:4 capture is a follow-up.)
if want.chroma_444 && capture != CaptureBackend::Dda {
tracing::info!("4:4:4 session — using DDA capture (RGB source) instead of {capture:?}");
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>);
}
// P2 direct frame push (kill DDA): consume frames straight from the pf-vdisplay driver's shared
// ring — no Desktop Duplication, no win32u reparenting hook. Resolved once in the `SessionPlan`
// (was re-derived from `config().idd_push` here); `IddPush` takes the keepalive (owns the virtual
@@ -410,8 +432,15 @@ pub fn capture_virtual_output(
error = %format!("{e:#}"),
"IDD-push open/attach failed — falling back to DDA"
);
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false)
.map(|c| Box::new(c) as Box<dyn Capturer>);
return dxgi::DuplCapturer::open(
target,
pref,
keep,
want.gpu,
false,
want.chroma_444,
)
.map(|c| Box::new(c) as Box<dyn Capturer>);
}
}
}
@@ -422,7 +451,7 @@ pub fn capture_virtual_output(
// chosen backend (it owns the SudoVDA keepalive), so there's no open-time auto-fallback. The
// backend choice (`dda`/`dxgi`/`PUNKTFUNK_NO_WGC` → DDA, else WGC) is now resolved once in the plan.
if capture == CaptureBackend::Dda {
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false)
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>);
}
// WGC default, with a watchdog'd DDA fallback. WGC's Direct3D11CaptureFramePool::CreateFreeThreaded
@@ -433,6 +462,11 @@ pub fn capture_virtual_output(
// DDA is the safety net (+ the secure-desktop path). The encode thread is set MTA so the WGC
// objects built on the watchdog thread (also MTA) are usable here; the keepalive is handed to WGC
// only on success, else to DDA. A hung watchdog thread is abandoned (holds no keepalive).
// SAFETY: `RoInitialize` is a combase FFI call that initializes the WinRT apartment for the calling
// thread. It takes the `RO_INIT_MULTITHREADED` enum by value and borrows no memory, so there is no
// pointer/lifetime/aliasing obligation; it is safe on any thread and idempotent — a second call on a
// thread already in a compatible apartment returns S_FALSE / RPC_E_CHANGED_MODE, which we discard.
// Runs on the encode thread that goes on to use the WGC (WinRT) objects built by the watchdog thread.
unsafe {
let _ = windows::Win32::System::WinRT::RoInitialize(
windows::Win32::System::WinRT::RO_INIT_MULTITHREADED,
@@ -452,12 +486,12 @@ pub fn capture_virtual_output(
}
Ok(Err(e)) => {
tracing::warn!(error = %format!("{e:#}"), "WGC open failed — falling back to DDA");
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false)
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
Err(_) => {
tracing::warn!("WGC open timed out (CreateFreeThreaded hang on the virtual display) — falling back to DDA");
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false)
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
}
+147 -5
View File
@@ -17,6 +17,9 @@
//! instead of leaking it to process exit. The portal thread (when used) still parks on its zbus
//! connection until process exit.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, Context, Result};
use std::os::fd::OwnedFd;
@@ -37,6 +40,13 @@ pub struct PortalCapturer {
/// branch to tell "format never negotiated" (modifier/format mismatch) apart from "negotiated
/// but no buffers arrived" (compositor idle/unmapped) — the two black-screen root causes.
negotiated: Arc<AtomicBool>,
/// True only while the PipeWire stream is `Streaming`. [`try_latest`](Self::try_latest) reads it
/// to distinguish a static desktop (alive, no new buffers) from a dead source (left `Streaming`).
streaming: Arc<AtomicBool>,
/// When the stream first dropped out of `Streaming` with no new frame; used to grace a transient
/// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream
/// is `Streaming`.
stall_since: Option<std::time::Instant>,
/// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis.
node_id: u32,
/// Stops the PipeWire loop on teardown (sent in `Drop`). Without it a dropped or failed
@@ -79,21 +89,29 @@ impl PortalCapturer {
node_id,
"ScreenCast portal session started; connecting PipeWire"
);
Ok(spawn_pipewire(Some(fd), node_id, None)?.into_capturer(node_id, None))
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok(spawn_pipewire(Some(fd), node_id, None, true)?.into_capturer(node_id, None))
}
/// Build a capturer from an already-created virtual output ([`crate::vdisplay::VirtualOutput`]):
/// connect PipeWire to its node (`remote_fd` selects portal-remote vs. default-daemon) and
/// take ownership of its keepalive so the output lives exactly as long as this capturer. This
/// is how the client's requested resolution becomes the captured resolution without scaling.
pub fn from_virtual_output(vout: crate::vdisplay::VirtualOutput) -> Result<PortalCapturer> {
/// `allow_zerocopy` mirrors [`OutputFormat::gpu`](crate::capture::OutputFormat): `false` forces the
/// CPU mmap path (a 4:4:4 NVENC session needs CPU-resident RGB), `true` keeps the GPU zero-copy
/// path subject to `PUNKTFUNK_ZEROCOPY`.
pub fn from_virtual_output(
vout: crate::vdisplay::VirtualOutput,
allow_zerocopy: bool,
) -> Result<PortalCapturer> {
tracing::info!(
node_id = vout.node_id,
allow_zerocopy,
"connecting PipeWire to virtual output"
);
let node_id = vout.node_id;
Ok(
spawn_pipewire(vout.remote_fd, node_id, vout.preferred_mode)?
spawn_pipewire(vout.remote_fd, node_id, vout.preferred_mode, allow_zerocopy)?
.into_capturer(node_id, Some(vout.keepalive)),
)
}
@@ -106,6 +124,7 @@ struct PwHandles {
frames: Receiver<CapturedFrame>,
active: Arc<AtomicBool>,
negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>,
}
@@ -118,6 +137,8 @@ impl PwHandles {
frames: self.frames,
active: self.active,
negotiated: self.negotiated,
streaming: self.streaming,
stall_since: None,
node_id,
quit: Some(self.quit),
join: Some(self.join),
@@ -133,6 +154,12 @@ fn spawn_pipewire(
fd: Option<OwnedFd>,
node_id: u32,
preferred: Option<(u32, u32, u32)>,
// Allow GPU zero-copy capture (dmabuf→CUDA/VA). `false` forces the CPU mmap path even when
// `PUNKTFUNK_ZEROCOPY` is set — a 4:4:4 NVENC session needs CPU-resident RGB (the encoder
// swscales RGB→YUV444P; `hevc_nvenc` can't 4:4:4 from a CUDA RGB surface), so the session plan
// passes `gpu = false` for it. Without this, a 4:4:4 session under `PUNKTFUNK_ZEROCOPY=1` would
// get CUDA frames and the encoder would bail (`want_444 && cuda`).
allow_zerocopy: bool,
) -> Result<PwHandles> {
// Frames flow from the pipewire thread over a small bounded channel.
let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8);
@@ -140,11 +167,13 @@ fn spawn_pipewire(
let active_cb = active.clone();
let negotiated = Arc::new(AtomicBool::new(false));
let negotiated_cb = negotiated.clone();
let streaming = Arc::new(AtomicBool::new(false));
let streaming_cb = streaming.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope.
let (quit_tx, quit_rx) = ::pipewire::channel::channel::<()>();
let zerocopy = crate::zerocopy::enabled();
let zerocopy = allow_zerocopy && crate::zerocopy::enabled();
let join = thread::Builder::new()
.name("punktfunk-pipewire".into())
.spawn(move || {
@@ -154,6 +183,7 @@ fn spawn_pipewire(
frame_tx,
active_cb,
negotiated_cb,
streaming_cb,
zerocopy,
preferred,
quit_rx,
@@ -166,6 +196,7 @@ fn spawn_pipewire(
frames: frame_rx,
active,
negotiated,
streaming,
quit: quit_tx,
join,
})
@@ -216,6 +247,28 @@ impl Capturer for PortalCapturer {
}
}
}
if latest.is_some() || self.streaming.load(Ordering::Relaxed) {
// A frame arrived, or the source is alive but idle (static desktop) — normal. Clear any
// stall and repeat the last frame on `None`, exactly as before.
self.stall_since = None;
return Ok(latest);
}
// No new frame AND the stream has left `Streaming` (Paused/Unconnected/Error). The source
// went away — a compositor torn down on a Gaming↔Desktop switch, a removed virtual output.
// Grace a brief window (a transient mid-stream renegotiation can blip out of Streaming and
// back) before declaring it lost so the encode loop rebuilds in place rather than freezing
// on the last frame forever.
const STALL_GRACE: Duration = Duration::from_millis(1500);
let since = *self.stall_since.get_or_insert_with(std::time::Instant::now);
if since.elapsed() >= STALL_GRACE {
self.stall_since = None;
return Err(anyhow!(
"PipeWire source stalled (node {}): stream left Streaming for >{}ms with no frames \
— the compositor/virtual output went away (session switch?)",
self.node_id,
STALL_GRACE.as_millis()
));
}
Ok(latest)
}
@@ -464,6 +517,10 @@ mod pipewire {
/// Set once a video format is agreed (`param_changed`), so a first-frame timeout can tell
/// "format never negotiated" apart from "negotiated but no buffers arrived".
negotiated: Arc<AtomicBool>,
/// True only while the PipeWire stream is in `Streaming` (the source is alive). Goes false on
/// `Paused`/`Unconnected`/`Error` — the source vanished (compositor torn down on a session
/// switch). Read by [`PortalCapturer::try_latest`] to surface a sustained drop as a loss.
streaming: Arc<AtomicBool>,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer.
importer: Option<crate::zerocopy::EglImporter>,
/// VAAPI zero-copy: hand the raw dmabuf to the encoder (which imports + GPU-CSCs it) instead
@@ -498,6 +555,12 @@ mod pipewire {
impl DmabufMap {
fn new(fd: i32, len: usize) -> Option<DmabufMap> {
// SAFETY: a null `addr` lets the kernel choose the mapping address; `fd` is a caller-owned
// dmabuf/MemFd fd, valid for the duration of this call, and `len` is the requested map length.
// `mmap` reads no Rust memory — it installs a fresh PROT_READ/MAP_SHARED page mapping and
// returns its base (or MAP_FAILED, checked below before `DmabufMap` adopts it). The returned
// region is a brand-new VMA, so it aliases no live Rust object, and it keeps the underlying
// object mapped independently of `fd` (which may be closed after this returns).
let ptr = unsafe {
libc::mmap(
std::ptr::null_mut(),
@@ -514,6 +577,11 @@ mod pipewire {
impl Drop for DmabufMap {
fn drop(&mut self) {
// SAFETY: `self.ptr`/`self.len` are exactly the base+length of a successful `mmap` in
// `DmabufMap::new` (constructed only when `ptr != MAP_FAILED`). This `DmabufMap` uniquely owns
// that mapping and `drop` runs once, so `munmap` releases a live mapping exactly once — no
// double-unmap. Every `&[u8]` derived from the mapping is bounded by this `DmabufMap`'s
// lifetime, so no borrow outlives the unmap.
unsafe {
libc::munmap(self.ptr, self.len);
}
@@ -719,6 +787,14 @@ mod pipewire {
if !ud.active.load(Ordering::Relaxed) {
return;
}
// SAFETY: `spa_buf` is the `*mut spa_buffer` of the PipeWire buffer we dequeued and still hold for
// this `.process` callback (not requeued until after `consume_frame` returns), so it is live. The
// block null-checks `spa_buf`, requires `n_datas != 0`, and null-checks the `datas` array pointer
// before forming any slice. `(*spa_buf).datas` points to `n_datas` libspa `spa_data` structs, and
// `pw::spa::buffer::Data` is `#[repr(transparent)]` over `spa_data` (the same cast
// `Buffer::datas_mut` performs — see the function doc), so the pointer cast + length describe
// exactly that array, in bounds. The PipeWire loop is single-threaded and owns the buffer here, so
// this `&mut` slice is the only reference to it (no aliasing/data race).
let datas: &mut [pw::spa::buffer::Data] = unsafe {
if spa_buf.is_null() || (*spa_buf).n_datas == 0 || (*spa_buf).datas.is_null() {
&mut []
@@ -783,6 +859,10 @@ mod pipewire {
// dup the fd so it survives the SPA buffer recycle — the encode thread
// imports it. (Content stability across the brief map+CSC window relies on
// the compositor's buffer-pool depth, like any zero-copy capture.)
// SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid
// for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd,
// touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1).
// The original stays owned by PipeWire; the dup is a new fd we own (checked >= 0).
let dup =
unsafe { libc::fcntl(datas[0].fd() as i32, libc::F_DUPFD_CLOEXEC, 0) };
if dup >= 0 {
@@ -796,6 +876,10 @@ mod pipewire {
pts_ns,
format: fmt,
payload: FramePayload::Dmabuf(DmabufFrame {
// SAFETY: `dup` is the fresh fd `fcntl(F_DUPFD_CLOEXEC)` just returned
// (checked `dup >= 0`); nothing else owns it, so `OwnedFd` takes sole
// ownership and closes it exactly once on drop — no alias, no
// double-close.
fd: unsafe { OwnedFd::from_raw_fd(dup) },
fourcc,
modifier: ud.modifier,
@@ -930,6 +1014,11 @@ mod pipewire {
// cleanly if the real buffer is genuinely too small. MemPtr buffers (no fd) are same-process —
// trust `d.data()`.
let fd_len = if raw_fd > 0 {
// SAFETY: `libc::stat` is a C plain-old-data struct for which all-zero is a valid value, so
// `mem::zeroed()` is a sound initializer. `raw_fd` is the buffer's fd (`> 0` checked here) and
// valid for this callback; `fstat` writes metadata into `&mut st`, a live, aligned,
// correctly-sized stack `stat` that outlives the synchronous call. `st.st_size` is read only
// after the return value is confirmed `== 0`. `st` is a fresh local, so nothing aliases it.
unsafe {
let mut st: libc::stat = std::mem::zeroed();
(libc::fstat(raw_fd as i32, &mut st) == 0 && st.st_size > 0)
@@ -946,6 +1035,14 @@ mod pipewire {
match DmabufMap::new(raw_fd as i32, map_len) {
Some(m) => {
_mapping = m;
// SAFETY: `_mapping` is the `DmabufMap` just stored; its `ptr`/`len` come from a
// successful `mmap` of `map_len` PROT_READ bytes, so `ptr` is non-null, page-aligned,
// and the VMA is one allocated object of `len` bytes valid for reads. In the common
// path `map_len == fd_len` (the fd's real size from `fstat`), so the mapping spans the
// whole object; the de-pad copy below is further bounded by the `offset <= buf.len()`
// and `needed > avail` guards. The `&[u8]` borrows `_mapping`, which lives to the end
// of `consume_frame`, so the slice never outlives the mapping, and the memory is only
// read here, so there is no aliasing/mutation.
Some(unsafe {
std::slice::from_raw_parts(_mapping.ptr as *const u8, _mapping.len)
})
@@ -1013,6 +1110,7 @@ mod pipewire {
tx: SyncSender<CapturedFrame>,
active: Arc<AtomicBool>,
negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
zerocopy: bool,
preferred: Option<(u32, u32, u32)>,
quit_rx: pw::channel::Receiver<()>,
@@ -1107,6 +1205,7 @@ mod pipewire {
tx,
active,
negotiated,
streaming,
importer,
vaapi_passthrough,
nv12: crate::zerocopy::nv12_enabled(),
@@ -1131,8 +1230,17 @@ mod pipewire {
let _listener = stream
.add_local_listener_with_user_data(data)
.state_changed(|_stream, _ud, old, new| {
.state_changed(|_stream, ud, old, new| {
tracing::info!(?old, ?new, "pipewire stream state");
// Track whether the node is actively producing. A live source sits in `Streaming`
// (a static desktop just sends no buffers); anything else — `Paused`/`Unconnected`/
// `Error` — means the source went away (compositor died, virtual output removed on a
// Gaming↔Desktop switch). `try_latest` turns a sustained non-Streaming state into a
// capture-loss so the encode loop rebuilds instead of freezing on the last frame.
ud.streaming.store(
matches!(new, pw::stream::StreamState::Streaming),
Ordering::Relaxed,
);
})
.param_changed(|_stream, ud, id, param| {
let Some(param) = param else { return };
@@ -1177,24 +1285,43 @@ mod pipewire {
// Latest-frame-only (OBS pattern): Mutter delivers buffers in bursts and
// recycles its pool; an older queued buffer carries a STALE frame. Drain all
// queued buffers, requeue the older ones, keep only the newest.
// SAFETY: `stream` is the live stream PipeWire passes into this `.process` callback on
// the loop thread, where `pw_stream_dequeue_buffer` is the documented call. It returns
// a `*mut pw_buffer` owned by the stream (or null when the queue is drained),
// null-checked before any use. The loop is single-threaded, so no concurrent access.
let mut newest = unsafe { stream.dequeue_raw_buffer() };
if newest.is_null() {
return;
}
let mut drained = 1u32;
loop {
// SAFETY: same stream/loop-thread contract as the dequeue above; each call returns
// the next stream-owned `*mut pw_buffer` or null (null-checked before use).
let next = unsafe { stream.dequeue_raw_buffer() };
if next.is_null() {
break;
}
// SAFETY: `newest` is a non-null `*mut pw_buffer` previously dequeued from this same
// stream and not yet requeued; `pw_stream_queue_buffer` hands ownership back to the
// stream. We immediately overwrite `newest = next`, so the requeued pointer is never
// touched again (no use-after-requeue). Loop thread, single-threaded.
unsafe { stream.queue_raw_buffer(newest) };
newest = next;
drained += 1;
}
// SAFETY: `newest` is the non-null buffer we still own (dequeued, not requeued);
// `.buffer` is a `*mut spa_buffer` field libpipewire populated. This is a single field
// load through a valid pointer — no mutation or aliasing.
let spa_buf = unsafe { (*newest).buffer };
// Inspect the newest buffer's header + first chunk for the diagnostic and the
// CORRUPTED skip. SPA_META_Header is optional — `hdr` may be null.
// SAFETY: `spa_buf` is the `*mut spa_buffer` of the buffer we still hold.
// `spa_buffer_find_meta_data` scans that buffer's metadata array for a `SPA_META_Header`
// of at least `size_of::<spa_meta_header>()` bytes and returns a pointer into the held
// buffer's metadata (or null). The size argument matches the struct the result is cast
// to, and the pointer stays valid as long as the buffer is held (until requeue). Null is
// handled below.
let hdr = unsafe {
spa::sys::spa_buffer_find_meta_data(
spa_buf,
@@ -1205,11 +1332,20 @@ mod pipewire {
let hdr_flags = if hdr.is_null() {
0u32
} else {
// SAFETY: reached only when `hdr` is non-null; it points to a `spa_meta_header`
// inside the live buffer's metadata (returned for a size >=
// `size_of::<spa_meta_header>()`, so `.flags` is in bounds). A single field read
// while the buffer is still held.
unsafe { (*hdr).flags }
};
// First data chunk's size + flags (used for the diagnostic + CORRUPTED check)
// and its data type (a dmabuf legitimately reports chunk size 0, so the size-0
// stale skip only applies to mappable SHM buffers).
// SAFETY: every dereference is guarded in order before any field read — `spa_buf`
// non-null, `n_datas > 0`, the `datas` (`*mut spa_data`) array non-null, and the first
// element's `chunk` (`*mut spa_chunk`) non-null. `d0` is that first `spa_data` and `c`
// its chunk; reading `(*d0).type_`, `(*c).size`, `(*c).flags` are in-bounds field loads
// of libspa structs inside the buffer we still hold. Single-threaded loop, no mutation.
let (chunk_size, chunk_flags, is_dmabuf) = unsafe {
if !spa_buf.is_null()
&& (*spa_buf).n_datas > 0
@@ -1246,11 +1382,17 @@ mod pipewire {
"capture: skipped a stale CORRUPTED/cursor buffer (GNOME)"
);
}
// SAFETY: `newest` is the non-null buffer we own (dequeued, never requeued on this
// skip path); hand it back to the stream exactly once and return without touching it
// again. Loop thread inside `.process`.
unsafe { stream.queue_raw_buffer(newest) };
return;
}
consume_frame(ud, spa_buf);
// SAFETY: `consume_frame` has finished reading `spa_buf` (and the `datas` borrows derived
// from `newest`), so requeuing the owned `newest` exactly once here is sound — no
// use-after-requeue. Loop thread inside `.process`.
unsafe { stream.queue_raw_buffer(newest) };
}));
if outcome.is_err() {
@@ -15,6 +15,9 @@
//! composed while a session is live). Effectiveness can be build/driver-dependent; gated by
//! `PUNKTFUNK_FORCE_COMPOSED` (default ON; set =0 to disable).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use windows::core::w;
@@ -48,6 +51,10 @@ impl ForceComposedFlip {
let st = stop.clone();
std::thread::Builder::new()
.name("composed-flip".into())
// SAFETY: `run` is this module's `unsafe fn` (it owns a desktop+window lifecycle via Win32
// FFI); it takes ownership of `st` (the stop `Arc<AtomicBool>`) and has no caller-side memory
// precondition. It is designed to own its thread for its whole duration — exactly the
// dedicated `composed-flip` thread spawned here.
.spawn(move || unsafe { run(st) })
.ok()?;
tracing::info!("force-composed-flip overlay started (Winlogon-aware)");
@@ -62,6 +69,9 @@ impl Drop for ForceComposedFlip {
}
extern "system" fn wndproc(hwnd: HWND, msg: u32, wp: WPARAM, lp: LPARAM) -> LRESULT {
// SAFETY: this is the window procedure the OS invokes with the window's own `hwnd` and a real
// message `(msg, wp, lp)`. `DefWindowProcW` performs default processing for exactly those
// parameters (all passed straight through by value); it borrows no Rust memory and is synchronous.
unsafe { DefWindowProcW(hwnd, msg, wp, lp) }
}
@@ -1,5 +1,5 @@
//! Input-desktop watcher (Windows) — the authoritative "normal vs secure desktop" signal for the
//! two-process secure-desktop design (docs/windows-secure-desktop.md).
//! two-process secure-desktop design (design/archive/windows-secure-desktop.md).
//!
//! Windows switches the *input desktop* to "Winlogon" (the secure desktop) for UAC elevation, the
//! lock screen and the login screen, and back to "Default" for the normal session. WGC captures only
@@ -7,6 +7,9 @@
//! desktop's NAME (WTS session notifications miss UAC entirely, so the name is the reliable signal)
//! and publishes it as an atomic the capture mux + input path read.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use std::sync::atomic::{AtomicBool, AtomicU8, Ordering};
use std::sync::Arc;
use std::time::Duration;
@@ -33,6 +36,10 @@ impl DesktopWatcher {
// mux) sees the real state immediately. Otherwise a session that begins already on the secure
// desktop (e.g. a reconnect to a locked box) would read DESKTOP_NORMAL for the first poll
// interval and relay one stale normal-desktop frame — the "flash of the login screen" bug.
// SAFETY: `is_secure_desktop` is this module's `unsafe fn` — unsafe only because it calls Win32
// desktop FFI (`OpenInputDesktop`/`GetUserObjectInformationW`/`CloseDesktop`), with no caller
// precondition; it opens, names, and closes the input-desktop handle internally and is safe to
// call from any thread (here, on the thread running `DesktopWatcher::start`).
let initial = if unsafe { is_secure_desktop() } {
DESKTOP_SECURE
} else {
@@ -53,6 +60,9 @@ impl DesktopWatcher {
let mut candidate = initial;
let mut stable = 0u32;
while !st.load(Ordering::Relaxed) {
// SAFETY: same as in `start` — `is_secure_desktop` is self-contained Win32 desktop
// FFI with no caller precondition, called here on the dedicated `desktop-watch`
// polling thread.
let v = if unsafe { is_secure_desktop() } {
DESKTOP_SECURE
} else {
@@ -7,6 +7,9 @@
//! Validates only with a real GPU + an *activated* SudoVDA monitor (`DuplicateOutput` needs a live
//! WDDM output). Compiles on the GPU-less VM; the pure helpers are unit-tested there.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use std::ffi::c_void;
@@ -69,7 +72,12 @@ pub struct D3d11Frame {
pub texture: ID3D11Texture2D,
pub device: ID3D11Device,
}
// COM pointers, used only from the single owning thread.
// SAFETY: `D3d11Frame` owns an `ID3D11Texture2D` + `ID3D11Device`, which are COM interface pointers.
// D3D11 devices/resources use thread-safe (interlocked) COM reference counting, and the device is
// created free-threaded (`make_device` passes no `D3D11_CREATE_DEVICE_SINGLETHREADED`), so handing
// ownership of the frame to another thread — the capture→encode handoff — and releasing it there is
// sound. The value is moved, never aliased (no `Sync`), so there is no concurrent use of the
// single-threaded immediate context.
unsafe impl Send for D3d11Frame {}
pub fn pack_luid(luid: LUID) -> i64 {
@@ -295,6 +303,12 @@ unsafe fn d3dkmt_set_scheduling_priority_class(
fn elevate_process_gpu_priority() {
use std::sync::Once;
static ONCE: Once = Once::new();
// SAFETY: the closure calls two of this module's `unsafe fn`s — `enable_inc_base_priority`
// (adjusts the current-process token; it has no caller precondition and builds all its FFI args
// locally) and `d3dkmt_set_scheduling_priority_class` (loads gdi32 by name and calls the export).
// The latter requires `process` to be a valid process handle; `GetCurrentProcess()` returns the
// current-process pseudo-handle, which is always valid and needs no close. Runs once via
// `Once::call_once`; no raw pointers are dereferenced here.
ONCE.call_once(|| unsafe {
use windows::Win32::System::Threading::GetCurrentProcess;
let Some(prio) = configured_gpu_priority_class() else {
@@ -538,6 +552,17 @@ unsafe extern "system" fn hybrid_query_hook(gpu_preference: *mut u32) -> i32 {
pub(crate) fn install_gpu_pref_hook() {
use std::sync::Once;
static HOOK: Once = Once::new();
// SAFETY: this one-time hook install only touches a region it has just validated.
// `LoadLibraryA("win32u.dll")` + `GetProcAddress("NtGdiDdDDIGetCachedHybridQueryValue")` yield the
// live base of the real exported function, so `target` is a valid executable code pointer to at
// least the 12 bytes the patch overwrites (an x64 prologue, per Apollo's verified hook). The two
// `ptr::copy_nonoverlapping`s each move exactly 12 bytes between the 12-byte stack arrays
// (`patch`/`readback`) and `target`, which `VirtualProtect(target, 12, PAGE_EXECUTE_READWRITE, …)`
// has just made writable (and is restored to `old` after) — source and dest never overlap (stack
// vs. loaded module image), so every access stays in mapped, in-bounds memory.
// `FlushInstructionCache` gets the current-process pseudo-handle + that same range. The DPI calls
// take by-value context handles / fill the live local `&mut old`/`&mut restore` for the duration of
// each synchronous call. Runs once via `Once::call_once`, before any DXGI use.
HOOK.call_once(|| unsafe {
use windows::Win32::System::LibraryLoader::{GetProcAddress, LoadLibraryA};
use windows::Win32::System::Memory::{
@@ -1389,6 +1414,14 @@ pub fn hdr_p010_selftest() -> Result<()> {
}
}
// SAFETY: this self-test creates its own D3D11 device + immediate context (`D3D11CreateDevice`,
// both checked non-null) and uses ONLY that device for the rest of the block: every
// `CreateTexture2D`/`CreateShaderResourceView`/`HdrP010Converter::{new,convert}`/`CopyResource`/
// `Map` is invoked on that device or its context, so all resources share one device and run on this
// single thread. The source texture's `D3D11_SUBRESOURCE_DATA` points at `fp16`, a live
// `Vec<u16>` of `W*H*4` samples with `SysMemPitch = W*8`, matching the W×H R16G16B16A16 texture;
// `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are
// proven individually at the `read_u16` closure below.
unsafe {
// Hardware D3D11 device (no adapter pin — the default GPU is fine for the self-test).
let mut device: Option<ID3D11Device> = None;
@@ -1977,6 +2010,10 @@ pub struct DuplCapturer {
/// first, retried (legacy DuplicateOutput can't capture HDR). Set for the secure-desktop DDA leg
/// when the SudoVDA is in HDR; threaded into every (re)duplication incl. ACCESS_LOST recovery.
want_hdr: bool,
/// Full-chroma 4:4:4 session: deliver packed RGB (`Bgra` SDR / `Rgb10a2` HDR) and SKIP the
/// video-engine RGB→YUV (NV12/P010) conversion — NVENC reconstructs 4:4:4 only from a full-chroma
/// source, so we hand it the RGB texture and it CSCs to YUV444 at encode (chroma_format_idc=3).
chroma_444: bool,
/// HDR (scRGB FP16) capture state. Set when the duplication surface is `R16G16B16A16_FLOAT`
/// (the desktop has HDR on). The frame can't be `CopyResource`d into a BGRA target, so the HDR
/// path copies it into an FP16 SRV texture, composites the cursor, then runs [`HdrConverter`] to
@@ -2038,7 +2075,11 @@ pub struct DuplCapturer {
dbg_cursor: u64,
_keepalive: Box<dyn Send>,
}
// COM objects used only from the one thread that owns the capturer (the encode thread).
// SAFETY: `DuplCapturer` holds D3D11 device/context/duplication COM pointers plus plain data. The
// device is created free-threaded (`make_device` sets no `D3D11_CREATE_DEVICE_SINGLETHREADED`) and
// COM reference counting is interlocked, so moving ownership of the whole capturer to another thread
// is sound. It is used by exactly one thread (the encode thread) at a time — moved to it once, never
// shared (no `Sync`) — so the single-threaded immediate context is never touched concurrently.
unsafe impl Send for DuplCapturer {}
impl DuplCapturer {
@@ -2050,7 +2091,16 @@ impl DuplCapturer {
// stage 5) so the capturer never re-derives the encode backend itself.
gpu: bool,
want_hdr: bool,
// 4:4:4 session → deliver RGB, skip the NV12/P010 video-engine conversion (see the field doc).
chroma_444: bool,
) -> Result<Self> {
// SAFETY: runs on the capture thread that will own this `DuplCapturer`. `install_gpu_pref_hook()`
// and the DPI-context calls take by-value handles / no args and touch only thread/process state;
// `SetThreadExecutionState` takes a flags bitmask by value. `CreateDXGIFactory1` yields a live
// `IDXGIFactory1`, and every subsequent COM method (`EnumAdapters1`/`EnumOutputs`/`GetDesc1`/
// `GetDesc`/`cast`) is called on that factory or on an adapter/output it returned — each obtained
// through a checked `while let Ok(..)`/`?` — all from this one thread. No raw pointers are
// dereferenced; the borrowed strings/locals outlive each synchronous call.
unsafe {
// Stop DXGI hybrid-GPU output reparenting BEFORE we create the factory / enumerate outputs
// (the cause of the 0x887A0026 ACCESS_LOST churn on this hybrid box: RTX 4090 + AMD iGPU).
@@ -2267,6 +2317,7 @@ impl DuplCapturer {
gpu_copy: None,
last_present: None,
want_hdr,
chroma_444,
hdr_fp16: is_hdr_init,
hdr_meta: hdr_meta_init,
fp16_src: None,
@@ -3044,7 +3095,10 @@ impl DuplCapturer {
// Video-engine path: scRGB FP16 → BT.2020 PQ P010 on the VIDEO engine (no 3D shader, and
// NVENC encodes P010 natively). Fall back to the HdrConverter pixel shader (3D) only if the
// video processor is unavailable.
if let Some(p010) = self.convert_to_yuv(&src, true) {
if let Some(p010) = (!self.chroma_444)
.then(|| self.convert_to_yuv(&src, true))
.flatten()
{
self.last_present = Some((p010.clone(), PixelFormat::P010));
return Ok(CapturedFrame {
width: self.width,
@@ -3104,7 +3158,10 @@ impl DuplCapturer {
// conversion AND NVENC's encode stay OFF the 3D engine — the only way to keep up when a
// game pins the 3D engine at ~100%. Fall back to handing NVENC the BGRA texture (it then
// does RGB→YUV internally on the 3D/compute engine).
if let Some(nv12) = self.convert_to_yuv(&gpu, false) {
if let Some(nv12) = (!self.chroma_444)
.then(|| self.convert_to_yuv(&gpu, false))
.flatten()
{
self.last_present = Some((nv12.clone(), PixelFormat::Nv12));
return Ok(CapturedFrame {
width: self.width,
@@ -3207,6 +3264,11 @@ impl Capturer for DuplCapturer {
// the duplication up to 12 s). Better a few seconds of frozen-last-frame than dropping the stream.
let mut deadline = Instant::now() + Duration::from_secs(20);
loop {
// SAFETY: `acquire` is an `unsafe fn` because it drives the D3D11 immediate context + the
// output duplication, which must be touched only from the capturer's owning thread.
// `next_frame` runs on that one thread — `DuplCapturer` is `Send` but not `Sync`, so it is
// owned by a single (encode) thread for its whole life — and `&mut self` gives exclusive
// access for the call, satisfying that contract.
if let Some(f) = unsafe { self.acquire() }? {
self.ever_got_frame = true;
return Ok(f);
@@ -3253,6 +3315,8 @@ impl Capturer for DuplCapturer {
}
fn try_latest(&mut self) -> Result<Option<CapturedFrame>> {
// SAFETY: as in `next_frame` — `acquire` must run on the capturer's single owning thread, and
// `try_latest` is called on it (`DuplCapturer` is `Send`, not `Sync`); `&mut self` is exclusive.
unsafe { self.acquire() }
}
@@ -3264,11 +3328,19 @@ impl Capturer for DuplCapturer {
impl Drop for DuplCapturer {
fn drop(&mut self) {
if self.holding_frame {
// SAFETY: `self.dupl` is the live `IDXGIOutputDuplication` this capturer created and owns;
// `ReleaseFrame` is a valid COM method on it, called only when `holding_frame` records that a
// frame was acquired and not yet released (so it is not an unbalanced release). Drop runs on
// whichever thread owns the capturer — its sole owner, since it is `!Sync` — and the `&`
// borrow of the duplication outlives this synchronous call.
unsafe {
let _ = self.dupl.as_ref().map(|d| d.ReleaseFrame());
}
}
// Release the display/system-required execution state we took at open().
// SAFETY: `SetThreadExecutionState` is a Win32 FFI call taking an execution-state flag bitmask
// by value (`ES_CONTINUOUS` clears the display/system-required state taken at open); it borrows
// no Rust memory and is safe to call from any thread.
unsafe {
SetThreadExecutionState(ES_CONTINUOUS);
}
@@ -10,7 +10,10 @@
//! [`pf_driver_proto::frame`] (which OWNS the contract, with `const` size asserts) — both sides
//! `use` it, so drift is a compile error rather than a "must match" comment.
use super::dxgi::{make_device, D3d11Frame, HdrConverter, WinCaptureTarget};
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::dxgi::{make_device, D3d11Frame, HdrP010Converter, VideoConverter, WinCaptureTarget};
use super::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{bail, Context, Result};
use pf_driver_proto::frame;
@@ -20,13 +23,12 @@ use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use windows::core::{w, Interface, HSTRING};
use windows::Win32::Foundation::{HANDLE, INVALID_HANDLE_VALUE, LUID};
use windows::Win32::Graphics::Direct3D11::{
ID3D11Device, ID3D11DeviceContext, ID3D11RenderTargetView, ID3D11ShaderResourceView,
ID3D11Texture2D, D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE,
D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX, D3D11_RESOURCE_MISC_SHARED_NTHANDLE,
D3D11_TEXTURE2D_DESC, D3D11_USAGE_DEFAULT,
ID3D11Device, ID3D11DeviceContext, ID3D11ShaderResourceView, ID3D11Texture2D,
D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE, D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX,
D3D11_RESOURCE_MISC_SHARED_NTHANDLE, D3D11_TEXTURE2D_DESC, D3D11_USAGE_DEFAULT,
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_FORMAT, DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_R10G10B10A2_UNORM,
DXGI_FORMAT, DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_NV12, DXGI_FORMAT_P010,
DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_SAMPLE_DESC,
};
use windows::Win32::Graphics::Dxgi::{
@@ -205,21 +207,33 @@ pub struct IddPushCapturer {
/// cleared when a fresh frame resumes. If it stays set past the recovery window, `try_consume` drops
/// the session (recover-or-drop, no DDA).
recovering_since: Option<Instant>,
/// Host-owned ROTATING output ring NVENC encodes (texture + RTV per slot). Rotating it per frame is
/// the precondition for pipelining the encode loop: while NVENC encodes frame N's texture on the
/// ASIC, frame N+1's convert/copy writes a DIFFERENT texture on the 3D engine — the two overlap. The
/// HDR convert and the SDR copy both write into the current slot. Format = `out_format()` (Rgb10a2 in
/// HDR, Bgra in SDR); rebuilt on a display-mode flip. Built lazily.
out_ring: Vec<(ID3D11Texture2D, ID3D11RenderTargetView)>,
/// Host-owned ROTATING output ring NVENC encodes (one YUV texture per slot). Rotating it per frame
/// is the precondition for pipelining the encode loop: while NVENC encodes frame N's texture on the
/// ASIC, frame N+1's convert writes a DIFFERENT texture — the two overlap. Format = `out_format()`:
/// NV12 (SDR, BT.709 limited) or P010 (HDR, BT.2020 PQ limited), so NVENC takes native YUV and skips
/// its internal RGB→YUV CSC on the SM/3D engine the game saturates (plan §5.A). Rebuilt on a
/// display-mode flip. Built lazily.
out_ring: Vec<ID3D11Texture2D>,
out_idx: usize,
/// FP16 scRGB → `Rgb10a2` BT.2020 PQ converter, used while the display is HDR. Built lazily.
hdr_conv: Option<HdrConverter>,
/// BGRA slot → NV12 (BT.709 limited) on the dedicated D3D11 VIDEO engine, used while the display is
/// SDR — keeps the colour-convert OFF the contended 3D/compute engine. Built lazily; rebuilt on a
/// size/HDR flip.
video_conv: Option<VideoConverter>,
/// FP16 scRGB slot → P010 (BT.2020 PQ limited) via two shader passes, used while the display is HDR
/// (NVIDIA's VideoProcessor can't do RGB→P010). The passes run on the 3D engine, but it still skips
/// NVENC's internal SM-side CSC. Built lazily.
hdr_p010_conv: Option<HdrP010Converter>,
last_seq: u64,
last_present: Option<(ID3D11Texture2D, PixelFormat)>,
status_logged: bool,
_keepalive: Box<dyn Send>,
}
// COM objects used only from the owning (encode) thread.
// SAFETY: `IddPushCapturer` is `!Send` only because of its `*mut SharedHeader`/`*mut DebugBlock` raw
// pointers (and the COM interfaces). It is created, used, and dropped by a SINGLE thread — the owning
// capture/encode thread — never shared: the `ID3D11DeviceContext` is the device's IMMEDIATE context
// (single-threaded by D3D11 contract) and is only ever touched from that thread, and the header/
// dbg_block pointers (into mappings this struct owns) are only dereferenced there. `Send` transfers
// ownership to one thread at a time with NO concurrent access; we do not (and must not) claim `Sync`.
unsafe impl Send for IddPushCapturer {}
/// Build a permissive (Everyone:GenericAll) `SECURITY_ATTRIBUTES` so the restricted WUDFHost driver
@@ -337,6 +351,9 @@ impl IddPushCapturer {
// a fullscreen game can hold the virtual display at a different mode (esp. across a reconnect), so
// matching the actual mode lets the first frame flow instead of being dropped (game-capture bug
// GB1). Falls back to the negotiated mode when the CCD read is unavailable.
// SAFETY: `active_resolution` is an `unsafe fn` (Win32 CCD `QueryDisplayConfig`) that takes only a
// copy of the plain `u32` CCD target id and returns owned `(w, h)` values; it forms no borrows from
// us and validates the id internally, returning `None` on any failure (handled by `unwrap_or`).
let (w, h) =
unsafe { crate::win_display::active_resolution(target.target_id) }.unwrap_or((pw, ph));
if (w, h) != (pw, ph) {
@@ -355,6 +372,27 @@ impl IddPushCapturer {
// PROACTIVELY enable advanced color so HDR streams without the user toggling anything; an
// SDR-only client leaves the display alone (and still gets a tone-mapped picture, never a freeze,
// if the user does enable HDR).
// SAFETY: one block over the whole ring setup; every operation in it is sound:
// - `set_advanced_color`/`advanced_color_enabled` are `unsafe fn`s taking only a copy of the plain
// `u32` target id; they read/flip CCD display config and return owned values, borrowing nothing.
// - `CreateDXGIFactory1`, `EnumAdapterByLuid`, `make_device`, `permissive_sa`, `CreateFileMappingW`,
// `MapViewOfFile`, `CreateEventW`, and `create_ring_slots` are all `?`-checked, so every returned
// interface/handle/view is non-error before use; `&sa`/`&adapter`/`&device`/the `&HSTRING` names
// are live borrows that outlive each synchronous call, and `sa.lpSecurityDescriptor` stays valid
// because its backing `_psd` is held in scope for the whole block.
// - The header mapping is created AND viewed at `bytes == size_of::<SharedHeader>().max(64)`; the
// view's null is checked (`bail!` on failure, after which the owned `map` closes the mapping). The
// OS view base is page-aligned, so `section.ptr::<SharedHeader>()` is suitably aligned for a
// `SharedHeader`, and `write_bytes(.., 0, bytes)` plus the `(*header).field = ..` writes all stay
// within those `bytes` and write THROUGH the raw pointer without forming any `&mut`. The debug
// section is the same pattern at `dbg_bytes == size_of::<DebugBlock>()`, only entered when its
// own view is non-null.
// - The `magic` publish stores through `addr_of!((*header).magic) as *const AtomicU32`: `addr_of!`
// takes the field address without a reference; the field is a 4-aligned `u32` (valid for
// `AtomicU32`), and the `Release` store after the `Release` fence is the cross-process handshake
// that orders all preceding writes before the driver may observe `MAGIC`.
// - `header`/`dbg_block` point into the OS mappings, NOT into the `MappedSection` structs, so moving
// `section`/`dbg_section` into `me` leaves them valid (see the `MappedSection` doc comment).
unsafe {
// If we ENABLE advanced color for a 10-bit client, trust it (the driver will compose FP16) and
// size the ring FP16 directly — don't race the advanced_color_enabled poll, which may not have
@@ -504,7 +542,8 @@ impl IddPushCapturer {
recovering_since: None,
out_ring: Vec::new(),
out_idx: 0,
hdr_conv: None,
video_conv: None,
hdr_p010_conv: None,
last_seq: 0,
last_present: None,
status_logged: false,
@@ -523,7 +562,7 @@ impl IddPushCapturer {
/// Block (bounded) until the driver has ATTACHED to the host ring (`DRV_STATUS_OPENED`) **and published
/// a first frame**, else fail so the caller can fall back to DDA (audit §5.1 +
/// `docs/windows-host-rewrite.md` §2.5 — the GB1 game-capture fix).
/// `design/windows-host-rewrite.md` §2.5 — the GB1 game-capture fix).
///
/// Requiring the first frame — not just the attach — catches the *reconnect-into-a-broken-state* case:
/// a fullscreen game can leave the virtual display in a format/size that the driver's `publish()` guard
@@ -534,10 +573,16 @@ impl IddPushCapturer {
fn wait_for_attach(&self) -> Result<()> {
let deadline = Instant::now() + Duration::from_secs(4);
loop {
// Plain read: the driver writes this u32; an aligned u32 read can't tear (same access as
// SAFETY: `self.header` points into the live shared-header mapping this capturer owns (sized
// `>= size_of::<SharedHeader>()`, page-aligned), so the field read is in-bounds + aligned, and
// no reference into the shared region is formed. Plain read: the driver writes this `u32`
// cross-process, but an aligned `u32` read can't tear and `driver_status` is best-effort
// diagnostics — the real handshake is the atomic `magic`/`latest` (same access as
// log_driver_status_once).
let st = unsafe { (*self.header).driver_status };
if matches!(st, DRV_STATUS_TEX_FAIL | DRV_STATUS_NO_DEVICE1) {
// SAFETY: as above — an in-bounds, aligned `u32` read of a best-effort diagnostic field
// through the owned, live header mapping; no reference into the shared region is formed.
let detail = unsafe { (*self.header).driver_status_detail };
bail!(
"IDD-push driver failed to attach (driver_status={st} detail=0x{detail:08x} — \
@@ -560,6 +605,10 @@ impl IddPushCapturer {
#[inline]
fn latest(&self) -> u64 {
// SAFETY: `self.header` is the live, owned shared-header mapping (page-aligned, sized for a
// `SharedHeader`). `addr_of!((*self.header).latest)` forms the address of the `latest` field
// WITHOUT a reference; it is an 8-aligned `u64` (so valid for `AtomicU64`), and the `Acquire` load
// is the consumer half of the cross-process publish handshake (pairs with the driver's `Release`).
unsafe {
(*(std::ptr::addr_of!((*self.header).latest) as *const AtomicU64))
.load(Ordering::Acquire)
@@ -571,6 +620,10 @@ impl IddPushCapturer {
if self.status_logged {
return;
}
// SAFETY: four in-bounds, aligned reads of the live, owned shared-header mapping. The driver writes
// these `u32`/`i32` diagnostic fields cross-process, but aligned word reads can't tear and these are
// best-effort status (the real handshake is the atomic `magic`/`latest`); no `&`/`&mut` reference
// into the shared region is formed.
let (status, detail, lo, hi) = unsafe {
(
(*self.header).driver_status,
@@ -610,6 +663,11 @@ impl IddPushCapturer {
tracing::warn!("IDD push DEBUG: no debug block");
return;
}
// SAFETY: `self.dbg_block` was just checked non-null (the early return above); it points into the
// owned `dbg_section` mapping sized exactly `size_of::<DebugBlock>()` and page-aligned, so it is
// valid + aligned for `DebugBlock`. `d` is a short-lived SHARED reference used only to read the
// fields below; we never form `&mut` into this region, and the driver's cross-process writes are
// aligned `u32`s that don't tear (best-effort bring-up diagnostics).
let d = unsafe { &*self.dbg_block };
tracing::error!(
run_core_entries = d.run_core_entries,
@@ -625,16 +683,17 @@ impl IddPushCapturer {
);
}
/// The output texture format + the [`PixelFormat`] it presents as, driven SOLELY by the DISPLAY's
/// HDR state (like the WGC path): HDR → `Rgb10a2` BT.2020 PQ → NVENC Main10, and the client
/// auto-detects PQ from the HEVC VUI; SDR → 8-bit `Bgra`. We do NOT gate HDR on the client's
/// advertised `VIDEO_CAP_10BIT` — clients under-report it (e.g. the Mac advertises 10-bit only when
/// its OWN display is HDR), yet all decode Main10 + auto-switch, exactly as on the WGC path.
/// The output texture format + the [`PixelFormat`] NVENC encodes, driven SOLELY by the DISPLAY's HDR
/// state (like the WGC path): HDR → `P010` (BT.2020 PQ 10-bit limited) → NVENC Main10, and the client
/// auto-detects PQ from the HEVC VUI; SDR → `Nv12` (BT.709 8-bit limited). Both are native YUV so
/// NVENC skips its internal RGB→YUV CSC on the contended SM (plan §5.A). We do NOT gate HDR on the
/// client's advertised `VIDEO_CAP_10BIT` — clients under-report it (e.g. the Mac advertises 10-bit
/// only when its OWN display is HDR), yet all decode Main10 + auto-switch, exactly as on the WGC path.
fn out_format(&self) -> (DXGI_FORMAT, PixelFormat) {
if self.display_hdr {
(DXGI_FORMAT_R10G10B10A2_UNORM, PixelFormat::Rgb10a2)
(DXGI_FORMAT_P010, PixelFormat::P010)
} else {
(DXGI_FORMAT_B8G8R8A8_UNORM, PixelFormat::Bgra)
(DXGI_FORMAT_NV12, PixelFormat::Nv12)
}
}
@@ -658,6 +717,10 @@ impl IddPushCapturer {
self.height = new_h;
let fmt = self.ring_format();
let new_gen = IDD_GENERATION.fetch_add(1, Ordering::Relaxed);
// SAFETY: `create_ring_slots` is an `unsafe fn` (it makes D3D11/DXGI COM calls); we pass a live
// borrow of `self.device` (the capturer's own device, on which the slots are created) plus plain
// `u32`/`DXGI_FORMAT` values, and `?` propagates any failure before the slots are used. Every
// returned slot's texture + keyed mutex belongs to that same `self.device`.
let new_slots = unsafe {
Self::create_ring_slots(
&self.device,
@@ -668,6 +731,12 @@ impl IddPushCapturer {
fmt,
)?
};
// SAFETY: `self.header` is the live, owned shared-header mapping (page-aligned, sized for a
// `SharedHeader`). The `latest`/`generation` stores go through `addr_of!`-formed field pointers (no
// references) of correctly-aligned `u64`/`u32` fields, valid for `AtomicU64`/`AtomicU32`; the
// `dxgi_format`/`width`/`height` writes are in-bounds raw writes through the pointer (no `&mut`).
// The `Release` fence + the `Release` `generation` store publish all preceding writes so the driver
// only re-attaches (`Acquire`) once the new textures + format are in place.
unsafe {
// Clear `latest` to the 0 sentinel (generation 0, which try_consume rejects). The real guard
// against consuming an unwritten new-ring slot is the generation tag in `latest`: a stale
@@ -688,6 +757,8 @@ impl IddPushCapturer {
self.generation = new_gen;
self.last_seq = 0;
self.out_ring.clear(); // the output format changed → rebuild lazily at the new format
self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode
self.hdr_p010_conv = None;
self.out_idx = 0;
self.last_present = None;
Ok(())
@@ -701,9 +772,13 @@ impl IddPushCapturer {
return;
}
self.last_acm_poll = Instant::now();
// SAFETY: `advanced_color_enabled` is an `unsafe fn` taking only a copy of the plain `u32` target
// id; it performs a read-only CCD query and returns an owned `bool`, borrowing nothing from us.
let now_hdr = unsafe { crate::win_display::advanced_color_enabled(self.target_id) };
// Follow the display's ACTUAL resolution too — a fullscreen game can mode-set the virtual display
// out from under the negotiated size (game-capture bug GB1). Unknown read → keep our current size.
// SAFETY: `active_resolution` is an `unsafe fn` taking only a copy of the plain `u32` target id; it
// performs a read-only CCD query and returns owned `(w, h)` values, borrowing nothing from us.
let (now_w, now_h) = unsafe { crate::win_display::active_resolution(self.target_id) }
.unwrap_or((self.width, self.height));
if now_hdr == self.display_hdr && now_w == self.width && now_h == self.height {
@@ -742,31 +817,46 @@ impl IddPushCapturer {
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
// RENDER_TARGET: the VIDEO processor (NV12) and the P010 shader passes both write here, and
// NVENC registers it as encode input — matching the WGC YUV ring.
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
CPUAccessFlags: 0,
MiscFlags: 0,
};
for _ in 0..OUT_RING {
let mut t: Option<ID3D11Texture2D> = None;
let mut rtv: Option<ID3D11RenderTargetView> = None;
// SAFETY: `CreateTexture2D` is called on `self.device` (the capturer's live D3D11 device);
// `&desc` is a fully-initialized stack `D3D11_TEXTURE2D_DESC`, the data arg is `None` (no
// initial data), and `Some(&mut t)` is a live out-parameter the call fills. `?` rejects a failed
// HRESULT before `t` is unwrapped, and the created texture belongs to `self.device`.
unsafe {
self.device
.CreateTexture2D(&desc, None, Some(&mut t))
.context("CreateTexture2D(IDD out ring)")?;
let t = t.context("null out-ring texture")?;
self.device
.CreateRenderTargetView(&t, None, Some(&mut rtv))
.context("CreateRenderTargetView(IDD out ring)")?;
self.out_ring.push((t, rtv.context("null out-ring rtv")?));
self.out_ring.push(t.context("null out-ring texture")?);
}
}
Ok(())
}
/// Build the HDR converter if not already built (HDR-display path only — an SDR display is a copy).
/// Build the per-mode YUV converter if not already built: a VIDEO-engine BGRA→NV12 processor on an
/// SDR display, or the FP16→P010 shader on an HDR display. Both keep NVENC's RGB→YUV CSC off the SM.
fn ensure_converter(&mut self) -> Result<()> {
if self.hdr_conv.is_none() {
self.hdr_conv = Some(unsafe { HdrConverter::new(&self.device)? });
if self.display_hdr {
if self.hdr_p010_conv.is_none() {
// SAFETY: `HdrP010Converter::new` is `unsafe` (it compiles D3D11 shaders + creates
// resources); we pass a live borrow of `self.device`, the device the converter's resources
// belong to, and `?` propagates any failure before the converter is stored.
self.hdr_p010_conv = Some(unsafe { HdrP010Converter::new(&self.device)? });
}
} else if self.video_conv.is_none() {
// SAFETY: `VideoConverter::new` is `unsafe` (it sets up the D3D11 VIDEO processor); we pass live
// borrows of `self.device` + its immediate `self.context` (single-threaded, this thread) plus
// plain `u32` dimensions, and `?` propagates any failure before it is stored. The converter's
// resources belong to that same device/context.
self.video_conv = Some(unsafe {
VideoConverter::new(&self.device, &self.context, self.width, self.height, false)?
});
}
Ok(())
}
@@ -801,16 +891,11 @@ impl IddPushCapturer {
return Ok(None);
}
self.ensure_out_ring()?;
// Build the HDR converter BEFORE acquiring the slot so nothing between Acquire and Release can
// Build the converter BEFORE acquiring the slot so nothing between Acquire and Release can
// `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot).
if self.display_hdr {
self.ensure_converter()?;
}
self.ensure_converter()?;
let i = self.out_idx;
let (out, out_rtv) = {
let (t, rtv) = &self.out_ring[i];
(t.clone(), rtv.clone())
};
let out = self.out_ring[i].clone();
let (_, pf) = self.out_format();
// Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the
@@ -824,16 +909,27 @@ impl IddPushCapturer {
let Some(_lock) = KeyedMutexGuard::acquire(&s.mutex, 0, 8) else {
return Ok(None);
};
// SAFETY: convert/copy on the owning (encode) thread's immediate context, holding the slot lock.
// SAFETY: convert on the owning (encode) thread's immediate context, holding the slot lock.
// A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing
// the slot back to the driver.
unsafe {
if self.display_hdr {
// Sample the FP16 slot's SRV directly (no scratch copy) → BT.2020 PQ Rgb10a2.
if let Some(conv) = self.hdr_conv.as_ref() {
conv.convert(&self.context, &s.srv, &out_rtv, self.width, self.height);
// HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010.
if let Some(conv) = self.hdr_p010_conv.as_ref() {
conv.convert(
&self.device,
&self.context,
&s.srv,
&out,
self.width,
self.height,
)?;
}
} else {
// SDR: the slot is already 8-bit BGRA — one copy into the out-ring (hidden by pipelining).
self.context.CopyResource(&out, &s.tex);
// SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC.
if let Some(conv) = self.video_conv.as_ref() {
conv.convert(&s.tex, &out)?;
}
}
}
// `_lock` drops here → `ReleaseSync(0)`.
@@ -861,7 +957,7 @@ impl IddPushCapturer {
// OUT_RING(3) > the max pipeline_depth(2) guarantees the rotated slot is not in flight.
let (src, pf) = self.last_present.clone()?;
let i = self.out_idx;
let dst = self.out_ring.get(i)?.0.clone();
let dst = self.out_ring.get(i)?.clone();
// SAFETY: GPU copy on the owning thread's immediate context; src/dst are our out-ring textures of
// identical format/size (src is a previous out-ring slot; dst the next).
unsafe {
@@ -922,6 +1018,8 @@ pub fn spawn_observer(target: WinCaptureTarget, preferred: Option<(u32, u32, u32
/// The discrete render GPU LUID (where NVENC runs), falling back to the monitor's `OsAdapterLuid`.
fn resolve_render_adapter_luid_or(fallback_packed: i64) -> LUID {
// SAFETY: `resolve_render_adapter_luid` is an `unsafe fn` (it enumerates DXGI adapters) that takes no
// arguments and returns an owned `Option<LUID>`, borrowing nothing.
if let Some(l) = unsafe { crate::win_adapter::resolve_render_adapter_luid() } {
return l;
}
@@ -935,6 +1033,9 @@ impl Capturer for IddPushCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
let deadline = Instant::now() + Duration::from_secs(20);
loop {
// SAFETY: `self.event` is the live frame-ready `OwnedHandle` this capturer owns; its raw value
// (borrowed for the call, so it outlives this synchronous wait) is a valid auto-reset event
// handle. `WaitForSingleObject` only reads the handle; the 16 ms timeout bounds the wait.
let _ = unsafe { WaitForSingleObject(HANDLE(self.event.as_raw_handle()), 16) };
if let Some(f) = self.try_consume()? {
return Ok(f);
@@ -944,6 +1045,9 @@ impl Capturer for IddPushCapturer {
}
if Instant::now() > deadline {
self.log_debug_block();
// SAFETY: four in-bounds, aligned reads of the live, owned shared-header mapping — the same
// best-effort diagnostic fields as `log_driver_status_once` (aligned word reads can't tear;
// no reference into the shared region is formed).
let (st, detail, lo, hi) = unsafe {
(
(*self.header).driver_status,
@@ -16,6 +16,9 @@
//! Limitation: WGC cannot capture the secure desktop (lock / UAC / login) — the caller falls back to
//! the DDA backend ([`super::dxgi::DuplCapturer`]) for those (see capture.rs).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::dxgi::{
find_output, hdr_shader_p010_enabled, make_device, nudge_cursor_onto, D3d11Frame, HdrConverter,
HdrP010Converter, VideoConverter, WinCaptureTarget,
@@ -92,6 +95,10 @@ struct Deimpersonate(Option<HANDLE>);
impl Drop for Deimpersonate {
fn drop(&mut self) {
if let Some(tok) = self.0.take() {
// SAFETY: `RevertToSelf` takes no arguments and undoes the thread impersonation set during
// WGC activation; `tok` is the impersonation token `HANDLE` from `impersonate_active_user`,
// owned by this `Deimpersonate` and closed exactly once here (taken out of the `Option`, so
// no double-close). Both are FFI calls borrowing no Rust memory.
unsafe {
let _ = RevertToSelf();
let _ = CloseHandle(tok);
@@ -174,7 +181,12 @@ pub struct WgcCapturer {
_keepalive: Option<Box<dyn Send>>,
}
// COM + WinRT pointers; confined to the single owning (encode) thread, like DuplCapturer.
// SAFETY: like `DuplCapturer`. `WgcCapturer` holds D3D11 (free-threaded device/context) plus WGC WinRT
// objects (`Direct3D11CaptureFramePool` etc., created free-threaded via `CreateFreeThreaded`). COM/WinRT
// reference counting is interlocked, and the capturer is owned + used by exactly one encode thread,
// moved to it once and never shared (no `Sync`), so transferring ownership across threads is sound. The
// free-threaded `FrameArrived` callback touches only the `Arc<WgcSignal>` (itself `Send + Sync`), not
// the capturer's COM fields.
unsafe impl Send for WgcCapturer {}
impl WgcCapturer {
@@ -182,6 +194,15 @@ impl WgcCapturer {
/// [`attach_keepalive`](Self::attach_keepalive) only after open succeeds, so a failure leaves the
/// keepalive with the caller to hand to the DDA fallback.
pub fn open(target: WinCaptureTarget, preferred: Option<(u32, u32, u32)>) -> Result<Self> {
// SAFETY: runs on the thread opening the WGC session. `RoInitialize` inits this thread's WinRT
// apartment (idempotent; result ignored). `impersonate_active_user()` and `find_output()` are
// this module's `unsafe fn`s whose contracts (call on the activating thread; pass a GDI name)
// are met, and the impersonation is reverted by `_deimp`'s Drop on every return path. Every
// COM/WinRT call thereafter operates on an object obtained + `?`-checked earlier in this same
// block on this single thread — the `IDXGIOutput1` from `find_output`, the device/context from
// `make_device`, the factory/interop/item/pool/session — and the `TypedEventHandler` closure
// captures an `Arc<WgcSignal>` (Send+Sync) by move. No raw pointers are dereferenced; borrowed
// locals outlive their synchronous calls.
unsafe {
// WGC is WinRT — the calling thread needs a COM/WinRT apartment for the GraphicsCaptureItem
// activation factory (RoGetActivationFactory). Initialize MTA; ignore "already initialized"
@@ -585,6 +606,15 @@ impl WgcCapturer {
}
fn process_frame(&mut self, frame: Direct3D11CaptureFrame) -> Result<CapturedFrame> {
// SAFETY: runs on the capturer's single owning thread. `frame` is a live
// `Direct3D11CaptureFrame` from `self.pool`; `frame.Surface().cast::<IDirect3DDxgiInterfaceAccess
// >().GetInterface()` yields the frame's backing `ID3D11Texture2D`, which belongs to
// `self.device` (the pool was created on it via `CreateDirect3D11DeviceFromDXGIDevice`). Every
// helper called here — `hdr_to_p010`, `convert_to_yuv`, `ensure_fp16_src`, `ensure_out_ring`,
// `HdrConverter::convert`, `CopyResource`, `CreateRenderTargetView` — operates on
// `self.device`/`self.context` and that same-device texture, so all resources share one device.
// The frame is held in `self.held` until its async GPU read completes for the zero-copy paths.
// Single-threaded immediate-context use; borrowed textures/SRVs/RTVs outlive each synchronous call.
unsafe {
let surface = frame.Surface().context("frame Surface")?;
let access: IDirect3DDxgiInterfaceAccess = surface
@@ -1,5 +1,5 @@
//! Host-side WGC helper relay (Windows two-process secure-desktop design,
//! docs/windows-secure-desktop.md — step 4).
//! design/archive/windows-secure-desktop.md — step 4).
//!
//! WGC won't activate under the SYSTEM account, so the SYSTEM host can't capture the normal desktop
//! itself. Instead it spawns `punktfunk-host wgc-helper` in the **interactive user session** (so WGC works)
@@ -13,6 +13,9 @@
//! Wire framing (must match `wgc_helper::write_au`): per AU
//! `[u32 magic "PFAU" LE][u32 len LE][u64 pts_ns LE][u8 keyframe][len bytes data]`.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use crate::capture::dxgi::WinCaptureTarget;
use anyhow::{bail, Context, Result};
use std::io::{BufRead, BufReader, Read};
@@ -56,9 +59,15 @@ pub struct HelperRelay {
rx: Receiver<RelayAu>,
}
// HANDLEs are just kernel handle values; we own them for the relay's lifetime and close them on Drop.
// SAFETY: every field is itself `Send`: the `proc`/`thread` `HANDLE`s are process-global kernel
// handle values (plain integers valid from any thread, owned for the relay's lifetime and closed once
// on Drop), `stdin_w` is a `Mutex<HANDLE>`, and `rx` is an mpsc `Receiver<RelayAu>` (which is `Send`).
// The relay is moved to one thread and owned there, so transferring it across threads is sound.
unsafe impl Send for HelperRelay {}
unsafe impl Sync for HelperRelay {}
// NOTE: `HelperRelay` is deliberately NOT `Sync`. Its `rx: Receiver<RelayAu>` is `!Sync` (std mpsc
// is single-consumer), and the relay is only ever a single-owner local in the punktfunk1 two-process
// mux loop — never shared by `&` across threads — so `Sync` is neither sound nor needed. (A prior
// `unsafe impl Sync` here asserted more than the fields support; removed.)
/// Control byte on the helper's stdin: force the next encoded frame to be an IDR (client decode
/// recovery). Mirrors `enc.request_keyframe()` in the single-process path.
@@ -84,6 +93,10 @@ impl HelperRelay {
);
tracing::info!(cmd = %cmdline, "spawning WGC helper in user session");
// SAFETY: `spawn_inner` is an `unsafe fn` only because it drives raw Win32 token/pipe/process
// FFI; it imposes no caller-side memory precondition beyond valid arguments. `cmdline` is a live
// `&str` borrowed for the synchronous call and `(w, h, hz)` are plain `u32`s. It validates its
// own runtime requirements (active console session, SYSTEM token) and returns `Err` otherwise.
unsafe { spawn_inner(&cmdline, w, h, hz) }
}
@@ -108,6 +121,11 @@ impl HelperRelay {
pub fn request_keyframe(&self) {
let h = self.stdin_w.lock().unwrap();
let mut written = 0u32;
// SAFETY: `*h` is the host's write end of the helper's stdin pipe — a live `HANDLE` owned by
// this `HelperRelay` (held under the `stdin_w` Mutex, locked here), closed only in Drop.
// `WriteFile` reads the 1-byte `&[CTL_KEYFRAME]` buffer and writes the byte count into
// `written`; both are live locals that outlive the synchronous call. A failure (helper gone) is
// discarded as documented.
unsafe {
let _ = windows::Win32::Storage::FileSystem::WriteFile(
*h,
@@ -121,6 +139,10 @@ impl HelperRelay {
impl Drop for HelperRelay {
fn drop(&mut self) {
// SAFETY: `self.proc`/`self.thread` are the child process/thread `HANDLE`s from
// `CreateProcessAsUserW`, and `stdin_w` is the host's pipe write end — all owned by this
// `HelperRelay` and closed exactly once here in Drop (no double-close). `TerminateProcess` and
// the three `CloseHandle`s are FFI calls taking those handles by value, borrowing no Rust memory.
unsafe {
// Terminate the child first so its WGC capture + NVENC session tear down, then close our
// handles (the reader threads end on the resulting broken pipe).
@@ -364,10 +386,17 @@ fn au_reader(mut r: HandleReader, tx: SyncSender<RelayAu>) {
/// Minimal `Read` over a Win32 pipe HANDLE (the windows crate doesn't impl `Read` on HANDLE).
struct HandleReader(HANDLE);
// SAFETY: `HandleReader` owns a single pipe `HANDLE` (a process-global kernel handle value, valid from
// any thread). It is moved into the dedicated reader thread and used only there (and closed once on
// Drop), never shared — so transferring ownership across threads is sound.
unsafe impl Send for HandleReader {}
impl Read for HandleReader {
fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
let mut read = 0u32;
// SAFETY: `self.0` is the live read end of an anonymous pipe owned by this `HandleReader`
// (closed only in Drop). `ReadFile` fills the caller-provided `buf` (writing at most `buf.len()`
// bytes) and stores the count in `read`; both outlive the synchronous call. A broken pipe
// surfaces as `Err` and is mapped to EOF below.
let ok = unsafe {
windows::Win32::Storage::FileSystem::ReadFile(self.0, Some(buf), Some(&mut read), None)
};
@@ -380,6 +409,8 @@ impl Read for HandleReader {
}
impl Drop for HandleReader {
fn drop(&mut self) {
// SAFETY: `self.0` is the pipe `HANDLE` this `HandleReader` owns; `CloseHandle` (an FFI call
// taking the handle by value) is invoked exactly once here in Drop, so there is no double-close.
unsafe {
let _ = CloseHandle(self.0);
}
@@ -391,6 +422,13 @@ impl Drop for HandleReader {
pub fn running_as_system() -> bool {
use windows::Win32::Security::{GetTokenInformation, TokenUser, TOKEN_QUERY, TOKEN_USER};
use windows::Win32::System::Threading::{GetCurrentProcess, OpenProcessToken};
// SAFETY: `OpenProcessToken(GetCurrentProcess(), TOKEN_QUERY, &mut token)` opens the current-process
// token (the pseudo-handle is always valid) into `token`, which is closed once before each return.
// The first `GetTokenInformation` (null buffer) queries the required `len`; `buf` is then a
// `Vec<u8>` of exactly `len` bytes and the second call fills it, so `&*(buf.as_ptr() as *const
// TOKEN_USER)` reads a `TOKEN_USER` the kernel just wrote into a sufficiently-sized buffer (the
// variable-length SID it points at also lies within `buf`, which outlives the borrow).
// `is_local_system_sid` is this module's `unsafe fn`, given that in-buffer `PSID`. Safe on any thread.
unsafe {
let mut token = HANDLE::default();
if OpenProcessToken(GetCurrentProcess(), TOKEN_QUERY, &mut token).is_err() {
+7 -2
View File
@@ -4,10 +4,10 @@
//! environment before the host starts, and **for the knobs captured here the environment is constant for the
//! process lifetime**, so a lazily-parsed global is equivalent to "parsed once at startup".
//!
//! **Goal-1 stages 12** (`docs/windows-host-rewrite.md` §2.2): stage 1 stood this up; stage 2 migrated the
//! **Goal-1 stages 12** (`design/windows-host-rewrite.md` §2.2): stage 1 stood this up; stage 2 migrated the
//! genuinely-constant operator/dispatch knobs onto it (the dispatch-disagreement bug class: `idd_push`,
//! `capture_backend`, `encoder_pref`, `render_adapter`, `no_wgc`, the vdisplay backend select — plus the
//! plan-named `secure_dda`/`idd_depth`/`zerocopy`/`ten_bit` and the multi-site `perf`/`compositor`/
//! plan-named `secure_dda`/`idd_depth`/`zerocopy`/`ten_bit`/`four_four_four` and the multi-site `perf`/`compositor`/
//! `video_source`/`gamepad`). `SessionPlan` (stage 3) consumes it as the single owner of the
//! capture/topology/encoder decision.
//!
@@ -63,6 +63,10 @@ pub struct HostConfig {
pub zerocopy: bool,
/// `PUNKTFUNK_10BIT` — host policy gate for HEVC Main10 (only honored when the client also advertised 10-bit).
pub ten_bit: bool,
/// `PUNKTFUNK_444` — host policy gate for full-chroma HEVC 4:4:4 (Range Extensions). Honored only
/// when the client also advertised 4:4:4, the codec is HEVC, and the GPU/driver supports a 4:4:4
/// encode (probed) — otherwise the session stays 4:2:0. Independent of `ten_bit` (chroma vs depth).
pub four_four_four: bool,
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
pub perf: bool,
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic).
@@ -112,6 +116,7 @@ impl HostConfig {
.unwrap_or(2),
zerocopy: flag("PUNKTFUNK_ZEROCOPY"),
ten_bit: flag("PUNKTFUNK_10BIT"),
four_four_four: flag("PUNKTFUNK_444"),
perf: flag("PUNKTFUNK_PERF"),
video_source: val("PUNKTFUNK_VIDEO_SOURCE"),
compositor: val("PUNKTFUNK_COMPOSITOR"),
+137 -3
View File
@@ -3,6 +3,9 @@
//! RGB→YUV on the GPU, so no host-side CSC) and VAAPI on AMD/Intel (`*_vaapi`; the CPU-input
//! fallback swscales RGB→NV12, the zero-copy path imports the capture dmabuf straight into a
//! VA surface). One [`Encoder`] trait, selected in [`open_video`].
// Every unsafe block in this module tree carries a `// SAFETY:` proof; enforce it (unsafe-proof
// program). As a parent module this also covers the child modules (encode::windows/linux::*).
#![deny(clippy::undocumented_unsafe_blocks)]
use crate::capture::{CapturedFrame, PixelFormat};
use anyhow::Result;
@@ -26,6 +29,33 @@ pub enum Codec {
Av1,
}
/// Chroma subsampling the encoder emits, negotiated with the client (the `PUNKTFUNK_444` gate + the
/// client's `VIDEO_CAP_444` + a GPU probe). `Yuv420` is the universal default; `Yuv444` is HEVC-only,
/// native-protocol-only (GameStream stays 4:2:0), and the host only ever passes it after
/// [`can_encode_444`] confirmed the active backend supports it.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum ChromaFormat {
#[default]
Yuv420,
Yuv444,
}
impl ChromaFormat {
/// The HEVC `chroma_format_idc` this maps to: `1` (4:2:0) or `3` (4:4:4). Also the wire value
/// echoed in [`punktfunk_core::quic::Welcome::chroma_format`].
pub fn idc(self) -> u8 {
match self {
ChromaFormat::Yuv420 => punktfunk_core::quic::CHROMA_IDC_420,
ChromaFormat::Yuv444 => punktfunk_core::quic::CHROMA_IDC_444,
}
}
/// True for full-chroma 4:4:4.
pub fn is_444(self) -> bool {
matches!(self, ChromaFormat::Yuv444)
}
}
impl Codec {
/// The FFmpeg NVENC encoder name (selected by name, not codec id — the latter would
/// pick the software encoder).
@@ -86,6 +116,13 @@ pub struct EncoderCaps {
/// When `false`, `set_hdr_meta` is a no-op and no in-band grade reaches the client. Only the
/// Windows direct-NVENC path attaches it today.
pub supports_hdr_metadata: bool,
/// The opened encoder is actually producing a full-chroma 4:4:4 (`chroma_format_idc = 3`) stream.
/// `false` on every 4:2:0 session (the default) and on a backend that declined 4:4:4. Set by the
/// NVENC backends (Linux + Windows). The chroma is committed to the wire (`Welcome::chroma_format`)
/// from the pre-open probe, so this is a *post-open cross-check*: the session glue logs loudly if
/// the encoder's real chroma disagrees with what was negotiated (the in-band SPS is authoritative
/// for the decoder either way).
pub chroma_444: bool,
}
/// A hardware encoder. One per session; runs on the encode thread.
@@ -190,8 +227,21 @@ pub fn open_video(
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Box<dyn Encoder>> {
validate_dimensions(codec, width, height)?;
// 4:4:4 is HEVC-only. The negotiator should never pass `Yuv444` for another codec (it gates on
// `codec == H265`), but defend the contract here so a future caller can't silently emit a stream
// no decoder expects: a non-HEVC 4:4:4 request degrades to 4:2:0 with a warning.
let chroma = if chroma.is_444() && codec != Codec::H265 {
tracing::warn!(
?codec,
"4:4:4 requested for a non-HEVC codec — encoding 4:2:0"
);
ChromaFormat::Yuv420
} else {
chroma
};
#[cfg(target_os = "linux")]
{
// Pick the GPU encode backend. NVIDIA → NVENC/CUDA (the original path, unchanged);
@@ -200,8 +250,17 @@ pub fn open_video(
// its errors crisply instead of silently trying the other).
let pref = crate::config::config().encoder_pref.as_str();
let open_vaapi = || -> Result<Box<dyn Encoder>> {
vaapi::VaapiEncoder::open(codec, format, width, height, fps, bitrate_bps, bit_depth)
.map(|e| Box::new(e) as Box<dyn Encoder>)
vaapi::VaapiEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| Box::new(e) as Box<dyn Encoder>)
};
match pref {
"nvenc" | "nvidia" | "cuda" => open_nvenc_probed(
@@ -213,6 +272,7 @@ pub fn open_video(
bitrate_bps,
cuda,
bit_depth,
chroma,
),
"vaapi" | "amd" | "intel" => open_vaapi(),
"auto" | "" => {
@@ -228,6 +288,7 @@ pub fn open_video(
bitrate_bps,
cuda,
bit_depth,
chroma,
)
} else {
open_vaapi()
@@ -257,6 +318,7 @@ pub fn open_video(
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| Box::new(e) as Box<dyn Encoder>)
}
@@ -286,6 +348,7 @@ pub fn open_video(
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| Box::new(e) as Box<dyn Encoder>)
}
@@ -330,6 +393,7 @@ pub fn open_video(
bitrate_bps,
cuda,
bit_depth,
chroma,
);
anyhow::bail!("video encode requires Linux or Windows")
}
@@ -352,6 +416,7 @@ fn open_nvenc_probed(
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Box<dyn Encoder>> {
const MIN_PROBE_BPS: u64 = 50_000_000;
let mut candidates = vec![bitrate_bps];
@@ -366,7 +431,9 @@ fn open_nvenc_probed(
}
let mut last: Option<anyhow::Error> = None;
for (i, &b) in candidates.iter().enumerate() {
match linux::NvencEncoder::open(codec, format, width, height, fps, b, cuda, bit_depth) {
match linux::NvencEncoder::open(
codec, format, width, height, fps, b, cuda, bit_depth, chroma,
) {
Ok(enc) => {
if i > 0 {
tracing::warn!(
@@ -443,6 +510,65 @@ pub fn vaapi_codec_support() -> CodecSupport {
})
}
/// Whether the active GPU encode backend can actually produce a full-chroma **4:4:4** HEVC stream.
/// Resolved (and cached, once) *before* the Welcome so the host advertises the chroma it will really
/// encode — the honest-downgrade channel. 4:4:4 is HEVC-only; the probe opens a tiny encoder on the
/// active backend (NVENC FREXT is broad on NVIDIA, but VAAPI / AMF / QSV 4:4:4 is hardware-specific,
/// so it must be probed, never assumed). Non-HEVC codecs are always `false`.
#[cfg(any(target_os = "linux", target_os = "windows"))]
pub fn can_encode_444(codec: Codec) -> bool {
use std::sync::OnceLock;
if codec != Codec::H265 {
return false;
}
static CACHE: OnceLock<bool> = OnceLock::new();
*CACHE.get_or_init(|| {
let supported = {
#[cfg(target_os = "linux")]
{
// Mirror open_video's backend dispatch: VAAPI (AMD/Intel) vs NVENC (NVIDIA).
if linux_zero_copy_is_vaapi() {
vaapi::probe_can_encode_444(codec)
} else {
linux::probe_can_encode_444(codec)
}
}
#[cfg(target_os = "windows")]
{
match windows_resolved_backend() {
WindowsBackend::Nvenc => {
#[cfg(feature = "nvenc")]
{
nvenc::probe_can_encode_444(codec)
}
#[cfg(not(feature = "nvenc"))]
{
false
}
}
WindowsBackend::Amf | WindowsBackend::Qsv => {
#[cfg(feature = "amf-qsv")]
{
let vendor = match windows_resolved_backend() {
WindowsBackend::Qsv => ffmpeg_win::WinVendor::Qsv,
_ => ffmpeg_win::WinVendor::Amf,
};
ffmpeg_win::probe_can_encode_444(vendor, codec)
}
#[cfg(not(feature = "amf-qsv"))]
{
false
}
}
WindowsBackend::Software => false,
}
}
};
tracing::info!(supported, "HEVC 4:4:4 encode capability probed");
supported
})
}
// ---------------------------------------------------------------------------------------------
// Windows backend selection (the analogue of the Linux nvidia_present / linux_zero_copy_is_vaapi
// logic). NVIDIA → NVENC, AMD → AMF, Intel → QSV; `auto` (default) reads the DXGI adapter vendor.
@@ -505,6 +631,14 @@ fn windows_gpu_vendor() -> Option<GpuVendor> {
CreateDXGIFactory1, IDXGIFactory1, DXGI_ADAPTER_FLAG_SOFTWARE,
};
static CACHE: OnceLock<Option<GpuVendor>> = OnceLock::new();
// SAFETY: `CreateDXGIFactory1` returns a fresh owned `IDXGIFactory1` COM object (refcounted by the
// windows-rs wrapper, Released when the local drops); `.ok()?` bails on failure so `factory` is a
// valid interface before any use. `EnumAdapters1(i)` hands back the i-th adapter as an owned
// `IDXGIAdapter1` (or an error past the last adapter, which ends the loop). `GetDesc1()` returns the
// `DXGI_ADAPTER_DESC1` by value (no out-pointer), so reading `desc.Flags`/`desc.VendorId` is plain
// field access. Every call only touches COM objects this closure owns; the `OnceLock` runs the
// closure once (no data race) and all interfaces are Released as the locals drop. No raw pointer is
// dereferenced and nothing is aliased.
*CACHE.get_or_init(|| unsafe {
let factory: IDXGIFactory1 = CreateDXGIFactory1().ok()?;
let mut i = 0u32;
+257 -10
View File
@@ -8,8 +8,10 @@
//! does *not* accept — we expand it to `rgb0` (one padding byte/pixel, no colour math).
//! The encoder is opened *without* a global header so VPS/SPS/PPS are emitted in-band on
//! every IDR — the output is both a playable raw Annex-B stream and self-contained AUs.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{Codec, EncodedFrame, Encoder};
use super::{ChromaFormat, Codec, EncodedFrame, Encoder};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use ffmpeg::format::Pixel;
@@ -17,9 +19,33 @@ use ffmpeg::util::frame::Video as VideoFrame;
use ffmpeg::{codec, encoder, Dictionary, Packet, Rational};
use ffmpeg_next as ffmpeg;
use std::os::raw::c_int;
use std::ptr;
use ffmpeg::ffi; // = ffmpeg_sys_next
/// swscale: nearest-neighbour scaler flag (`SWS_POINT`). We never rescale (src dims == dst dims), so
/// the resampler choice only governs the colour-conversion path; POINT is the cheapest.
const SWS_POINT: c_int = 0x10;
/// swscale colorspace id for ITU-R BT.709 (`SWS_CS_ITU709`) — the CSC coefficients for our RGB→YUV.
const SWS_CS_ITU709: c_int = 1;
/// The swscale *source* pixel format for a captured packed RGB/BGR layout (the real byte order, not
/// the NVENC-padded `*0` form). Used by the 4:4:4 RGB→YUV444P conversion path. Mirrors the VAAPI
/// CPU-input mapping; YUV/10-bit inputs can't feed this path (the 4:4:4 session forces packed RGB).
fn sws_src_pixel(format: PixelFormat) -> Result<Pixel> {
Ok(match format {
PixelFormat::Bgrx => Pixel::BGRZ, // bgr0
PixelFormat::Rgbx => Pixel::RGBZ, // rgb0
PixelFormat::Bgra => Pixel::BGRA,
PixelFormat::Rgba => Pixel::RGBA,
PixelFormat::Rgb => Pixel::RGB24,
PixelFormat::Bgr => Pixel::BGR24,
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 => {
bail!("NVENC 4:4:4 CPU-input path supports packed RGB/BGR only; got {format:?}")
}
})
}
/// `AVCUDADeviceContext` (libavutil/hwcontext_cuda.h) — not in the ffmpeg-sys bindings (the
/// crate doesn't allowlist that header), so mirror its stable 3-pointer layout. We set the
/// first field to *our* `CUcontext` so NVENC shares the context the EGL importer maps into.
@@ -79,6 +105,12 @@ impl CudaHw {
impl Drop for CudaHw {
fn drop(&mut self) {
// SAFETY: `frames_ref`/`device_ref` are the two non-null `AVBufferRef`s `CudaHw::new` created
// (it bails before returning `Self` if either alloc fails, so a live `CudaHw` always holds
// both). `av_buffer_unref` drops one reference and nulls the pointer through the `&mut`. This
// `Drop` runs exactly once and `CudaHw` owns these refs exclusively → no double-free /
// use-after-free. Frames are unref'd before the device (the frames ctx internally refs the
// device; refcounted, so the order is sound regardless).
unsafe {
ffi::av_buffer_unref(&mut self.frames_ref);
ffi::av_buffer_unref(&mut self.device_ref);
@@ -123,6 +155,10 @@ pub struct NvencEncoder {
frame: Option<VideoFrame>,
/// Zero-copy path: CUDA hwdevice/hwframes contexts (the encoder takes `AV_PIX_FMT_CUDA`).
cuda: Option<CudaHw>,
/// 4:4:4 path only: swscale context converting the captured packed RGB/BGR → planar YUV444P
/// (BT.709 limited) into [`Self::frame`], because `hevc_nvenc` only emits 4:4:4 from a YUV444
/// *input* (RGB-in is always 4:2:0). `None` on the ordinary 4:2:0 RGB path. Freed in `Drop`.
sws_444: Option<*mut ffi::SwsContext>,
src_format: PixelFormat,
expand: bool,
width: u32,
@@ -134,8 +170,17 @@ pub struct NvencEncoder {
force_kf: bool,
}
// `CudaHw` holds raw `AVBufferRef`s; the encoder lives on a single thread. The CPU encoder is
// already `Send` via ffmpeg-next; assert it for the CUDA fields too.
// `CudaHw` holds raw `AVBufferRef`s and `sws_444` a raw `SwsContext`; the encoder lives on a single
// thread. The CPU encoder is already `Send` via ffmpeg-next; assert it for the raw fields too.
// SAFETY: `NvencEncoder` owns an ffmpeg-next `Encoder`/`VideoFrame` (already `Send`) plus a `CudaHw`
// holding raw `AVBufferRef`s and an optional raw `SwsContext`, none of which are `Send` by default.
// The `SwsContext` is a self-contained swscale state object with no thread affinity, touched only
// through `&mut self` on the one encode thread. The encoder is owned and driven by
// exactly ONE thread — the per-session encode thread it is moved to — and is only touched through
// `&mut self` methods, so it is never aliased or accessed concurrently. The wrapped libav contexts
// (and the shared `CUcontext` the `CudaHw` references) have no thread affinity, so transferring
// ownership across threads is sound. This asserts `Send` (transfer) only, extending ffmpeg-next's
// existing `Send` to the raw CUDA fields; `Sync` (shared `&`) is deliberately NOT implemented.
unsafe impl Send for NvencEncoder {}
impl NvencEncoder {
@@ -149,6 +194,7 @@ impl NvencEncoder {
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Self> {
// TODO(hdr): Linux 10-bit parity. Unlike the Windows raw-SDK path (which upconverts 8-bit
// ARGB → Main10 via pixelBitDepthMinus8), libavcodec hevc_nvenc needs a 10-bit input pixel
@@ -160,14 +206,36 @@ impl NvencEncoder {
"Linux NVENC 10-bit not yet wired — encoding 8-bit"
);
}
// Full-chroma 4:4:4 (HEVC Range Extensions). `hevc_nvenc` only emits 4:4:4 from a YUV444
// *input* frame — feeding RGB always subsamples to 4:2:0 regardless of profile (verified on
// the RTX 5070 Ti). So a 4:4:4 session swscales the captured RGB → YUV444P (BT.709 limited)
// and feeds that with `profile=rext`. The negotiator gates this to HEVC + the single-process
// CPU-capture topology, so `cuda` must be false here; defend the contract.
let want_444 = chroma.is_444() && codec == Codec::H265;
if want_444 && cuda {
bail!(
"NVENC 4:4:4 needs CPU RGB frames (the session forces non-zero-copy capture for \
4:4:4); got a CUDA frame capture/encoder negotiation mismatch"
);
}
ffmpeg::init().context("ffmpeg init")?;
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
// SAFETY: `av_log_set_level` sets libav's global integer log level; `48` (= AV_LOG_DEBUG)
// is a valid level with no pointer args, and libav was just initialized by `ffmpeg::init()`
// above — always sound.
unsafe { ffi::av_log_set_level(48) }; // AV_LOG_DEBUG — surface NVENC hw-frame rejects
}
let name = codec.nvenc_name();
let av_codec = encoder::find_by_name(name)
.ok_or_else(|| anyhow!("{name} not built into libavcodec"))?;
let (nvenc_pixel, expand) = nvenc_input(format);
let (rgb_pixel, rgb_expand) = nvenc_input(format);
// 4:4:4 feeds NVENC a planar YUV444P frame we produce by swscale; the ordinary path feeds the
// captured RGB straight in and lets NVENC's internal CSC subsample to 4:2:0.
let (nvenc_pixel, expand) = if want_444 {
(Pixel::YUV444P, false)
} else {
(rgb_pixel, rgb_expand)
};
let mut video = codec::context::Context::new_with_codec(av_codec)
.encoder()
@@ -195,6 +263,11 @@ impl NvencEncoder {
.unwrap_or(1.0);
let vbv_bits = ((bitrate_bps as f64 / fps.max(1) as f64) * vbv_frames as f64)
.clamp(1.0, i32::MAX as f64);
// SAFETY: `video` is the ffmpeg-next encoder builder wrapping a freshly-allocated
// `AVCodecContext` that we hold by value and have not opened yet; `video.as_mut_ptr()` returns
// that non-null, properly-aligned, exclusively-owned context. Writing the plain `rc_buffer_size`
// int field before `open_with` is the supported way to set a field ffmpeg-next exposes no
// setter for. Sole owner → no aliasing; synchronous in-bounds scalar write.
unsafe {
(*video.as_mut_ptr()).rc_buffer_size = vbv_bits as i32;
}
@@ -204,16 +277,23 @@ impl NvencEncoder {
// "freeze". NVENC emits one IDR at stream start, then P-frames only; `forced-idr` (below)
// turns a client recovery request (RFI, via `request_keyframe`) into an IDR on demand.
// This is the Moonlight/Sunshine low-latency model.
// SAFETY: same `video` builder as above — a non-null, properly-aligned, sole-owned, not-yet-
// opened `AVCodecContext`. We write the plain `gop_size` int field (= -1, infinite GOP) before
// `open_with`, which ffmpeg-next has no setter for. No aliasing; synchronous scalar write.
unsafe {
(*video.as_mut_ptr()).gop_size = -1;
}
// NV12 path: we did the RGB→YUV conversion ourselves as BT.709 *limited* range, so signal
// that in the bitstream VUI (colorspace/range/primaries/transfer) — otherwise the client
// decoder assumes a default and the picture comes out washed-out / wrong-contrast. The
// RGB-input paths leave these unset (NVENC's internal CSC writes its own VUI). Matches the
// Windows NV12 path's BT.709 limited-range signalling.
if matches!(format, PixelFormat::Nv12) {
// NV12 / 4:4:4 paths: we do the RGB→YUV conversion ourselves as BT.709 *limited* range
// (swscale), so signal that in the bitstream VUI (colorspace/range/primaries/transfer) —
// otherwise the client decoder assumes a default and the picture comes out washed-out /
// wrong-contrast. The RGB-input 4:2:0 path leaves these unset (NVENC's internal CSC writes
// its own VUI). Matches the Windows NV12 path's BT.709 limited-range signalling.
if matches!(format, PixelFormat::Nv12) || want_444 {
// SAFETY: same `video` builder — `raw = video.as_mut_ptr()` is the non-null, properly-
// aligned, sole-owned, not-yet-opened `AVCodecContext`. We set its four VUI colour enum
// fields to valid `AVColorSpace`/`AVColorRange`/`AVColorPrimaries`/`AVColorTransfer-
// Characteristic` variants before `open_with`. Sole owner → no aliasing; synchronous writes.
unsafe {
let raw = video.as_mut_ptr();
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT709;
@@ -228,7 +308,17 @@ impl NvencEncoder {
// *before* open (NVENC derives the device from `hw_frames_ctx`).
let cuda_hw = if cuda {
let cu_ctx = crate::zerocopy::cuda::context().context("shared CUDA context")?;
// SAFETY: `CudaHw::new` (an `unsafe fn`) requires libav initialized (the `ffmpeg::init()`
// above ran) and a valid `CUcontext`; `cu_ctx` is the shared importer context from
// `zerocopy::cuda::context()?`, non-null on the `Ok` path. `nvenc_pixel` is a valid `Pixel`
// and `width`/`height` are the validated positive dims. It returns a RAII `CudaHw` wrapping
// (not owning) `cu_ctx` and owning two `AVBufferRef`s freed on drop.
let hw = unsafe { CudaHw::new(cu_ctx, nvenc_pixel, width, height)? };
// SAFETY: `raw = video.as_mut_ptr()` is the non-null, sole-owned, not-yet-opened
// `AVCodecContext`. We set `pix_fmt = CUDA` and attach NEW refs (`av_buffer_ref`) of
// `hw.device_ref`/`hw.frames_ref` — both non-null (`CudaHw::new` guarantees) and from the
// live `hw`, which is moved into `NvencEncoder.cuda` next to `enc` and so outlives the
// encoder. The context owns its own refs (freed when the context closes). No aliasing.
unsafe {
let raw = video.as_mut_ptr();
(*raw).pix_fmt = ffi::AVPixelFormat::AV_PIX_FMT_CUDA;
@@ -240,6 +330,45 @@ impl NvencEncoder {
None
};
// 4:4:4: build the RGB→YUV444P swscale (BT.709 limited, no rescale). Mirrors the VAAPI CPU
// path's RGB→NV12 scaler, but the dst is full-chroma planar 4:4:4.
let sws_444 = if want_444 {
let src_av = pixel_to_av(sws_src_pixel(format)?);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated, packed-RGB-only source), the
// dst is YUV444P. The trailing filter/param pointers are null = "use defaults" (documented
// as accepted). No Rust memory is borrowed; the returned pointer is null-checked below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
height as c_int,
src_av,
width as c_int,
height as c_int,
ffi::AVPixelFormat::AV_PIX_FMT_YUV444P,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
ptr::null(),
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→YUV444P) failed");
}
// SAFETY: `sws` is the non-null context from the call above (null-checked). The ITU-709
// coefficient table from `sws_getCoefficients` is a process-lifetime libswscale static,
// reused for src+dst matrices; `sws_setColorspaceDetails` only reads it and writes scalar
// CSC settings into `sws` (limited-range dst: dstRange = 0). No Rust memory is passed.
unsafe {
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, 0, 0, 1 << 16, 1 << 16);
}
Some(sws)
} else {
None
};
// Low-latency NVENC tuning (plan §7 / linux-setup doc).
let mut opts = Dictionary::new();
opts.set("preset", "p1"); // fastest
@@ -248,6 +377,12 @@ impl NvencEncoder {
opts.set("bf", "0");
opts.set("delay", "0");
opts.set("forced-idr", "1"); // RFI/request_keyframe → real IDR under the infinite GOP
if want_444 {
// HEVC Range Extensions — the profile that carries chroma_format_idc=3. With a YUV444P
// input `hevc_nvenc` auto-selects it, but pin it explicitly so the chroma is never silently
// dropped on a future libavcodec.
opts.set("profile", "rext");
}
// Split-frame encode across both NVENC engines (GB203 has 2) when the pixel rate exceeds
// a single engine's HEVC capacity (~1 Gpix/s); e.g. 5120x1440@240 = 1.77 Gpix/s needs it,
@@ -281,6 +416,7 @@ impl NvencEncoder {
enc,
frame,
cuda: cuda_hw,
sws_444,
src_format: format,
expand,
width,
@@ -293,6 +429,15 @@ impl NvencEncoder {
}
impl Encoder for NvencEncoder {
fn caps(&self) -> super::EncoderCaps {
super::EncoderCaps {
// 4:4:4 iff this session opened the RGB→YUV444P swscale path (FREXT). RFI/HDR-SEI stay
// unsupported on libavcodec NVENC (the trait defaults).
chroma_444: self.sws_444.is_some(),
..super::EncoderCaps::default()
}
}
fn submit(&mut self, captured: &CapturedFrame) -> Result<()> {
anyhow::ensure!(
captured.width == self.width && captured.height == self.height,
@@ -371,6 +516,47 @@ impl NvencEncoder {
bytes.len(),
src_row * h
);
// 4:4:4: swscale the packed RGB straight into the planar YUV444P input frame (BT.709 limited),
// then send it — no byte-expand. The 4:2:0 RGB path (below) feeds NVENC packed RGB directly.
if let Some(sws) = self.sws_444 {
let frame = self
.frame
.as_mut()
.context("CPU frame missing (encoder opened in CUDA mode)")?;
// SAFETY: `format == self.src_format` and `bytes.len() >= src_row * h` (the `ensure!`s
// above), so `sws_scale` reads `h` rows of `src_row` bytes from `src_data[0] = bytes`
// (packed RGB is single-plane; the other src planes are null/0) — all in bounds. `sws` is
// the non-null context built in `open`. The dst is `frame`'s underlying `AVFrame`: its
// `data`/`linesize` in-struct arrays were sized for YUV444P by `VideoFrame::new`, and the
// 3 planes are each `width`×`height`. All pointers are live locals for this synchronous
// call; the encoder runs only on this thread (`unsafe impl Send`), so no aliasing/race.
unsafe {
let dst_av = frame.as_mut_ptr();
let src_data: [*const u8; 4] =
[bytes.as_ptr(), ptr::null(), ptr::null(), ptr::null()];
let src_stride: [c_int; 4] = [src_row as c_int, 0, 0, 0];
let r = ffi::sws_scale(
sws,
src_data.as_ptr(),
src_stride.as_ptr(),
0,
h as c_int,
(*dst_av).data.as_ptr(),
(*dst_av).linesize.as_ptr(),
);
if r < 0 {
bail!("sws_scale(RGB→YUV444P) failed ({r})");
}
}
frame.set_pts(Some(pts));
frame.set_kind(if idr {
ffmpeg::picture::Type::I
} else {
ffmpeg::picture::Type::None
});
self.enc.send_frame(frame).context("send_frame(444)")?;
return Ok(());
}
let frame = self
.frame
.as_mut()
@@ -428,6 +614,19 @@ impl NvencEncoder {
// The device→device copy below uses our shared context directly; make it current on the
// encode thread (ffmpeg pushes its own around the pool alloc, so order is fine).
crate::zerocopy::cuda::make_current().context("CUDA context current (encode thread)")?;
// SAFETY: `frames_ref` is the non-null CUDA frames ctx from `self.cuda` (unwrapped via
// `.context(..)?` above), and the shared CUDA context was just made current on THIS thread
// (`make_current()?`), the precondition for the device-pointer copies below.
// * `av_frame_alloc` → `f` (null-checked). `av_hwframe_get_buffer(frames_ref, f, 0)` fills `f`
// with a pooled CUDA surface (sets `data[]`/`linesize[]`/`buf[0]`/`hw_frames_ctx`); on
// failure we free `f` and bail.
// * For NV12 we read `(*f).data[0..2]` / `linesize[0..2]` (Y + interleaved UV), else
// `data[0]`/`linesize[0]` — in-struct fields of the non-null `f`, valid for the surface dims
// ffmpeg allocated — and pass them to the cuda copy helpers, which device→device copy `buf`
// (the imported `DeviceBuffer`, owned by the caller and live for this call) into the surface.
// * On copy error we free `f` and return. Otherwise we write `pts`/`pict_type` through `f` and
// `avcodec_send_frame` it into the live owned `self.enc` context (which takes its own ref of
// the pooled surface), then free our `f` ref exactly once. Single-threaded encoder → no race.
unsafe {
let mut f = ffi::av_frame_alloc();
if f.is_null() {
@@ -473,3 +672,51 @@ impl NvencEncoder {
Ok(())
}
}
impl Drop for NvencEncoder {
fn drop(&mut self) {
if let Some(sws) = self.sws_444.take() {
// SAFETY: `sws` is the non-null `SwsContext` allocated by `sws_getContext` in `open` and
// owned exclusively by this encoder (taken out of the field so it can't be freed twice).
// `sws_freeContext` frees it; nothing else references it after this single-threaded drop.
unsafe { ffi::sws_freeContext(sws) };
}
}
}
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode HEVC **4:4:4** (Range
/// Extensions). Opens a tiny real `hevc_nvenc` 4:4:4 session — the exact path [`NvencEncoder::open`]
/// takes for a live 4:4:4 stream — and reports whether it succeeded. HEVC-only; the result is cached
/// by the caller ([`crate::encode::can_encode_444`]). A GPU/driver/ffmpeg without RExt 4:4:4 fails
/// the open here, so the host resolves the session to 4:2:0 before the Welcome (honest downgrade).
pub fn probe_can_encode_444(codec: Codec) -> bool {
if codec != Codec::H265 {
return false;
}
if ffmpeg::init().is_err() {
return false;
}
// Quiet ffmpeg's open error on a GPU that lacks 4:4:4 — the probe failing is an expected outcome.
// SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
// (no pointer args) and are always sound post-init.
let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
let ok = NvencEncoder::open(
codec,
PixelFormat::Bgra,
640,
480,
30,
2_000_000,
false, // CPU input (the 4:4:4 path never uses CUDA)
8,
ChromaFormat::Yuv444,
)
.is_ok();
// SAFETY: restore the saved global log level (scalar arg, no pointers).
unsafe { ffi::av_log_set_level(prev) };
ok
}
+169 -3
View File
@@ -19,6 +19,8 @@
//! hwdevice/hwframes/buffersrc/buffersink calls go through `ffmpeg::ffi` (= `ffmpeg_sys_next`),
//! as the CUDA encode path and the clients' decode paths already do. The encoder is opened
//! *without* a global header, so VPS/SPS/PPS are in-band on every IDR.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{Codec, EncodedFrame, Encoder};
use crate::capture::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat};
@@ -133,6 +135,14 @@ pub fn probe_can_encode(codec: Codec) -> bool {
if ffmpeg::init().is_err() {
return false;
}
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `av_log_get_level`/
// `av_log_set_level` only read/write libav's global integer log level (no pointer args) and are
// always sound to call post-init. `VaapiHw::new` (an `unsafe fn`) builds a VAAPI device + NV12
// frames pool from the literal NV12/640x480/pool=2 args and hands back a RAII handle that unrefs
// both `AVBufferRef`s on drop. `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/
// `hw.frames_ref` — the two non-null refs `VaapiHw::new` just created — and `av_buffer_ref`s them
// into the encoder; `hw` is a live local for the whole match arm, so the borrows outlive the
// synchronous call, and both `hw` and the probe encoder are dropped (RAII) when the arm ends.
unsafe {
// A missing VA device (non-VAAPI host, GPU-less CI) is an expected probe outcome — quiet
// ffmpeg's "No VA display found" error for the probe, then restore the level.
@@ -150,6 +160,18 @@ pub fn probe_can_encode(codec: Codec) -> bool {
}
}
/// Whether the active VAAPI GPU can encode HEVC **4:4:4** (Range Extensions). **Deferred in v1 —
/// always `false`.** VAAPI HEVC 4:4:4 encode is narrow and vendor-specific (the lab's AMD Phoenix1 /
/// RDNA3 exposes only `VAProfileHEVCMain`/`Main10` `EncSlice`, no `Main444`), and there is no
/// validated hardware to build + verify the 4:4:4 surface/profile path against. Returning `false`
/// keeps the negotiation honest: a VAAPI host resolves every session to 4:2:0 before the Welcome, so
/// the client never builds a 4:4:4 decoder it would only get 4:2:0 frames for. (Follow-up: implement
/// + validate on an Intel Arc / RDNA4-class box that advertises a HEVC 4:4:4 encode entrypoint.)
pub fn probe_can_encode_444(_codec: Codec) -> bool {
tracing::info!("VAAPI HEVC 4:4:4 encode is not implemented yet — declining (encoding 4:2:0)");
false
}
/// Drain the encoder for one packet (shared poll logic).
fn poll_encoder(enc: &mut encoder::video::Encoder, fps: u32) -> Result<Option<EncodedFrame>> {
let mut pkt = Packet::empty();
@@ -224,6 +246,12 @@ impl VaapiHw {
impl Drop for VaapiHw {
fn drop(&mut self) {
// SAFETY: `frames_ref`/`device_ref` are the two non-null `AVBufferRef`s `VaapiHw::new`
// created (it bails before constructing `Self` if either alloc fails, so a live `VaapiHw`
// always holds both). `av_buffer_unref` drops one reference and nulls the pointer through the
// `&mut`. This `Drop` runs exactly once and `VaapiHw` owns these refs exclusively, so there
// is no double-free / use-after-free. Frames are unref'd before the device because the frames
// ctx internally holds a ref on the device (refcounted, so the order is sound either way).
unsafe {
ffi::av_buffer_unref(&mut self.frames_ref);
ffi::av_buffer_unref(&mut self.device_ref);
@@ -252,7 +280,16 @@ impl CpuInner {
) -> Result<Self> {
let src_pixel = vaapi_sws_src(format)?;
const POOL: c_int = 16;
// SAFETY: `VaapiHw::new` (an `unsafe fn`) requires libav initialized — guaranteed because the
// only path here is `VaapiEncoder::open` → `ensure_inner` → `CpuInner::open`, and `open` ran
// `ffmpeg::init()`. The args are valid: NV12 sw_format, the validated positive `width`/`height`,
// pool=16. It returns a RAII `VaapiHw` that unrefs its two `AVBufferRef`s on drop.
let hw = unsafe { VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, width, height, POOL)? };
// SAFETY: `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` — both
// non-null (`VaapiHw::new` guarantees it) and from the `hw` just built above, which is a live
// local that outlives this synchronous call. The fn `av_buffer_ref`s them into the encoder, so
// the encoder holds its own references; `hw` is also moved into the returned `CpuInner` next to
// `enc`, keeping the device/frames alive for the encoder's whole lifetime.
let enc = unsafe {
open_vaapi_encoder(
codec,
@@ -266,6 +303,12 @@ impl CpuInner {
};
// swscale RGB→NV12, BT.709 limited (matches the VUI), no rescale.
let src_av = pixel_to_av(src_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dimensions and
// pixel formats. All four dims are the encoder's positive `width`/`height` cast to `c_int`;
// `src_av` is a valid `AVPixelFormat` (from `pixel_to_av` of the `vaapi_sws_src`-validated
// `src_pixel`), the dst is NV12. The three trailing pointers (srcFilter, dstFilter, param) are
// explicitly null = "use defaults", which the API documents as accepted. No Rust memory is
// borrowed — only by-value ints/enums — and the returned pointer is null-checked just below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
@@ -283,10 +326,23 @@ impl CpuInner {
if sws.is_null() {
bail!("sws_getContext(RGB→NV12) failed");
}
// SAFETY: `sws` is the non-null `SwsContext` from `sws_getContext` above (the `is_null()`
// check immediately preceding returned false). `sws_getCoefficients(SWS_CS_ITU709)` returns a
// pointer into a libswscale static const coefficient table valid for the whole process, reused
// here for both the inverse (src) and forward (dst) matrices. `sws_setColorspaceDetails` only
// reads those tables and writes scalar CSC settings into `sws`; the table pointer outlives the
// synchronous call and no Rust memory is passed.
unsafe {
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, 0, 0, 1 << 16, 1 << 16);
}
// SAFETY: `av_frame_alloc` returns a fresh, uniquely-owned heap `AVFrame` (null-checked — on
// null we free the already-built `sws` and bail). We then write the plain `format`/`width`/
// `height` fields through the non-null, properly-aligned `f` (sole owner, not yet shared).
// `av_frame_get_buffer(f, 0)` allocates backing storage for those dims/format; on failure we
// free `f` and `sws` (unwinding the half-built state) and bail. On success `f` is a fully-owned
// NV12 frame stored in `CpuInner.nv12` and freed once in `CpuInner::drop`. `f` is a unique
// fresh pointer, so none of these writes alias anything.
let nv12 = unsafe {
let f = ffi::av_frame_alloc();
if f.is_null() {
@@ -329,6 +385,18 @@ impl CpuInner {
let h = self.height as usize;
let src_row = w * self.src_format.bytes_per_pixel();
anyhow::ensure!(bytes.len() >= src_row * h, "captured buffer too small");
// SAFETY: The `ensure!`s above guarantee `format == self.src_format` and
// `bytes.len() >= src_row * h`. `sws_scale` reads `h` rows of `src_row` bytes from
// `src_data[0] = bytes.as_ptr()` (the other planes null/0 — packed RGB is single-plane), all
// in bounds; `bytes`, `src_data`, `src_stride` are live locals for this synchronous call.
// `self.sws` is the non-null context built in `open`; it writes into `self.nv12` (a non-null
// owned frame whose `data`/`linesize` in-struct arrays were sized by `av_frame_get_buffer`).
// `av_frame_alloc` (null-checked) yields a fresh `hwf`; `av_hwframe_get_buffer` pulls a pooled
// VAAPI surface from the live non-null `self.hw.frames_ref`; `av_hwframe_transfer_data` uploads
// the staged NV12 into it — both frames live, failures free `hwf` and bail. We then write
// `pts`/`pict_type` through the non-null `hwf` and `avcodec_send_frame` it into the live
// owned `self.enc` context (which takes its own ref), then free our `hwf` ref exactly once.
// The encoder runs only on this thread (see `unsafe impl Send`), so no aliasing/data race.
unsafe {
let src_data: [*const u8; 4] = [bytes.as_ptr(), ptr::null(), ptr::null(), ptr::null()];
let src_stride: [c_int; 4] = [src_row as c_int, 0, 0, 0];
@@ -374,6 +442,12 @@ impl CpuInner {
impl Drop for CpuInner {
fn drop(&mut self) {
// SAFETY: `self.nv12` (an owned `AVFrame`) and `self.sws` (an owned `SwsContext`) are each
// freed exactly once here, guarded by `is_null()` so a never-set pointer is skipped (no double
// free). `CpuInner` owns both exclusively and `Drop` runs once. `av_frame_free` takes `&mut`
// and nulls the pointer. `self.enc`/`self.hw` are freed afterward by their own `Drop` impls;
// the encoder holds its own `av_buffer_ref`'d device/frames copies, so field-drop order is
// irrelevant to soundness.
unsafe {
if !self.nv12.is_null() {
ffi::av_frame_free(&mut self.nv12);
@@ -417,6 +491,31 @@ impl DmabufInner {
let drm_fourcc = crate::zerocopy::drm_fourcc(format)
.ok_or_else(|| anyhow!("no DRM fourcc for {format:?} (VAAPI zero-copy)"))?;
let node = render_node();
// SAFETY: libav is initialized (`VaapiEncoder::open` ran `ffmpeg::init()` before
// `ensure_inner` → `DmabufInner::open`). Every raw pointer dereferenced below is either freshly
// allocated by the immediately-preceding ffmpeg call and null-checked, or an in-struct field of
// such an object:
// * `node` is a `CString` (from `render_node`) live for the whole block; its `.as_ptr()` is a
// NUL-terminated path read only during `av_hwdevice_ctx_create`.
// * `av_hwdevice_ctx_create(&mut drm_device, DRM, …)` / `…_create_derived(&mut vaapi_device,
// VAAPI, drm_device, …)`: on `r < 0` the out-param stays null and we bail (the derive path
// unrefs `drm_device` first); on success each is a non-null owned `AVBufferRef`.
// * `av_hwframe_ctx_alloc(drm_device)` → `drm_frames` (null-checked); `(*drm_frames).data` is
// its `AVHWFramesContext` payload, written before `av_hwframe_ctx_init`.
// * `avfilter_graph_alloc` → `graph` (null-checked); `avfilter_get_by_name` returns a static
// const `AVFilter` (process-lifetime) or null; `avfilter_graph_alloc_filter` allocates each
// filter ctx inside `graph`; the four are null-checked together. `inst`/arg strings are
// 'static C literals.
// * `(*hwmap/scale).hw_device_ctx = av_buffer_ref(vaapi_device)` attaches a NEW ref owned by
// the filter (freed by `avfilter_graph_free`); our `vaapi_device` ref is untouched.
// * `av_buffersink_get_hw_frames_ctx(sink)` → `nv12_ctx` is a borrowed ref owned by the sink,
// valid while `graph` lives (and `graph` is moved into the returned `DmabufInner`).
// * `open_vaapi_encoder` borrows `vaapi_device` (our live owned ref) and `nv12_ctx` (sink's
// live ref) and `av_buffer_ref`s both into the encoder.
// Every early-error path unref's the allocated buffers and frees the graph in the right order
// before bailing; on success the four `AVBufferRef`s + `graph` + `src`/`sink` are moved into
// `DmabufInner` and freed in its `Drop`. (Two non-UB leaks noted below: `av_buffersrc_*` and
// the final `?`.)
unsafe {
// DRM device (source dmabuf frames) + a VAAPI device derived from it (same GPU) for
// hwmap/scale_vaapi/the encoder.
@@ -509,7 +608,12 @@ impl DmabufInner {
num: 1,
den: fps as c_int,
};
(*par).hw_frames_ctx = ffi::av_buffer_ref(drm_frames);
// Assign `drm_frames` BORROWED (no extra ref): `av_buffersrc_parameters_set` takes its
// own ref of `par->hw_frames_ctx` (via av_buffer_replace), and `av_free(par)` frees only
// the struct, not the ref. Our single owned `drm_frames` ref is retained, lives in
// `DmabufInner`, and is unref'd in `Drop`. Wrapping it in `av_buffer_ref` here would leak
// that extra ref every session (the persistent listener would accumulate them).
(*par).hw_frames_ctx = drm_frames;
let r = ffi::av_buffersrc_parameters_set(src, par);
ffi::av_free(par as *mut _);
if r < 0 {
@@ -564,7 +668,12 @@ impl DmabufInner {
ffi::av_buffer_unref(&mut drm_device);
bail!("filter sink has no VAAPI frames context");
}
let enc = open_vaapi_encoder(
// On encoder-open failure, free the graph + our owned buffer refs before bailing (matching
// every error path above) so a failed session doesn't leak them. `nv12_ctx` is borrowed
// from the sink (owned by `graph`), so `avfilter_graph_free` reclaims it — don't unref it
// separately. On success the encoder takes its own ref of `vaapi_device`, and `drm_frames`/
// `vaapi_device`/`drm_device`/`graph` move into `DmabufInner` (freed in `Drop`).
let enc = match open_vaapi_encoder(
codec,
width,
height,
@@ -572,7 +681,16 @@ impl DmabufInner {
bitrate_bps,
vaapi_device,
nv12_ctx,
)?;
) {
Ok(enc) => enc,
Err(e) => {
ffi::avfilter_graph_free(&mut graph);
ffi::av_buffer_unref(&mut drm_frames);
ffi::av_buffer_unref(&mut vaapi_device);
ffi::av_buffer_unref(&mut drm_device);
return Err(e);
}
};
tracing::info!(
encoder = codec.vaapi_name(),
@@ -600,6 +718,23 @@ impl DmabufInner {
dmabuf.fourcc,
self.fourcc
);
// SAFETY: The `ensure!` above checked `dmabuf.fourcc == self.fourcc`.
// * `std::mem::zeroed::<AVDRMFrameDescriptor>()` is sound: it is a `#[repr(C)]` POD of ints and
// nested int-struct arrays (no `NonNull`/refs), for which all-zero is a valid bit pattern;
// `Box` puts it on the heap with a unique owner.
// * `dmabuf.fd.as_raw_fd()` is the fd of the caller's `&DmabufFrame`, which owns it for the
// whole synchronous `submit`; we describe one object/layer/plane from its
// fourcc/modifier/offset/stride and pass `object.size = 0` (ffmpeg queries the real size).
// * `av_frame_alloc` → `drm` (null-checked); we set its scalar fields and
// `hw_frames_ctx = av_buffer_ref(self.drm_frames)` (new ref of the live owned ctx).
// * `data[0] = Box::into_raw(desc)` transfers the box into the frame; `buf[0] =
// av_buffer_create(.., free_desc, ..)` registers a destructor that reclaims it exactly once
// when the buffer's refcount hits zero — matched alloc/free, no leak/double-free.
// * `av_buffersrc_add_frame_flags(self.src, drm, KEEP_REF)` pushes a ref into the live
// buffersrc; KEEP_REF keeps our own `drm` ref, which we then `av_frame_free`. We pull the
// converted surface with `av_buffersink_get_frame(self.sink, nv12)` BEFORE returning, so the
// dmabuf (owned by the caller) is read while still valid. `nv12` is sent into the live owned
// `self.enc` (takes its own ref) and our ref freed once. Single-threaded encoder → no race.
unsafe {
// Build a DRM-PRIME AVFrame describing the dmabuf (one object/fd, one layer/plane).
let mut desc: Box<ffi::AVDRMFrameDescriptor> = Box::new(std::mem::zeroed());
@@ -626,6 +761,11 @@ impl DmabufInner {
// Own the descriptor so it frees with the frame (the fd is owned by the DmabufFrame,
// which outlives this call — the graph reads the surface before submit returns).
extern "C" fn free_desc(_opaque: *mut std::ffi::c_void, data: *mut u8) {
// SAFETY: `data` is exactly the pointer produced by `Box::into_raw(desc)` and passed as
// `av_buffer_create`'s first arg, which libav hands back verbatim to this callback. It
// is a valid, uniquely-owned `Box<AVDRMFrameDescriptor>` raw pointer; libav invokes the
// callback exactly once (when the last buffer ref drops), so `from_raw` + `drop`
// reclaims it exactly once — no double-free. `_opaque` is unused (we passed null).
unsafe { drop(Box::from_raw(data as *mut ffi::AVDRMFrameDescriptor)) };
}
(*drm).buf[0] = ffi::av_buffer_create(
@@ -673,6 +813,13 @@ impl DmabufInner {
impl Drop for DmabufInner {
fn drop(&mut self) {
// SAFETY: `graph`/`drm_frames`/`vaapi_device`/`drm_device` are the non-null objects
// `DmabufInner::open` built and moved into `self` (open bails before constructing `Self` if any
// alloc fails). `avfilter_graph_free` frees the graph (and the per-filter device refs it owns);
// each `av_buffer_unref` drops one ref and nulls the pointer via `&mut`. `DmabufInner` owns all
// four exclusively and `Drop` runs once → no double-free/use-after-free. The graph is freed
// first (it holds refs on the devices), then frames, then the derived VAAPI device, then DRM.
// (`self.enc` drops via ffmpeg-next afterward, holding its own refs.)
unsafe {
ffi::avfilter_graph_free(&mut self.graph);
ffi::av_buffer_unref(&mut self.drm_frames);
@@ -703,9 +850,17 @@ pub struct VaapiEncoder {
}
// Raw FFI pointers; the encoder lives on a single thread (same contract as `NvencEncoder`).
// SAFETY: `VaapiEncoder`'s `Inner` holds raw FFI pointers (`SwsContext`, `AVFrame`, `AVBufferRef`,
// `AVFilterContext`, `AVCodecContext`) that are not `Send` by default. The encoder is owned and
// driven by exactly ONE thread — the host's per-session encode thread it is moved (transferred) to —
// and is only ever touched through `&mut self` methods, so it is never aliased or accessed
// concurrently from two threads. None of the underlying libav/libswscale objects have thread
// affinity (they are not thread-local), so transferring ownership across threads is sound. This
// asserts `Send` (transfer) only; `Sync` (shared `&`) is deliberately NOT implemented.
unsafe impl Send for VaapiEncoder {}
impl VaapiEncoder {
#[allow(clippy::too_many_arguments)]
pub fn open(
codec: Codec,
format: PixelFormat,
@@ -714,12 +869,23 @@ impl VaapiEncoder {
fps: u32,
bitrate_bps: u64,
bit_depth: u8,
chroma: super::ChromaFormat,
) -> Result<Self> {
if bit_depth != 8 {
tracing::warn!(bit_depth, "VAAPI 10-bit not yet wired — encoding 8-bit");
}
// VAAPI 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel hardware in the
// lab exposes a HEVC 4:4:4 encode entrypoint, and the probe returns false so the host never
// negotiates 4:4:4 for a VAAPI session. If a request slips through, fall back to 4:2:0 rather
// than emit an unverified stream — the host signalled 4:2:0 in the Welcome anyway.
if chroma.is_444() {
tracing::warn!("VAAPI 4:4:4 encode not implemented — encoding 4:2:0");
}
ffmpeg::init().context("ffmpeg init")?;
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
// SAFETY: `av_log_set_level` sets libav's global integer log level; `48` (= AV_LOG_DEBUG)
// is a valid level and there are no pointer args. libav was just initialized by the
// `ffmpeg::init()` above, so the call is always sound.
unsafe { ffi::av_log_set_level(48) };
}
// Validate the codec/format up front so a bad request fails at open, not on the first frame.
@@ -28,8 +28,10 @@
//! through `ffmpeg::ffi` (= `ffmpeg_sys_next`), exactly as the Linux CUDA/VAAPI paths do. The
//! `AVD3D11VADeviceContext`/`AVD3D11VAFramesContext` layouts are mirrored (the bindings don't
//! allowlist `hwcontext_d3d11va.h`), as [`super::linux`] mirrors `AVCUDADeviceContext`.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{Codec, EncodedFrame, Encoder};
use super::{ChromaFormat, Codec, EncodedFrame, Encoder};
use crate::capture::{dxgi::D3d11Frame, CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use ffmpeg::format::Pixel;
@@ -239,10 +241,28 @@ unsafe fn open_win_encoder(
/// driver/runtime rejects codecs the video engine can't do (AV1 on pre-RDNA3 AMD / pre-Arc Intel,
/// or HEVC on a very old part). Used to build the GameStream codec advertisement so a client never
/// negotiates a codec the encoder can't open. Torn down immediately.
/// Whether the active AMD (AMF) / Intel (QSV) GPU can encode HEVC **4:4:4**. **Deferred in v1 —
/// always `false`.** AMF/QSV HEVC 4:4:4 encode is narrow (AMD RDNA3+, Intel Arc/Xe2+) and the
/// libavcodec profile/pixel-format incantation is vendor- and driver-specific — a wrong profile
/// `avcodec_open2` *silently* falls back to 4:2:0, so a positive probe would need a verify-by-frame,
/// and there is no AMD/Intel Windows box in the lab to build + validate that against. Returning
/// `false` keeps the negotiation honest: an AMF/QSV host resolves every session to 4:2:0 before the
/// Welcome. (Follow-up: implement + validate on an RDNA3+/Arc Windows box.)
pub fn probe_can_encode_444(_vendor: WinVendor, _codec: Codec) -> bool {
tracing::info!("AMF/QSV HEVC 4:4:4 encode is not implemented yet — declining (encoding 4:2:0)");
false
}
pub fn probe_can_encode(vendor: WinVendor, codec: Codec) -> bool {
if ffmpeg::init().is_err() {
return false;
}
// SAFETY: `ffmpeg::init()` succeeded above, so libav's global state is initialised.
// `av_log_get_level`/`av_log_set_level` are global scalar getters/setters with no pointer args.
// `open_win_encoder` (the `unsafe fn`) is called with null `device_ref`/`frames_ref` (the system
// path), so it touches no D3D11/hwcontext — it only allocates and opens a self-contained
// libavcodec encoder that is dropped at the end of `.is_ok()`. We restore the prior log level and
// no raw pointer escapes the block.
unsafe {
// A missing AMF/QSV runtime (wrong-vendor host, GPU-less CI) is an expected probe outcome —
// quiet ffmpeg's open error for the probe, then restore the level.
@@ -337,6 +357,10 @@ impl SystemInner {
} else {
ffi::AVPixelFormat::AV_PIX_FMT_NV12
};
// SAFETY: calls the `unsafe fn open_win_encoder` with null `device_ref`/`frames_ref`, so the
// system path is taken (no hw device/frames context is touched); all other args are scalars.
// The returned `encoder::video::Encoder` owns its `AVCodecContext` and frees it on drop; no raw
// pointer is aliased.
let enc = unsafe {
open_win_encoder(
vendor,
@@ -352,6 +376,11 @@ impl SystemInner {
ptr::null_mut(),
)?
};
// SAFETY: `av_frame_alloc` returns a freshly-allocated, uniquely-owned `AVFrame` (null-checked
// before any deref); writing `format`/`width`/`height` through `*f` stays inside that
// allocation. `av_frame_get_buffer(f, 0)` allocates the backing planes — on failure we
// `av_frame_free` the sole owner (no double-free) and bail; on success the raw `f` is moved into
// `self.sw_frame` and freed exactly once in `Drop`.
let sw_frame = unsafe {
let f = ffi::av_frame_alloc();
if f.is_null() {
@@ -467,6 +496,18 @@ impl SystemInner {
} else {
DXGI_FORMAT_NV12
};
// SAFETY: `ensure_staging` builds a STAGING texture (CPU_ACCESS_READ) matching `dxgi_fmt` on
// `frame.device` — the same `ID3D11Device` that owns `frame.texture` — and caches that device's
// immediate context in `self.ctx`. `src`/`dst` are that device's textures of identical NV12/P010
// format and dimensions, so `CopyResource` on the single-threaded immediate context is valid.
// `Map(.., D3D11_MAP_READ)` succeeds on a staging texture and yields `map.pData` valid for the
// whole resource; for NV12/P010 the luma plane is `H` rows at `RowPitch` and the chroma plane
// follows at byte offset `RowPitch*H` (`H/2` rows), so `total = pitch*(H+⌈H/2⌉)` is exactly the
// mapped extent and `from_raw_parts(base, total)` stays in-bounds. Each `copy_nonoverlapping`
// reads a bounds-checked `mapped[..]` sub-slice (`row_bytes ≤ pitch`) and writes `row_bytes ≤
// linesize` into the `av_frame_get_buffer`-allocated plane at row `y < H`, so every destination
// offset is inside the frame's plane allocation; src and dst never alias. `Unmap` pairs `Map`,
// then `send` (the `unsafe fn`) hands `sw_frame` to the encoder.
unsafe {
self.ensure_staging(&frame.device, dxgi_fmt)?;
let staging = self.staging.clone().context("staging texture")?;
@@ -510,6 +551,14 @@ impl SystemInner {
if self.ten_bit {
bail!("ffmpeg_win: BGRA readback is 8-bit only (HDR needs the P010 capture path)");
}
// SAFETY: `ensure_staging` builds a B8G8R8A8 STAGING texture on `frame.device` and caches that
// device's immediate context; `src`/`dst` are that device's textures of matching BGRA format,
// so `CopyResource` on the single-threaded context is valid. `Map(READ)` on the staging texture
// yields `base` valid for `pitch` × `h` rows. `ensure_sws` lazily builds the BGRA→NV12 context;
// `sws_scale` reads `h` rows of `pitch` bytes from `base` (in-bounds — the staging surface is
// `≥ pitch*h`) into the `sw_frame` planes addressed by its `data`/`linesize` (allocated for
// `width`×`height` NV12). `Unmap` pairs `Map`; the cached `sws` is freed once in `Drop`. The
// mapped read region never aliases the owned encoder frame.
unsafe {
self.ensure_staging(&frame.device, DXGI_FORMAT_B8G8R8A8_UNORM)?;
let staging = self.staging.clone().context("staging texture")?;
@@ -552,6 +601,13 @@ impl SystemInner {
/// R10 shader output instead of P010. DXGI `R10G10B10A2_UNORM` (R in the low 10 bits, X2 alpha in
/// the top 2) == FFmpeg `AV_PIX_FMT_X2BGR10LE`. UNTESTED on glass (no AMD/Intel Windows box).
fn readback_rgb10(&mut self, frame: &D3d11Frame, pts: i64, idr: bool) -> Result<()> {
// SAFETY: same shape as `readback_yuv`/`readback_bgra` — `ensure_staging` builds an
// R10G10B10A2 STAGING texture on `frame.device` and caches its immediate context; `src`/`dst`
// are that device's matching-format textures, so `CopyResource` on the single-threaded context
// is valid. `Map(READ)` yields `base` valid for `pitch` × `h` rows. `ensure_sws` builds the
// X2BGR10LE→P010 (BT.2020) context; `sws_scale` reads `h` rows of `pitch` bytes from `base`
// (in-bounds) into the `sw_frame` P010 planes (`data`/`linesize`, allocated `width`×`height`).
// `Unmap` pairs `Map`; `sws` is freed once in `Drop`. No aliasing between read and write.
unsafe {
self.ensure_staging(&frame.device, DXGI_FORMAT_R10G10B10A2_UNORM)?;
let staging = self.staging.clone().context("staging texture")?;
@@ -605,6 +661,12 @@ impl SystemInner {
let h = self.height as usize;
let src_row = w * format.bytes_per_pixel();
anyhow::ensure!(bytes.len() >= src_row * h, "captured buffer too small");
// SAFETY: `ensure_sws` lazily builds the (packed RGB/BGR)→NV12 context for this fixed src/dst
// format pair. `src_data[0] = bytes.as_ptr()` with `src_stride[0] = src_row`; the `ensure!`
// above guarantees `bytes` holds at least `src_row*h` bytes, so `sws_scale` reads `h` rows of
// `src_row` bytes in-bounds and writes the `sw_frame` NV12 planes (`data`/`linesize`, allocated
// `width`×`height`). `bytes` is borrowed for the call only and never aliases the owned
// `sw_frame`. `send` then hands `sw_frame` to the encoder.
unsafe {
self.ensure_sws(
pixel_to_av(sws_src(format)?),
@@ -667,6 +729,10 @@ impl SystemInner {
impl Drop for SystemInner {
fn drop(&mut self) {
// SAFETY: `sw_frame` is the `AVFrame` allocated in `open` (or null) — `av_frame_free` drops it
// once and nulls the pointer through the `&mut`; `sws` is the cached `SwsContext` (or null) —
// `sws_freeContext` frees it once. This `Drop` runs exactly once and `SystemInner` owns both
// exclusively, so there is no double-free or use-after-free.
unsafe {
if !self.sw_frame.is_null() {
ffi::av_frame_free(&mut self.sw_frame);
@@ -745,6 +811,12 @@ impl D3d11Hw {
impl Drop for D3d11Hw {
fn drop(&mut self) {
// SAFETY: `frames_ref`/`device_ref` are the two non-null `AVBufferRef`s `D3d11Hw::new` created
// (it bails before constructing `Self` if either alloc/init fails, so a live `D3d11Hw` always
// holds both). `av_buffer_unref` drops one reference and nulls the pointer through the `&mut`.
// This `Drop` runs exactly once and `D3d11Hw` owns these refs exclusively → no double-free /
// use-after-free. Frames are unref'd before the device because the frames ctx internally holds
// a ref on the device (refcounted, so the order is sound either way).
unsafe {
ffi::av_buffer_unref(&mut self.frames_ref);
ffi::av_buffer_unref(&mut self.device_ref);
@@ -800,6 +872,18 @@ impl ZeroCopyInner {
WinVendor::Qsv => (D3D11_BIND_DECODER.0 | D3D11_BIND_VIDEO_ENCODER.0) as u32,
};
const POOL: c_int = 8;
// SAFETY: `D3d11Hw::new` wraps the capturer's `device` as a D3D11VA hwdevice (handing FFmpeg an
// owned AddRef of it, balanced by FFmpeg's teardown Release) and builds an owned
// device_ref/frames_ref pair freed by `D3d11Hw::Drop`; `hw` is a local, so it is dropped (and
// both refs freed) on every early `return Err`. For QSV, `av_hwdevice_ctx_create_derived` and
// `av_hwframe_ctx_create_derived` fill the null-initialised `qsv_device`/`qsv_frames` out-params
// only on success (`r >= 0` checked); on the frames-derive failure we unref the already-created
// `qsv_device` before bailing. `open_win_encoder` internally `av_buffer_ref`s the dev/frames
// refs it is given (so ownership of `hw`'s and the derived refs stays here), and on its failure
// we unref the still-owned derived `qsv_frames`/`qsv_device` (null for AMF → skipped) and return
// — `hw` then drops its D3D11 refs. On success the derived refs are moved into `ZeroCopyInner`
// (freed in its `Drop`) and the encoder holds its own AddRef'd copies. Every `AVBufferRef` is
// unref'd exactly once across all paths — no leak, no double-free.
unsafe {
let hw = D3d11Hw::new(device, sw_av, bind_flags, width, height, POOL)?;
let (pix_fmt, dev_ref, frames_ref, mut qsv_device, mut qsv_frames) = match vendor {
@@ -887,6 +971,19 @@ impl ZeroCopyInner {
}
fn submit(&mut self, frame: &D3d11Frame, pts: i64, idr: bool) -> Result<()> {
// SAFETY: `d3d = av_frame_alloc()` is a fresh owned frame (null-checked) and is `av_frame_free`d
// exactly once on every path below. `av_hwframe_get_buffer` fills it from the pool — on failure
// we free it and bail. `(*d3d).data[0]` is the pool's texture-array and `data[1]` the array
// index; `from_raw_borrowed` borrows that `ID3D11Texture2D` WITHOUT taking ownership (no Release
// — the frame owns it) and is null-checked. `src` (the captured texture) and `dst` (the pooled
// slice) live on the SAME D3D11 device wrapped by `self.hw`, and the caller guarantees
// `captured.format == pool_format` before calling, so `CopySubresourceRegion(dst, dst_index, ..,
// src, 0, ..)` on the single-threaded immediate context `self.ctx` is a valid same-format GPU
// copy. For QSV the mapped `qsv` frame is a fresh owned frame whose `hw_frames_ctx` takes an
// `av_buffer_ref` of `self.qsv_frames`; it is `av_frame_free`d (releasing that ref) on both the
// map-failure and success paths. `avcodec_send_frame` only internally refs the input frame, so
// the `av_frame_free(d3d)`/`av_frame_free(qsv)` afterwards are the sole owning frees — no leak,
// no double-free, no use-after-free.
unsafe {
// Pull a pooled D3D11 surface; its data[0] is the pool's texture-ARRAY, data[1] the slice.
let mut d3d = ffi::av_frame_alloc();
@@ -959,6 +1056,11 @@ impl ZeroCopyInner {
impl Drop for ZeroCopyInner {
fn drop(&mut self) {
// SAFETY: `qsv_frames`/`qsv_device` are the derived QSV `AVBufferRef`s (or null for AMF); each
// is `av_buffer_unref`'d once here (nulling the pointer through the `&mut`) — `ZeroCopyInner`
// owns these handles exclusively and this `Drop` runs once, so no double-free. The `enc` and
// `hw` fields free the encoder's AddRef'd copies and the D3D11 device/frames refs through their
// own `Drop`, so all references stay balanced.
unsafe {
if !self.qsv_frames.is_null() {
ffi::av_buffer_unref(&mut self.qsv_frames);
@@ -996,9 +1098,17 @@ pub struct FfmpegWinEncoder {
}
// Raw FFI pointers + COM objects; the encoder lives on a single thread (same contract as NVENC/VAAPI).
// SAFETY: `FfmpegWinEncoder` owns raw libav pointers (`AVFrame`/`SwsContext`/`AVBufferRef`) and
// windows-rs COM handles (`ID3D11Device`/`ID3D11DeviceContext`/textures) that are not auto-`Send`. The
// session creates the encoder, drives `submit`/`poll`/`flush`, and drops it all on one dedicated encode
// thread; it is never shared by reference across threads, and the D3D11 immediate context is only ever
// touched from that thread. The only cross-thread action is the initial move to the encode thread,
// after which every interior pointer/COM ref is used single-threaded — the same contract the
// NVENC/VAAPI encoders rely on. No interior state is accessed concurrently.
unsafe impl Send for FfmpegWinEncoder {}
impl FfmpegWinEncoder {
#[allow(clippy::too_many_arguments)]
#[allow(clippy::too_many_arguments)]
pub fn open(
vendor: WinVendor,
@@ -1009,9 +1119,19 @@ impl FfmpegWinEncoder {
fps: u32,
bitrate_bps: u64,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Self> {
// AMF/QSV 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel Windows
// hardware in the lab, and the AMF/QSV HEVC 4:4:4 profile/format incantations are vendor- and
// driver-specific (a wrong profile silently encodes 4:2:0). The probe returns false so the host
// never negotiates 4:4:4 for an AMF/QSV session; if a request slips through, fall back to 4:2:0.
if chroma.is_444() {
tracing::warn!("AMF/QSV 4:4:4 encode not implemented — encoding 4:2:0");
}
ffmpeg::init().context("ffmpeg init")?;
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
// SAFETY: `ffmpeg::init()` ran on the line above, so libav is initialised; `av_log_set_level`
// is a global scalar setter with no pointer arguments.
unsafe { ffi::av_log_set_level(48) };
}
// Make sure the encoder name exists in this libavcodec build up front (clear error vs a
@@ -13,7 +13,10 @@
//! Needs a real NVIDIA GPU at runtime (session creation fails otherwise) — compiles GPU-less, but
//! `open`/`submit` only succeed on a GPU box. The software encoder (`super::sw`) is the fallback.
use super::{Codec, EncodedFrame, Encoder, EncoderCaps};
// Every `unsafe` block / impl in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use std::collections::{HashMap, VecDeque};
@@ -54,6 +57,15 @@ pub struct NvencD3d11Encoder {
buffer_fmt: nv::NV_ENC_BUFFER_FORMAT,
/// Encoded bit depth (8 or 10). 10 → HEVC Main10 (NVENC upconverts the 8-bit ARGB input).
bit_depth: u8,
/// Full-chroma 4:4:4 (HEVC Range Extensions, `chroma_format_idc = 3`) requested for this session.
/// NVENC ingests the RGB (ARGB/ABGR10) input and CSCs it to YUV444 internally — the `FREXT` profile
/// + `chromaFormatIDC = 3` in the encode config carry the chroma. Gated on the GPU's
/// `NV_ENC_CAPS_SUPPORT_YUV444_ENCODE` (cleared in `query_caps` on a card that lacks it) and on an
/// RGB input format (NV12/P010 capture can't reconstruct 4:4:4). HEVC-only.
chroma_444: bool,
/// `NV_ENC_CAPS_SUPPORT_YUV444_ENCODE` from the caps probe — whether this GPU can 4:4:4 encode at
/// all. `chroma_444` is forced off when this is false (graceful downgrade to 4:2:0).
yuv444_supported: bool,
/// HDR: the capturer is delivering BT.2020 PQ 10-bit (`PixelFormat::Rgb10a2`) frames. Sets the
/// `ABGR10` input format + the BT.2020/PQ colour VUI. Derived per-frame from the capture format
/// (HDR can toggle mid-session); a change re-inits the session.
@@ -88,10 +100,19 @@ pub struct NvencD3d11Encoder {
init_device: *mut c_void,
}
// Raw NVENC handle + COM ptrs; confined to the single encode thread (like the Linux encoder).
// SAFETY: the `!Send` fields are the raw NVENC session/device handles (`encoder`, `init_device`),
// the raw NVENC bitstream/registered/mapped pointers carried in `bitstreams`/`regs`/`pending`, and
// the `ID3D11Texture2D` COM refs — none of which may be touched concurrently from two threads. This
// encoder is owned by exactly one thread: it is moved onto the host encode thread once at
// construction, and every NVENC call and D3D11 access happens only from that thread thereafter
// (`submit`/`poll`/`invalidate_ref_frames`/`Drop` all run there, like the Linux encoder). Moving the
// handles across that single ownership-transfer boundary is sound because no NVENC/D3D11 call is in
// flight during the move and the session and its D3D11 immediate context are never shared (`&`) or
// used concurrently — so `Send` introduces no data race on the non-`Send` fields.
unsafe impl Send for NvencD3d11Encoder {}
impl NvencD3d11Encoder {
#[allow(clippy::too_many_arguments)]
pub fn open(
codec: Codec,
_format: PixelFormat,
@@ -100,6 +121,7 @@ impl NvencD3d11Encoder {
fps: u32,
bitrate_bps: u64,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Self> {
Ok(Self {
encoder: ptr::null_mut(),
@@ -111,6 +133,9 @@ impl NvencD3d11Encoder {
bitrate_bps,
buffer_fmt: nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ARGB,
bit_depth,
// 4:4:4 is HEVC-only; the GPU-support gate is applied in `query_caps`.
chroma_444: chroma.is_444() && codec == Codec::H265,
yuv444_supported: false,
hdr: false,
hdr_meta: None,
regs: HashMap::new(),
@@ -198,6 +223,7 @@ impl NvencD3d11Encoder {
let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX);
let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX);
let ten_bit = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_10BIT_ENCODE);
let yuv444 = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE);
let rfi = self.get_cap(
enc,
nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_REF_PIC_INVALIDATION,
@@ -224,6 +250,13 @@ impl NvencD3d11Encoder {
self.bit_depth = 8;
self.hdr = false;
}
// Same for 4:4:4: a card without YUV444 encode falls back to 4:2:0. (The host already probed
// this via `probe_can_encode_444` before the Welcome, so this is a belt-and-braces guard.)
self.yuv444_supported = yuv444 != 0;
if self.chroma_444 && !self.yuv444_supported {
tracing::warn!("NVENC: this GPU can't 4:4:4 encode — falling back to 4:2:0");
self.chroma_444 = false;
}
self.rfi_supported = rfi != 0;
self.custom_vbv = custom_vbv != 0;
tracing::info!(
@@ -302,9 +335,31 @@ impl NvencD3d11Encoder {
cfg.encodeCodecConfig.hevcConfig.tier = 1;
cfg.encodeCodecConfig.hevcConfig.level = 0;
// 10-bit HEVC Main10 (HDR foundation): NVENC upconverts the 8-bit input; 8-bit leaves the
// preset default (Main) untouched.
if self.bit_depth == 10 {
// Chroma + bit depth. Full-chroma 4:4:4 (HEVC Range Extensions) takes precedence and composes
// with 10-bit (Main 4:4:4 10): NVENC ingests the RGB input (ARGB / ABGR10) and CSCs it to
// YUV444 internally when `chromaFormatIDC = 3` under the FREXT profile. Only valid on an RGB
// input — a subsampled NV12/P010 source can't reconstruct full chroma (so the capturer is
// forced to RGB for a 4:4:4 session, and we guard on the input format here too).
//
// ON-GLASS TODO (RTX box): confirm ARGB + chromaFormatIDC=3 + FREXT yields a *true* 4:4:4
// stream. NVENC's RGB→YUV CSC is documented to honor chromaFormatIDC (unlike libavcodec's
// wrapper, which always subsamples RGB to 4:2:0 — hence the Linux path feeds planar YUV444
// instead). If on-glass shows 4:2:0, the follow-up is a BGRA→AYUV shader feeding the native
// `NV_ENC_BUFFER_FORMAT_AYUV` 4:4:4 input format.
let rgb_input = matches!(
self.buffer_fmt,
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ARGB
| nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ABGR10
);
if self.chroma_444 && rgb_input {
cfg.profileGUID = nv::NV_ENC_HEVC_PROFILE_FREXT_GUID;
cfg.encodeCodecConfig.hevcConfig.set_chromaFormatIDC(3);
if self.bit_depth == 10 {
cfg.encodeCodecConfig.hevcConfig.set_pixelBitDepthMinus8(2); // Main 4:4:4 10
}
} else if self.bit_depth == 10 {
// 10-bit HEVC Main10 (HDR foundation): NVENC upconverts the 8-bit input; 8-bit leaves the
// preset default (Main) untouched.
cfg.profileGUID = nv::NV_ENC_HEVC_PROFILE_MAIN10_GUID;
cfg.encodeCodecConfig.hevcConfig.set_pixelBitDepthMinus8(2); // 10 - 8
}
@@ -403,6 +458,17 @@ impl NvencD3d11Encoder {
/// Lazily create the session on the first frame's D3D11 device (so capture + encode share it).
fn init_session(&mut self, device: &ID3D11Device) -> Result<()> {
// SAFETY: every call below goes through a function pointer resolved once from the loaded
// `nvidia_video_codec_sdk::ENCODE_API` (`nvEncodeAPI`) table, or through this type's own
// `unsafe fn`s whose contract is met here. `query_caps`/`try_open_session` receive `device`,
// the live `ID3D11Device` the caller pulled off the first frame; each returns either a valid
// open NVENC session handle or an `Err`. `destroy_encoder` is only ever called on a handle a
// `try_open_session` just returned (and `best` only when `!best.is_null()`), so it never frees
// a dangling or null session. `create_bitstream_buffer` is passed `enc` — the one chosen live
// session — and `&mut cb`, a `#[repr(C)] NV_ENC_CREATE_BITSTREAM_BUFFER` whose `version` is set
// to `NV_ENC_CREATE_BITSTREAM_BUFFER_VER`; `cb` lives across the synchronous call and its
// returned `bitstreamBuffer` is copied into `self.bitstreams` before `cb` drops. No handle
// escapes the encode thread.
unsafe {
// Probe real GPU caps first (max dims / 10-bit / custom-VBV / RFI) so the config below is
// gated on what this card supports and an out-of-range mode fails with a clear error
@@ -589,6 +655,11 @@ impl Encoder for NvencD3d11Encoder {
new = format!("{}x{}", captured.width, captured.height),
"NVENC: capture device/size/HDR changed — re-initializing session"
);
// SAFETY: `teardown` (an `unsafe fn`) requires the encode thread with no NVENC call in
// flight and a session whose cached regs/bitstreams/pending all belong to `self.encoder`.
// All hold: this is the synchronous encode thread, `self.inited` so `self.encoder` is the
// live session every cached resource was created against, and the previous frame's encode
// has already been polled (synchronous submit→poll), so nothing is mid-encode.
unsafe { self.teardown() };
}
if !self.inited {
@@ -609,7 +680,14 @@ impl Encoder for NvencD3d11Encoder {
self.bit_depth = 10;
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ABGR10
}
PixelFormat::Nv12 => nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12,
PixelFormat::Nv12 => {
// NV12 is 8-bit 4:2:0. Force 8-bit so a transition from a prior P010 (10-bit) session
// — or a 10-bit-negotiated client on an SDR display — re-inits at the matching depth.
// Unlike ARGB (which NVENC upconverts to Main10), NV12 cannot feed a 10-bit session:
// `register_resource` rejects it as InvalidParam (the HDR→SDR-toggle stream drop).
self.bit_depth = 8;
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12
}
_ => nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ARGB,
};
let device = frame.device.clone();
@@ -618,6 +696,21 @@ impl Encoder for NvencD3d11Encoder {
}
let slot = self.next % POOL;
self.next += 1;
// SAFETY: every NVENC call goes through a function pointer from the loaded `ENCODE_API` table
// and takes `self.encoder`, the live session `init_session` just established (non-null on the
// path that reaches here). `NV_ENC_REGISTER_RESOURCE rr` has `version =
// NV_ENC_REGISTER_RESOURCE_VER` and registers `frame.texture` — a D3D11 texture from
// `frame.device`, which is the SAME device the session was opened against (any device change
// tears down and re-inits above, so `init_device == frame.device.as_raw()` here); the cloned
// `ID3D11Texture2D` is kept alive in `regs` so NVENC's registration never outlives the texture.
// `mp` (`NV_ENC_MAP_INPUT_RESOURCE`, version set) maps that registration and the map is recorded
// in `pending` to be unmapped exactly once in `poll`/`teardown`. `pic` (`NV_ENC_PIC_PARAMS`,
// version set) points `inputBuffer` at `mp.mappedResource` and `outputBitstream` at the live
// pool bitstream `bitstreams[slot]`; the optional SEI scratch (`mastering_sei`/`cll_sei` and the
// `sei` Vec whose `as_mut_ptr()` is written into the codec union) are stack locals that outlive
// the synchronous `encode_picture`. Every `#[repr(C)]` param is a live local borrowed `&mut`
// for the duration of its one synchronous call. (In-place encode without `CopyResource` is
// sound because the encode loop is synchronous, as the module docs state.)
unsafe {
// Register the capturer's texture with NVENC once (cached by raw pointer), then encode it
// IN PLACE — no `CopyResource` into an encoder-owned pool. This is the zero-copy win: the
@@ -738,6 +831,9 @@ impl Encoder for NvencD3d11Encoder {
EncoderCaps {
supports_rfi: self.rfi_supported,
supports_hdr_metadata: self.hdr,
// Reflects what the session actually configured (cleared in `query_caps` if the GPU lacks
// YUV444 encode), so the glue can confirm 4:4:4 vs the negotiated request.
chroma_444: self.chroma_444,
}
}
@@ -774,6 +870,12 @@ impl Encoder for NvencD3d11Encoder {
// We tag each input with `inputTimeStamp = frame_idx` (0,1,2,…), which is also the client's
// frame number (the packetizer numbers frames in submit order), so the client's lost-frame
// range maps 1:1 onto the timestamps NVENC invalidates here.
// SAFETY: `invalidate_ref_frames` is a function pointer from the loaded `ENCODE_API` table.
// `self.encoder` was checked non-null at the top of this fn and is the live session; this runs
// on the encode thread (like submit/poll), so there is no concurrent NVENC use. Each `ts` was
// clamped to `[oldest_in_dpb, frame_idx - 1]` above, so it names a frame still in the session's
// DPB; the call passes only that `u64` timestamp (no struct), so there is no struct-size or
// lifetime concern.
unsafe {
for ts in first..=last {
if (API.invalidate_ref_frames)(self.encoder, ts as u64)
@@ -792,6 +894,16 @@ impl Encoder for NvencD3d11Encoder {
let Some((bs, map, pts_ns)) = self.pending.pop_front() else {
return Ok(None);
};
// SAFETY: a non-empty `pending` implies `submit` ran, so `self.encoder` is the live session
// (`teardown` clears `pending` whenever it nulls the handle); all calls below use function
// pointers from the loaded `ENCODE_API` table on the encode thread. `NV_ENC_LOCK_BITSTREAM lock`
// (version = `NV_ENC_LOCK_BITSTREAM_VER`) locks `bs`, a pool bitstream a prior `encode_picture`
// targeted; `lock_bitstream` blocks until that encode finishes, so on success
// `lock.bitstreamBufferPtr` is non-null and points at `lock.bitstreamSizeInBytes` bytes of
// NVENC-owned, CPU-readable output valid until `unlock_bitstream`. The `from_raw_parts` slice is
// only read (copied via `to_vec()`) BEFORE `unlock_bitstream(bs)` — lock and unlock pair on the
// same buffer — so it never outlives the lock. `map` (the input resource paired with `bs` in
// `pending`) is unmapped here, after the encode completed, exactly once.
unsafe {
let mut lock = nv::NV_ENC_LOCK_BITSTREAM {
version: nv::NV_ENC_LOCK_BITSTREAM_VER,
@@ -831,6 +943,77 @@ impl Encoder for NvencD3d11Encoder {
impl Drop for NvencD3d11Encoder {
fn drop(&mut self) {
// SAFETY: `teardown` (an `unsafe fn`) needs the owning thread with no NVENC call in flight and
// a session whose cached resources all belong to `self.encoder`. At Drop this encoder is owned
// exclusively (no other reference can exist), runs on the encode thread it was confined to, and
// `teardown` early-returns when `self.encoder` is null; otherwise every cached reg/bitstream/
// pending was created against that live session. It runs exactly once (here).
unsafe { self.teardown() };
}
}
/// Probe whether the active NVIDIA GPU can encode HEVC **4:4:4** (`NV_ENC_CAPS_SUPPORT_YUV444_ENCODE`).
/// Creates a throwaway hardware D3D11 device + NVENC session, queries the cap, and tears down. HEVC-only;
/// the result is cached by the caller ([`crate::encode::can_encode_444`]) and read *before* the Welcome
/// so the host advertises the chroma it can really encode (honest downgrade to 4:2:0 on a card without it).
pub fn probe_can_encode_444(codec: Codec) -> bool {
use windows::Win32::Foundation::HMODULE;
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_FEATURE_LEVEL_11_0};
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_SDK_VERSION,
};
if codec != Codec::H265 {
return false;
}
// SAFETY: a self-contained probe owning every handle it creates. `D3D11CreateDevice` (HARDWARE
// driver, NULL adapter) fills `device` or returns Err (→ false). `open_encode_session_ex` opens an
// NVENC session against that device's raw pointer (valid while `device` is held) or errors (→ false,
// tearing nothing down). `get_encode_caps` reads one scalar cap into `val` via the loaded API table.
// `destroy_encoder` frees the session exactly once; `device`/its context drop with the COM wrappers.
// No handle escapes this call and nothing runs concurrently.
unsafe {
let mut device: Option<ID3D11Device> = None;
if D3D11CreateDevice(
None,
D3D_DRIVER_TYPE_HARDWARE,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
None,
)
.is_err()
{
return false;
}
let Some(device) = device else { return false };
let mut params = nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS {
version: nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER,
deviceType: nv::NV_ENC_DEVICE_TYPE::NV_ENC_DEVICE_TYPE_DIRECTX,
device: device.as_raw(),
apiVersion: nv::NVENCAPI_VERSION,
..Default::default()
};
let mut enc: *mut c_void = ptr::null_mut();
if (API.open_encode_session_ex)(&mut params, &mut enc)
.result_without_string()
.is_err()
{
return false;
}
let mut param = nv::NV_ENC_CAPS_PARAM {
version: nv::NV_ENC_CAPS_PARAM_VER,
capsToQuery: nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE,
reserved: [0; 62],
};
let mut val: i32 = 0;
let ok = (API.get_encode_caps)(enc, nv::NV_ENC_CODEC_HEVC_GUID, &mut param, &mut val)
.result_without_string()
.is_ok()
&& val != 0;
let _ = (API.destroy_encoder)(enc);
ok
}
}
@@ -2,6 +2,8 @@
//! fallback when NVENC is unavailable). Low-latency screen-content config: single-reference,
//! no B-frames (Baseline), bitrate rate-control, in-band SPS/PPS each IDR, BT.709 limited range.
//! Synchronous: `submit` encodes immediately and stashes the AU for `poll` (no internal queue).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{EncodedFrame, Encoder};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
@@ -30,6 +32,12 @@ pub struct OpenH264Encoder {
}
// openh264's Encoder holds a raw C handle (not auto-Send); it lives on the single encode thread.
// SAFETY: `OpenH264Encoder` wraps `Oh264` (openh264's `Encoder`), which holds a raw C handle to the
// openh264 `ISVCEncoder` and is not auto-`Send`; the other fields (`YUVBuffer`, `Vec`, scalars,
// `Option<EncodedFrame>`) are plain owned data. The session creates the encoder, calls
// `submit`/`poll`/`flush`, and drops it all on one dedicated encode thread, never sharing it by
// reference across threads, so the C handle is only ever touched from a single thread. Moving the
// whole value to that thread is therefore sound — there is no concurrent access to the handle.
unsafe impl Send for OpenH264Encoder {}
impl OpenH264Encoder {
+74 -223
View File
@@ -17,6 +17,9 @@
//! data packets are consumed immediately and missing parity only costs loss recovery — so
//! the validated stereo path stays byte-identical (data packets only, exactly as before).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
#[cfg(any(target_os = "linux", target_os = "windows", test))]
use crate::audio::SAMPLE_RATE;
#[cfg(any(target_os = "linux", target_os = "windows"))]
@@ -38,8 +41,6 @@ type Aes128CbcEnc = cbc::Encryptor<aes::Aes128>;
/// `RTP_PAYLOAD_TYPE_FEC 127`).
const AUDIO_PACKET_TYPE: u8 = 97;
const AUDIO_FEC_PACKET_TYPE: u8 = 127;
/// Stereo Opus bitrate (unchanged from the live-validated stereo path).
const OPUS_BITRATE: i32 = 128_000;
/// Audio FEC geometry (moonlight-common-c `RtpAudioQueue.h`: `RTPA_DATA_SHARDS 4`,
/// `RTPA_FEC_SHARDS 2`). Blocks are aligned: the client synthesizes the block base as
@@ -79,67 +80,20 @@ impl Default for AudioParams {
}
}
/// One Opus (multi)stream layout. Channel order is the GameStream/Moonlight order
/// FL FR FC LFE RL RR [SL SR]; `mapping` is the libopus multistream mapping we *encode*
/// with — identical to Sunshine's `audio.cpp stream_configs` (verified verbatim 2026-06-10):
/// identity mapping, so normal quality couples (FL,FR) and (FC,LFE) [+ (RL,RR) on 7.1] with
/// the remaining channels as mono streams; high quality is one mono stream per channel.
/// Bitrates are Sunshine's per-config values (stereo keeps punktfunk's existing 128 kbps).
pub struct OpusLayout {
pub channels: u8,
pub streams: u8,
pub coupled: u8,
pub mapping: &'static [u8],
pub bitrate: i32,
}
pub const LAYOUT_STEREO: OpusLayout = OpusLayout {
channels: 2,
streams: 1,
coupled: 1,
mapping: &[0, 1],
bitrate: OPUS_BITRATE,
};
pub const LAYOUT_51: OpusLayout = OpusLayout {
channels: 6,
streams: 4,
coupled: 2,
mapping: &[0, 1, 2, 3, 4, 5],
bitrate: 256_000,
};
pub const LAYOUT_51_HQ: OpusLayout = OpusLayout {
channels: 6,
streams: 6,
coupled: 0,
mapping: &[0, 1, 2, 3, 4, 5],
bitrate: 1_536_000,
};
pub const LAYOUT_71: OpusLayout = OpusLayout {
channels: 8,
streams: 5,
coupled: 3,
mapping: &[0, 1, 2, 3, 4, 5, 6, 7],
bitrate: 450_000,
};
pub const LAYOUT_71_HQ: OpusLayout = OpusLayout {
channels: 8,
streams: 8,
coupled: 0,
mapping: &[0, 1, 2, 3, 4, 5, 6, 7],
bitrate: 2_048_000,
// The Opus surround layout table (channel order FL FR FC LFE RL RR [SL SR], identity mapping,
// Sunshine's per-config bitrates) now lives in `punktfunk_core::audio`, shared with the native
// `punktfunk/1` path and every client decoder. Re-export the pieces the GameStream module + its
// RTSP SDP (`rtsp.rs`) reference; the GFE-specific `surround_params` SDP rotation stays below.
pub use punktfunk_core::audio::{
OpusLayout, LAYOUT_51, LAYOUT_51_HQ, LAYOUT_71, LAYOUT_71_HQ, LAYOUT_STEREO,
};
/// Pick the encoder layout for the negotiated session parameters. Unknown channel counts
/// fall back to stereo (the client can only request 2/6/8 — `AUDIO_CONFIGURATION_*` in
/// Pick the encoder layout for the negotiated session parameters. Thin wrapper over the shared
/// [`punktfunk_core::audio::layout_for`] keyed on this module's [`AudioParams`] (unknown channel
/// counts fall back to stereo; the client can only request 2/6/8 — `AUDIO_CONFIGURATION_*` in
/// Limelight.h).
pub fn layout_for(params: &AudioParams) -> &'static OpusLayout {
match (params.channels, params.high_quality) {
(6, false) => &LAYOUT_51,
(6, true) => &LAYOUT_51_HQ,
(8, false) => &LAYOUT_71,
(8, true) => &LAYOUT_71_HQ,
_ => &LAYOUT_STEREO,
}
punktfunk_core::audio::layout_for(params.channels, params.high_quality)
}
/// The `a=fmtp:97 surround-params=` digit string for a layout: channelCount, streams,
@@ -342,21 +296,21 @@ fn run(
}
/// Opus encoder for one session: the plain stereo encoder (the live-validated path, byte
/// identical) or a libopus multistream encoder for 5.1/7.1.
/// identical) or the safe `opus::MSEncoder` multistream encoder for 5.1/7.1. Both are
/// cross-platform (Linux + Windows) — surround no longer needs `audiopus_sys`.
#[cfg(any(target_os = "linux", target_os = "windows"))]
enum SessionEncoder {
Stereo(opus::Encoder),
// Surround needs the libopus *multistream* encoder via `audiopus_sys` (Linux-only dep).
#[cfg(target_os = "linux")]
Surround(MsEncoder),
Surround(opus::MSEncoder),
}
#[cfg(any(target_os = "linux", target_os = "windows"))]
impl SessionEncoder {
fn new(layout: &'static OpusLayout) -> Result<SessionEncoder> {
// RESTRICTED_LOWDELAY (`opus::Application::LowDelay`) + hard CBR, matching Sunshine — CBR
// keeps the Opus packet size constant, which the GameStream audio FEC (equal-length shards)
// relies on, and the client asserts a constant per-stream TOC.
if layout.channels == 2 {
// RESTRICTED_LOWDELAY + CBR, matching Sunshine — CBR keeps the Opus TOC byte
// constant, which the client asserts per stream.
let mut enc = opus::Encoder::new(
SAMPLE_RATE,
opus::Channels::Stereo,
@@ -367,113 +321,32 @@ impl SessionEncoder {
enc.set_vbr(false).ok();
Ok(SessionEncoder::Stereo(enc))
} else {
#[cfg(target_os = "linux")]
{
Ok(SessionEncoder::Surround(MsEncoder::new(layout)?))
}
#[cfg(not(target_os = "linux"))]
{
anyhow::bail!(
"surround audio ({} ch) needs the libopus multistream encoder (Linux only) — \
use a stereo session",
layout.channels
)
}
let mut enc = opus::MSEncoder::new(
SAMPLE_RATE,
layout.streams,
layout.coupled,
layout.mapping,
opus::Application::LowDelay,
)
.map_err(|e| anyhow::anyhow!("create Opus multistream encoder: {e}"))?;
enc.set_bitrate(opus::Bitrate::Bits(layout.bitrate)).ok();
enc.set_vbr(false).ok();
Ok(SessionEncoder::Surround(enc))
}
}
/// Encode one interleaved frame (`samples_per_channel * channels` f32s) into `out`,
/// returning the packet length.
fn encode_float(
&mut self,
frame: &[f32],
samples_per_channel: usize,
out: &mut [u8],
) -> Result<usize> {
// `samples_per_channel` only feeds the multistream (surround) encoder; stereo infers it.
#[cfg(not(target_os = "linux"))]
let _ = samples_per_channel;
/// Encode one interleaved frame into `out`, returning the packet length. Both encoders infer
/// the per-channel sample count from `frame.len()` and their channel count.
fn encode_float(&mut self, frame: &[f32], out: &mut [u8]) -> Result<usize> {
match self {
SessionEncoder::Stereo(enc) => enc.encode_float(frame, out).context("opus encode"),
#[cfg(target_os = "linux")]
SessionEncoder::Surround(enc) => enc.encode_float(frame, samples_per_channel, out),
SessionEncoder::Surround(enc) => enc
.encode_float(frame, out)
.context("opus multistream encode"),
}
}
}
/// RAII wrapper for `OpusMSEncoder` (the safe `opus` crate is stereo-only; the multistream
/// API comes from `audiopus_sys`, the same libopus the crate already links). Configured like
/// the stereo path: RESTRICTED_LOWDELAY, hard CBR, per-layout bitrate.
#[cfg(target_os = "linux")]
struct MsEncoder {
st: std::ptr::NonNull<audiopus_sys::OpusMSEncoder>,
}
// The raw encoder state has no thread affinity; the session owns it on one thread at a time.
#[cfg(target_os = "linux")]
unsafe impl Send for MsEncoder {}
#[cfg(target_os = "linux")]
impl MsEncoder {
fn new(layout: &OpusLayout) -> Result<MsEncoder> {
use std::os::raw::c_int;
let mut err: c_int = 0;
let st = unsafe {
audiopus_sys::opus_multistream_encoder_create(
SAMPLE_RATE as i32,
layout.channels as c_int,
layout.streams as c_int,
layout.coupled as c_int,
layout.mapping.as_ptr(),
audiopus_sys::OPUS_APPLICATION_RESTRICTED_LOWDELAY,
&mut err,
)
};
let st = std::ptr::NonNull::new(st)
.filter(|_| err == audiopus_sys::OPUS_OK)
.ok_or_else(|| anyhow::anyhow!("opus_multistream_encoder_create failed ({err})"))?;
unsafe {
audiopus_sys::opus_multistream_encoder_ctl(
st.as_ptr(),
audiopus_sys::OPUS_SET_BITRATE_REQUEST,
layout.bitrate as c_int,
);
audiopus_sys::opus_multistream_encoder_ctl(
st.as_ptr(),
audiopus_sys::OPUS_SET_VBR_REQUEST,
0 as c_int, // hard CBR (constant packet size — also what audio FEC relies on)
);
}
Ok(MsEncoder { st })
}
fn encode_float(
&mut self,
frame: &[f32],
samples_per_channel: usize,
out: &mut [u8],
) -> Result<usize> {
let n = unsafe {
audiopus_sys::opus_multistream_encode_float(
self.st.as_ptr(),
frame.as_ptr(),
samples_per_channel as std::os::raw::c_int,
out.as_mut_ptr(),
out.len() as i32,
)
};
anyhow::ensure!(n > 0, "opus_multistream_encode_float failed ({n})");
Ok(n as usize)
}
}
#[cfg(target_os = "linux")]
impl Drop for MsEncoder {
fn drop(&mut self) {
unsafe { audiopus_sys::opus_multistream_encoder_destroy(self.st.as_ptr()) }
}
}
#[cfg(any(target_os = "linux", target_os = "windows"))]
fn audio_body(
cap: &mut dyn AudioCapturer,
@@ -537,7 +410,7 @@ fn audio_body(
*s = (*s * gain).clamp(-1.0, 1.0);
}
}
let n = enc.encode_float(&frame, samples_per_channel, &mut out)?;
let n = enc.encode_float(&frame, &mut out)?;
// AES-128-CBC the Opus payload (RTP header stays plaintext). Per-packet IV =
// BE32(rikeyid + seq) in [0..4], zero elsewhere; PKCS7 padding.
let iv_seq = (rikeyid as u32).wrapping_add(seq as u32);
@@ -747,37 +620,33 @@ mod tests {
/// Real-codec proof of the 5.1 mapping math: encode with our encoder layout, decode with
/// the mapping a stock Moonlight client derives from our advertised surround-params
/// (parse → GFE swap), and verify a tone fed into each input channel comes out on the
/// same output channel.
#[cfg(target_os = "linux")]
/// same output channel. Cross-platform via the safe `opus` crate — this also guards the
/// (now un-gated) Windows GameStream surround build.
#[test]
fn multistream_51_roundtrip_channel_identity() {
let layout = &LAYOUT_51;
let samples = 240; // 5 ms
let ch = layout.channels as usize;
// Client-side decoder mapping derived exactly as moonlight-common-c does.
// Client-side decoder mapping derived exactly as moonlight-common-c does (GFE swap).
let s = surround_params(layout, false);
let digits: Vec<u8> = s.bytes().map(|b| b - b'0').collect();
let client_mapping = client_swap(&digits[3..]);
let mut err = 0i32;
let dec = unsafe {
audiopus_sys::opus_multistream_decoder_create(
SAMPLE_RATE as i32,
ch as i32,
layout.streams as i32,
layout.coupled as i32,
client_mapping.as_ptr(),
&mut err,
)
};
assert_eq!(err, audiopus_sys::OPUS_OK);
assert!(!dec.is_null());
let mut dec =
opus::MSDecoder::new(SAMPLE_RATE, layout.streams, layout.coupled, &client_mapping)
.expect("multistream decoder");
for tone_ch in 0..ch {
let mut enc = MsEncoder::new(layout).unwrap();
let mut enc = opus::MSEncoder::new(
SAMPLE_RATE,
layout.streams,
layout.coupled,
layout.mapping,
opus::Application::LowDelay,
)
.expect("multistream encoder");
let mut out = vec![0u8; 1400];
let mut decoded = vec![0f32; samples * ch];
let mut energy = vec![0f64; ch];
// A few frames so the codec converges past its startup transient.
for f in 0..8 {
@@ -787,23 +656,15 @@ mod tests {
/ SAMPLE_RATE as f32;
frame[t * ch + tone_ch] = 0.5 * phase.sin();
}
let n = enc.encode_float(&frame, samples, &mut out).unwrap();
let n = enc.encode_float(&frame, &mut out).unwrap();
assert!(n > 0);
let got = unsafe {
audiopus_sys::opus_multistream_decode_float(
dec,
out.as_ptr(),
n as i32,
decoded.as_mut_ptr(),
samples as i32,
0,
)
};
assert_eq!(got as usize, samples);
let mut decoded = vec![0f32; samples * ch];
let got = dec.decode_float(&out[..n], &mut decoded, false).unwrap();
assert_eq!(got, samples);
if f >= 4 {
for t in 0..samples {
for c in 0..ch {
energy[c] += (decoded[t * ch + c] as f64).powi(2);
for (c, e) in energy.iter_mut().enumerate() {
*e += (decoded[t * ch + c] as f64).powi(2);
}
}
}
@@ -817,7 +678,6 @@ mod tests {
(energies: {energy:?})"
);
}
unsafe { audiopus_sys::opus_multistream_decoder_destroy(dec) };
}
/// Live 5.1 capture → multistream encode → decode, against a real PipeWire session.
@@ -830,7 +690,15 @@ mod tests {
fn surround_capture_live() {
let mut cap = crate::audio::open_audio_capture(6).expect("open 6ch capture");
let layout = &LAYOUT_51;
let mut enc = MsEncoder::new(layout).unwrap();
let mut enc = opus::MSEncoder::new(
SAMPLE_RATE,
layout.streams,
layout.coupled,
layout.mapping,
opus::Application::LowDelay,
)
.unwrap();
enc.set_vbr(false).ok(); // hard CBR so packet sizes are constant (audio FEC relies on it)
let mut out = vec![0u8; 1400];
let mut acc: Vec<f32> = Vec::new();
let frame_len = 240 * 6;
@@ -841,41 +709,24 @@ mod tests {
acc.extend_from_slice(&chunk);
while acc.len() >= frame_len && packets < 100 {
let frame: Vec<f32> = acc.drain(..frame_len).collect();
let n = enc.encode_float(&frame, 240, &mut out).unwrap();
let n = enc.encode_float(&frame, &mut out).unwrap();
sizes.insert(n);
packets += 1;
}
}
// Hard CBR: every multistream packet must be the same size (audio FEC relies on it).
assert_eq!(sizes.len(), 1, "CBR sizes: {sizes:?}");
// And a stock client's decoder must accept them.
// And a stock client's GFE-derived decoder must accept them.
let s = surround_params(layout, false);
let digits: Vec<u8> = s.bytes().map(|b| b - b'0').collect();
let client_mapping = client_swap(&digits[3..]);
let mut err = 0i32;
let dec = unsafe {
audiopus_sys::opus_multistream_decoder_create(
48000,
6,
layout.streams as i32,
layout.coupled as i32,
client_mapping.as_ptr(),
&mut err,
)
};
assert_eq!(err, audiopus_sys::OPUS_OK);
let mut dec =
opus::MSDecoder::new(SAMPLE_RATE, layout.streams, layout.coupled, &client_mapping)
.unwrap();
let mut pcm = vec![0f32; 240 * 6];
let got = unsafe {
audiopus_sys::opus_multistream_decode_float(
dec,
out.as_ptr(),
*sizes.first().unwrap() as i32,
pcm.as_mut_ptr(),
240,
0,
)
};
unsafe { audiopus_sys::opus_multistream_decoder_destroy(dec) };
let got = dec
.decode_float(&out[..*sizes.first().unwrap()], &mut pcm, false)
.unwrap();
assert_eq!(got, 240);
}
}
@@ -1,7 +1,7 @@
//! Pairing crypto primitives (control plane only — distinct from `punktfunk_core`'s AES-GCM
//! data-plane sealing). GameStream pairing uses: AES-128-**ECB** with **no padding**,
//! SHA-256 (host appversion major ≥ 7), and RSA-PKCS1v15-SHA256 signatures. See the
//! `serverinfo + pairing` section of `docs/research/gamestream-protocol-research.json`.
//! `serverinfo + pairing` section of `design/research/gamestream-protocol-research.json`.
use aes::cipher::generic_array::GenericArray;
use aes::cipher::{BlockDecrypt, BlockEncrypt, KeyInit};
+21 -8
View File
@@ -1,7 +1,7 @@
//! GameStream (P1) control plane — what a stock Moonlight/Artemis client talks to around
//! the media streams: mDNS discovery, the nvhttp serverinfo + pairing HTTP(S) API, RTSP,
//! and the ENet control stream. `tokio`/`axum` live here (control plane, I/O-bound — never
//! the per-frame hot path; that is `punktfunk_core`'s P1 wire codec). See `docs/gamestream-host-plan.md`.
//! the per-frame hot path; that is `punktfunk_core`'s P1 wire codec). See `design/gamestream-host-plan.md`.
//!
//! Status: P1.1 — mDNS `_nvstream._tcp` advertisement + `/serverinfo`. Pairing, RTSP, and
//! the media streams follow (see the GameStream host task list / plan).
@@ -125,12 +125,21 @@ pub struct AppState {
/// (avoids a PipeWire stream setup per reconnect); drained on reuse so no stale audio is
/// sent, dropped + reopened when a session negotiates a different channel count.
pub audio_cap: std::sync::Arc<std::sync::Mutex<Option<Box<dyn crate::audio::AudioCapturer>>>>,
/// Shared streaming-stats recorder (web-console capture/graph). The GameStream encode loop
/// reads `is_armed()` per frame and emits samples; the same `Arc` is shared with the mgmt API
/// and the native punktfunk/1 loops so one capture spans whichever path is streaming.
pub stats: Arc<crate::stats_recorder::StatsRecorder>,
}
impl AppState {
/// Fresh control-plane state: no active session; the pairing allow-list is loaded from
/// disk (pairings persist across restarts).
pub fn new(host: Host, identity: cert::ServerIdentity) -> AppState {
/// disk (pairings persist across restarts). `stats` is the shared recorder handed to both the
/// mgmt API and the streaming loops.
pub fn new(
host: Host,
identity: cert::ServerIdentity,
stats: Arc<crate::stats_recorder::StatsRecorder>,
) -> AppState {
AppState {
host,
identity,
@@ -145,6 +154,7 @@ impl AppState {
rfi_range: std::sync::Arc::new(std::sync::Mutex::new(None)),
video_cap: std::sync::Arc::new(std::sync::Mutex::new(None)),
audio_cap: std::sync::Arc::new(std::sync::Mutex::new(None)),
stats,
}
}
}
@@ -166,7 +176,10 @@ pub fn serve(
) -> Result<()> {
let host = Host::detect()?;
let identity = cert::ServerIdentity::load_or_create().context("host certificate")?;
let state = Arc::new(AppState::new(host, identity));
// The shared streaming-stats recorder: one handle for the mgmt API, the GameStream encode loop
// (via `AppState`), and the native punktfunk/1 loops (passed to `punktfunk1::serve`).
let stats = crate::stats_recorder::StatsRecorder::new(crate::stats_recorder::default_dir());
let state = Arc::new(AppState::new(host, identity, stats.clone()));
// The native plane always runs, so the shared native-pairing handle (linking the QUIC ceremony
// and the management API) always exists.
let np = Arc::new(
@@ -206,8 +219,8 @@ pub fn serve(
);
tokio::try_join!(
nvhttp::run(state.clone()),
crate::mgmt::run(state.clone(), mgmt, Some(np.clone())),
crate::punktfunk1::serve(native_opts, np),
crate::mgmt::run(state.clone(), mgmt, Some(np.clone()), stats.clone()),
crate::punktfunk1::serve(native_opts, np, stats.clone()),
)?;
} else {
// Secure default: native punktfunk/1 + management API only (no GameStream surface).
@@ -217,8 +230,8 @@ pub fn serve(
(GameStream OFF pass --gamestream for stock-Moonlight compat)"
);
tokio::try_join!(
crate::mgmt::run(state.clone(), mgmt, Some(np.clone())),
crate::punktfunk1::serve(native_opts, np),
crate::mgmt::run(state.clone(), mgmt, Some(np.clone()), stats.clone()),
crate::punktfunk1::serve(native_opts, np, stats.clone()),
)?;
}
Ok(())
@@ -291,7 +291,10 @@ mod tests {
https_port: HTTPS_PORT,
};
let identity = super::super::cert::ServerIdentity::ephemeral().expect("ephemeral identity");
Arc::new(AppState::new(host, identity))
let stats = crate::stats_recorder::StatsRecorder::new(
std::env::temp_dir().join(format!("pf-nvhttp-stats-{}", std::process::id())),
);
Arc::new(AppState::new(host, identity, stats))
}
fn fp_of(der: &[u8]) -> String {
@@ -1,7 +1,7 @@
//! The 4-phase GameStream pairing state machine (over HTTP), keyed by `uniqueid`. Proves
//! both sides know the PIN (via the SHA-256(salt||pin) AES-ECB key) and own their certs
//! (RSA signatures), then pins the client cert. The final `pairchallenge` happens over
//! HTTPS (handled in `nvhttp`). Byte-exact spec: `docs/research/…-research.json`.
//! HTTPS (handled in `nvhttp`). Byte-exact spec: `design/research/…-research.json`.
use super::cert::ServerIdentity;
use super::crypto;
@@ -234,6 +234,7 @@ fn handle_request(req: &Request, state: &AppState) -> String {
state.force_idr.clone(),
state.rfi_range.clone(),
state.video_cap.clone(),
state.stats.clone(),
);
}
Some(_) => tracing::info!("RTSP PLAY — stream already running"),
+319 -50
View File
@@ -3,6 +3,9 @@
//! either real portal desktop capture (`PUNKTFUNK_VIDEO_SOURCE=portal`, the portal PipeWire path) or
//! a synthetic test pattern (default). Runs on its own native thread.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::video::{FrameType, VideoPacketizer};
use super::VIDEO_PORT;
use crate::capture::{self, Capturer, FastSyntheticCapturer};
@@ -45,6 +48,7 @@ pub fn start(
force_idr: Arc<AtomicBool>,
rfi_range: RfiSlot,
video_cap: CapturerSlot,
stats: Arc<crate::stats_recorder::StatsRecorder>,
) {
let _ = std::thread::Builder::new()
.name("punktfunk-video".into())
@@ -57,6 +61,7 @@ pub fn start(
&force_idr,
&rfi_range,
&video_cap,
&stats,
) {
tracing::error!(error = %format!("{e:#}"), "video stream failed");
}
@@ -65,6 +70,7 @@ pub fn start(
});
}
#[allow(clippy::too_many_arguments)]
fn run(
cfg: StreamConfig,
app: Option<&super::apps::AppEntry>,
@@ -72,6 +78,9 @@ fn run(
force_idr: &AtomicBool,
rfi_range: &std::sync::Mutex<Option<(i64, i64)>>,
video_cap: &std::sync::Mutex<Option<Box<dyn Capturer>>>,
// Shared stats recorder for the web-console capture/graph. Threaded into `stream_body` (the
// encode loop); per-frame sample emission is wired by a later pass.
stats: &Arc<crate::stats_recorder::StatsRecorder>,
) -> Result<()> {
// GameStream capture/encode thread: apply Windows session tuning (no-op off Windows).
crate::session_tuning::on_hot_thread();
@@ -97,18 +106,20 @@ fn run(
sock.connect(client)
.context("connect client video endpoint")?;
tracing::info!(%client, "video: client endpoint learned");
// Short label for web-console stats captures: the client's peer IP.
let client_label = client.ip().to_string();
// Native client-resolution source: create a compositor virtual output sized to the client's
// request and capture it (no scaling). Self-contained — deliberately NOT pooled in
// `video_cap`, since a reconnect at a different resolution needs a freshly-sized output; the
// output is released when this capturer drops at stream end (RAII via its keepalive).
if crate::config::config().video_source.as_deref() == Some("virtual") {
// The launched app picks the compositor (e.g. gamescope for game entries) and the
// nested command.
let compositor = app
.and_then(|a| a.compositor)
.map(Ok)
.unwrap_or_else(|| crate::vdisplay::detect().context("detect compositor"))?;
// Open the virtual-display source: pick the live compositor, normalize the session env
// (apply_session_env/apply_input_env — gamescope ATTACH/resize + KWin/Mutter retargeting,
// exactly like the native plane), create a virtual output at the client mode, and capture it.
// Re-runnable: the encode loop calls it again on a mid-stream capture loss to FOLLOW a
// Desktop<->Game switch.
let (mut capturer, compositor) = open_gs_virtual_source(cfg, app)?;
tracing::info!(
?compositor,
app = ?app.map(|a| &a.title),
@@ -116,32 +127,41 @@ fn run(
h = cfg.height,
"video source: virtual display (native client resolution)"
);
let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?;
// Carry the resolved launch command on the backend instance (per-session) rather than a
// process-global env var, so concurrent sessions can't stomp each other's launch target.
vd.set_launch_command(app.and_then(|a| a.cmd.clone()));
let vout = vd
.create(punktfunk_core::Mode {
width: cfg.width,
height: cfg.height,
refresh_hz: cfg.fps,
})
.context("create virtual output at client resolution")?;
// `want_hdr=false`: the IDD-push backend (opt-in PUNKTFUNK_IDD_PUSH) has no monitor-HDR
// auto-detection — it converts its always-FP16 ring per this flag — and GameStream HDR is not
// negotiated into StreamConfig here, so an IDD-push GameStream session streams SDR even on an
// HDR desktop. (The default WGC backend DOES auto-detect HDR from the output colorspace, but
// IDD-push bypasses WGC.) Acceptable for the experimental IDD-push A/B path; HDR over IDD-push
// is wired only for punktfunk/1 (want_hdr = negotiated bit_depth >= 10). TODO: derive want_hdr
// from a GameStream HDR flag once StreamConfig carries one.
let mut capturer = capture::capture_virtual_output(
vout,
capture::OutputFormat::resolve(false),
crate::session_plan::CaptureBackend::resolve(),
)
.context("capture virtual output")?;
capturer.set_active(true);
return stream_body(&mut *capturer, &sock, cfg, running, force_idr, rfi_range);
// Launch the app's command now that capture is live, for the backends that DON'T nest it via
// set_launch_command above: Windows (no gamescope) and Linux kwin/mutter/wlroots (which stream
// the existing desktop, so the app must be spawned into the session to land on the streamed
// output). Linux gamescope already nested it via set_launch_command, so skip it there.
#[cfg(windows)]
let launch_here = true;
#[cfg(target_os = "linux")]
let launch_here = compositor != crate::vdisplay::Compositor::Gamescope;
#[cfg(any(windows, target_os = "linux"))]
if launch_here {
if let Some(cmd) = app
.and_then(|a| a.cmd.as_deref())
.filter(|c| !c.trim().is_empty())
{
if let Err(e) = crate::library::launch_gamestream_command(cmd) {
tracing::warn!(command = %cmd, error = %e, "gamestream: could not launch app");
}
}
}
// Rebuild closure: re-open the source on a mid-stream capture loss, RE-DETECTING the live
// compositor — so a Desktop<->Game switch (at the client's fixed mode) is FOLLOWED in place
// without a Moonlight reconnect. (A resolution change can't be followed mid-stream on
// GameStream — WxH is locked at ANNOUNCE — but a session toggle keeps the negotiated mode.)
let rebuild = || open_gs_virtual_source(cfg, app).map(|(c, _)| c);
return stream_body(
&mut capturer,
Some(&rebuild),
&sock,
cfg,
running,
force_idr,
rfi_range,
stats,
&client_label,
);
}
// Reuse the persistent capturer (one screencast session → clean reconnect); create it on
@@ -161,12 +181,70 @@ fn run(
}
};
capturer.set_active(true);
let result = stream_body(&mut *capturer, &sock, cfg, running, force_idr, rfi_range);
// Portal/synthetic source: no compositor virtual output to re-detect, so no rebuild closure.
let result = stream_body(
&mut capturer,
None,
&sock,
cfg,
running,
force_idr,
rfi_range,
stats,
&client_label,
);
capturer.set_active(false);
*video_cap.lock().unwrap() = Some(capturer);
result
}
/// Open the virtual-display video source for a GameStream session: pick the LIVE compositor + normalize
/// the session env (apply_session_env/apply_input_env — gamescope ATTACH/resize, KWin/Mutter
/// retargeting) exactly like the native plane (punktfunk1.rs resolve_compositor), create a virtual
/// output at the client's mode, and capture it. Returns the capturer (it owns the output's keepalive;
/// the stateless VirtualDisplay factory is dropped here) plus the resolved compositor. An apps.json
/// entry can PIN a compositor (skips the live detect/retarget). Re-run on a mid-stream capture loss to
/// FOLLOW a Desktop<->Game switch: it re-detects the now-live compositor and re-targets at it. Does NOT
/// launch the app (that happens once at stream start; a rebuild must not re-spawn it).
fn open_gs_virtual_source(
cfg: StreamConfig,
app: Option<&super::apps::AppEntry>,
) -> Result<(Box<dyn Capturer>, crate::vdisplay::Compositor)> {
let compositor = if let Some(c) = app.and_then(|a| a.compositor) {
c
} else {
let active = crate::vdisplay::detect_active_session();
crate::vdisplay::apply_session_env(&active);
let c = crate::vdisplay::compositor_for_kind(active.kind)
.map(Ok)
.unwrap_or_else(crate::vdisplay::detect)
.context("detect compositor")?;
crate::vdisplay::apply_input_env(c);
c
};
let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?;
// Carry the resolved launch command on the backend instance (per-session) rather than a
// process-global env var, so concurrent sessions can't stomp each other's launch target.
vd.set_launch_command(app.and_then(|a| a.cmd.clone()));
let vout = vd
.create(punktfunk_core::Mode {
width: cfg.width,
height: cfg.height,
refresh_hz: cfg.fps,
})
.context("create virtual output at client resolution")?;
// want_hdr=false: GameStream HDR is not negotiated into StreamConfig here (the default WGC backend
// still auto-detects HDR from the output colorspace; only the opt-in IDD-push path streams SDR).
let capturer = capture::capture_virtual_output(
vout,
capture::OutputFormat::resolve(false),
crate::session_plan::CaptureBackend::resolve(),
)
.context("capture virtual output")?;
capturer.set_active(true);
Ok((capturer, compositor))
}
/// One frame's packets, handed from the encode thread to the send thread.
type PacketBatch = Vec<Vec<u8>>;
@@ -188,6 +266,10 @@ fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
let mut hdrs: Vec<libc::mmsghdr> = iovs
.iter_mut()
.map(|iov| {
// SAFETY: `libc::mmsghdr` is a plain `#[repr(C)]` struct of integers and raw
// pointers, for which an all-zero bit pattern is valid (null pointers / zero
// lengths); the fields we rely on (`msg_iov`, `msg_iovlen`) are overwritten on the
// next two lines before the struct is handed to the kernel.
let mut h: libc::mmsghdr = unsafe { std::mem::zeroed() };
h.msg_hdr.msg_iov = iov;
h.msg_hdr.msg_iovlen = 1;
@@ -196,6 +278,13 @@ fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
.collect();
let mut off = 0usize;
while off < hdrs.len() {
// SAFETY: `fd` is `sock`'s live raw fd (`sock` outlives the call). `hdrs[off..]
// .as_mut_ptr()` is a live slice of `(hdrs.len() - off)` `mmsghdr`s — exactly the count
// passed — into which the kernel writes each `msg_len`. Each header's `msg_iov` points
// into `iovs` (a local that outlives this call, with `msg_iovlen == 1` matching its one
// entry) and each `iovec.iov_base` points into the `chunk` packet buffers (the caller's
// `pkts`, alive for the call); the kernel only reads those payloads. Flags 0; the return
// is error-/progress-checked before advancing `off`.
let n = unsafe {
libc::sendmmsg(fd, hdrs[off..].as_mut_ptr(), (hdrs.len() - off) as u32, 0)
};
@@ -293,15 +382,36 @@ fn spawn_sender(
Ok(())
}
/// Percentile of a slice (sorts it in place first). `q` in `0.0..=1.0`. Used for the web-console
/// stats sample's per-stage p50/p99.
fn percentile(v: &mut [u32], q: f64) -> u32 {
if v.is_empty() {
return 0;
}
v.sort_unstable();
let i = ((v.len() as f64 * q) as usize).min(v.len() - 1);
v[i]
}
/// The encode → packetize loop, over a borrowed capturer. Sending runs on a dedicated thread
/// (see [`spawn_sender`]) so a send spike can never stall capture/encode.
#[allow(clippy::too_many_arguments)]
fn stream_body(
capturer: &mut dyn Capturer,
// `&mut Box` (not `&mut dyn`) so a mid-stream capture-loss rebuild can SWAP the capturer in place.
capturer: &mut Box<dyn Capturer>,
// Re-open the video source on capture loss (virtual-display path → follow a Desktop<->Game switch);
// `None` for the portal/synthetic source, which has nothing to re-detect (propagate the error).
rebuild: Option<&dyn Fn() -> Result<Box<dyn Capturer>>>,
sock: &UdpSocket,
cfg: StreamConfig,
running: &Arc<AtomicBool>,
force_idr: &AtomicBool,
rfi_range: &std::sync::Mutex<Option<(i64, i64)>>,
// Shared stats recorder. The encode loop reads `stats.is_armed()` per frame to decide whether
// to accumulate the per-stage split, then emits a `StatsSample` at its 1 s aggregation boundary.
stats: &Arc<crate::stats_recorder::StatsRecorder>,
// Short client label (peer IP) seeded into the capture meta on the first armed registration.
client_label: &str,
) -> Result<()> {
// The first frame establishes the authoritative size/format for the encoder.
let mut frame = capturer.next_frame().context("capture first frame")?;
@@ -321,6 +431,9 @@ fn stream_body(
cfg.bitrate_kbps as u64 * 1000,
frame.is_cuda(),
8, // GameStream/Moonlight path: 8-bit (its own codec negotiation)
// GameStream/Moonlight stays 4:2:0 — stock Moonlight clients can't decode 4:4:4, and the
// protocol has no chroma negotiation. 4:4:4 is punktfunk/1-native only.
encode::ChromaFormat::Yuv420,
)
.context("open video encoder for stream")?;
// FEC overhead percent (Sunshine default 20). Override with PUNKTFUNK_FEC_PCT (0 = data-only).
@@ -365,19 +478,100 @@ fn stream_body(
let perf = crate::config::config().perf;
let (mut mx_cap, mut mx_enc, mut mx_pkt, mut mx_send, mut mx_pkts, mut uniq) =
(0u128, 0u128, 0u128, 0u128, 0usize, 0u32);
// Web-console stats accumulation (active when `perf` OR a capture is armed): per-stage vectors
// for p50/p99, the goodput bytes queued to the sender this window, the previous window's
// dropped-frame count for delta computation, and the registration id cached on the first sample.
let codec_name = match cfg.codec {
Codec::H264 => "h264",
Codec::H265 => "hevc",
Codec::Av1 => "av1",
};
let mut sid: Option<u32> = None;
let (mut v_cap, mut v_enc, mut v_pkt, mut v_send): (Vec<u32>, Vec<u32>, Vec<u32>, Vec<u32>) =
(Vec::new(), Vec::new(), Vec::new(), Vec::new());
let mut bytes_win: u64 = 0;
let mut last_dropped_batches: u64 = 0;
// Absolute next-frame deadline — the single pacing clock for the loop.
let mut next_frame = Instant::now();
// RFI capability is fixed for the session (probed at encoder open). Query it once so the
// recovery path skips the always-`false` invalidate call on encoders without NVENC RFI and
// forces a keyframe directly instead.
let supports_rfi = enc.caps().supports_rfi;
let mut supports_rfi = enc.caps().supports_rfi;
// Bound consecutive capture-loss rebuilds (a delivered frame clears the counter) so a permanently
// dead source can't loop forever — it ends the stream after the cap, falling back to a reconnect.
const MAX_REBUILDS: u32 = 5;
let mut rebuilds: u32 = 0;
while running.load(Ordering::SeqCst) {
let tick = Instant::now();
// Measure per-stage timing when `PUNKTFUNK_PERF` is set OR a web-console stats capture is
// armed (cheap Relaxed atomic, re-read each frame).
let measure = perf || stats.is_armed();
// Advance to the freshest captured frame if one arrived; otherwise reuse the last.
if let Some(f) = capturer.try_latest().context("capture frame")? {
frame = f;
uniq += 1;
match capturer.try_latest() {
Ok(Some(f)) => {
frame = f;
uniq += 1;
rebuilds = 0; // a delivered frame clears the consecutive-loss counter
}
Ok(None) => {} // no new frame — reuse the last (static/idle desktop)
Err(e) => {
// The capture source went away — the compositor was torn down on a Desktop<->Game
// switch, or the virtual output was removed. On the virtual-display path, re-detect the
// now-live compositor and re-attach IN PLACE (the send thread + packetizer + socket +
// RTP clock all survive), then force an IDR so Moonlight resyncs — so the stream FOLLOWS
// the switch with no client reconnect. Build the new source BEFORE dropping the old.
// Bounded by a counter + a ~40s budget; on exhaustion, end the stream (Moonlight
// reconnect). The portal/synthetic path has no rebuild closure → propagate as before.
let Some(rebuild) = rebuild else {
return Err(e).context("capture frame");
};
rebuilds += 1;
if rebuilds > MAX_REBUILDS {
return Err(e).context("capture lost — rebuild attempts exhausted");
}
tracing::warn!(error = %format!("{e:#}"), rebuild = rebuilds,
"gamestream: capture lost — rebuilding source in place (following a session switch)");
let rebuild_deadline = Instant::now() + Duration::from_secs(40);
let new_cap = loop {
match rebuild() {
Ok(c) => break c,
Err(e2) => {
if !running.load(Ordering::SeqCst) || Instant::now() >= rebuild_deadline
{
return Err(e2)
.context("capture lost — no source within the rebuild budget");
}
tracing::warn!(error = %format!("{e2:#}"),
"gamestream: source not up yet — retrying");
std::thread::sleep(Duration::from_millis(500));
}
}
};
*capturer = new_cap;
capturer.set_active(true);
frame = capturer.next_frame().context("first frame after rebuild")?;
// Re-open the encoder for the new source (same negotiated WxH → same SPS profile) and
// force an IDR so Moonlight resyncs on the first emitted AU.
enc = encode::open_video(
cfg.codec,
frame.format,
frame.width,
frame.height,
cfg.fps,
cfg.bitrate_kbps as u64 * 1000,
frame.is_cuda(),
8,
encode::ChromaFormat::Yuv420, // GameStream stays 4:2:0
)
.context("reopen encoder after rebuild")?;
supports_rfi = enc.caps().supports_rfi;
enc.request_keyframe();
next_frame = Instant::now();
tracing::info!("gamestream: source rebuilt — stream continues");
continue;
}
}
let t_cap = tick.elapsed();
// Honor a client recovery request. Prefer reference-frame invalidation (the encoder
@@ -414,9 +608,19 @@ fn stream_body(
// Hand the frame's packets to the send thread; never block here. A full queue means
// the sender is behind — drop this batch (FEC/RFI covers the client) and keep encoding.
let n = batch.len();
// Goodput this window = bytes actually queued to the sender (a dropped batch never reaches
// the wire, so it's excluded). Summed only when measuring, to keep the idle path free.
let batch_bytes: u64 = if measure {
batch.iter().map(|p| p.len() as u64).sum()
} else {
0
};
if n > 0 {
match batch_tx.try_send(batch) {
Ok(()) => sent_batches += 1,
Ok(()) => {
sent_batches += 1;
bytes_win += batch_bytes;
}
Err(std::sync::mpsc::TrySendError::Full(_)) => {
dropped_batches += 1;
if dropped_batches.is_power_of_two() {
@@ -428,17 +632,26 @@ fn stream_body(
}
}
}
if perf {
if measure {
let t_send = tick.elapsed();
mx_cap = mx_cap.max(t_cap.as_micros());
mx_enc = mx_enc.max((t_enc - t_cap).as_micros());
mx_pkt = mx_pkt.max((t_pkt - t_enc).as_micros());
mx_send = mx_send.max((t_send - t_pkt).as_micros());
let cap_us = t_cap.as_micros();
let enc_us = (t_enc - t_cap).as_micros();
let pkt_us = (t_pkt - t_enc).as_micros();
let send_us = (t_send - t_pkt).as_micros();
mx_cap = mx_cap.max(cap_us);
mx_enc = mx_enc.max(enc_us);
mx_pkt = mx_pkt.max(pkt_us);
mx_send = mx_send.max(send_us);
mx_pkts = mx_pkts.max(n);
v_cap.push(cap_us as u32);
v_enc.push(enc_us as u32);
v_pkt.push(pkt_us as u32);
v_send.push(send_us as u32);
}
fps_count += 1;
if fps_t.elapsed() >= Duration::from_secs(1) {
let secs = fps_t.elapsed().as_secs_f64();
if perf {
// Max µs/stage this second: cap=drain channel, enc=submit (zero-copy device
// copy + NVENC), pkt=poll+FEC+packetize, send=paced packet send. `uniq`=new
@@ -453,12 +666,6 @@ fn stream_body(
max_pkts = mx_pkts,
"video: streaming (perf)"
);
mx_cap = 0;
mx_enc = 0;
mx_pkt = 0;
mx_send = 0;
mx_pkts = 0;
uniq = 0;
} else {
tracing::info!(
fps = fps_count,
@@ -467,6 +674,68 @@ fn stream_body(
"video: streaming"
);
}
// Web-console capture: build the aggregated sample. The host send side exposes no
// receiver-side packet loss / FEC-recovery / send-buffer EAGAIN counters, so those stay
// 0 (not fabricated); `frames_dropped` is the per-frame send-queue overflow delta.
if stats.is_armed() {
let session_id = *sid.get_or_insert_with(|| {
stats.register_session(
"gamestream",
cfg.width,
cfg.height,
cfg.fps,
codec_name,
client_label,
)
});
let sample = crate::stats_recorder::StatsSample {
t_ms: 0, // stamped by push_sample from the capture's monotonic start
session_id,
stages: vec![
crate::stats_recorder::StageTiming {
name: "capture".into(),
p50_us: percentile(&mut v_cap, 0.50) as f32,
p99_us: percentile(&mut v_cap, 0.99) as f32,
},
crate::stats_recorder::StageTiming {
name: "encode".into(),
p50_us: percentile(&mut v_enc, 0.50) as f32,
p99_us: percentile(&mut v_enc, 0.99) as f32,
},
crate::stats_recorder::StageTiming {
name: "packetize".into(),
p50_us: percentile(&mut v_pkt, 0.50) as f32,
p99_us: percentile(&mut v_pkt, 0.99) as f32,
},
crate::stats_recorder::StageTiming {
name: "send".into(),
p50_us: percentile(&mut v_send, 0.50) as f32,
p99_us: percentile(&mut v_send, 0.99) as f32,
},
],
fps: (uniq as f64 / secs) as f32,
repeat_fps: (fps_count.saturating_sub(uniq) as f64 / secs) as f32,
mbps: (bytes_win as f64 * 8.0 / secs / 1_000_000.0) as f32,
bitrate_kbps: cfg.bitrate_kbps,
frames_dropped: dropped_batches.saturating_sub(last_dropped_batches) as u32,
packets_dropped: 0,
send_dropped: 0,
fec_recovered: 0,
};
stats.push_sample(session_id, sample);
}
mx_cap = 0;
mx_enc = 0;
mx_pkt = 0;
mx_send = 0;
mx_pkts = 0;
uniq = 0;
v_cap.clear();
v_enc.clear();
v_pkt.clear();
v_send.clear();
bytes_win = 0;
last_dropped_batches = dropped_batches;
fps_count = 0;
fps_t = Instant::now();
}
@@ -3,7 +3,7 @@
//! `RTP_PACKET(12, big-endian) + reserved[4] + NV_VIDEO_PACKET(16, little-endian) + payload`
//! and the frame's bitstream is prefixed with an 8-byte `video_short_frame_header_t`, then
//! striped into ≤4 FEC blocks of ≤255 shards. Byte-exact spec:
//! `docs/research/gamestream-protocol-research.json` (video plane).
//! `design/research/gamestream-protocol-research.json` (video plane).
//!
//! FEC (P1.5): each block carries `m = ⌈k·pct/100⌉` ReedSolomon parity shards generated by
//! `punktfunk_core::fec::Gf8Coder` (the nanors-compatible Cauchy GF(2⁸) coder). Crucially, RS runs
+41 -9
View File
@@ -24,6 +24,9 @@ pub trait InputInjector {
pub enum Backend {
/// wlroots virtual pointer + keyboard Wayland protocols — the headless-Sway path.
WlrVirtual,
/// KWin `org_kde_kwin_fake_input` — direct injection, no RemoteDesktop portal / approval dialog
/// (authorized by the host's `.desktop`). The headless KDE-Desktop path; what krdpserver uses.
KwinFakeInput,
/// libei via `reis` — Wayland-native (RemoteDesktop portal). Not yet implemented.
Libei,
/// libei directly against gamescope's own EIS socket (no portal): input lands in the
@@ -47,6 +50,16 @@ pub fn open(backend: Backend) -> Result<Box<dyn InputInjector>> {
anyhow::bail!("wlroots virtual input requires Linux + a Wayland compositor")
}
}
Backend::KwinFakeInput => {
#[cfg(target_os = "linux")]
{
Ok(Box::new(kwin_fake_input::KwinFakeInjector::open()?))
}
#[cfg(not(target_os = "linux"))]
{
anyhow::bail!("KWin fake_input requires Linux + a KWin Wayland session")
}
}
Backend::Libei => {
#[cfg(target_os = "linux")]
{
@@ -90,12 +103,18 @@ pub fn open(backend: Backend) -> Result<Box<dyn InputInjector>> {
/// Pick the injection backend for the current session. gamescope hosts its own EIS server (no
/// portal), so a gamescope session injects directly into it. wlroots/Sway only implements the
/// ScreenCast portal (no RemoteDesktop), so libei can't run there — use the wlr virtual-input
/// protocols. KWin and GNOME implement RemoteDesktop but not the wlr protocols, so use libei.
/// `PUNKTFUNK_INPUT_BACKEND=wlr|libei|gamescope|uinput` overrides the auto-detection.
/// protocols. **KWin** exposes `org_kde_kwin_fake_input` (direct injection, no portal / approval
/// dialog — the only headless-capable path; what krdpserver uses), so prefer it there. **GNOME**
/// has neither fake_input nor the wlr protocols, so it uses libei via the RemoteDesktop portal
/// (which needs a user to approve, or a pre-seeded grant — not truly headless).
/// `PUNKTFUNK_INPUT_BACKEND=wlr|kwin|libei|gamescope|uinput` overrides the auto-detection.
pub fn default_backend() -> Backend {
if let Ok(v) = std::env::var("PUNKTFUNK_INPUT_BACKEND") {
match v.trim().to_ascii_lowercase().as_str() {
"wlr" | "wlroots" | "wlrvirtual" => return Backend::WlrVirtual,
"kwin" | "fakeinput" | "fake_input" | "kwin-fake-input" => {
return Backend::KwinFakeInput
}
"libei" | "ei" | "portal" => return Backend::Libei,
"gamescope" | "gamescope-ei" => return Backend::GamescopeEi,
"uinput" => return Backend::Uinput,
@@ -112,16 +131,26 @@ pub fn default_backend() -> Backend {
}
#[cfg(not(target_os = "windows"))]
{
if crate::config::config()
.compositor
.as_deref()
.is_some_and(|v| v.trim().eq_ignore_ascii_case("gamescope"))
{
return Backend::GamescopeEi;
// An explicit compositor pick (set per connect / mid-stream) is the strongest signal.
let compositor = crate::config::config().compositor.clone();
if let Some(c) = compositor.as_deref() {
let c = c.trim();
if c.eq_ignore_ascii_case("gamescope") {
return Backend::GamescopeEi;
}
if c.eq_ignore_ascii_case("kwin") {
return Backend::KwinFakeInput;
}
if c.eq_ignore_ascii_case("wlroots") || c.eq_ignore_ascii_case("sway") {
return Backend::WlrVirtual;
}
// mutter (GNOME) falls through to the XDG_CURRENT_DESKTOP check below.
}
let desktop = std::env::var("XDG_CURRENT_DESKTOP").unwrap_or_default();
let d = desktop.to_ascii_uppercase();
if d.contains("KDE") || d.contains("GNOME") {
if d.contains("KDE") {
Backend::KwinFakeInput
} else if d.contains("GNOME") {
Backend::Libei
} else {
Backend::WlrVirtual
@@ -478,6 +507,9 @@ pub mod gamepad {
}
}
#[cfg(target_os = "linux")]
#[path = "inject/linux/kwin_fake_input.rs"]
mod kwin_fake_input;
#[cfg(target_os = "linux")]
#[path = "inject/linux/libei.rs"]
mod libei;
#[cfg(target_os = "windows")]
@@ -15,6 +15,9 @@
//! `<linux/uinput.h>` on x86_64. `/dev/uinput` needs a udev rule + `input` group membership
//! (see `scripts/60-punktfunk.rules`); creation fails with a clear error otherwise.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use crate::gamestream::gamepad::{self, GamepadFrame, MAX_PADS};
use anyhow::{bail, Result};
use std::collections::HashMap;
@@ -215,6 +218,11 @@ const _: () = {
};
fn ioctl_int(fd: i32, req: libc::c_ulong, arg: libc::c_int, what: &str) -> Result<()> {
// SAFETY: every caller passes one of UI_SET_EVBIT/KEYBIT/FFBIT/UI_DEV_CREATE/UI_DEV_DESTROY as
// `req` — all integer-argument ioctls whose third arg the kernel takes BY VALUE, so nothing is
// dereferenced through `arg` and no memory must outlive the call. The only precondition is `fd`
// being a valid open descriptor; callers pass the live `/dev/uinput` fd, and even a stale fd
// would merely return -1/EBADF (reported below), never UB.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
@@ -222,6 +230,12 @@ fn ioctl_int(fd: i32, req: libc::c_ulong, arg: libc::c_int, what: &str) -> Resul
}
fn ioctl_ptr<T>(fd: i32, req: libc::c_ulong, arg: *mut T, what: &str) -> Result<()> {
// SAFETY: `fd` is the caller's live `/dev/uinput` fd. Every call site passes `&mut x` for a live,
// uniquely-borrowed `#[repr(C)]` `x: T` whose size matches the struct the request number encodes
// (UI_DEV_SETUP=0x405c_5503 → 0x5c=92=size_of::<UinputSetup>(); UI_ABS_SETUP → 0x1c=28; the FF
// upload/erase ioctls → 0x68/0x0c — all pinned by the `size_of` asserts above). The kernel copies
// exactly that many bytes in/out through `arg`; the `&mut` keeps the pointee alive and unaliased
// for the whole synchronous call.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
@@ -251,6 +265,9 @@ pub struct VirtualPad {
impl VirtualPad {
pub fn create(index: usize, identity: PadIdentity) -> Result<VirtualPad> {
use std::os::fd::FromRawFd;
// SAFETY: `c"/dev/uinput"` is a 'static NUL-terminated C string literal; `as_ptr()` yields a
// valid pointer the kernel only reads as a filesystem path. `open` returns a fresh fd (or -1)
// and retains nothing; no Rust memory is aliased or handed to the kernel beyond that 'static path.
let raw = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
@@ -264,6 +281,9 @@ impl VirtualPad {
std::io::Error::last_os_error()
);
}
// SAFETY: `raw >= 0` here (the `< 0` branch above already bailed), so it is a freshly-opened fd
// from `libc::open` that is not stored or owned anywhere else. Transferring it to `OwnedFd` makes
// this the unique owner, which will `close` it exactly once on drop (no double-close, no leak).
let fd = unsafe { OwnedFd::from_raw_fd(raw) };
ioctl_int(raw, UI_SET_EVBIT, EV_KEY as i32, "UI_SET_EVBIT(EV_KEY)")?;
@@ -356,6 +376,11 @@ impl VirtualPad {
code,
value,
};
// SAFETY: `ev` is a live local `#[repr(C)]` struct of all-integer fields with no padding bytes
// (timeval=16 + u16 + u16 + i32 = 24, the size asserted above), so every byte is initialized and
// valid to read as `u8`. The pointer is non-null and `u8`-aligned (align 1), the length is exactly
// `size_of::<InputEventRaw>()` so the slice spans precisely `ev`'s bytes (in bounds), and `ev`
// outlives `bytes` (used immediately below) with no concurrent mutation (single-threaded local).
let bytes = unsafe {
std::slice::from_raw_parts(
&ev as *const _ as *const u8,
@@ -363,6 +388,10 @@ impl VirtualPad {
)
};
// Best-effort: a full kernel queue drops the event; the next frame re-syncs state.
// SAFETY: `self.fd` is the live uinput `OwnedFd` (borrowed via `as_raw_fd`, so it stays open for
// the call); `bytes` is the slice above backed by the still-live local `ev`. `write` only READS
// exactly `bytes.len()` bytes from `bytes.as_ptr()` (in bounds) and retains nothing past return,
// so the buffer outlives the synchronous call and the read-only access cannot race or alias.
let _ = unsafe {
libc::write(
self.fd.as_raw_fd(),
@@ -404,6 +433,10 @@ impl VirtualPad {
let raw = self.fd.as_raw_fd();
let mut buf = [0u8; std::mem::size_of::<InputEventRaw>()];
loop {
// SAFETY: `raw` is the live raw fd of `self.fd` (the non-blocking uinput device). `buf` is a
// live local `[u8; size_of::<InputEventRaw>()]`; `buf.as_mut_ptr()` is a valid writable pointer
// to its `buf.len()` bytes. `read` writes AT MOST `buf.len()` bytes (in bounds), the buffer
// outlives this synchronous call, and `buf` is borrowed uniquely here (no alias/race).
let n = unsafe { libc::read(raw, buf.as_mut_ptr() as *mut libc::c_void, buf.len()) };
if n != buf.len() as isize {
break; // EAGAIN / short read — queue drained
@@ -415,6 +448,10 @@ impl VirtualPad {
unsafe { std::ptr::read_unaligned(buf.as_ptr() as *const InputEventRaw) };
match (ev.type_, ev.code) {
(EV_UINPUT, UI_FF_UPLOAD) => {
// SAFETY: `UinputFfUpload` is `#[repr(C)]` over integers (`u32`, `i32`) and two
// `FfEffect`s (integers + `[u8; 32]`); all-zero is a valid bit pattern for every field
// (no bool/NonZero/enum/reference niche), so `zeroed` yields a fully-initialized valid
// value — `request_id` is then set below and the rest filled by UI_BEGIN_FF_UPLOAD.
let mut up: UinputFfUpload = unsafe { std::mem::zeroed() };
up.request_id = ev.value as u32;
if ioctl_ptr(raw, UI_BEGIN_FF_UPLOAD, &mut up, "UI_BEGIN_FF_UPLOAD").is_ok() {
@@ -442,6 +479,9 @@ impl VirtualPad {
}
}
(EV_UINPUT, UI_FF_ERASE) => {
// SAFETY: `UinputFfErase` is `#[repr(C)]` over three integer fields (`u32`, `i32`,
// `u32`); all-zero is a valid bit pattern for each, so `zeroed` produces a fully-valid
// initialized value — `request_id` is set below and `effect_id` filled by the ioctl.
let mut er: UinputFfErase = unsafe { std::mem::zeroed() };
er.request_id = ev.value as u32;
if ioctl_ptr(raw, UI_BEGIN_FF_ERASE, &mut er, "UI_BEGIN_FF_ERASE").is_ok() {
@@ -492,6 +532,9 @@ impl VirtualPad {
impl Drop for VirtualPad {
fn drop(&mut self) {
// SAFETY: `self.fd` is still the live owned uinput fd here (the `OwnedFd` field is closed only
// AFTER this `drop` body returns), borrowed by `as_raw_fd`. UI_DEV_DESTROY takes its argument
// (0) BY VALUE, so nothing is dereferenced or aliased; the ioctl just tears down the device.
let _ = unsafe { libc::ioctl(self.fd.as_raw_fd(), UI_DEV_DESTROY, 0) };
}
}
@@ -0,0 +1,209 @@
//! Headless input injection on KWin via the privileged `org_kde_kwin_fake_input` protocol — the
//! exact path KDE's own headless RDP server (`krdpserver`) uses. KWin advertises this restricted
//! global only to a client authorized through its installed `.desktop` `X-KDE-Wayland-Interfaces`
//! (we ship `io.unom.Punktfunk.Host.desktop`, which lists `org_kde_kwin_fake_input` alongside
//! `zkde_screencast_unstable_v1`). Binding the global IS the authorization, so injection needs **no
//! RemoteDesktop portal and no "Allow remote control?" dialog** — it works with no user present,
//! which the libei/portal path cannot. We connect as an ordinary Wayland client on the KWin session's
//! `$WAYLAND_DISPLAY` and translate events into fake-input requests; keyboard keys are raw Linux
//! evdev codes that KWin resolves through the session's own keymap (no keymap upload, unlike the wlr
//! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space — which
//! on a headless box (single per-session virtual output at the origin, scale 1) equals the streamed
//! output's pixels.
#![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{gs_button_to_evdev, vk_to_evdev, InputEvent, InputInjector};
use anyhow::{Context, Result};
use punktfunk_core::input::InputKind;
use wayland_client::protocol::wl_registry::{self, WlRegistry};
use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle};
// Generate the client bindings for the vendored protocol XML inline (no build.rs), exactly like the
// KWin virtual-output backend. Path is relative to CARGO_MANIFEST_DIR.
#[allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
pub mod fake {
use wayland_client;
use wayland_client::protocol::*;
pub mod __interfaces {
use wayland_client::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("protocols/fake-input.xml");
}
use self::__interfaces::*;
wayland_scanner::generate_client_code!("protocols/fake-input.xml");
}
use fake::org_kde_kwin_fake_input::OrgKdeKwinFakeInput as FakeInput;
/// Highest interface version we drive. `keyboard_key` arrived at v4; KWin advertises ≥4.
const MAX_VERSION: u32 = 4;
/// `wl_pointer.axis` values used by `axis`.
const AXIS_VERTICAL: u32 = 0;
const AXIS_HORIZONTAL: u32 = 1;
/// `code` value marking a horizontal scroll event (mirrors `gamestream::input` / the wlr backend).
const SCROLL_HORIZONTAL: u32 = 1;
/// Registry-bound globals (the Wayland dispatch state).
#[derive(Default)]
struct State {
fake: Option<FakeInput>,
}
impl Dispatch<WlRegistry, ()> for State {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wl_registry::Event,
_: &(),
_: &Connection,
qh: &QueueHandle<Self>,
) {
if let wl_registry::Event::Global {
name,
interface,
version,
} = event
{
if interface == "org_kde_kwin_fake_input" {
state.fake = Some(registry.bind(name, version.min(MAX_VERSION), qh, ()));
}
}
}
}
// fake_input emits no events.
impl Dispatch<FakeInput, ()> for State {
fn event(
_: &mut Self,
_: &FakeInput,
_: <FakeInput as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
pub struct KwinFakeInjector {
conn: Connection,
queue: EventQueue<State>,
state: State,
fake: FakeInput,
}
impl KwinFakeInjector {
pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env()
.context("connect to KWin Wayland (is WAYLAND_DISPLAY set to the KWin socket?)")?;
let mut queue = conn.new_event_queue();
let qh = queue.handle();
let _registry = conn.display().get_registry(&qh, ());
let mut state = State::default();
queue
.roundtrip(&mut state)
.context("Wayland registry roundtrip")?;
let fake = state.fake.clone().context(
"KWin does not expose org_kde_kwin_fake_input to this client — install the host's \
.desktop (io.unom.Punktfunk.Host.desktop, X-KDE-Wayland-Interfaces) and re-login so \
KWin authorizes it (the grant is cached per-exe on first connect), or this is not a \
KWin session",
)?;
// Authenticate (the legacy handshake; for an interface-authorized client KWin accepts it
// without a dialog — same as krdpserver/krfb headless).
fake.authenticate("punktfunk".into(), "remote streaming input".into());
queue
.roundtrip(&mut state)
.context("fake_input authenticate roundtrip")?;
conn.flush().ok();
tracing::info!("KWin fake_input ready (headless keyboard/mouse/touch — no portal)");
Ok(Self {
conn,
queue,
state,
fake,
})
}
}
impl InputInjector for KwinFakeInjector {
fn inject(&mut self, event: &InputEvent) -> Result<()> {
match event.kind {
InputKind::MouseMove => {
self.fake.pointer_motion(event.x as f64, event.y as f64);
}
InputKind::MouseMoveAbs => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
self.fake.pointer_motion_absolute(x, y);
}
}
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
if let Some(btn) = gs_button_to_evdev(event.code) {
let st = u32::from(event.kind == InputKind::MouseButtonDown);
self.fake.button(btn, st);
}
}
InputKind::MouseScroll => {
// GameStream sends WHEEL_DELTA(120)-scaled units; a notch ≈ 15px. Vertical flips
// sign on the Wayland axis, horizontal passes through — same as the wlr backend.
let horizontal = event.code == SCROLL_HORIZONTAL;
let axis = if horizontal {
AXIS_HORIZONTAL
} else {
AXIS_VERTICAL
};
let notches = event.x as f64 / 120.0;
let sign = if horizontal { 1.0 } else { -1.0 };
self.fake.axis(axis, sign * notches * 15.0);
}
InputKind::KeyDown | InputKind::KeyUp => {
// Raw evdev keycode; KWin resolves it through the session's own keymap (and tracks
// modifier state itself, so no separate modifiers request is needed).
if let Some(evdev) = vk_to_evdev(event.code as u8) {
let st = u32::from(event.kind == InputKind::KeyDown);
self.fake.keyboard_key(evdev as u32, st);
} else {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
}
}
// Touch: id = event.code, coords in the client surface w×h packed into flags (same
// absolute mapping as MouseMoveAbs). Each event is its own frame.
InputKind::TouchDown | InputKind::TouchMove => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
if event.kind == InputKind::TouchDown {
self.fake.touch_down(event.code, x, y);
} else {
self.fake.touch_motion(event.code, x, y);
}
self.fake.touch_frame();
}
}
InputKind::TouchUp => {
self.fake.touch_up(event.code);
self.fake.touch_frame();
}
// Gamepads are injected through uinput, not the compositor.
InputKind::GamepadButton | InputKind::GamepadAxis => {}
}
// Surface protocol errors / disconnects, then push the batch to the compositor.
self.queue
.dispatch_pending(&mut self.state)
.context("wayland dispatch")?;
self.conn.flush().context("wayland flush")?;
Ok(())
}
}
@@ -5,6 +5,9 @@
//! keymap, and translate events into virtual pointer/keyboard requests, tracking modifier state
//! so the compositor resolves shifted keysyms correctly.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{gs_button_to_evdev, vk_to_evdev, InputEvent, InputInjector};
use anyhow::{bail, Context, Result};
use punktfunk_core::input::InputKind;
@@ -264,10 +267,17 @@ impl InputInjector for WlrootsInjector {
/// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd).
fn memfd_with(s: &str) -> Result<std::fs::File> {
let name = b"punktfunk-keymap\0";
// SAFETY: `name` is a byte-string literal with an explicit trailing NUL, so `name.as_ptr()` is a
// valid NUL-terminated C string; `memfd_create` only reads that name (copying it) and creates an
// anonymous file, returning a fresh fd (or -1). `MFD_CLOEXEC` is a valid flag. The 'static literal
// outlives the synchronous call and nothing aliases it. The result is checked `< 0` below.
let fd = unsafe { libc::memfd_create(name.as_ptr() as *const libc::c_char, libc::MFD_CLOEXEC) };
if fd < 0 {
bail!("memfd_create failed: {}", std::io::Error::last_os_error());
}
// SAFETY: `fd` is the fresh memfd `memfd_create` just returned and checked `>= 0`; it is a unique
// open fd nothing else owns, so `File` takes sole ownership and closes it exactly once on drop —
// no alias, no double-close.
let mut f = unsafe { std::fs::File::from_raw_fd(fd) };
f.write_all(s.as_bytes()).context("write keymap")?;
f.write_all(&[0]).context("write keymap NUL")?;
@@ -41,7 +41,8 @@ pub(super) const SHM_MAGIC: u32 = pf_driver_proto::gamepad::PAD_MAGIC; // "PFDS"
pub(super) const OFF_INPUT: usize = core::mem::offset_of!(pf_driver_proto::gamepad::PadShm, input);
pub(super) const OFF_OUT_SEQ: usize =
core::mem::offset_of!(pf_driver_proto::gamepad::PadShm, out_seq);
pub(super) const OFF_OUTPUT: usize = core::mem::offset_of!(pf_driver_proto::gamepad::PadShm, output);
pub(super) const OFF_OUTPUT: usize =
core::mem::offset_of!(pf_driver_proto::gamepad::PadShm, output);
/// Device-type selector the driver reads to choose which HID identity/descriptor it serves: 0 =
/// DualSense (the default — the section is zeroed), 1 = DualShock 4.
pub(super) const OFF_DEVTYPE: usize =
@@ -108,7 +109,7 @@ pub(super) struct SwDeviceProfile<'a> {
/// `profile.instance`). The returned `HSWDEVICE` owns it — `SwDeviceClose` removes it on drop, so the
/// pad appears/disappears with the session and nothing persists.
///
/// **Game-detection identity** (see `docs/windows-dualsense-game-detection.md`). `HIDD_ATTRIBUTES`
/// **Game-detection identity** (see `design/windows-dualsense-game-detection.md`). `HIDD_ATTRIBUTES`
/// alone (VID/PID via the IOCTL) satisfies SDL/HIDAPI/RawInput, but a native PS5 path (libScePad-
/// style raw HID) classifies the *connection type* by walking from the HID child to its parent
/// (`CM_Get_Parent`) and string-matching `"USB"`/`"BTHENUM"` in that parent's
@@ -187,8 +187,10 @@ impl XusbWinPad {
#[allow(clippy::too_many_arguments)]
fn write_state(&mut self, buttons: u16, lt: u8, rt: u8, lx: i16, ly: i16, rx: i16, ry: i16) {
self.packet = self.packet.wrapping_add(1);
// SAFETY: base points at SHM_SIZE bytes; all offsets are in range.
let base = self.shm.base();
// SAFETY: `base` is the start of the mapped section (`SHM_SIZE` bytes, owned by `Shm`); every
// `OFF_*` is a fixed in-range offset into it and `write_unaligned` handles the unaligned field
// writes. Single owner (`&mut self`), so no concurrent writer races these stores.
unsafe {
std::ptr::write_unaligned(base.add(OFF_BUTTONS) as *mut u16, buttons);
*base.add(OFF_LT) = lt;
@@ -5,6 +5,9 @@
//! thread stays bound to its desktop and only reattaches (`OpenInputDesktop`/`SetThreadDesktop`) when
//! `SendInput` reports a short write (the input desktop switched) — no per-event reattach overhead.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::Result;
use punktfunk_core::input::{InputEvent, InputKind};
use std::mem::size_of;
@@ -35,7 +38,12 @@ pub struct SendInputInjector {
desktop: Option<HDESK>,
}
// Only ever used from the host's single injector thread.
// SAFETY: `SendInputInjector` holds only an `Option<HDESK>` (a desktop handle). The host creates
// and drives it from a single dedicated injector thread; the handle is opened, rebound, and closed
// on whichever thread owns the value, and the type is not `Sync`, so there is never concurrent
// access. A desktop `HDESK` is not thread-affine for ownership (`CloseDesktop` works from any
// thread; `SetThreadDesktop` rebinds the current thread), so transferring ownership via `Send` is
// sound.
unsafe impl Send for SendInputInjector {}
impl SendInputInjector {
@@ -49,6 +57,12 @@ impl SendInputInjector {
/// Bind this thread to the desktop currently receiving input. UAC / lock screen / Ctrl-Alt-Del
/// swap the input desktop; `SendInput` silently no-ops unless our thread is on it.
fn reattach_input_desktop(&mut self) {
// SAFETY: `OpenInputDesktop`/`SetThreadDesktop`/`CloseDesktop` are FFI calls passed only
// by-value args (constant desktop flags, a `bool`, an access mask). `OpenInputDesktop`
// yields an owned `HDESK` only on `Ok`; we then either install it with `SetThreadDesktop`
// (closing the previously-owned handle exactly once) or close the fresh handle on failure —
// so every handle is closed exactly once and none is used after close. `SetThreadDesktop`
// only rebinds this calling thread, which is where the injector runs.
unsafe {
match OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
@@ -75,12 +89,17 @@ impl SendInputInjector {
/// switched out from under us, e.g. into UAC/lock) do we reattach to the now-current input desktop
/// and retry once. This serves both the normal and secure desktops with no steady-state overhead.
fn send(&mut self, inputs: &[INPUT]) -> Result<()> {
// SAFETY: `inputs` is a live `&[INPUT]` slice that outlives this synchronous `SendInput`
// call; `size_of::<INPUT>()` is the exact per-element stride Win32 requires as `cbSize`. The
// call only reads the array (one event per element) and returns the count injected.
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
if n as usize == inputs.len() {
return Ok(());
}
// Short write → the input desktop likely changed. Reattach + retry once.
self.reattach_input_desktop();
// SAFETY: same as the first `SendInput` — `inputs` is the identical live slice outliving the
// call and `cbSize == size_of::<INPUT>()`; only re-issued after reattaching the input desktop.
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
if n as usize != inputs.len() {
anyhow::bail!(
@@ -95,6 +114,9 @@ impl SendInputInjector {
impl Drop for SendInputInjector {
fn drop(&mut self) {
if let Some(h) = self.desktop.take() {
// SAFETY: `h` is the `HDESK` this injector owned (moved out of `self.desktop`);
// `CloseDesktop` runs once here in `Drop` on that still-valid handle, with no later use —
// no double close.
unsafe {
let _ = CloseDesktop(h);
}
@@ -216,7 +238,11 @@ impl InputInjector for SendInputInjector {
}
InputKind::KeyDown | InputKind::KeyUp => {
let down = event.kind == InputKind::KeyDown;
let vk = (event.code & 0xff) as u16; // client sends Windows VK
// client sends Windows VK
let vk = (event.code & 0xff) as u16;
// SAFETY: `MapVirtualKeyExW` is a pure value translation (VK → scancode); all three
// args are by-value (`u32`, the `MAPVK_VK_TO_VSC_EX` map-type constant, a `None`
// HKL). It dereferences no pointer and returns a `u32` — FFI-`unsafe` only.
let sc_ex = unsafe { MapVirtualKeyExW(vk as u32, MAPVK_VK_TO_VSC_EX, None) };
if sc_ex == 0 {
return Ok(()); // unmappable -> drop
@@ -264,6 +290,8 @@ fn key(ki: KEYBDINPUT) -> INPUT {
}
fn virtual_desktop_rect() -> (i32, i32, i32, i32) {
// SAFETY: each `GetSystemMetrics` takes a single by-value `SYSTEM_METRICS_INDEX` constant and
// returns an `i32`; it dereferences no pointer and has no side effects — FFI-`unsafe` only.
unsafe {
(
GetSystemMetrics(SM_XVIRTUALSCREEN),
+806
View File
@@ -548,6 +548,621 @@ fn heroic_launch_prefix() -> Option<String> {
flatpak.then(|| "flatpak run com.heroicgameslauncher.hgl".into())
}
// ---------------------------------------------------------------------------------------
// Epic Games Store (Windows) — reads the launcher's local `.item` manifests under ProgramData
// (no auth, launcher need not run). Cover art from the base64 `catcache.bin` (public Epic CDN).
// ---------------------------------------------------------------------------------------
/// Reads the Epic Games Launcher's local install manifests. Windows-only. Best-effort: empty when
/// the launcher (or its manifest dir) isn't present.
#[cfg(windows)]
pub struct EpicProvider;
#[cfg(windows)]
impl LibraryProvider for EpicProvider {
fn store(&self) -> &'static str {
"epic"
}
fn list(&self) -> Vec<GameEntry> {
let data = epic_data_dir();
let Ok(rd) = std::fs::read_dir(data.join("Manifests")) else {
return Vec::new();
};
// Parse the (best-effort) artwork cache ONCE: catalogItemId -> Artwork.
let art = epic_art_index(&data.join("Catalog").join("catcache.bin"));
let mut games = Vec::new();
for entry in rd.flatten() {
let p = entry.path();
if p.extension().and_then(|e| e.to_str()) != Some("item") {
continue;
}
let Ok(text) = std::fs::read_to_string(&p) else {
continue;
};
let Ok(v) = serde_json::from_str::<serde_json::Value>(&text) else {
continue;
};
if let Some(g) = epic_entry(&v, &art) {
games.push(g);
}
}
games
}
}
/// `%ProgramData%\Epic\EpicGamesLauncher\Data` (machine-wide, SYSTEM-readable).
#[cfg(windows)]
fn epic_data_dir() -> PathBuf {
std::env::var_os("ProgramData")
.map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from("C:\\ProgramData"))
.join("Epic")
.join("EpicGamesLauncher")
.join("Data")
}
/// Map one `.item` manifest to a [`GameEntry`], or `None` if it isn't a launchable game. Uses
/// Playnite's proven EXCLUSION filter (skip `UE_*` Unreal components; skip a DLC/addon unless it is
/// `addons/launchable`) rather than a positive `games`-category match, which can drop legit titles.
#[cfg(windows)]
fn epic_entry(
v: &serde_json::Value,
art: &std::collections::HashMap<String, Artwork>,
) -> Option<GameEntry> {
let s = |k: &str| v.get(k).and_then(|x| x.as_str());
let app_name = s("AppName")?.to_string();
if app_name.starts_with("UE_") {
return None; // Unreal Engine component, not a game
}
let cats: Vec<&str> = v
.get("AppCategories")
.and_then(|c| c.as_array())
.map(|a| a.iter().filter_map(|x| x.as_str()).collect())
.unwrap_or_default();
if cats.contains(&"addons") && !cats.contains(&"addons/launchable") {
return None; // non-launchable DLC/addon
}
// Drop stale records whose install dir is gone.
let install = s("InstallLocation")?;
if !Path::new(install).is_dir() {
return None;
}
let title = s("DisplayName").unwrap_or(&app_name).to_string();
let namespace = s("CatalogNamespace").unwrap_or("");
let catalog = s("CatalogItemId").unwrap_or("");
// The robust launch form is the namespace:catalogItemId:appName triple; fall back to the bare
// appName when those ids are absent (some manifests lack them) — never drop the launch entirely.
let value = if !namespace.is_empty() && !catalog.is_empty() {
format!("{namespace}:{catalog}:{app_name}")
} else {
app_name.clone()
};
Some(GameEntry {
id: format!("epic:{app_name}"),
store: "epic".into(),
title,
art: art.get(catalog).cloned().unwrap_or_default(),
launch: Some(LaunchSpec {
kind: "epic".into(),
value,
}),
})
}
/// Best-effort parse of `catcache.bin` (base64-encoded JSON array of catalog items) into
/// catalogItemId → [`Artwork`] from each item's `keyImages`. Empty map on any read/decode failure
/// (the format is community-reverse-engineered + can lag a fresh install → titles just show no art).
#[cfg(windows)]
fn epic_art_index(catcache: &Path) -> std::collections::HashMap<String, Artwork> {
use base64::Engine as _;
let mut map = std::collections::HashMap::new();
let Ok(raw) = std::fs::read(catcache) else {
return map;
};
let Ok(decoded) = base64::engine::general_purpose::STANDARD.decode(raw) else {
return map;
};
let Ok(items) = serde_json::from_slice::<serde_json::Value>(&decoded) else {
return map;
};
let Some(arr) = items.as_array() else {
return map;
};
for item in arr {
let Some(cat) = item
.get("id")
.or_else(|| item.get("catalogItemId"))
.and_then(|v| v.as_str())
else {
continue;
};
let Some(images) = item.get("keyImages").and_then(|v| v.as_array()) else {
continue;
};
let mut art = Artwork::default();
for img in images {
let (Some(ty), Some(url)) = (
img.get("type").and_then(|v| v.as_str()),
img.get("url").and_then(|v| v.as_str()),
) else {
continue;
};
if !(url.starts_with("http://") || url.starts_with("https://")) {
continue;
}
match ty {
"DieselGameBoxTall" => art.portrait = Some(url.to_string()),
"DieselGameBox" => art.hero = Some(url.to_string()),
"DieselGameBoxLogo" => art.logo = Some(url.to_string()),
_ => {}
}
}
if art.portrait.is_some() || art.hero.is_some() || art.logo.is_some() {
map.insert(cat.to_string(), art);
}
}
map
}
/// Build the `com.epicgames.launcher://` launch URI from a stored launch value — the triple
/// `<namespace>:<catalogItemId>:<appName>` (colons URL-encoded), or a bare `<appName>` fallback.
/// Each part is charset-validated (host-derived, but belt-and-suspenders) so no shell/URI injection.
#[cfg(windows)]
fn epic_launch_uri(value: &str) -> Option<String> {
let ok = |s: &str| {
!s.is_empty()
&& s.bytes()
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'-'))
};
let inner = match value.split(':').collect::<Vec<_>>().as_slice() {
[ns, cat, app] if ok(ns) && ok(cat) && ok(app) => format!("{ns}%3A{cat}%3A{app}"),
[app] if ok(app) => (*app).to_string(),
_ => return None,
};
Some(format!(
"com.epicgames.launcher://apps/{inner}?action=launch&silent=true"
))
}
// ---------------------------------------------------------------------------------------
// GOG (Windows) — registry-indexed installs + each game's `goggame-<id>.info` for a direct-exe
// launch (no Galaxy needed, dodges its cold-start/anti-cheat). Art (api.gog.com) is a follow-up.
// ---------------------------------------------------------------------------------------
/// Reads the GOG.com install registry + per-game `.info` files. Windows-only. Best-effort: empty
/// when GOG isn't installed.
#[cfg(windows)]
pub struct GogProvider;
#[cfg(windows)]
impl LibraryProvider for GogProvider {
fn store(&self) -> &'static str {
"gog"
}
fn list(&self) -> Vec<GameEntry> {
gog_games()
}
}
#[cfg(windows)]
fn gog_games() -> Vec<GameEntry> {
use winreg::enums::HKEY_LOCAL_MACHINE;
use winreg::RegKey;
// 32-bit GOG writes under WOW6432Node; a 64-bit process reads the explicit path directly.
let Ok(games_key) =
RegKey::predef(HKEY_LOCAL_MACHINE).open_subkey("SOFTWARE\\WOW6432Node\\GOG.com\\Games")
else {
return Vec::new();
};
let mut out = Vec::new();
for sub in games_key.enum_keys().flatten() {
// The subkey name IS the GOG product id.
let Ok(k) = games_key.open_subkey(&sub) else {
continue;
};
let Ok(path) = k.get_value::<String, _>("PATH") else {
continue;
};
if !Path::new(&path).is_dir() {
continue;
}
let title = k
.get_value::<String, _>("GAMENAME")
.unwrap_or_else(|_| sub.clone());
// Resolve the primary play task (exe + args + workdir) from goggame-<id>.info; skip if absent.
let Some((exe, args, workdir)) = gog_play_task(&path, &sub) else {
continue;
};
let id = format!("gog:{sub}");
// Art (public api.gog.com) is resolved off the hot path by the background warmer; read
// whatever it has cached (title-only until warmed).
let art = cached_art(&id).unwrap_or_default();
out.push(GameEntry {
id,
store: "gog".into(),
title,
art,
launch: Some(LaunchSpec {
kind: "gog".into(),
value: format!("{exe}\t{args}\t{workdir}"),
}),
});
}
out
}
/// The primary play task from `<install>\goggame-<id>.info`: `(absolute exe, args, working dir)`.
/// Prefers `isPrimary` + `FileTask`, else the first `FileTask`. Paths are resolved against `install`.
#[cfg(windows)]
fn gog_play_task(install: &str, id: &str) -> Option<(String, String, String)> {
let text =
std::fs::read_to_string(Path::new(install).join(format!("goggame-{id}.info"))).ok()?;
let v: serde_json::Value = serde_json::from_str(&text).ok()?;
let tasks = v.get("playTasks")?.as_array()?;
let is_file =
|t: &serde_json::Value| t.get("type").and_then(|s| s.as_str()) == Some("FileTask");
let pick = tasks
.iter()
.find(|t| {
t.get("isPrimary")
.and_then(|b| b.as_bool())
.unwrap_or(false)
&& is_file(t)
})
.or_else(|| tasks.iter().find(|t| is_file(t)))?;
let rel = pick.get("path").and_then(|s| s.as_str())?;
let exe = Path::new(install).join(rel);
let args = pick
.get("arguments")
.and_then(|s| s.as_str())
.unwrap_or("")
.to_string();
let workdir = pick
.get("workingDir")
.and_then(|s| s.as_str())
.map(|w| Path::new(install).join(w))
.unwrap_or_else(|| Path::new(install).to_path_buf());
Some((
exe.to_string_lossy().into_owned(),
args,
workdir.to_string_lossy().into_owned(),
))
}
/// Build the spawn `(command line, working dir)` for a `gog` launch value (`exe \t args \t workdir`,
/// all host-resolved from the operator's own disk). Direct exe — no shell, no Galaxy.
#[cfg(windows)]
fn gog_spawn(value: &str) -> Option<(String, Option<PathBuf>)> {
let mut parts = value.split('\t');
let exe = parts.next().filter(|s| !s.is_empty())?;
let args = parts.next().unwrap_or("");
let workdir = parts.next().filter(|s| !s.is_empty()).map(PathBuf::from);
let cmdline = if args.trim().is_empty() {
format!("\"{exe}\"")
} else {
format!("\"{exe}\" {args}")
};
Some((cmdline, workdir))
}
// ---------------------------------------------------------------------------------------
// Xbox / Microsoft Store / Game Pass (Windows) — scans the flat-file `XboxGames` install dirs
// (no auth) for GDK games (each has a Content\MicrosoftGame.config). Launch via the AUMID
// (shell:AppsFolder\<PFN>!<AppId>) in the interactive session. Cover art (displaycatalog) deferred.
// ---------------------------------------------------------------------------------------
/// Reads installed Xbox / Game Pass / Store GDK games from the flat-file install dirs. Windows-only.
/// Best-effort: empty when no `XboxGames` dir exists.
#[cfg(windows)]
pub struct XboxProvider;
#[cfg(windows)]
impl LibraryProvider for XboxProvider {
fn store(&self) -> &'static str {
"xbox"
}
fn list(&self) -> Vec<GameEntry> {
xbox_games()
}
}
/// Scan each fixed drive's default `<drive>:\XboxGames` for GDK games — the presence of
/// `Content\MicrosoftGame.config` is the game marker (so we list games, not ordinary UWP apps). A
/// custom install folder (set via the undocumented `.GamingRoot`) isn't covered; the default folder
/// is the common case. Non-GDK pure-UWP Store games (under the ACL-locked WindowsApps) are missed too.
#[cfg(windows)]
fn xbox_games() -> Vec<GameEntry> {
let mut games = Vec::new();
for letter in b'C'..=b'Z' {
let root = PathBuf::from(format!("{}:\\XboxGames", letter as char));
let Ok(rd) = std::fs::read_dir(&root) else {
continue;
};
for entry in rd.flatten() {
let title_dir = entry.path();
let cfg = title_dir.join("Content").join("MicrosoftGame.config");
if !cfg.is_file() {
continue;
}
let Ok(text) = std::fs::read_to_string(&cfg) else {
continue;
};
let folder = title_dir
.file_name()
.map(|f| f.to_string_lossy().into_owned());
let Some((name, app_id, title, store_id)) = xbox_parse_config(&text, folder.as_deref())
else {
continue;
};
let Some(pfn) = xbox_pfn(&name) else {
tracing::debug!(package = %name, "xbox: no AppRepository entry → can't resolve PFN, skipping");
continue;
};
let id_key = if store_id.is_empty() {
pfn.clone()
} else {
store_id
};
let id = format!("xbox:{id_key}");
// Art (unofficial displaycatalog, keyed by StoreId) is resolved off the hot path by the
// background warmer; read whatever it has cached (title-only until warmed / if no StoreId).
let art = cached_art(&id).unwrap_or_default();
games.push(GameEntry {
id,
store: "xbox".into(),
title,
art,
launch: Some(LaunchSpec {
kind: "aumid".into(),
value: format!("{pfn}!{app_id}"),
}),
});
}
}
games.sort_by(|a, b| a.id.cmp(&b.id));
games.dedup_by(|a, b| a.id == b.id); // same game on two drives → one entry
games
}
/// Parse the fields we need from a `MicrosoftGame.config`: `(Identity Name, AppId, title, StoreId)`.
/// AppId is the `<Executable>`'s `Id` (the AUMID app id, typically "Game"). The title prefers
/// `ShellVisuals@DefaultDisplayName`, but that can be an unresolved `ms-resource:` ref → fall back to
/// the install folder name, then the package name.
#[cfg(windows)]
fn xbox_parse_config(text: &str, folder: Option<&str>) -> Option<(String, String, String, String)> {
let doc = roxmltree::Document::parse(text).ok()?;
let root = doc.root_element();
let name = root
.children()
.find(|n| n.has_tag_name("Identity"))?
.attribute("Name")?
.to_string();
let app_id = root
.children()
.find(|n| n.has_tag_name("ExecutableList"))
.and_then(|el| {
el.children()
.filter(|n| n.has_tag_name("Executable"))
.find_map(|e| e.attribute("Id"))
})?
.to_string();
let ddn = root
.children()
.find(|n| n.has_tag_name("ShellVisuals"))
.and_then(|sv| sv.attribute("DefaultDisplayName"))
.filter(|s| !s.is_empty() && !s.starts_with("ms-resource"));
let title = ddn
.map(String::from)
.or_else(|| folder.map(String::from))
.unwrap_or_else(|| name.clone());
let store_id = root
.children()
.find(|n| n.has_tag_name("StoreId"))
.and_then(|n| n.text())
.unwrap_or("")
.to_string();
Some((name, app_id, title, store_id))
}
/// Resolve a package's PackageFamilyName by finding its
/// `AppRepository\Packages\<PackageFullName>` dir (machine-wide, SYSTEM-readable) and reducing the
/// full name to `Name_PublisherHash`. This READS the authoritative PFN — never compute the hash.
#[cfg(windows)]
fn xbox_pfn(identity: &str) -> Option<String> {
let pkgs = PathBuf::from(std::env::var_os("ProgramData")?)
.join("Microsoft")
.join("Windows")
.join("AppRepository")
.join("Packages");
let prefix = format!("{identity}_");
for e in std::fs::read_dir(&pkgs).ok()?.flatten() {
let dn = e.file_name().to_string_lossy().into_owned();
if dn.starts_with(&prefix) {
if let Some(pfn) = pfn_from_full(&dn, identity) {
return Some(pfn);
}
}
}
None
}
/// PackageFamilyName from a PackageFullName dir name
/// (`Name_Version_Arch_ResourceId_PublisherHash`) → `Name_PublisherHash`. The hash is the last
/// `_`-segment; `Name` is the caller's identity.
#[cfg(windows)]
fn pfn_from_full(dir_name: &str, identity: &str) -> Option<String> {
let hash = dir_name.rsplit('_').next()?;
(!hash.is_empty() && hash != dir_name).then(|| format!("{identity}_{hash}"))
}
// ---------------------------------------------------------------------------------------
// Cover-art resolver + cache (shared by the Windows GOG + Xbox providers, which have no local
// art). A disk cache is the source of truth read by all_games() (so the list/launch path never
// blocks on the network); a host-lifetime background warmer fetches uncached art (GOG's public
// api.gog.com + Xbox's displaycatalog, both no-auth) and persists it. Cross-platform so the
// HTTP/JSON code is compiled + checked everywhere; the warmer simply finds nothing to fetch on a
// host whose stores all carry their own art (Steam CDN / Heroic CDN / Lutris data: URLs).
// ---------------------------------------------------------------------------------------
/// The persisted art cache: GameEntry id → resolved [`Artwork`]. An entry's PRESENCE means "already
/// resolved" (even an empty Artwork = fetched, none found) so the warmer never re-fetches it.
fn art_cache() -> &'static std::sync::Mutex<std::collections::HashMap<String, Artwork>> {
static CACHE: std::sync::OnceLock<
std::sync::Mutex<std::collections::HashMap<String, Artwork>>,
> = std::sync::OnceLock::new();
CACHE.get_or_init(|| {
let loaded = std::fs::read_to_string(art_cache_path())
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
std::sync::Mutex::new(loaded)
})
}
/// The art cache lives in the canonical HOST config dir (`%ProgramData%\punktfunk` on Windows /
/// `~/.config/punktfunk` on Linux — gamestream::config_dir, NOT the legacy XDG/HOME `config_dir`
/// below that the custom store still uses).
fn art_cache_path() -> PathBuf {
crate::gamestream::config_dir().join("library-art-cache.json")
}
/// The cached art for a library id, if it has been resolved (positive or negative). `None` = not yet
/// warmed → the provider shows title-only until the warmer fills it in.
fn cached_art(id: &str) -> Option<Artwork> {
art_cache().lock().unwrap().get(id).cloned()
}
/// Record resolved art for a library id + persist the cache (write-then-rename; best-effort).
fn store_art(id: &str, art: Artwork) {
let mut cache = art_cache().lock().unwrap();
cache.insert(id.to_string(), art);
if let Ok(json) = serde_json::to_string(&*cache) {
let path = art_cache_path();
if let Some(dir) = path.parent() {
let _ = std::fs::create_dir_all(dir);
}
let tmp = path.with_extension("json.tmp");
if std::fs::write(&tmp, json).is_ok() {
let _ = std::fs::rename(&tmp, &path);
}
}
}
/// Start the host-lifetime cover-art warmer: every few minutes, fetch + cache art for any library
/// entry whose store needs a network lookup (GOG / Xbox) and isn't cached yet. Idempotent — once
/// everything is cached a pass makes no network calls (and a host with only self-art stores never
/// fetches at all). Call once from `serve()`; the returned handle can be dropped to detach it.
pub fn start_art_warmer() -> std::thread::JoinHandle<()> {
std::thread::Builder::new()
.name("pf-art-warmer".into())
.spawn(|| loop {
warm_art_once();
std::thread::sleep(std::time::Duration::from_secs(300));
})
.expect("spawn art warmer thread")
}
/// One warming pass: resolve uncached GOG/Xbox art. Other stores carry their own art (Steam CDN
/// template, Heroic CDN URLs, Lutris data: URLs, custom user URLs) and are skipped.
fn warm_art_once() {
for g in all_games() {
if cached_art(&g.id).is_some() {
continue;
}
let Some((store, localid)) = g.id.split_once(':') else {
continue;
};
let art = match store {
"gog" => fetch_gog_art(localid),
// The xbox id is the StoreId when present, else the PFN (contains '_', no displaycatalog
// entry) → cache empty for those so they aren't retried every pass.
"xbox" if !localid.contains('_') => fetch_xbox_art(localid),
"xbox" => Artwork::default(),
_ => continue, // steam/heroic/lutris/custom resolve their own art
};
store_art(&g.id, art);
}
}
/// HTTP GET + parse JSON with a bounded timeout. `None` on any network/parse failure (best-effort —
/// art is non-essential, so a failure just leaves the title-only card).
fn fetch_json(url: &str) -> Option<serde_json::Value> {
let agent = ureq::AgentBuilder::new()
.timeout(std::time::Duration::from_secs(10))
.build();
let body = agent.get(url).call().ok()?.into_string().ok()?;
serde_json::from_str(&body).ok()
}
/// Make a protocol-relative URL (`//host/...`, common in GOG + MS catalog responses) absolute https.
fn abs_url(u: &str) -> String {
u.strip_prefix("//")
.map(|rest| format!("https://{rest}"))
.unwrap_or_else(|| u.to_string())
}
/// GOG cover art via the public (no-auth) product API. Field names / URL shapes are GOG-specific and
/// best-effort (worth on-box confirmation); a wrong URL just degrades to the title card client-side.
fn fetch_gog_art(product_id: &str) -> Artwork {
let Some(v) = fetch_json(&format!(
"https://api.gog.com/products/{product_id}?expand=images"
)) else {
return Artwork::default();
};
let img = |k: &str| {
v.get("images")
.and_then(|i| i.get(k))
.and_then(|u| u.as_str())
.map(abs_url)
};
Artwork {
portrait: img("verticalCover"),
hero: img("background"),
logo: img("logo2x"),
header: img("logo"),
}
}
/// Xbox cover art via the (unofficial, no-auth) Microsoft display catalog, keyed by StoreId. Best-
/// effort: the endpoint is internal/unstable, so on drift this just yields no art (title-only).
fn fetch_xbox_art(store_id: &str) -> Artwork {
let Some(v) = fetch_json(&format!(
"https://displaycatalog.mp.microsoft.com/v7.0/products/{store_id}?market=US&languages=en-us&fieldsTemplate=Details"
)) else {
return Artwork::default();
};
let images = v
.get("Products")
.and_then(|p| p.as_array())
.and_then(|a| a.first())
.and_then(|p| p.get("LocalizedProperties"))
.and_then(|l| l.as_array())
.and_then(|a| a.first())
.and_then(|lp| lp.get("Images"))
.and_then(|i| i.as_array());
let mut art = Artwork::default();
for img in images.into_iter().flatten() {
let (Some(purpose), Some(uri)) = (
img.get("ImagePurpose").and_then(|v| v.as_str()),
img.get("Uri").and_then(|v| v.as_str()),
) else {
continue;
};
let url = abs_url(uri);
match purpose {
"Poster" => art.portrait = Some(url),
"SuperHeroArt" | "Hero" => art.hero = Some(url),
"Logo" => art.logo = Some(url),
"BoxArt" => art.header = Some(url),
_ => {}
}
}
art
}
// ---------------------------------------------------------------------------------------
// Custom store (user-curated entries, persisted + CRUD'd via the mgmt API)
// ---------------------------------------------------------------------------------------
@@ -768,6 +1383,32 @@ fn windows_launch_for(spec: &LaunchSpec) -> Option<(String, Option<std::path::Pa
};
Some((cmdline, None))
}
// Epic: open the (host-built, validated) com.epicgames.launcher:// URI via explorer.exe — a
// concrete EXE that resolves the registered protocol handler as the user; the URI is a single
// argv element (no shell, no cmd /c). Same pattern as the steam explorer fallback.
"epic" => epic_launch_uri(&spec.value).map(|uri| (format!("explorer.exe \"{uri}\""), None)),
// GOG: spawn the resolved game exe directly (host-derived from goggame-<id>.info), no Galaxy.
"gog" => gog_spawn(&spec.value),
// Xbox/Game Pass: activate the UWP/GDK package by its AUMID (<PFN>!<AppId>) via explorer's
// shell:AppsFolder — which runs in the interactive user session (UWP activation fails as
// SYSTEM/session-0; spawn_in_active_session uses the user token). Guard the charset (the value
// is host-derived from MicrosoftGame.config + AppRepository, but belt-and-suspenders).
"aumid" => {
let valid = spec.value.split_once('!').is_some_and(|(pfn, app)| {
let part = |s: &str| {
!s.is_empty()
&& s.bytes()
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'-'))
};
part(pfn) && part(app)
});
valid.then(|| {
(
format!("explorer.exe \"shell:AppsFolder\\{}\"", spec.value),
None,
)
})
}
// Operator-typed custom command (host-owned, never client-set): run it through the shell in the
// interactive session. `cmd.exe /c` is acceptable here precisely because the value is operator
// input — the same trust as the operator typing it — not a client-influenced string.
@@ -795,6 +1436,38 @@ fn steam_exe() -> Option<std::path::PathBuf> {
None
}
/// Launch a GameStream `apps.json` command (operator-typed, trusted — never client-set) into the live
/// session, AFTER capture is up. Used by the GameStream path for the backends that DON'T nest the
/// command via [`VirtualDisplay::set_launch_command`]: Windows (no gamescope) and Linux
/// kwin/mutter/wlroots (which stream the existing desktop). The caller skips this for Linux gamescope,
/// which already nested it. On Windows it runs in the interactive USER session (the host is SYSTEM);
/// on Linux the host is already inside the user's graphical session, so a plain spawn lands the app on
/// the streamed (primary) output.
#[cfg(any(windows, target_os = "linux"))]
pub fn launch_gamestream_command(cmd: &str) -> Result<()> {
let cmd = cmd.trim();
anyhow::ensure!(!cmd.is_empty(), "empty command");
#[cfg(windows)]
{
// cmd.exe /c is fine here: the value is the host operator's own apps.json command, not a
// client-influenced string (same trust as the custom-store `command` kind).
let pid = crate::interactive::spawn_in_active_session(&format!("cmd.exe /c {cmd}"), None)
.context("spawn gamestream command in the interactive session")?;
tracing::info!(command = %cmd, pid, "gamestream: launched app in the interactive session");
Ok(())
}
#[cfg(target_os = "linux")]
{
let child = std::process::Command::new("sh")
.arg("-c")
.arg(cmd)
.spawn()
.context("spawn gamestream command")?;
tracing::info!(command = %cmd, pid = child.id(), "gamestream: launched app into the session");
Ok(())
}
}
/// The full library: every store's titles merged + the custom entries, sorted by title.
pub fn all_games() -> Vec<GameEntry> {
let mut games = SteamProvider.list();
@@ -805,6 +1478,13 @@ pub fn all_games() -> Vec<GameEntry> {
games.extend(LutrisProvider.list());
games.extend(HeroicProvider.list());
}
// Windows store providers (their launchers are Windows-only): Epic + GOG + Xbox/Game Pass.
#[cfg(windows)]
{
games.extend(EpicProvider.list());
games.extend(GogProvider.list());
games.extend(XboxProvider.list());
}
games.extend(load_custom().into_iter().map(GameEntry::from));
games.sort_by_key(|g| g.title.to_lowercase());
games
@@ -1048,6 +1728,20 @@ mod tests {
windows_launch_for(&cmd).unwrap().0,
"cmd.exe /c notepad.exe"
);
// Xbox AUMID → explorer shell:AppsFolder activation; a value without '!' is rejected.
let aumid = LaunchSpec {
kind: "aumid".into(),
value: "Microsoft.X_8wekyb3d8bbwe!Game".into(),
};
assert_eq!(
windows_launch_for(&aumid).unwrap().0,
"explorer.exe \"shell:AppsFolder\\Microsoft.X_8wekyb3d8bbwe!Game\""
);
assert!(windows_launch_for(&LaunchSpec {
kind: "aumid".into(),
value: "no-bang".into()
})
.is_none());
// Empty / unknown kinds → no recipe.
assert!(windows_launch_for(&LaunchSpec {
kind: "command".into(),
@@ -1060,4 +1754,116 @@ mod tests {
})
.is_none());
}
#[cfg(windows)]
#[test]
fn epic_filters_and_builds_launch() {
let dir = std::env::temp_dir().join(format!("pf-epic-test-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let inst = dir.to_string_lossy().into_owned();
let empty = std::collections::HashMap::new();
// Normal game with the full triple → kept, triple launch value.
let game = serde_json::json!({
"AppName": "Fortnite", "DisplayName": "Fortnite", "CatalogNamespace": "fn",
"CatalogItemId": "abc123", "InstallLocation": inst.clone(),
"AppCategories": ["public", "games", "applications"]
});
let e = epic_entry(&game, &empty).expect("game kept");
assert_eq!(e.id, "epic:Fortnite");
assert_eq!(e.launch.as_ref().unwrap().value, "fn:abc123:Fortnite");
// UE component, non-launchable addon, and a missing install dir are all skipped.
let ue = serde_json::json!({"AppName":"UE_5.3","InstallLocation":inst.clone(),"AppCategories":["engines"]});
assert!(epic_entry(&ue, &empty).is_none());
let dlc =
serde_json::json!({"AppName":"DLC","InstallLocation":inst,"AppCategories":["addons"]});
assert!(epic_entry(&dlc, &empty).is_none());
let gone = serde_json::json!({"AppName":"Gone","InstallLocation":"C:\\nope-xyz","AppCategories":["games"]});
assert!(epic_entry(&gone, &empty).is_none());
std::fs::remove_dir_all(&dir).ok();
}
#[cfg(windows)]
#[test]
fn epic_launch_uri_triple_bare_and_guard() {
assert_eq!(
epic_launch_uri("fn:abc:Fortnite").as_deref(),
Some("com.epicgames.launcher://apps/fn%3Aabc%3AFortnite?action=launch&silent=true")
);
assert_eq!(
epic_launch_uri("Fortnite").as_deref(),
Some("com.epicgames.launcher://apps/Fortnite?action=launch&silent=true")
);
assert!(epic_launch_uri("bad part:x:y").is_none()); // a space → rejected
assert!(epic_launch_uri("").is_none());
}
#[cfg(windows)]
#[test]
fn gog_spawn_parses_and_guards() {
let (cmd, wd) = gog_spawn("C:\\Games\\W3\\witcher3.exe\t--skip\tC:\\Games\\W3").unwrap();
assert_eq!(cmd, "\"C:\\Games\\W3\\witcher3.exe\" --skip");
assert_eq!(wd, Some(std::path::PathBuf::from("C:\\Games\\W3")));
let (cmd2, wd2) = gog_spawn("C:\\g.exe").unwrap();
assert_eq!(cmd2, "\"C:\\g.exe\"");
assert!(wd2.is_none());
assert!(gog_spawn("").is_none());
}
#[cfg(windows)]
#[test]
fn gog_play_task_picks_primary_filetask() {
let dir = std::env::temp_dir().join(format!("pf-gog-test-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let id = "1207658924";
std::fs::write(
dir.join(format!("goggame-{id}.info")),
r#"{"playTasks":[
{"isPrimary":false,"type":"FileTask","path":"other.exe"},
{"isPrimary":true,"type":"FileTask","path":"bin\\game.exe","arguments":"-w","workingDir":"bin"}
]}"#,
)
.unwrap();
let (exe, args, wd) = gog_play_task(&dir.to_string_lossy(), id).unwrap();
std::fs::remove_dir_all(&dir).ok();
assert!(exe.ends_with("bin\\game.exe"), "exe={exe}");
assert_eq!(args, "-w");
assert!(wd.ends_with("bin"), "wd={wd}");
}
#[cfg(windows)]
#[test]
fn xbox_parse_config_and_pfn() {
let xml = r#"<?xml version="1.0" encoding="utf-8"?>
<Game configVersion="1">
<Identity Name="Microsoft.624F8B84B80" Publisher="CN=Microsoft" Version="1.0.0.0" />
<ExecutableList>
<Executable Name="gamelaunchhelper.exe" Id="Game" />
</ExecutableList>
<StoreId>9NBLGGH4R315</StoreId>
<ShellVisuals DefaultDisplayName="Halo Infinite" Square150x150Logo="x.png" />
</Game>"#;
let (name, app_id, title, store_id) = xbox_parse_config(xml, Some("HaloInfinite")).unwrap();
assert_eq!(name, "Microsoft.624F8B84B80");
assert_eq!(app_id, "Game");
assert_eq!(title, "Halo Infinite");
assert_eq!(store_id, "9NBLGGH4R315");
// An ms-resource DefaultDisplayName is unresolvable → fall back to the install folder name.
let xml2 = r#"<Game><Identity Name="Pkg.Name"/>
<ExecutableList><Executable Id="App"/></ExecutableList>
<ShellVisuals DefaultDisplayName="ms-resource:DisplayName"/></Game>"#;
let (_, app2, title2, sid2) = xbox_parse_config(xml2, Some("MyGameFolder")).unwrap();
assert_eq!(app2, "App");
assert_eq!(title2, "MyGameFolder");
assert_eq!(sid2, "");
// PackageFamilyName reduced from a PackageFullName dir name (the hash is the last segment).
assert_eq!(
pfn_from_full(
"Microsoft.624F8B84B80_1.0.0.0_x64__8wekyb3d8bbwe",
"Microsoft.624F8B84B80"
)
.as_deref(),
Some("Microsoft.624F8B84B80_8wekyb3d8bbwe")
);
assert!(pfn_from_full("NoUnderscore", "NoUnderscore").is_none());
}
}
@@ -13,6 +13,9 @@
//! attaches none, the export yields an already-signaled sync_file (poll returns immediately) — no
//! wait, no harm, and `waited=false` tells us the driver doesn't fence (so zero-copy would still race).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use std::os::fd::RawFd;
// linux/dma-buf.h ioctls on the DMA_BUF_BASE ('b' = 0x62) magic. _IOWR = dir(3)<<30 | size<<16 | base<<8 | nr.
@@ -40,6 +43,11 @@ pub fn wait_read_ready(dmabuf_fd: RawFd, timeout_ms: i32) -> std::io::Result<boo
flags: DMA_BUF_SYNC_READ,
fd: -1,
};
// SAFETY: `dmabuf_fd` is a live dmabuf fd supplied by the caller (borrowed for this call; we
// never close it). `DMA_BUF_IOCTL_EXPORT_SYNC_FILE` encodes `size_of::<DmaBufExportSyncFile>()`
// — the exact byte count the kernel copies — and `&mut req` is a live, correctly-sized
// `#[repr(C)]` struct the EXPORT_SYNC_FILE ioctl reads (`flags`) and writes (`fd`). `req`
// outlives this synchronous call and is not aliased elsewhere.
let r = unsafe { libc::ioctl(dmabuf_fd, DMA_BUF_IOCTL_EXPORT_SYNC_FILE, &mut req) };
if r < 0 {
return Err(std::io::Error::last_os_error());
@@ -54,11 +62,21 @@ pub fn wait_read_ready(dmabuf_fd: RawFd, timeout_ms: i32) -> std::io::Result<boo
revents: 0,
};
// Non-blocking probe: not-yet-signaled (poll==0) means the producer is still rendering.
// SAFETY: `&mut pfd` points at a single live `libc::pollfd` and `nfds == 1` matches that one
// element; `pfd.fd` is `sync_fd`, the sync_file fd just exported (already checked `>= 0`).
// `poll` reads `fd`/`events` and writes `revents` for this non-blocking (timeout 0) probe, then
// returns — `pfd` outlives the call and aliases nothing.
let pending = unsafe { libc::poll(&mut pfd, 1, 0) } == 0;
if pending {
pfd.revents = 0;
// SAFETY: same live single-element `pfd` (its `revents` reset to 0 just above), `nfds == 1`,
// and `sync_fd` still open. This blocking `poll` (up to `timeout_ms`) waits for the render
// fence to signal; it reads `fd`/`events`, writes `revents`, and returns before `pfd` ends.
unsafe { libc::poll(&mut pfd, 1, timeout_ms) }; // block until the render fence signals
}
// SAFETY: `sync_fd` is the sync_file fd the EXPORT_SYNC_FILE ioctl created and handed us to own;
// this point is reached only when `sync_fd >= 0`, this `close` runs exactly once on it, and it is
// never used afterward — no double-close or use-after-close.
unsafe { libc::close(sync_fd) };
Ok(pending)
}
@@ -8,6 +8,8 @@
//! verified (ioctl numbers + a live signal→wait round trip), ready to wire in the moment a producer
//! gains working `SPA_META_SyncTimeline`.
#![allow(dead_code)]
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
//!
//! Compositors that render directly into the PipeWire buffer pool (Mutter's virtual
//! monitors) hand buffers over at GPU-submit time; on drivers without implicit dmabuf
@@ -81,6 +83,8 @@ pub struct DrmSync {
impl DrmSync {
pub fn open() -> Result<DrmSync> {
let path = c"/dev/dri/renderD128";
// SAFETY: `path` is a 'static NUL-terminated C string literal; `open` only reads it as a
// filesystem path and returns an fd (or -1). No Rust memory is aliased or handed to the kernel.
let fd = unsafe { libc::open(path.as_ptr(), libc::O_RDWR | libc::O_CLOEXEC) };
if fd < 0 {
bail!("open /dev/dri/renderD128 for syncobj ops: {}", errno());
@@ -94,6 +98,9 @@ impl DrmSync {
fd: syncobj_fd,
..Default::default()
};
// SAFETY: `self.fd` is the live render-node fd from `open`; the request number encodes
// `size_of::<DrmSyncobjHandle>()` (the bytes the kernel copies), and `&mut req` is a live,
// correctly-sized `#[repr(C)]` struct the FD_TO_HANDLE ioctl reads (`fd`) and writes (`handle`).
let r = unsafe { libc::ioctl(self.fd, DRM_IOCTL_SYNCOBJ_FD_TO_HANDLE, &mut req) };
if r < 0 {
bail!("SYNCOBJ_FD_TO_HANDLE: {}", errno());
@@ -106,6 +113,8 @@ impl DrmSync {
handle,
..Default::default()
};
// SAFETY: `self.fd` is the live render-node fd; `DRM_IOCTL_SYNCOBJ_DESTROY` encodes
// `size_of::<DrmSyncobjDestroy>()`, and `&mut req` is a live correctly-sized struct the kernel reads.
unsafe { libc::ioctl(self.fd, DRM_IOCTL_SYNCOBJ_DESTROY, &mut req) };
}
@@ -117,6 +126,8 @@ impl DrmSync {
tv_sec: 0,
tv_nsec: 0,
};
// SAFETY: `CLOCK_MONOTONIC` is a valid clock id and `&mut now` is a live `libc::timespec` the
// kernel fills in; the call returns before `now` is read, so there is no aliasing/lifetime issue.
unsafe { libc::clock_gettime(libc::CLOCK_MONOTONIC, &mut now) };
let deadline = now.tv_sec * 1_000_000_000 + now.tv_nsec + timeout_ms as i64 * 1_000_000;
let handles = [handle];
@@ -129,6 +140,11 @@ impl DrmSync {
flags: DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT,
..Default::default()
};
// SAFETY: `self.fd` is the live render-node fd; the request number encodes
// `size_of::<DrmSyncobjTimelineWait>()`; `&mut req` is a live correctly-sized struct. Its
// `handles`/`points` u64 fields hold the addresses of the local `handles`/`points` arrays, which
// outlive this synchronous call, and `count_handles == 1` matches their length — so every kernel
// read through those addresses stays in bounds.
let r = unsafe { libc::ioctl(self.fd, DRM_IOCTL_SYNCOBJ_TIMELINE_WAIT, &mut req) };
let saved = errno();
self.destroy(handle);
@@ -151,6 +167,10 @@ impl DrmSync {
count_handles: 1,
flags: 0,
};
// SAFETY: `self.fd` is the live render-node fd; the request number encodes
// `size_of::<DrmSyncobjTimelineArray>()`; `&mut req` is a live correctly-sized struct whose
// `handles`/`points` u64 fields address the local `handles`/`points` arrays (alive for this
// synchronous call, `count_handles == 1` matching their length).
let r = unsafe { libc::ioctl(self.fd, DRM_IOCTL_SYNCOBJ_TIMELINE_SIGNAL, &mut req) };
let saved = errno();
self.destroy(handle);
@@ -163,6 +183,8 @@ impl DrmSync {
impl Drop for DrmSync {
fn drop(&mut self) {
// SAFETY: `self.fd` is the fd `open` returned; this `DrmSync` owns it exclusively and `close`
// runs exactly once (here, in `Drop`), so there is no double-close or use-after-close.
unsafe { libc::close(self.fd) };
}
}
@@ -203,14 +225,19 @@ mod tests {
const CREATE: u64 = iowr(0xBF, std::mem::size_of::<Create>());
const HANDLE_TO_FD: u64 = iowr(0xC1, std::mem::size_of::<DrmSyncobjHandle>());
let mut c = Create::default();
// SAFETY: `sync.fd` is the live render-node fd; `CREATE` encodes `size_of::<Create>()`, and
// `&mut c` is a live correctly-sized struct the kernel fills (`handle`).
assert!(unsafe { libc::ioctl(sync.fd, CREATE, &mut c) } >= 0);
let mut h = DrmSyncobjHandle {
handle: c.handle,
..Default::default()
};
// SAFETY: `sync.fd` is live; `HANDLE_TO_FD` encodes `size_of::<DrmSyncobjHandle>()`; `&mut h`
// is a live correctly-sized struct (the kernel reads `handle`, writes `fd`).
assert!(unsafe { libc::ioctl(sync.fd, HANDLE_TO_FD, &mut h) } >= 0);
sync.signal_point(h.fd, 1).expect("signal");
sync.wait_point(h.fd, 1, 100).expect("wait after signal");
// SAFETY: `h.fd` is the fd HANDLE_TO_FD just exported; we own it and close it exactly once here.
unsafe { libc::close(h.fd) };
sync.destroy(c.handle);
}
@@ -11,6 +11,8 @@
//! thread) and ffmpeg's `hevc_nvenc` (encode thread); each thread makes it current before use.
#![allow(non_camel_case_types, non_snake_case)]
// Every `unsafe` block/impl below carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{bail, Result};
use std::os::raw::{c_int, c_uint, c_void};
@@ -128,8 +130,14 @@ struct CudaApi {
) -> CUresult,
cuDestroyExternalMemory: unsafe extern "C" fn(CUexternalMemory) -> CUresult,
}
// The resolved fn pointers are plain addresses into a process-lifetime mapping; safe to share.
// SAFETY: every field is a bare `extern "C" fn` address into the leaked, process-lifetime
// `libcuda` mapping (`cuda_api` `forget`s the `Library`, so it is never unloaded) — an immutable
// value with no interior mutability and no thread affinity. Moving the table to another thread
// cannot dangle (the code it points at stays mapped) or race (the fields are read-only).
unsafe impl Send for CudaApi {}
// SAFETY: as above — the table is a set of immutable fn-pointer addresses with no interior
// mutability, so concurrent shared reads from multiple threads cannot race; the driver entry
// points they address are themselves thread-safe.
unsafe impl Sync for CudaApi {}
/// `CUresult` returned by the wrappers when `libcuda` isn't loaded (no NVIDIA driver). Non-zero so
@@ -143,6 +151,14 @@ static CUDA_API: OnceLock<Option<CudaApi>> = OnceLock::new();
/// (the expected case on AMD/Intel hosts) — logged at debug, not an error.
fn cuda_api() -> Option<&'static CudaApi> {
CUDA_API
// SAFETY: `Library::new` runs `libcuda.so.1`'s initializers — it is the trusted NVIDIA
// driver library, so loading has no unexpected effects; `?`/`None` handle its absence.
// Each `lib.get::<T>(name)` asserts the symbol's real ABI equals `T`: every NUL-terminated
// name is a documented CUDA Driver API entry point and `T` is the exact
// `unsafe extern "C" fn(..)` signature from cuda.h/cudaGL.h (`_v2` for ctx/mem ops). Each
// `Symbol` only borrows `lib` until the end of the struct-literal statement; we deref-copy
// the raw fn-pointer out first, then `forget(lib)` leaks the mapping so those addresses
// stay valid for the whole process. Runs once under the `OnceLock` init — no aliasing.
.get_or_init(|| unsafe {
let lib = libloading::Library::new("libcuda.so.1")
.or_else(|_| libloading::Library::new("libcuda.so"))
@@ -361,6 +377,12 @@ pub fn read_plane_to_host(
Height: height,
..Default::default()
};
// SAFETY: `copy_blocking` is unsafe because it issues a CUDA copy; its contract is a valid
// descriptor with the shared context current (the caller's responsibility — self-test path).
// `&copy` is a live local `#[repr(C)] CUDA_MEMCPY2D` that outlives the synchronous call:
// `srcDevice`/`srcPitch` are the caller's live pitched device plane, `dstHost` addresses the
// freshly-allocated `host` `Vec` of exactly `width_bytes*height` bytes, and `WidthInBytes`×
// `Height` fit both. The copy is synchronous, so `host` is fully written before we return it.
unsafe { copy_blocking(&copy, "cuMemcpy2DAsync_v2(dev->host)")? };
Ok(host)
}
@@ -369,7 +391,13 @@ pub fn read_plane_to_host(
/// in a `OnceLock`; the raw `CUcontext` is thread-safe to make current from any thread.
#[derive(Clone, Copy)]
pub struct Context(pub CUcontext);
// SAFETY: `CUcontext` is an opaque CUDA driver handle, not a dereferenceable Rust pointer. It is
// created once and never destroyed (process lifetime), and the only thing done with it is
// `cuCtxSetCurrent`, which the Driver API explicitly allows from any thread — so transferring the
// handle to another thread cannot dangle or race (the driver owns the synchronization).
unsafe impl Send for Context {}
// SAFETY: as above — the wrapped handle is an immutable opaque address and the driver does all the
// synchronization, so sharing `&Context` across threads is sound.
unsafe impl Sync for Context {}
static CONTEXT: OnceLock<Context> = OnceLock::new();
@@ -382,6 +410,12 @@ pub fn context() -> Result<CUcontext> {
if cuda_api().is_none() {
bail!("libcuda.so.1 not available — no NVIDIA driver (CUDA zero-copy disabled)");
}
// SAFETY: we returned above unless `cuda_api()` is `Some`, so every wrapper here forwards into
// the live, leaked `libcuda` table rather than the not-loaded stub. `cuInit(0)` passes the
// API-required flags value 0. `&mut dev`/`&mut ctx` are live, zero/null-initialized stack
// out-params the driver writes the device handle / new context into; each outlives its
// synchronous call and they are distinct locals (no aliasing). `cuCtxCreate_v2` yields a valid
// `CUcontext` on success (`ck` bails otherwise), which becomes the block's value.
let ctx = unsafe {
ck(cuInit(0), "cuInit")?;
let mut dev: CUdevice = 0;
@@ -401,6 +435,10 @@ pub fn context() -> Result<CUcontext> {
/// Make the shared context current on the calling thread (required before any CUDA op here).
pub fn make_current() -> Result<()> {
let ctx = context()?;
// SAFETY: `ctx` came from `context()?`, so it is the live shared `CUcontext` and the driver
// table is present. `cuCtxSetCurrent` binds that opaque handle to the calling thread; it takes
// no Rust-memory pointer and is thread-safe (affects only this thread's current context), so
// there is no aliasing or lifetime hazard.
unsafe { ck(cuCtxSetCurrent(ctx), "cuCtxSetCurrent") }
}
@@ -423,6 +461,12 @@ fn copy_stream() -> CUstream {
if let Some(s) = cell.get() {
return s;
}
// SAFETY: `copy_stream` runs with the shared context current (its doc contract), so the
// wrappers forward into the live `libcuda` table. `&mut least`/`&mut greatest` are live
// stack `i32`s the driver fills with the priority range; `&mut s` is a live null-init
// `CUstream` the driver writes the new stream into. All out-params outlive their
// synchronous calls and are distinct locals. On any non-zero result we fall back to a null
// (NULL-stream) value and never read an uninitialized handle.
let stream = unsafe {
let (mut least, mut greatest) = (0i32, 0i32);
if cuCtxGetStreamPriorityRange(&mut least, &mut greatest) != 0 {
@@ -459,6 +503,11 @@ unsafe fn copy_blocking(copy: &CUDA_MEMCPY2D, what: &str) -> Result<()> {
fn alloc_pitched(width: u32, height: u32) -> Result<(CUdeviceptr, usize)> {
let mut ptr: CUdeviceptr = 0;
let mut pitch: usize = 0;
// SAFETY: `cuMemAllocPitch_v2` allocates a pitched device buffer (the wrapper forwards to the
// live table on any path that reached allocation). `&mut ptr` (`CUdeviceptr`) and `&mut pitch`
// (`usize`) are live, distinct stack out-params the driver writes the allocation pointer and
// its pitch into; both outlive the synchronous call. Width/height/element-size are by-value
// ints. No aliasing — two separate locals.
unsafe {
ck(
cuMemAllocPitch_v2(
@@ -486,6 +535,10 @@ fn alloc_pitched_nv12(
let mut y_pitch: usize = 0;
let mut uv_ptr: CUdeviceptr = 0;
let mut uv_pitch: usize = 0;
// SAFETY: two independent `cuMemAllocPitch_v2` calls (wrapper → live table). `&mut y_ptr`/
// `&mut y_pitch` and `&mut uv_ptr`/`&mut uv_pitch` are live, distinct stack out-params the
// driver writes each plane's pointer and pitch into; all outlive their synchronous calls. The
// dimension/element-size args are by-value ints. No aliasing — four separate locals.
unsafe {
ck(
cuMemAllocPitch_v2(
@@ -524,6 +577,13 @@ struct PoolInner {
impl Drop for PoolInner {
fn drop(&mut self) {
// SAFETY: the pool only exists because allocation succeeded, so the driver table is live.
// `PoolInner` drops only once every `DeviceBuffer` that referenced it (each holds an `Arc`
// clone) has been recycled, so `free`/`free_uv` hold every outstanding allocation exactly
// once and nothing else still uses them — no double-free or use-after-free. We make the
// shared context current first (drop may run off the allocating thread) so `cuMemFree_v2`
// targets the right context. Each `p` is a `CUdeviceptr` previously returned by
// `cuMemAllocPitch_v2`; results are ignored (best-effort teardown).
unsafe {
if let Some(c) = CONTEXT.get() {
let _ = cuCtxSetCurrent(c.0);
@@ -697,6 +757,12 @@ impl Drop for DeviceBuffer {
}
} else {
// The buffer may be freed on the encode thread; cuMemFree needs a current context.
// SAFETY: this is the un-pooled branch (`pool` is `None`), so this `DeviceBuffer`
// exclusively owns `self.ptr` (and `self.uv`'s `uv_ptr`), each returned by
// `cuMemAllocPitch_v2` and freed exactly once here — `drop` runs once and the
// `self.ptr == 0` guard above skips the sentinel/empty case, so no double-free. We set
// the shared context current first because drop may run on a thread where it isn't, and
// `cuMemFree_v2` needs it. Wrapper → live table; results ignored (teardown).
unsafe {
if let Some(c) = CONTEXT.get() {
let _ = cuCtxSetCurrent(c.0);
@@ -745,6 +811,16 @@ impl RegisteredTexture {
/// unmap. The copy is synchronized (on our priority stream) before unmap so `dst` is ready
/// before the source dmabuf is recycled. Always unmaps, even if the copy errors.
pub fn copy_mapped_to(&mut self, dst: &DeviceBuffer) -> Result<()> {
// SAFETY: `self.resource` is the valid `CUgraphicsResource` from a successful `register_gl`
// (its only constructor), so the wrappers forward to the live table; the caller holds the
// GL+CUDA contexts current (the registration's contract). `cuGraphicsMapResources` maps
// `count == 1` resource via `&mut self.resource` (a live field) on the default stream;
// `cuGraphicsSubResourceGetMappedArray` writes the mapped `CUarray` into the live local
// `array` (index 0, mip 0). On failure we unmap and bail (balanced). `&copy` is a live
// local `CUDA_MEMCPY2D` outliving the synchronous `copy_blocking`: `srcArray` is valid
// while mapped, `dstDevice`/`dstPitch` are `dst`'s live allocation, `width*4`×`height` fit
// both. `copy_blocking` syncs before we unmap, so the array stays valid through the copy;
// we always unmap afterward (even on error), keeping the map/unmap pair balanced.
unsafe {
ck(
cuGraphicsMapResources(1, &mut self.resource, std::ptr::null_mut()),
@@ -783,6 +859,14 @@ impl RegisteredTexture {
width_bytes: usize,
height: usize,
) -> Result<()> {
// SAFETY: identical contract to `copy_mapped_to` — `self.resource` is the valid
// `CUgraphicsResource` from `register_gl` (wrappers → live table; caller holds GL+CUDA
// contexts current). Map `count == 1` resource via the live `&mut self.resource`; the
// mapped `CUarray` is written into the live local `array` (index 0, mip 0); on failure we
// unmap and bail (balanced). `&copy` is a live local outliving the synchronous
// `copy_blocking`: `srcArray` valid while mapped, `dstDevice`/`dstPitch` are the caller's
// live plane, `width_bytes`×`height` fit it. We always unmap afterward, even on copy error,
// so the map/unmap pair stays balanced and the array outlives the copy.
unsafe {
ck(
cuGraphicsMapResources(1, &mut self.resource, std::ptr::null_mut()),
@@ -847,6 +931,10 @@ pub fn copy_device_to_device(
Height: src.height as usize,
..Default::default()
};
// SAFETY: `copy_blocking` is unsafe (issues a CUDA copy); the caller must have the shared
// context current (documented). `&copy` is a live local device→device `CUDA_MEMCPY2D` outliving
// the synchronous call: `srcDevice`/`srcPitch` are `src`'s live allocation, `dstDevice`/
// `dstPitch` the caller's live region, `width*4`×`height` within both. Wrapper → live table.
unsafe { copy_blocking(&copy, "cuMemcpy2DAsync_v2(dev->dev)") }
}
@@ -888,6 +976,12 @@ pub fn copy_nv12_to_device(
Height: h / 2,
..Default::default()
};
// SAFETY: two unsafe `copy_blocking` device→device copies; the caller must have the shared
// context current (documented). `&y`/`&uv` are live local `CUDA_MEMCPY2D`s outliving each
// synchronous call. All four device pointers are valid: `src.ptr`/`src_uv_ptr` come from a live
// NV12 `DeviceBuffer` (its `.uv` presence was checked via `ok_or_else`), `y_dst`/`uv_dst` are
// the caller's live NVENC surface planes; the luma copy is `w`×`h`, the chroma copy
// `(w/2)*2`×`h/2`, each within its planes. Wrappers → live table.
unsafe {
copy_blocking(&y, "cuMemcpy2DAsync_v2(nv12 Y dev->dev)")?;
copy_blocking(&uv, "cuMemcpy2DAsync_v2(nv12 UV dev->dev)")
@@ -897,6 +991,12 @@ pub fn copy_nv12_to_device(
impl Drop for RegisteredTexture {
fn drop(&mut self) {
if !self.resource.is_null() {
// SAFETY: `self.resource` is non-null (just checked) and is the valid
// `CUgraphicsResource` from `register_gl`, owned exclusively by this `RegisteredTexture`
// and unregistered exactly once here (drop runs once) — no use-after-free or
// double-unregister. `cuGraphicsUnregisterResource` releases the GL↔CUDA registration;
// wrapper → live table (the resource exists ⇒ the driver was present). Result ignored
// (best-effort teardown).
unsafe {
let _ = cuGraphicsUnregisterResource(self.resource);
}
@@ -913,7 +1013,11 @@ pub struct ExternalDmabuf {
pub size: u64,
}
// Raw driver handles; used from the single capture thread but moved with the importer.
// SAFETY: the fields are opaque CUDA driver handles — an external-memory handle and a device
// pointer — not dereferenceable Rust memory, and the value is uniquely owned (no `Clone`). It is
// used from a single capture thread but constructed on / moved between threads with the importer;
// transferring these handles is sound because uniqueness rules out aliasing and they are destroyed
// exactly once in `Drop`. Only `Send` (not `Sync`) is asserted, matching the single-thread use.
unsafe impl Send for ExternalDmabuf {}
impl ExternalDmabuf {
@@ -921,6 +1025,9 @@ impl ExternalDmabuf {
/// from then on) and map its full `size` bytes to a device pointer. The shared context
/// must be current.
pub fn import(fd: i32, size: u64) -> Result<ExternalDmabuf> {
// SAFETY: `libc::dup` only reads the integer `fd` and returns a new descriptor (or -1); it
// touches no Rust memory and `fd` is the caller's still-owned dmabuf fd (not consumed
// here). No aliasing or lifetime concern — a pure syscall on an integer.
let dup = unsafe { libc::dup(fd) };
if dup < 0 {
bail!("dup(dmabuf fd) failed");
@@ -938,8 +1045,17 @@ impl ExternalDmabuf {
};
desc.handle[0] = dup as u32 as u64; // union member `int fd` (little-endian low bytes)
let mut ext: CUexternalMemory = std::ptr::null_mut();
// SAFETY: `cuImportExternalMemory` imports the memory described by `&desc`, a live local
// `#[repr(C)] CUDA_EXTERNAL_MEMORY_HANDLE_DESC` (cuda.h 64-bit layout) that outlives this
// synchronous call: `type_` is OPAQUE_FD, `handle[0]` holds the dup'd fd in the union's
// `int fd` low bytes, `size` is set. `&mut ext` is a live null-init out-param the driver
// writes the imported handle into. The driver takes ownership of the fd only on success.
// Distinct locals → no aliasing. Wrapper → live table (caller holds the context current).
let r = unsafe { cuImportExternalMemory(&mut ext, &desc) };
if r != 0 {
// SAFETY: import failed (`r != 0`), so the driver did NOT take ownership of `dup`; we
// still own it and close it exactly once here on the error path (the success path never
// closes it — the driver does). `libc::close` acts on the integer fd alone.
unsafe { libc::close(dup) }; // import failed → the driver did not take the fd
bail!("cuImportExternalMemory failed ({r}) — LINEAR dmabuf import unsupported?");
}
@@ -949,8 +1065,17 @@ impl ExternalDmabuf {
..Default::default()
};
let mut ptr: CUdeviceptr = 0;
// SAFETY: maps a device pointer from `ext` (the valid `CUexternalMemory` just imported) per
// `&buf`, a live local `CUDA_EXTERNAL_MEMORY_BUFFER_DESC` (offset 0, full `size`) that
// outlives this synchronous call. `&mut ptr` is a live zero-init out-param the driver writes
// the mapped device address into; distinct locals → no aliasing. Wrapper → live table
// (context current).
let r = unsafe { cuExternalMemoryGetMappedBuffer(&mut ptr, ext, &buf) };
if r != 0 {
// SAFETY: mapping failed; `ext` is the valid `CUexternalMemory` we imported and
// exclusively own. We destroy it exactly once here on the error path (the success path
// instead moves it into the returned `ExternalDmabuf`, whose `Drop` destroys it),
// releasing the fd the driver took — no double-destroy or use-after-free.
unsafe {
let _ = cuDestroyExternalMemory(ext);
}
@@ -962,6 +1087,12 @@ impl ExternalDmabuf {
impl Drop for ExternalDmabuf {
fn drop(&mut self) {
// SAFETY: this `ExternalDmabuf` only exists after a successful import, so the driver table
// is live. It exclusively owns `self.ptr` (the mapped buffer) and `self.ext` (the external
// memory), each torn down exactly once here (drop runs once; guarded by `!= 0` / `!null`) —
// no double-free or use-after-free. We make the shared context current first because drop
// may run off the import thread, and we free the mapped buffer before destroying its
// backing external memory. Results ignored (best-effort teardown).
unsafe {
if let Some(c) = CONTEXT.get() {
let _ = cuCtxSetCurrent(c.0);
@@ -996,5 +1127,10 @@ pub fn copy_pitched_to_buffer(
};
// copy_blocking syncs our priority stream before returning, so the copy is complete before the
// dmabuf is requeued to the producer.
// SAFETY: `copy_blocking` is unsafe (issues a CUDA copy); the caller must have the shared
// context current (documented). `&copy` is a live local device→device `CUDA_MEMCPY2D` outliving
// the synchronous call: `srcDevice`/`srcPitch` are the caller's live mapped span (e.g. an
// `ExternalDmabuf`), `dstDevice`/`dstPitch` are `dst`'s live allocation, `width*4`×`height`
// within both. Wrapper → live table.
unsafe { copy_blocking(&copy, "cuMemcpy2DAsync_v2(ext->dev)") }
}

Some files were not shown because too many files have changed in this diff Show More