45 Commits

Author SHA1 Message Date
enricobuehler 36259b264f docs(security): record remediation status for the 2026-06-28 host audit
apple / swift (push) Successful in 1m6s
ci / rust (push) Failing after 56s
ci / web (push) Successful in 52s
android / android (push) Successful in 3m24s
ci / docs-site (push) Successful in 1m4s
apple / screenshots (push) Successful in 5m23s
windows-host / package (push) Successful in 7m36s
deb / build-publish (push) Successful in 2m52s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m59s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m43s
14/18 fixed (3532e35 Linux-verified + 6f903f7 Windows DACL paths pending
CI/box); #5 deferred (needs on-box validation), #9/#13 accepted, S7
acknowledged (no upstream rsa fix).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:16:25 +00:00
enricobuehler 6f903f79bc fix(host/security): Windows DACL hardening — close audit #2, #3, #8, #11
Windows local-privilege findings from design/security-review-2026-06-28.md.
These are #[cfg(windows)] paths (verify in CI / on the box; this Linux dev
VM can't compile MSVC). They follow the existing write_secret_file/icacls
patterns; the cross-platform parts are cargo check/clippy/test green.

- #2 [HIGH]: route the mgmt bearer token write through the shared
  write_secret_file so it gets the SAME Windows DACL (SYSTEM/Administrators)
  as the host key — it was cfg(unix)-only and left Users-readable, leaking
  full mgmt admin authority to any local user.
- #3 [HIGH]: create_private_dir now applies a restrictive DACL to the
  %ProgramData%\punktfunk config directory (re-owns to Administrators to
  defeat a pre-creation, strips inheritance, SYSTEM/Admins/OWNER full +
  Users read-only) so a local user can't plant host.env/apps.json that the
  SYSTEM service trusts (env/arg-injection LPE). host.env is now written
  DACL-locked via write_secret_file; the config + logs dirs go through
  create_private_dir.
- #8 [LOW]: write the web-console password file empty, icacls-lock it, THEN
  write the secret — closes the brief write-then-icacls TOCTOU window.
- #11 [LOW]: the SYSTEM logs dir is DACL-locked (Users read-only, no
  create), so a local user can't pre-plant host.log as a reparse/hardlink to
  redirect SYSTEM's writes (subsumed by the #3 dir lockdown).

Deferred: #5 (host<->UMDF gamepad/IDD shared-section Everyone:GENERIC_ALL).
The section SDDL is intentionally permissive because the UMDF driver opens
it under a restricted token of unknown SID/integrity; scoping it blind would
likely break the live-validated gamepad/IDD pipeline, so it needs on-box
validation first. Tracked in the report.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:14:19 +00:00
enricobuehler 3532e35b75 fix(host/security): close audit findings S1,#1,#4,#10,#12,#7,#6,S2-S6 (Linux/cross-platform)
Remediations from design/security-review-2026-06-28.md verified on Linux
(cargo check/clippy/test green; Windows-gated paths verify in CI):

- S1 [HIGH]: bump quinn-proto 0.11.14 -> 0.11.15 (RUSTSEC-2026-0185,
  pre-auth out-of-order STREAM reassembly memory exhaustion on the
  always-on default QUIC listener).
- #1 [HIGH]: remove the unauthenticated nvhttp `GET /pin` endpoint; the
  GameStream PIN is delivered ONLY via the bearer-gated mgmt API, so a
  network client can no longer submit its own displayed PIN and self-pair.
- #4 [HIGH->MED]: gate the unauthenticated RTSP/UDP media plane on a paired
  `/launch` and bind it to the launching client's source IP (threaded
  through the HTTPS handler), so an unpaired peer can neither start capture
  on an idle host nor ride a paired client's active launch.
- #12: bound concurrent parked pairing waiters (MAX_PARKED_WAITERS) so a
  pre-auth peer can't pin unbounded 300s handshakes. +regression test.
- #10: throttle the per-packet ENet control GCM-decrypt-failed warn
  (exponential backoff) so a junk flood can't spam the log.
- #7 [MED->LOW]: serialize all process-global env mutation on the
  session-setup path under a new vdisplay::ENV_LOCK (apply_session_env /
  apply_input_env / the launch-cmd set_var / the gamescope env read), so
  concurrent native sessions can't race set_var/getenv (data-race UB ->
  host-wide DoS). Full per-session SessionContext threading remains a
  follow-up for cross-session value confusion.
- #6 [MED]: move the gamescope EIS socket relay from world-writable /tmp to
  $XDG_RUNTIME_DIR (per-user 0700) and reject a symlinked relay file, so a
  local user can't intercept (keylog) or deny the remote session's input.
- S2: a malformed client Opus mic frame now drops that frame instead of
  tearing down the shared host-lifetime virtual mic (cross-session DoS).
- S3: track held buttons/keys in capped HashSets (was unbounded Vec with
  O(n) scans) so a paired client can't grow per-session input state.
- S5: reject fps==0/absurd at the open_video chokepoint (covers Hello,
  ANNOUNCE, Reconfigure) so the encoder time_base/pts math can't div-by-0.
- S6: bound the shared mic mpsc (drop-newest when full).
- S4: cap Epic launcher-cache reads (catcache.bin/.item) so a planted giant
  can't OOM the host during library enumeration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:06:24 +00:00
enricobuehler 6b846913f5 docs(security): 2026-06-28 host security audit (follow-up) report
Multi-agent follow-up audit of the privileged streaming host: 18 attack
surfaces, every finding adversarially double-verified, plus a coverage
critic. Records 15 confirmed + 9 partial findings and a prior-fix
re-verification of the 2026-06-21 review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:05:58 +00:00
enricobuehler 26c6c939a2 fix(ci/apple): set CMAKE_POLICY_VERSION_MINIMUM=3.5 for the vendored libopus
apple / swift (push) Successful in 1m6s
release / apple (push) Successful in 8m50s
ci / rust (push) Successful in 1m17s
ci / web (push) Successful in 52s
apple / screenshots (push) Successful in 5m40s
ci / docs-site (push) Successful in 1m27s
android / android (push) Successful in 3m46s
deb / build-publish (push) Successful in 2m53s
decky / build-publish (push) Successful in 10s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m0s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m45s
With cmake now found, Homebrew's CMake 4 refuses the vendored libopus's
`cmake_minimum_required(VERSION <3.5)` ("Compatibility with CMake < 3.5 has been
removed"). Export CMAKE_POLICY_VERSION_MINIMUM=3.5 (the same knob the Windows
build uses) so the cmake crate's child cmake configures the audiopus_sys libopus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 21:49:27 +00:00
enricobuehler b6e6f2bff5 fix(ci/apple): locate Homebrew explicitly for the cmake install
apple / swift (push) Failing after 31s
release / apple (push) Failing after 8s
apple / screenshots (push) Has been skipped
android / android (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / web (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m30s
The self-hosted macOS runner runs steps with `bash --noprofile --norc`, so
Homebrew's bin dir is not on PATH — the previous `brew install cmake` died with
`brew: command not found` (exit 127). Find brew at its known prefix, install cmake
only if missing, and export the brew bin dir to $GITHUB_PATH so the subsequent
xcframework build (audiopus_sys → vendored libopus) actually finds `cmake`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 21:47:05 +00:00
enricobuehler e3034958ee fix(ci): unbreak the Apple + Windows-client builds after the surround-audio merge
apple / swift (push) Failing after 2s
release / apple (push) Failing after 3s
apple / screenshots (push) Has been skipped
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m17s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m15s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m2s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m6s
android / android (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / web (push) Has been cancelled
ci / rust (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
decky / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
The 5.1/7.1 surround commit (75627c8) added in-core Opus, which broke two CI jobs
that the merge didn't touch:

  * Windows MSIX client: clients/windows/src/main.rs's headless `SessionParams`
    initializer was missing the new `audio_channels` field (the GUI path sets it
    from settings). Default the CLI/test path to stereo (2), matching trust.rs.
  * Apple xcframework (apple.yml + release.yml): in-core Opus decode pulls
    `audiopus_sys`, which builds a vendored *static* libopus via CMake when
    pkg-config finds no system Opus — keeping the xcframework self-contained (no
    runtime libopus.dylib on end-user Macs/devices). The self-hosted macOS runner
    lacked `cmake`; install it self-healing before every xcframework build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 21:44:44 +00:00
enricobuehler 8672026e97 fix(host): clear clippy doc_lazy_continuation in the 4:4:4 docs
apple / swift (push) Failing after 7s
apple / screenshots (push) Has been skipped
android / android (push) Successful in 3m17s
ci / rust (push) Successful in 1m17s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 58s
windows-host / package (push) Successful in 7m27s
deb / build-publish (push) Successful in 2m54s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m39s
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
A line-wrap put `+`/`*`-style markers at the start of two doc lines, which
clippy (Windows host job, rust 1.96) reads as markdown list items whose
unindented follow-on lines trip `doc_lazy_continuation` under `-D warnings`:

  - encode/windows/nvenc.rs `chroma_444` field doc (the failing Windows-host
    clippy job): "+ chromaFormatIDC = 3" → "and chromaFormatIDC = 3".
  - encode/linux/vaapi.rs `probe_can_encode_444` doc: "+ validate" → "and
    validate" (last line, didn't fire yet, but fragile — fixed pre-emptively).

Pure doc rewording, no behaviour change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 21:38:07 +00:00
enricobuehler 75627c8afe feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients
apple / swift (push) Failing after 10s
release / apple (push) Failing after 7s
apple / screenshots (push) Has been skipped
audit / cargo-audit (push) Failing after 1m19s
windows-host / package (push) Failing after 2m44s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 39s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Failing after 39s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 45s
android / android (push) Successful in 5m17s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 45s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 56s
ci / rust (push) Successful in 9m19s
ci / bench (push) Successful in 4m40s
decky / build-publish (push) Successful in 26s
deb / build-publish (push) Successful in 2m57s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m56s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m35s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m20s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
flatpak / build-publish (push) Successful in 4m22s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m51s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m50s
Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client
(previously stereo-only):

- core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream
  mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome
  `audio_channels` negotiation via the trailing-byte back-compat pattern (old
  peers fall back to stereo); C-ABI `punktfunk_connect_ex6`,
  `punktfunk_connection_audio_channels`, and in-core multistream decode
  `punktfunk_connection_next_audio_pcm` for embedders without a multistream
  Opus decoder. Real-libopus channel-identity round-trip test.
- host: native audio thread captures + Opus-(multi)stream-encodes at the
  negotiated count (with a cross-session cached-capturer channel-mismatch fix);
  GameStream surround unified onto the safe `opus::MSEncoder`, dropping
  `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround;
  WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask.
- clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via
  `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM →
  AVAudioEngine with an explicit wire-order channel layout; each gains a
  Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless
  validator.

Verified on Linux: core/host/linux/probe test suites + the Android Rust
(cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple
builds, all on-glass checks, and the live native loopback are pending (CI / a
free box).

Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it
shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so
cannot be committed separately from the surround changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 21:11:05 +00:00
enricobuehler 6383e5f4fd feat(client/android): CI screenshot capture via Roborazzi
Play-listing/marketing screenshots of the Compose client rendered on the host JVM
by Roborazzi (Robolectric Native Graphics) — no emulator, GPU, KVM, host, or JNI
core. Five scenes render the REAL composables with embedded mock state under a
forced brand palette (Material You has no wallpaper to seed from on the JVM):
hosts grid, settings, TOFU + PIN dialogs, and the live stats HUD. Validated 5/5
locally.

- New JVM unit-test source set (app/src/test) + Roborazzi/Robolectric test deps;
  @Config(sdk=36) is mandatory (no android-all jar for compileSdk 37) and the
  animation clock is paused so a text-bearing scene reaches idle.
- kit: `-PskipRustBuild` skips the cargo-ndk native build so the JVM-only test job
  needs no Rust/NDK; normal APK/AAR builds are unchanged.
- Widen BrandDark / StatsOverlay to internal so the tests can use them.
- Standalone best-effort tag-gated workflow; PNGs upload as a 30-day artifact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:05:54 +00:00
enricobuehler 6a93d164a0 feat(client/linux): CI screenshot capture
Host-free UI screenshots of the GTK4/libadwaita client under a virtual X display
(clients/linux/tools/screenshots.sh) — Xvfb + software GL (llvmpipe) + a root-window
grab, one app launch per scene. PUNKTFUNK_SHOT_SCENE routes build_ui to render one
mock-populated REAL view (hosts grid / settings dialog / TOFU + PIN dialogs) and
print PF_SHOT_READY once it has settled; the saved-hosts grid is driven by a seeded
client-known-hosts.json. NON_UNIQUE in shot mode so back-to-back launches don't
collide. The stream scene is deferred — its page needs a live NativeClient.

Gated to stable release tags in a standalone best-effort workflow that builds the
client in the rust-ci image and captures under Xvfb; PNGs upload as a 30-day
artifact, not committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:05:38 +00:00
enricobuehler 9e98618e5f feat(web): CI screenshot capture for the mgmt console
Marketing/store screenshots of the console, captured from the built Storybook
with headless Chromium (web/tools/screenshots.mjs) — every Pages/* + Shell/*
story rendered at 1440x900@2x. The page stories render from fixtures, so no live
mgmt API, login, or GPU is needed (the web analogue of apple.yml's screenshots
job). Gated to stable release tags in a standalone best-effort workflow; PNGs
upload as a 30-day artifact, not committed.

- Add Stats + Pairing stories (the two pages that lacked them) with stats/pairing
  fixtures typed against the generated models.
- Extract a pure PairingView (index.tsx -> view.tsx), matching the
  Dashboard/Clients/Stats split, so the page renders host-free from mock state
  instead of racing its polling queries. Container wiring is behaviour-identical.
- Playwright driver + a chromium-capable tag-gated job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:05:27 +00:00
enricobuehler 1bd60ffb34 refactor(docs): use shared @unom/app-ui/footer component
apple / swift (push) Successful in 1m2s
android / android (push) Successful in 4m23s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 13s
ci / rust (push) Successful in 4m47s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 58s
apple / screenshots (push) Successful in 5m16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
ci / bench (push) Successful in 4m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m29s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m23s
docker / deploy-docs (push) Successful in 18s
The docs footer was a hand-maintained mirror of the marketing site's. Both now
render the same @unom/app-ui/footer component, so they stay in sync. The shared
view themes itself through @unom/style tokens (which the docs already map onto
their Fumadocs surfaces), and a resolveHref hook rebases root-relative links
onto the marketing-site origin. Footer types now come from the library too.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 14:34:45 +00:00
enricobuehler 30d0d36efe feat(decky): self-update without the store + Gaming-Mode launch polish, and ship the Steam Deck docs
apple / swift (push) Successful in 1m4s
apple / screenshots (push) Successful in 5m26s
android / android (push) Successful in 3m27s
ci / web (push) Successful in 1m7s
ci / docs-site (push) Successful in 1m16s
ci / rust (push) Successful in 4m21s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
ci / bench (push) Successful in 4m46s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m0s
flatpak / build-publish (push) Successful in 4m55s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m38s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m25s
Plugin self-update (no Decky store): CI publishes a per-channel manifest.json
({version, immutable per-version artifact, sha256}) beside the zip and bakes
update.json {channel, manifest} into the plugin. main.py `check_update` reads the
installed version from package.json (the value Decky reports — not plugin.json),
fetches the channel manifest, and the frontend shows an "Update to vX" button that
drives Decky Loader's own install RPC (root downloads + SHA-256-verifies + hot-reloads).
CI now stamps a plain-numeric semver (0.3.<run> canary / X.Y.Z stable) into
package.json — a -ciN suffix would mis-order under compare-versions.

Linux client: `--fullscreen` (plus SteamDeck/gamescope env fallback) enters GTK
fullscreen on stream start so Gaming-Mode chrome is hidden; native-mode resolution
falls back to the display's first monitor when the window isn't mapped yet (was
dropping to the 1080p floor — wrong on the Deck's 1280×800); add a confirmed
"Remove saved host" action (KnownHosts::remove_by_fp).

Docs: new docs/steam-deck.md (Decky install/pair/stream/self-update/troubleshooting),
wired into meta.json nav, and cross-linked from clients/install-client/channels. This
is the page docs.punktfunk.unom.io/docs/steam-deck — the website's download link
pointed at it before it existed; committing it makes that link resolve.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 13:03:44 +00:00
enricobuehler 3947d5b07a fix(host/audio): drive the Linux virtual mic with RT_PROCESS (was silent)
apple / swift (push) Successful in 1m1s
ci / rust (push) Successful in 4m36s
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 55s
apple / screenshots (push) Successful in 5m9s
ci / bench (push) Successful in 4m36s
windows-host / package (push) Successful in 7m8s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m19s
release / apple (push) Successful in 9m52s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m16s
android / android (push) Successful in 3m21s
decky / build-publish (push) Successful in 11s
deb / build-publish (push) Successful in 2m45s
flatpak / build-publish (push) Successful in 4m11s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m48s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m35s
docker / deploy-docs (push) Successful in 18s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
The punktfunk-mic PipeWire source connected without RT_PROCESS, so it ran as an
async/main-loop node. In the host's busy multi-stream graph (desktop audio + video
capture + the session) it never acquired a driver, stayed suspended, and its
process() callback never fired — every recorder reading the remote mic heard pure
silence (the long-standing "Linux host mic broken"). Connect the mic stream with
RT_PROCESS so it is a synchronous node that joins its consumer's driver group and
is actually driven.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:46:06 +00:00
enricobuehler 238501597e feat(host/gamestream): follow Desktop<->Game session switches
android / android (push) Successful in 4m49s
ci / web (push) Successful in 55s
apple / swift (push) Successful in 59s
ci / rust (push) Successful in 4m52s
ci / docs-site (push) Successful in 56s
apple / screenshots (push) Successful in 5m16s
windows-host / package (push) Successful in 7m1s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 42s
ci / bench (push) Successful in 4m41s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m59s
The GameStream/Moonlight video plane is a separate encode loop that lacked the
session-following the native punktfunk/1 plane has, so a mid-stream Desktop<->Game
switch killed the stream ("video stream failed") instead of following it.

* Normalize the session env like the native plane: extract open_gs_virtual_source,
  which detects the LIVE compositor + apply_session_env/apply_input_env (gamescope
  ATTACH default -> resize-on-attach to the box's own game-mode session at the
  client mode; KWin/Mutter retargeting). GameStream previously ran a bare detect()
  against raw process env, so in game mode it bare-spawned a COMPETING gamescope
  instead of attaching to the box's session.

* In-place capture-loss rebuild: replace the `?` that ended the stream with a
  bounded rebuild (re-detect the live compositor via the same factory, build the
  new source BEFORE dropping the old, reopen the encoder, force an IDR) — keeping
  the send thread + packetizer + socket + RTP clock. A same-resolution
  Desktop<->Game toggle is now FOLLOWED with no Moonlight reconnect.

Protocol limit (unchanged): a mid-stream RESOLUTION change is impossible on
GameStream (WxH locked at ANNOUNCE; no Reconfigure) — a session toggle keeps the
negotiated mode, so this isn't hit. The portal/synthetic source passes no rebuild
closure (propagates as before).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:22:12 +00:00
enricobuehler 04dd3e3a19 docs: refresh Windows host page for new users; drop stale Status/NVIDIA-only/SudoVDA
Rewrite the Windows host docs page for first-time setup, on par with the
other host guides: remove the standout "Status:" banner, restructure into
Requirements / Install (web console + pairing + configure) / How it works /
Notes & limits.

Bring the content up to date with the shipping host:
- encode is all-vendor (NVENC/AMF/QSV + software fallback), not NVIDIA-only
- virtual display is punktfunk's own pf-vdisplay IDD (SudoVDA removed)
- gamepads need no prerequisite — UMDF drivers bundled; ViGEmBus is gone
- add HDR10 + Vulkan-game HDR layer coverage

Fix the same stale claims where other pages cross-reference the Windows host
(requirements, running-as-a-service, install, roadmap, status).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:22:50 +00:00
enricobuehler 61aa1053e7 feat(host/gamescope): headless game mode that follows the box + matches the client
apple / swift (push) Successful in 1m2s
android / android (push) Successful in 4m43s
ci / rust (push) Successful in 4m53s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 5m6s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
windows-host / package (push) Successful in 9m2s
ci / bench (push) Successful in 4m41s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m6s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m43s
Make Steam game mode work on a display-less streaming host and stream it at the
client's resolution:

* Ship /etc/gamescope-session-plus/sessions.d/steam (packaging/bazzite/
  gamescope-headless-session, installed by the RPM + Arch PKGBUILD): fall back to
  gamescope's headless backend when no display is connected, so "Switch to Game
  Mode" boots offscreen instead of crashing on the missing panel (and 5-striking
  back to desktop). No-op on display-attached boxes; only sets unset values so
  the host's per-client mode still wins.

* Default Bazzite/SteamOS to ATTACH (PUNKTFUNK_GAMESCOPE_ATTACH=1 in host.env):
  the box owns its session (Desktop<->Game, persistent), the host follows +
  captures it and never tears it down — so switching is rock-solid and a
  disconnect leaves the box in its mode (reconnect returns there).

* Resize-on-attach (gamescope.rs): on connect, ensure the box's own game-mode
  session runs at the CLIENT's resolution — reuse it when already matching (fast
  path, no restart), else reconfigure + restart the box's own autologin
  gamescope-session-plus@<client> at the client mode (cooperative: no competing
  unit, so no autologin-respawn fight). Detect the live gamescope's -W/-H via
  argv[0] in /proc (its /proc/<pid>/exe is unreadable for that process).

Validated live on a headless bazzite-deck-nvidia box: game mode boots headless +
stable (0 strikes); the host attaches + streams video/audio/EIS input; a
5120x1440 client reuses the matching session and streams at 5120x1440.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:09:45 +00:00
enricobuehler 50e17b3508 fix(host/capture): hold the session through a slow compositor switch
apple / swift (push) Successful in 1m1s
ci / docs-site (push) Successful in 54s
apple / screenshots (push) Successful in 5m14s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 11s
android / android (push) Successful in 4m41s
ci / rust (push) Successful in 4m52s
ci / web (push) Successful in 49s
windows-host / package (push) Successful in 7m54s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m34s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m10s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m7s
A Bazzite/SteamOS Gaming↔Desktop switch tears the old compositor down and can
take 15s+ to bring the new one up — longer than the capture-loss rebuild's
~10s window, so the session failed mid-switch ("disconnect — session failed")
and forced the client to cold-reconnect. Retry the rebuild within a 40s budget
instead of giving up after one round, and re-detect the live compositor on
each attempt so the stream follows the box to whatever session comes up (a new
instance of the same compositor, or a different one — the kind-change case).
The QUIC keepalive runs on its own thread, so the client stays connected
(frozen on the last frame) and the stream resumes when the new output appears,
with no reconnect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 09:31:47 +00:00
enricobuehler 94c556f0e3 fix(host/capture): recover from compositor loss instead of freezing
apple / screenshots (push) Successful in 5m7s
apple / swift (push) Successful in 1m1s
windows-host / package (push) Successful in 7m26s
android / android (push) Successful in 4m50s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
decky / build-publish (push) Successful in 11s
ci / rust (push) Successful in 4m51s
deb / build-publish (push) Successful in 2m29s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m37s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m1s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m47s
When the compositor is torn down mid-stream (a Gaming↔Desktop switch removes
the virtual output), its PipeWire stream leaves Streaming for Paused rather
than disconnecting. try_latest treated that as Ok(None) ("static desktop —
repeat the last frame"), so the stream froze on the last frame forever and
neither recovery path fired: the capture-loss rebuild keys on Err, and the
session watcher keys on a session-KIND change (a desktop→desktop new KWin
instance is the same kind).

Track the PipeWire stream state via state_changed (a `streaming` flag) and,
in try_latest, surface a sustained non-Streaming state (1.5s grace for a
transient renegotiation blip) as a capture-loss Err — which the encode loop
already handles by rebuilding the pipeline in place. A static desktop stays
Streaming, so no false trigger. Complements the now-default session watcher.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 09:00:35 +00:00
enricobuehler 32c1929948 feat(host/session-watch): default Gaming↔Desktop follow on for Bazzite/SteamOS
apple / swift (push) Successful in 1m2s
android / android (push) Successful in 4m52s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 54s
decky / build-publish (push) Successful in 22s
windows-host / package (push) Successful in 9m7s
ci / bench (push) Successful in 4m40s
apple / screenshots (push) Successful in 5m20s
deb / build-publish (push) Successful in 2m31s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m40s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m39s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m24s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 47s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m19s
docker / deploy-docs (push) Successful in 22s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m29s
The mid-stream session watcher (rebuild the backend in place when the box
flips Gaming↔Desktop) was opt-in via PUNKTFUNK_SESSION_WATCH, so it never
ran on a stock Bazzite/SteamOS box — switching modes froze the stream on the
now-dead compositor. Default it ON when os-release ID/ID_LIKE is
bazzite/steamos (the platforms that flip sessions); still off on plain
desktops. Also parse the env properly so PUNKTFUNK_SESSION_WATCH=0 actually
disables it (was: any value, including "0", enabled it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 08:43:27 +00:00
enricobuehler 3915a82780 fix(host/input): route KWin auto-detect to the fake_input backend
apple / swift (push) Successful in 1m1s
apple / screenshots (push) Successful in 5m2s
windows-host / package (push) Successful in 6m56s
android / android (push) Successful in 4m42s
ci / rust (push) Successful in 4m52s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 56s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m4s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m0s
apply_input_env() hard-pinned PUNKTFUNK_INPUT_BACKEND=libei for KWin, and
default_backend() reads that env first — so the auto-detecting host (the
normal `serve` service) ignored the new KwinFakeInput backend and fell back
to the RemoteDesktop portal path that needs a user to approve. Route KWin to
"kwin" (org_kde_kwin_fake_input); GNOME/Mutter stay on libei (no fake_input
there).

Validated live on a Bazzite KDE box via the auto-detect path:
backend=KwinFakeInput, "KWin fake_input ready (no portal)", input events
forwarded with no errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:52:02 +00:00
enricobuehler a4833e4780 feat(android/touch): trackpad-relative cursor (default), with a direct-touch toggle
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 4m53s
ci / rust (push) Successful in 5m1s
ci / web (push) Successful in 58s
ci / docs-site (push) Successful in 55s
apple / screenshots (push) Successful in 5m28s
deb / build-publish (push) Successful in 2m30s
windows-host / package (push) Successful in 8m41s
decky / build-publish (push) Successful in 29s
ci / bench (push) Successful in 4m27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m43s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m35s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m25s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 48s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m46s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m1s
docker / deploy-docs (push) Successful in 24s
One-finger touch was absolute "direct pointing" — the host cursor jumped to the
finger and was recomputed from each touch-start, so you couldn't precisely reach a
target. Now a relative trackpad: the cursor stays put on touch-down and moves by the
finger delta (host MouseMove via nativeSendPointerMove, already supported — no
protocol change), with mild pointer acceleration and sub-pixel remainder
accumulation so slow precise moves aren't lost to Int truncation. Swipe, lift, and
re-swipe to walk it across; tap = left-click at the cursor's current position.
Two-finger scroll / right-click, three-finger HUD toggle, and tap-then-hold-drag are
preserved unchanged; finger-id re-anchoring keeps multi-touch transitions jump-free.

Added Settings → Pointer → "Trackpad mode" (default on); turning it off restores the
old direct-pointing path verbatim.

:app:compileDebugKotlin green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:34:03 +00:00
enricobuehler 4e79e6cdad fix(android/audio): kill the AAudio crackle (RT-safe ring + deeper buffer + XRun sizing)
The jitter ring was a port of the Linux client's, but Linux runs on PipeWire
(adaptive resampling masks host↔DAC drift + a shallow buffer); AAudio hands us a
raw realtime callback and we own the buffer, so the same code crackled only on
Android. Three converging causes, all fixed:

- Heap free on the realtime audio thread every quantum (Android's Scudo free() has
  unbounded tail latency → XRun → click). Decoded buffers are now recycled back to
  the producer via a free-list instead of freed on the audio thread; the ring is
  pre-reserved so extend() never reallocates there.
- The ring collapsed to ~15 ms on the tiny LowLatency burst and re-primed (a fresh
  silence) on every single empty callback. Now ~40 ms prime / ~150 ms hard cap,
  decoupled from the burst size, with de-prime hysteresis (re-prime only after a
  sustained drain).
- AAudio's anti-glitch knobs were unused: prime the HW buffer above its 2-burst
  default and grow it on getXRunCount(). The post-open log now reports
  perf/sharing/buffer so a fall to a resampled legacy path is visible.

Steady-state audio latency ~15 → ~40 ms (within lip-sync tolerance; matches the
Moonlight/Sunshine operating point). cargo-ndk build both ABIs + fmt + clippy green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:33:51 +00:00
enricobuehler f74bc4a3f1 feat(host/input): headless KDE input via org_kde_kwin_fake_input
Desktop-mode (KWin) streaming had no input: the path was libei via the
RemoteDesktop portal, which (a) isn't reachable from the host service env
and (b) requires a human to approve "Allow remote control?" — a
non-starter on a headless box. KWin's own headless RDP server (krdpserver)
solves this with org_kde_kwin_fake_input, authorized by the exact same
.desktop X-KDE-Wayland-Interfaces grant we already ship
(org_kde_kwin_fake_input is listed alongside zkde_screencast_unstable_v1).

Add a fake_input injector: vendor the protocol XML, bind the global as an
ordinary Wayland client, authenticate (auto-accepted for an
interface-authorized client — no dialog), and translate pointer (rel/abs),
button, scroll, keyboard (raw evdev keycodes resolved by KWin's own keymap)
and touch. Select it for KWin (compositor=="kwin" or XDG_CURRENT_DESKTOP
KDE); GNOME stays on libei (it has neither fake_input nor the wlr
protocols). PUNKTFUNK_INPUT_BACKEND=kwin forces it.

cargo check + clippy + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:26:04 +00:00
enricobuehler 8e18d01af5 fix(host/kwin): authorize Desktop-mode streaming via a shipped .desktop
Streaming the KDE *Desktop* (KWin) session failed on a real interactive
Plasma session with "KWin does not expose zkde_screencast_unstable_v1":
KWin treats the screencast/virtual-output and fake_input globals as
restricted and advertises them only to a client whose installed .desktop
lists them under X-KDE-Wayland-Interfaces (matched by /proc/<pid>/exe ->
Exec, and cached per-executable on first connect). The host shipped no
.desktop, so it was permanently denied; it only ever worked on the
headless dev box via KWIN_WAYLAND_NO_PERMISSION_CHECKS=1.

Ship packaging/linux/io.unom.Punktfunk.Host.desktop (least-privilege:
only the host, only zkde_screencast_unstable_v1 + org_kde_kwin_fake_input)
and install it from the RPM/.deb/Arch host packaging so it is present
before the host first connects. Drop the blunt session-wide
NO_PERMISSION_CHECKS hack from kde-desktop-setup.sh (it now only seeds the
RemoteDesktop input grant) and fix the now-misleading kwin.rs docs/errors.

Validated live on a Bazzite Kinoite box (KWin 6.6.4): probe-compositor +
spike --source kwin-virtual succeed against a KWin running WITHOUT the
permission bypass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:15:39 +00:00
enricobuehler 3477cbe7ce fix(audio/windows): stop the client mic echoing back through the loopback
The Windows virtual mic fakes a capture endpoint by writing the client's
uplinked PCM into a virtual device's *render* endpoint, while the
desktop-audio plane loopback-captures the *default render* endpoint — with
no mutual exclusion between the two. WASAPI loopback captures the mixed
output of an endpoint (everything any app renders to it, including our mic
writes), so when both resolve to the same device — VB-CABLE used for both,
or the auto-installed Steam Streaming Microphone being the default render on
a headless box — the injected mic is captured straight back into the
host->client audio stream: an infinite echo.

find_device() now resolves the loopback's endpoint id (default render) and
skips any candidate matching it, scanning on to the next non-loopback match,
so the mic can never land on the device the loopback reads. The auto-install
path now provisions the full Steam pair (Streaming Microphone + Streaming
Speakers) so a bare host gets two distinct devices instead of one shared
one. Errors distinguish "no device" from "only candidate is the loopback
device". Linux was already immune (its mic is a dedicated Audio/Source node,
structurally separate from the monitored sink).

Windows-only (#[cfg(windows)]); rustfmt-clean, compile-checked in
windows-host CI, needs on-glass validation on the RTX box. Does not force
the system default playback onto Steam Streaming Speakers (IPolicyConfig) —
not required to break the echo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 23:51:46 +00:00
enricobuehler 5a2e07e865 style(windows): rustfmt install.rs to unbreak cargo fmt --all --check
apple / swift (push) Successful in 1m3s
ci / rust (push) Successful in 4m52s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 59s
apple / screenshots (push) Successful in 5m12s
ci / bench (push) Successful in 4m40s
windows-host / package (push) Successful in 6m28s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m17s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m13s
release / apple (push) Successful in 10m9s
deb / build-publish (push) Successful in 2m44s
decky / build-publish (push) Successful in 11s
android / android (push) Successful in 3m33s
flatpak / build-publish (push) Successful in 4m9s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m5s
docker / deploy-docs (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m41s
The pnputil /add-driver call in windows/install.rs was committed unwrapped;
`cargo fmt --all --check` (which checks cfg(windows) files too) flagged it and
failed the `rust` CI job at the Format step, skipping clippy/build/test. Apply
rustfmt — no behavior change. Clears the way to cut the v0.2.0 release from
green main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 23:19:12 +00:00
enricobuehler 6e949b6748 fix(readme): make the logo readable on light + dark themes
apple / swift (push) Successful in 1m3s
apple / screenshots (push) Successful in 5m25s
ci / rust (push) Failing after 1m5s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 1m0s
android / android (push) Successful in 3m59s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
ci / bench (push) Successful in 4m35s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m55s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
docker / deploy-docs (push) Successful in 7s
The wordmark was light violet only — low-contrast on a light README
background. Swap to a single theme-adaptive SVG: an internal
`prefers-color-scheme` media query paints it deep violet (the brand-mark
palette) on light backgrounds and the original light violet on dark, so it
reads on both GitHub/Gitea themes with no markup change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:54:03 +00:00
enricobuehler 8ae161fe61 docs(windows): README - install via punktfunk-host.exe driver install / web setup (not .ps1)
apple / swift (push) Successful in 1m0s
windows-host / package (push) Successful in 6m20s
apple / screenshots (push) Successful in 5m26s
ci / rust (push) Failing after 26s
ci / web (push) Successful in 54s
deb / build-publish (push) Successful in 2m30s
ci / docs-site (push) Successful in 1m3s
android / android (push) Successful in 3m19s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m35s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m2s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m48s
docker / deploy-docs (push) Successful in 6s
Option A removed install-pf-vdisplay.ps1 / install-gamepad-drivers.ps1 / web-setup.ps1;
the installer now calls the exe subcommands. Drop the stale table rows + reword the
install-flow + 'thin installer' notes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:46:05 +00:00
enricobuehler 3a89ee8cd7 docs(readme): add logo banner + refresh Windows-host status
- Add the centered punktfunk wordmark banner at the top (assets/punktfunk-logo.svg,
  the same logo + layout the marketing site's README uses).
- Refresh the now-stale Windows-host facts: all-vendor (NVENC + AMF/QSV), its own
  all-Rust pf-vdisplay IddCx virtual display (was SudoVDA), bundled UMDF virtual-gamepad
  drivers (ViGEmBus gone), HDR incl. Vulkan-game HDR; x64-only, no longer NVIDIA-only.
- Note punktfunk-host covers Linux + Windows; point design/ at its new README index.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:45:29 +00:00
enricobuehler dac0fee4e3 docs(windows): reflect the install-via-exe (Option A) landing in the build/packaging doc
apple / swift (push) Successful in 1m3s
apple / screenshots (push) Successful in 5m31s
ci / web (push) Successful in 49s
decky / build-publish (push) Successful in 14s
ci / rust (push) Failing after 32s
ci / docs-site (push) Successful in 1m1s
android / android (push) Successful in 3m21s
deb / build-publish (push) Successful in 2m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
ci / bench (push) Successful in 4m49s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m32s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m47s
docker / deploy-docs (push) Successful in 6s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:44:47 +00:00
enricobuehler 125a51d81d feat(windows-installer): move driver + web install into the host exe (ASCII root fix)
apple / swift (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m16s
windows-host / package (push) Successful in 6m25s
ci / rust (push) Failing after 28s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m1s
android / android (push) Successful in 3m21s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m2s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m50s
docker / deploy-docs (push) Successful in 17s
Port the three install-time PowerShell *files* (install-pf-vdisplay.ps1,
install-gamepad-drivers.ps1, web-setup.ps1) into punktfunk-host.exe subcommands:
`driver install [--gamepad] --dir <stage>` and `web setup --app-dir <app>
[--password-file <f>]` (windows/install.rs).

Why: PowerShell 5.1 reads a BOM-less .ps1 FILE in the machine ANSI codepage, so a
stray non-ASCII byte mis-decodes and aborts on a non-English box - exactly how the
pf-vdisplay driver install silently failed. A compiled subcommand drives the same
external tools (certutil/pnputil/nefconc/schtasks/netsh/icacls) as fixed string
literals, with no file-codepage surface. (The .iss's INLINE -Command PowerShell is a
command-line string, not a file read, so it's unaffected and stays.)

- windows/install.rs: faithful port - cert trust, gated nefconc node create + pnputil
  for pf-vdisplay; pnputil per-inf for gamepads; web-password ACL, the PunktfunkWeb task
  (generated UTF-16 XML), firewall rule, start. Best-effort (a hiccup warns, never aborts).
- punktfunk-host.iss [Run]: call the exe instead of `powershell -File`; drop the
  web-setup.ps1 staging + WebSetup define; WebSetupParams emits --app-dir/--password-file.
- pack-host-installer.ps1: stop copying the three install scripts into the stages.
- delete the three .ps1 files.

The `mod install;` + dispatch arms in main.rs landed in the preceding docs commit
(swept up by a concurrent commit); this commit adds the module + installer wiring.
CI-compile-validated via windows-host; the install path is on-glass-validated on the
next canary install (the test box is offline).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:43:18 +00:00
enricobuehler 7b99b41ede docs(design): trim shipped plans, consolidate cluster, add index
Much of design/ described work that has since shipped. Trim each doc to
its durable rationale + still-open items (the code is the source of truth
for shipped detail; git history holds the full originals).

- Shipped plans -> status stubs: stats-capture, gamestream-host-plan,
  apple-stage2-presenter, windows-service.
- Trimmed completed-out / open-kept: implementation-plan, hdr-pipeline,
  host-latency, gpu-contention (fixed stale status table), game-library,
  linux-setup (fixed m0->spike + stale zero-copy claim),
  session-aware-host-followups, windows-client-bootstrap,
  windows-dualsense-{scoping,game-detection}, windows-virtual-display,
  security-review (per-finding status table; #12 still open),
  apollo-comparison (shipped backlog collapsed to one-liners).
- Windows-host cluster consolidated: windows-host.md -> redirect into
  windows-host-rewrite.md (whose stale scorecard is corrected -- goal1 is
  merged, M4 done); windows-secure-desktop.md archived (now a fallback
  behind IDD-push primary).
- Kept evergreen: ci.md, gamescope-multiuser.md, windows-build-and-packaging.md.
- New design/README.md: per-doc status table + consolidated open-items
  roll-up so nothing is tracked in only one buried doc.
- Repoint 5 code comments to the archived secure-desktop doc path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:39:06 +00:00
enricobuehler 9ea2c17419 docs(windows): add design/windows-build-and-packaging.md + refresh packaging README
apple / swift (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m19s
windows-host / package (push) Successful in 6m20s
android / android (push) Successful in 4m42s
ci / rust (push) Successful in 4m47s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 58s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
ci / bench (push) Successful in 4m40s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m16s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m3s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m0s
docker / deploy-docs (push) Successful in 22s
A single repo-internal source of truth for the Windows build/packaging: what ships, the
all-Rust driver workspace built FROM SOURCE in CI (+ the anti-stale rationale), the
toolchain (clang 22 + bindgen 0.72, no LLVM pin), the Inno installer, the web console
bundle, the CI workflows, signing, and the dev loop. (design/, not the docs-site.)

packaging/windows/README.md: drop the deleted vendored-driver dir + its "Vendored driver"
callout, add the build-* / install-gamepad / clear-force-integrity rows, point at the new
design doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:22:40 +00:00
enricobuehler a9cca82fb8 chore(windows): clean up build/packaging - drop vendored driver binaries + the LLVM-21 pin
windows-drivers-provision / provision (push) Successful in 13s
windows-drivers / probe-and-proto (push) Successful in 17s
android / android (push) Failing after 40s
apple / swift (push) Successful in 1m0s
ci / web (push) Successful in 58s
windows-drivers / driver-build (push) Successful in 1m9s
ci / docs-site (push) Successful in 1m18s
ci / rust (push) Successful in 4m25s
apple / screenshots (push) Successful in 5m24s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
deb / build-publish (push) Successful in 2m29s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
ci / bench (push) Successful in 4m48s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
windows-host / package (push) Successful in 6m38s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m24s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m31s
docker / deploy-docs (push) Successful in 18s
Now that the drivers build from source in CI, remove the dead checked-in binaries and
the toolchain cruft they left behind:

- Delete packaging/windows/{pf-vdisplay,gamepad-drivers}/ (the prebuilt .dll/.inf/.cat/.cer).
  pack-host-installer.ps1 builds + signs all three drivers from the drivers/ workspace and
  nothing reads the vendored dirs anymore; stage-pf-vdisplay.ps1's -VendorDir is now a
  mandatory build-output path, not a vendored default.
- Drop the LLVM-21 pin. The vendored bindgen 0.71->0.72 bump (the shipping pack already
  builds green on the runner-default clang 22) retired the bindgen-0.71 layout-test overflow
  that needed LLVM 21.1.2, so windows-drivers.yml + provision-windows-wdk.ps1 no longer
  install/point at C:\llvm-21 (~898 MB off a fresh provision) - both driver builds now use one
  toolchain (clang 22 + bindgen 0.72).
- pack -SkipBuild on the gamepad build (build-pf-vdisplay.ps1 already builds the whole
  workspace), build-web.ps1 reaps a stale node too, deploy-dev.ps1 nefconc path + comments.
- Reword the vendored-driver references (build scripts, .iss, READMEs, the vite web-bundle
  comment) to the build-from-source reality.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:16:46 +00:00
enricobuehler 7ab0661ddc fix(windows-installer): escape the brace in the [UninstallRun] PowerShell so ISCC compiles
windows-drivers / probe-and-proto (push) Successful in 21s
apple / swift (push) Successful in 1m4s
windows-drivers / driver-build (push) Successful in 1m9s
android / android (push) Successful in 4m25s
ci / web (push) Successful in 53s
apple / screenshots (push) Successful in 5m32s
ci / rust (push) Successful in 4m45s
ci / docs-site (push) Successful in 52s
windows-host / package (push) Successful in 6m47s
deb / build-publish (push) Successful in 2m28s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m45s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m55s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
The Bug C [UninstallRun] one-liner had `ForEach-Object { Stop-Process ... }`; Inno
Setup parses `{...}` as a constant in [Run]/[UninstallRun] sections, so ISCC aborted
with "Unknown constant" and the windows-host pack failed at the ISCC step (the host
build, clippy, driver build + web smoke-boot all passed). Escape `{` as `{{`. The
same one-liner in the [Code] StopWebConsole proc is inside a Pascal string literal,
so its brace is literal and must NOT be escaped. Validated: ISCC now parses past
[UninstallRun] + [Code] (fails only later on the absent dummy payload).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 15:15:07 +00:00
enricobuehler 92e68024f1 fix(windows-installer): build the gamepad drivers from source in CI too
Fold the pf-dualsense (DualSense / DualShock 4) and pf-xusb (Xbox 360 / XInput)
UMDF drivers into the in-tree drivers workspace (their source had stale
../../crates/wdk-* path-deps from before the wdk vendoring reorg and could no
longer build at all) and build them from source per release, exactly like
pf-vdisplay - same anti-stale reasoning. One `cargo build --release` now builds
all three drivers against the vendored wdk-sys (incl. the bindgen 0.72 pin), and
build-gamepad-drivers.ps1 signs pf_dualsense + pf_xusb (clear FORCE_INTEGRITY ->
sign dll -> stampinf -> Inf2Cat -> sign cat) with one shared cert + .cer,
matching the layout install-gamepad-drivers.ps1 expects. pack-host-installer.ps1
builds + stages them instead of the retired checked-in binaries.

Validated on the runner: the whole workspace (pf-vdisplay + pf-dualsense +
pf-xusb) builds with CARGO_TARGET_DIR=C:\t set, and build-gamepad-drivers.ps1
produces signed pf_dualsense.{dll,inf,cat} + pf_xusb.{dll,inf,cat} + the .cer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 15:08:40 +00:00
enricobuehler 64abce6daa fix(windows-installer): pf-vdisplay CI build - default target dir + non-fatal cat guard
apple / swift (push) Successful in 59s
android / android (push) Successful in 4m23s
ci / rust (push) Successful in 4m43s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
windows-host / package (push) Failing after 5m39s
apple / screenshots (push) Successful in 5m15s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m6s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m52s
The CI driver build panicked in wdk-sys's build script - "a Cargo.lock file should
exist in the same directory as the top-level Cargo.toml". wdk-build's
find_top_level_cargo_manifest() walks UP from OUT_DIR for the first ancestor holding a
Cargo.lock and explicitly does NOT support non-default target dirs - but
build-pf-vdisplay.ps1 pointed CARGO_TARGET_DIR at an out-of-tree dir (to isolate from
CI's shared C:\t), so no ancestor of OUT_DIR had a Cargo.lock. Build into the driver
workspace's DEFAULT target dir instead (its ancestors include the driver Cargo.lock);
the driver's own [workspace] already isolates it and it has no CMake deps needing C:\t.
Also make the Test-FileCatalog coverage guard non-fatal (it can't open a catalog
signed by a not-yet-trusted cert). Validated on the runner with CARGO_TARGET_DIR=C:\t.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 14:58:20 +00:00
enricobuehler bdfab8e0d5 fix(windows-installer): build pf-vdisplay from source in CI; ASCII scripts; upgrade-safe web console
windows-drivers / probe-and-proto (push) Successful in 24s
apple / swift (push) Successful in 1m4s
windows-drivers / driver-build (push) Successful in 1m8s
android / android (push) Successful in 4m4s
ci / rust (push) Successful in 4m39s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 53s
apple / screenshots (push) Successful in 5m10s
windows-host / package (push) Failing after 5m35s
deb / build-publish (push) Successful in 2m29s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
ci / bench (push) Successful in 4m42s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m57s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
The pf-vdisplay virtual-display driver shipped as a checked-in PREBUILT binary
that went stale - two field failures on a fresh install (live-repro'd on a
German-locale Dell laptop):

  * Bug A (every box): a repo-wide rename edited the vendored pf_vdisplay.inf
    but never re-signed pf_vdisplay.cat, so the catalog stopped covering the INF
    -> `pnputil /add-driver` fails SPAPI_E_FILE_HASH_NOT_IN_CATALOG -> driver
    never installs -> every session dies "pf-vdisplay driver interface not
    found".
  * the prebuilt binary also predated IOCTL_SET_RENDER_ADAPTER (added to the
    driver source after the vendor freeze) that the host needs to pin the IDD
    render GPU on hybrid/Optimus boxes.

Fix: build the driver FROM SOURCE every release (build-pf-vdisplay.ps1, wired
into pack-host-installer.ps1) so .dll/.inf/.cat are always in lockstep and
current driver features ship. The runner's clang 22 made the driver's pinned
bindgen 0.71 emit opaque structs (157 layout-assert errors), so bump the
vendored wdk-sys/wdk-build bindgen 0.71 -> 0.72 (+ lock). The build self-signs
the driver per build (installer trusts the bundled .cer); a stable
DRIVER_CERT_PFX_B64 secret can override.

  * Bug B (non-English boxes): the installer runs install-pf-vdisplay.ps1 etc.
    via powershell.exe (5.1), which reads a BOM-less script in the ANSI codepage
    - an em-dash's trailing 0x94 byte becomes a curly quote on German
    Windows-1252 and the script aborts "unterminated string", so the driver
    never installed (the gamepad script survived only because it was already
    ASCII). Scrub every installer-run .ps1/.cmd to ASCII + add a CI gate that
    fails on any non-ASCII so it can't regress.

  * Bug C (upgrades): nothing stopped the OLD web console before re-registering
    its task, so a stale server kept :3000 (the new one restart-looped on
    EADDRINUSE) and served a broken old bundle (500 on /login). Stop + reap it
    (runtime-agnostic, by the :3000 listener owner) in web-setup.ps1 and in the
    .iss before the file copy + on uninstall.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 14:33:34 +00:00
enricobuehler 8e87e617df fix(windows-host): force EXTEND topology so a new IddCx display isn't cloned
A freshly-added IddCx virtual display lands in CLONE/duplicate mode when a
physical display is already active (a laptop panel, an attached monitor): the
cloned output shares that display's source, so the OS never commits a distinct
path for it, never calls ASSIGN_SWAPCHAIN, and capture sees no frames - the
session fails "not an active display path / needs a WDDM GPU to activate" and
tears down with 0 frames (seen live on an Intel-iGPU + NVIDIA-Optimus laptop).

force_extend_topology() applies the EXTEND preset (the programmatic Win+P
"Extend") right after ADD so the IDD comes up as its own active path; the
existing resolve_gdi_name -> set_active_mode -> isolate_displays_ccd bring-up
then proceeds. Idempotent / no-op on a sole-display (headless single-GPU) box,
so it's safe on the path that already worked.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 14:33:15 +00:00
enricobuehler 5bf787eb2b feat(host): web-console performance capture — record stream stats, graph them
apple / swift (push) Successful in 1m1s
android / android (push) Successful in 4m13s
ci / rust (push) Successful in 4m42s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 53s
windows-host / package (push) Successful in 5m51s
apple / screenshots (push) Successful in 5m1s
deb / build-publish (push) Successful in 2m29s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
ci / bench (push) Successful in 4m35s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m9s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m10s
Arm streaming-perf-stats capture from the web console, play, stop, and review the
run as graphs; finished captures are saved to disk as browsable/exportable
recordings. Covers both the native punktfunk/1 path and GameStream.

- stats_recorder.rs: one shared Arc<StatsRecorder> ring (created in gamestream::serve,
  shared with the mgmt API + both streaming loops, mirroring NativePairing). The
  hot-path gate is a runtime AtomicBool that replaces the startup-only PUNKTFUNK_PERF
  for *recording* (PERF stdout logging unchanged); bounded ring (~3 h); atomic
  temp+rename writes to ~/.config/punktfunk/captures/*.json; path-traversal-safe ids;
  poison-resilient locks.
- native (punktfunk1.rs) + GameStream (stream.rs) emit a StatsSample at their existing
  ~2 s / ~1 s aggregation boundary — per-stage latency p50/p99, fps new/repeat, goodput,
  loss/FEC deltas — with no new per-frame work beyond the cheap atomic check.
  FrameMsg.was_measured keeps pre-arm in-flight frames out of the first window's
  percentiles (without zeroing the Windows-relay path's fps/encode).
- mgmt.rs: 7 bearer-only /api/v1/stats/* endpoints (capture start/stop/status/live;
  recordings list/get/delete); api/openapi.json regenerated, in sync.
- web: new "Performance" page (recharts, rendered SSR-safe) — capture control, live
  graphs while armed, recordings table (view / download-JSON / delete), and a detail
  view with the latency stacked-area bottleneck breakdown (p50/p99 toggle) + throughput
  + health. Charts adapt to either path's stage set.

Design: design/stats-capture-plan.md. Built and adversarially reviewed via a multi-agent
workflow; workspace build/clippy(-D warnings)/fmt/tests green, OpenAPI no-drift. Not yet
on-glass validated against a live session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 13:59:39 +00:00
enricobuehler 0a6c9d8852 docs: point Android install at Discord for beta access + add community links
apple / swift (push) Successful in 1m32s
apple / screenshots (push) Successful in 3m26s
android / android (push) Successful in 4m7s
ci / rust (push) Successful in 4m36s
ci / web (push) Successful in 44s
ci / docs-site (push) Successful in 53s
deb / build-publish (push) Successful in 2m18s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 42s
ci / bench (push) Successful in 4m42s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m12s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m8s
docker / deploy-docs (push) Successful in 6s
The Android app is in Google Play Internal Testing, so the public Play Store URL
doesn't resolve for non-testers. Lead the Android install instructions with a
"request a tester invite on Discord" CTA (the Play listing unlocks once a Google
account is added to the test track), and surface the Discord + r/Punktfunk
community links in the README, the docs intro, and the docs-site nav.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 11:59:25 +00:00
enricobuehler 0eedfb3c1f docs: first-class Linux + Windows positioning + IDD-push differentiator
apple / swift (push) Failing after 0s
apple / screenshots (push) Has been skipped
windows-drivers-provision / provision (push) Successful in 13s
windows-drivers / probe-and-proto (push) Successful in 17s
windows-drivers / driver-build (push) Successful in 1m10s
android / android (push) Successful in 3m19s
ci / web (push) Successful in 39s
ci / docs-site (push) Successful in 53s
windows-host / package (push) Successful in 6m6s
ci / rust (push) Successful in 11m12s
decky / build-publish (push) Successful in 11s
ci / bench (push) Successful in 5m9s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 21s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 43s
deb / build-publish (push) Successful in 7m31s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m14s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m12s
release / apple (push) Failing after 1s
docker / deploy-docs (push) Successful in 19s
flatpak / build-publish (push) Successful in 4m43s
Drop the "Linux-first" framing across the README and docs site in favor of
first-class Linux AND Windows hosts, and surface the Windows IDD-push
virtual-display path as a distinct differentiator (punktfunk's own indirect
display driver the host pushes frames into — a real virtual display, no physical
monitor or dummy plug, even on the secure desktop).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 11:53:02 +00:00
enricobuehler f6490f4c28 fix: complete the docs/→design/ and openapi→api/ rename references
The file moves (docs/ → design/, docs/api/openapi.json → api/openapi.json) landed
in d01a8fd, but the matching reference updates did not — so mgmt.rs's drift-test
`include_str!("../../../docs/api/openapi.json")` pointed at a path that no longer
exists and the host failed to build. This restores it and updates every reference:

  - mgmt.rs include_str! → ../../../api/openapi.json (fixes the build)
  - web/orval.config.ts codegen target, web/Dockerfile, .dockerignore
  - deb/rpm/Arch packaging install paths
  - CLAUDE.md, the .gitea CI workflows, code doc-comments, design-doc cross-links

docs-site route URLs (/docs/...) untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 11:53:02 +00:00
240 changed files with 12086 additions and 5111 deletions
+2 -2
View File
@@ -1,9 +1,9 @@
# Root build context is used only by web/Dockerfile, which needs web/ and
# docs/api/openapi.json. Allowlist those; keep everything else (target/, .git, crates)
# api/openapi.json. Allowlist those; keep everything else (target/, .git, crates)
# out of the context upload.
*
!web
!docs/api/openapi.json
!api/openapi.json
web/node_modules
web/.output
web/dist
+57
View File
@@ -0,0 +1,57 @@
# Android client screenshots for the Play listing / marketing. Roborazzi renders the real Compose
# UI with mock state on the host JVM via Robolectric — NO emulator, GPU, KVM, host, or JNI core
# (`-PskipRustBuild` skips the cargo-ndk native build). The Android analogue of apple.yml's
# `screenshots` job, gated to STABLE RELEASE tags only. Standalone + best-effort: a failure here
# reds nothing else. PNGs land as a 30-day artifact; not committed or published.
name: android-screenshots
on:
push:
tags: ["v*"]
workflow_dispatch:
jobs:
screenshots:
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- name: JDK 21 (AGP 9.2 + Robolectric's SDK-36 android-all jar both want 1721)
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "21"
- name: Android SDK
uses: android-actions/setup-android@v3
# No NDK/CMake — the screenshot unit tests are pure JVM. compileSdk 37 auto-downloads via AGP
# if the platform channel lacks it (same note as android.yml).
- name: platform-tools + platform 36 + build-tools
run: sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0"
- name: Cache (gradle)
uses: actions/cache@v4
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: android-screenshots-${{ hashFiles('clients/android/**/*.gradle.kts') }}
restore-keys: android-screenshots-
# Roborazzi renders Compose on the JVM (Robolectric Native Graphics). `-PskipRustBuild` keeps
# the cargo-ndk native build out of the graph — the tests never load libpunktfunk_android.so.
- name: Capture screenshots (Roborazzi)
working-directory: clients/android
run: ./gradlew :app:testDebugUnitTest -PskipRustBuild --stacktrace
- name: Upload screenshots
if: always()
# v3: Gitea's API rejects upload-artifact@v4 (see apple.yml). Download is a zip.
uses: actions/upload-artifact@v3
with:
name: punktfunk-android-screenshots
path: clients/android/app/build/outputs/roborazzi
retention-days: 30
+35
View File
@@ -32,6 +32,25 @@ jobs:
dirname "$RUSTUP" >> "$GITHUB_PATH"
"$RUSTUP" target add aarch64-apple-darwin x86_64-apple-darwin
# `punktfunk-core` now decodes Opus in-core for the Apple client (surround), pulling
# `audiopus_sys`, which builds a vendored static libopus via CMake when pkg-config can't find a
# system Opus — so the xcframework is self-contained (no runtime libopus.dylib on end-user Macs).
# CMake must be on PATH; install it self-healing on a fresh runner.
- name: CMake (for the vendored libopus audiopus_sys builds)
run: |
# Runner steps run with `bash --noprofile --norc`, so Homebrew's bin dir isn't on PATH —
# locate brew explicitly, install cmake if missing, and export its bin dir to GITHUB_PATH so
# the xcframework build step (audiopus_sys → vendored libopus) finds `cmake`.
for B in /opt/homebrew/bin/brew /usr/local/bin/brew; do [ -x "$B" ] && BREW="$B" && break; done
if [ -z "$BREW" ]; then echo "::error::Homebrew not found on the runner"; exit 1; fi
BREW_BIN="$(dirname "$BREW")"; export PATH="$BREW_BIN:$PATH"
command -v cmake >/dev/null || "$BREW" install cmake
echo "$BREW_BIN" >> "$GITHUB_PATH"
# Homebrew's CMake 4 dropped compatibility with the vendored libopus's pre-3.5
# `cmake_minimum_required`; treat 3.5 as the policy minimum (the cmake crate's child cmake
# inherits this from the env during the xcframework build).
echo "CMAKE_POLICY_VERSION_MINIMUM=3.5" >> "$GITHUB_ENV"
- name: Build PunktfunkCore.xcframework
run: bash scripts/build-xcframework.sh
@@ -71,6 +90,22 @@ jobs:
"$RUSTUP" target add aarch64-apple-darwin x86_64-apple-darwin \
aarch64-apple-ios aarch64-apple-ios-sim x86_64-apple-ios
# See the swift job: audiopus_sys (via the in-core Opus decode) builds vendored libopus with CMake.
- name: CMake (for the vendored libopus audiopus_sys builds)
run: |
# Runner steps run with `bash --noprofile --norc`, so Homebrew's bin dir isn't on PATH —
# locate brew explicitly, install cmake if missing, and export its bin dir to GITHUB_PATH so
# the xcframework build step (audiopus_sys → vendored libopus) finds `cmake`.
for B in /opt/homebrew/bin/brew /usr/local/bin/brew; do [ -x "$B" ] && BREW="$B" && break; done
if [ -z "$BREW" ]; then echo "::error::Homebrew not found on the runner"; exit 1; fi
BREW_BIN="$(dirname "$BREW")"; export PATH="$BREW_BIN:$PATH"
command -v cmake >/dev/null || "$BREW" install cmake
echo "$BREW_BIN" >> "$GITHUB_PATH"
# Homebrew's CMake 4 dropped compatibility with the vendored libopus's pre-3.5
# `cmake_minimum_required`; treat 3.5 as the policy minimum (the cmake crate's child cmake
# inherits this from the env during the xcframework build).
echo "CMAKE_POLICY_VERSION_MINIMUM=3.5" >> "$GITHUB_ENV"
- name: Build PunktfunkCore.xcframework (mac + iOS slices)
run: BUILD_IOS=1 bash scripts/build-xcframework.sh
+44 -13
View File
@@ -11,12 +11,18 @@
# punktfunk.zip
# punktfunk/ <- single top-level dir == plugin.json "name"
# plugin.json [required]
# package.json [required]
# package.json [required; CI stamps "version" — Decky reads the installed version here]
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
#
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
# plugin polls it (main.py check_update), and the frontend drives Decky's own install RPC to
# apply a newer build. See clients/decky/README.md "Updating".
#
# REGISTRY_TOKEN: repo Actions secret, a PAT with write:package scope (shared with deb/rpm/docker).
name: decky
@@ -56,20 +62,26 @@ jobs:
pnpm install --frozen-lockfile
pnpm run build # rollup -> clients/decky/dist/index.js
- name: Version + channel
# Tag vX.Y.Z -> X.Y.Z (stable `latest/` alias + Gitea Release); main push -> 0.3.0-ciN.g<sha>
# (`canary/` alias). Used for the registry version path + the zip name (the plugin.json
# version is the source of truth Decky reads after install — bump it in the release commit).
- name: Version + channel + stamp
# Tag vX.Y.Z -> X.Y.Z (stable `latest/` alias + Gitea Release); main push -> 0.3.<run>
# (`canary/` alias). Decky reads a plugin's INSTALLED version from package.json (NOT
# plugin.json), and the plugin's own update check (clients/decky/main.py check_update)
# compares against it — so the build version is STAMPED into package.json here (mirrored
# into plugin.json for store parity). Canary is a PLAIN numeric semver, never a
# `-ci<N>` prerelease: compare-versions orders prerelease identifiers lexically
# (ci10 < ci9), which would break update detection; the run number is monotonic.
working-directory: ${{ gitea.workspace }}
run: |
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; ALIAS=latest ;;
*) V="0.3.0-ci${GITHUB_RUN_NUMBER}.g${SHORT}"; ALIAS=canary ;;
*) V="0.3.${GITHUB_RUN_NUMBER}"; ALIAS=canary ;;
esac
BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE"
echo "VERSION=$V" >> "$GITHUB_ENV"
echo "ALIAS=$ALIAS" >> "$GITHUB_ENV"
echo "BASE=$BASE" >> "$GITHUB_ENV"
echo "decky version $V -> alias '$ALIAS'"
VERSION="$V" node -e 'const fs=require("fs");for(const f of ["clients/decky/package.json","clients/decky/plugin.json"]){const j=JSON.parse(fs.readFileSync(f,"utf8"));j.version=process.env.VERSION;fs.writeFileSync(f,JSON.stringify(j,null,2)+"\n");}'
- name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }}
@@ -89,9 +101,20 @@ jobs:
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip"
# The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias).
SHA=$(sha256sum "$RUNNER_TEMP/punktfunk.zip" | cut -d' ' -f1)
printf '{"version":"%s","artifact":"%s/%s/punktfunk.zip","sha256":"%s"}\n' \
"$VERSION" "$BASE" "$VERSION" "$SHA" > "$RUNNER_TEMP/manifest.json"
cat "$RUNNER_TEMP/manifest.json"
- name: Publish to the Gitea generic registry
working-directory: ${{ gitea.workspace }}
@@ -99,18 +122,26 @@ jobs:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
BASE="https://$REGISTRY/api/packages/$OWNER/generic/$PACKAGE"
# 1) Immutable, versioned URL.
# 1) Immutable, versioned URL + its update manifest (the manifest's `artifact` points
# here, so the published sha256 keeps matching what Decky later downloads).
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/punktfunk.zip" \
"$BASE/$VERSION/punktfunk.zip"
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/manifest.json" \
"$BASE/$VERSION/manifest.json"
echo "published $BASE/$VERSION/punktfunk.zip"
# 2) Channel alias (stable release -> latest/, canary main build -> canary/) — the link
# to paste into Decky's "install from URL". The generic registry rejects re-uploading
# an existing version/file (409), so delete the prior alias first (ignore 404 on run #1).
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
"$BASE/$ALIAS/punktfunk.zip" || true
# 2) Channel alias (stable release -> latest/, canary main build -> canary/) — the
# zip is the "install from URL" link; manifest.json is what the installed plugin
# polls for updates. The generic registry rejects re-uploading an existing
# version/file (409), so delete the prior alias copies first (ignore 404 on run #1).
for f in punktfunk.zip manifest.json; do
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE "$BASE/$ALIAS/$f" || true
done
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/punktfunk.zip" \
"$BASE/$ALIAS/punktfunk.zip"
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$RUNNER_TEMP/manifest.json" \
"$BASE/$ALIAS/manifest.json"
echo "install-from-URL link: $BASE/$ALIAS/punktfunk.zip"
echo "update manifest: $BASE/$ALIAS/manifest.json"
- name: Attach zip to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v')
+1 -1
View File
@@ -24,7 +24,7 @@ on:
push:
branches: [main]
# The flatpak is the CLIENT — only rebuild when the client/core/manifest change, not on every
# docs/host push (this is a heavy flatpak-builder run). Tags (v*, the client release) build too.
# design/host push (this is a heavy flatpak-builder run). Tags (v*, the client release) build too.
paths:
- 'clients/linux/**'
- 'crates/punktfunk-core/**'
@@ -0,0 +1,67 @@
# Native Linux client screenshots for the app/marketing listings. The client renders
# host-free mock scenes (PUNKTFUNK_SHOT_SCENE) under a virtual X display; the driver
# (clients/linux/tools/screenshots.sh) grabs each one — no host, GPU, or Wayland. The
# Linux analogue of apple.yml's `screenshots` job, gated to STABLE RELEASE tags only.
# Standalone + best-effort: a failure here reds nothing else. PNGs land as a 30-day
# artifact; they are not committed or published.
name: linux-client-screenshots
on:
push:
tags: ["v*"]
workflow_dispatch:
jobs:
screenshots:
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
# Same image as ci.yml/deb.yml — already carries the Rust toolchain + GTK/SDL build deps.
container:
image: git.unom.io/unom/punktfunk-rust-ci:latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
# Client link deps (baked into the image; kept here so the job is green across image
# rebuilds — a no-op once present) PLUS the headless-render extras: a virtual X server,
# software GL+Vulkan (llvmpipe/lavapipe), the icon theme + fonts the UI draws with, and a
# root-window grab tool.
- name: Client link + headless-render deps
run: |
apt-get update
apt-get install -y --no-install-recommends \
libgtk-4-dev libadwaita-1-dev libsdl3-dev \
xvfb x11-utils imagemagick scrot \
libgl1-mesa-dri mesa-vulkan-drivers \
adwaita-icon-theme fonts-cantarell fonts-dejavu-core
# Reuse the workspace cargo caches (same keys as ci.yml/deb.yml).
- name: Cache keys
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
- uses: actions/cache@v4
with:
path: |
/usr/local/cargo/registry
/usr/local/cargo/git
key: cargo-home-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-home-
- uses: actions/cache@v4
with:
path: target
key: cargo-target-v3-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-target-v3-${{ env.rustc }}-
- name: Build client
run: cargo build --release -p punktfunk-client-linux --locked
- name: Capture screenshots
run: bash clients/linux/tools/screenshots.sh
- name: Upload screenshots
if: always()
# v3: Gitea's API rejects upload-artifact@v4 (see apple.yml). Download is a zip.
uses: actions/upload-artifact@v3
with:
name: punktfunk-linux-client-screenshots
path: clients/linux/screenshots
retention-days: 30
+17
View File
@@ -118,6 +118,23 @@ jobs:
"$RUSTUP" toolchain install nightly --profile minimal
"$RUSTUP" component add rust-src --toolchain nightly
# The in-core Opus decode (surround) pulls audiopus_sys, which builds a vendored static libopus
# via CMake — keep the xcframework self-contained (no runtime libopus.dylib on end-user devices).
- name: CMake (for the vendored libopus audiopus_sys builds)
run: |
# Runner steps run with `bash --noprofile --norc`, so Homebrew's bin dir isn't on PATH —
# locate brew explicitly, install cmake if missing, and export its bin dir to GITHUB_PATH so
# the xcframework build step (audiopus_sys → vendored libopus) finds `cmake`.
for B in /opt/homebrew/bin/brew /usr/local/bin/brew; do [ -x "$B" ] && BREW="$B" && break; done
if [ -z "$BREW" ]; then echo "::error::Homebrew not found on the runner"; exit 1; fi
BREW_BIN="$(dirname "$BREW")"; export PATH="$BREW_BIN:$PATH"
command -v cmake >/dev/null || "$BREW" install cmake
echo "$BREW_BIN" >> "$GITHUB_PATH"
# Homebrew's CMake 4 dropped compatibility with the vendored libopus's pre-3.5
# `cmake_minimum_required`; treat 3.5 as the policy minimum (the cmake crate's child cmake
# inherits this from the env during the xcframework build).
echo "CMAKE_POLICY_VERSION_MINIMUM=3.5" >> "$GITHUB_ENV"
- name: Build PunktfunkCore.xcframework (mac + iOS + tvOS)
# tvOS is a tier-3 target (nightly -Zbuild-std): slow on the first build, then cached on
# the self-hosted runner. Built on canary too so the tvOS archive/upload below runs on the
+53
View File
@@ -0,0 +1,53 @@
# Management-console screenshots for the app/marketing listings. Captured from the
# built Storybook with headless Chromium (web/tools/screenshots.mjs) — the page
# stories render from fixtures, so no live mgmt API, login, or GPU is needed. This
# is the web analogue of apple.yml's `screenshots` job, but gated to STABLE RELEASE
# tags only (the console has no release workflow of its own — it ships inside the
# host packaging). Best-effort: a standalone workflow, so a failure here reds
# nothing else. PNGs land as a 30-day artifact; they are not committed or published.
name: web-screenshots
on:
push:
tags: ["v*"]
workflow_dispatch:
jobs:
screenshots:
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
container:
image: oven/bun:1
timeout-minutes: 30
defaults:
run:
working-directory: web
steps:
# oven/bun ships neither git nor a real node (the driver runs under node), and
# the slim Debian base lacks a CA bundle — without it actions/checkout's HTTPS
# fetch dies with "Problem with the SSL CA cert" (same as ci.yml's web job).
- name: Install git + node + CA certs
working-directory: /
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates git nodejs
- uses: actions/checkout@v4
# --ignore-scripts skips the prepare→codegen hook (mirrors ci.yml); run codegen
# explicitly since build-storybook has no prebuild hook of its own.
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
- name: Generate API client + i18n messages
run: bun run codegen
# Pulls the matching Chromium build + the apt libs it needs (root in-container).
- name: Install Chromium
run: bunx playwright install --with-deps chromium
- name: Build Storybook
run: bun run build-storybook
- name: Capture screenshots
run: bun run screenshots
- name: Upload screenshots
if: always()
# v3: Gitea's API rejects upload-artifact@v4 (see apple.yml). Download is a zip.
uses: actions/upload-artifact@v3
with:
name: punktfunk-web-console-screenshots
path: web/screenshots
retention-days: 30
@@ -1,5 +1,5 @@
# One-shot provisioning of the WDK + cargo-wdk onto the persistent self-hosted windows-amd64 runner, so
# the all-Rust UMDF drivers can build there (docs/windows-host-rewrite.md, M0). The runner has the base
# the all-Rust UMDF drivers can build there (design/windows-host-rewrite.md, M0). The runner has the base
# Windows SDK + MSVC + LLVM + Rust but NOT the WDK (no km/wdf/iddcx headers) or cargo-wdk.
#
# Dispatch manually (workflow_dispatch). Idempotent: re-running is a near no-op once provisioned. The
+6 -6
View File
@@ -1,5 +1,5 @@
# Windows driver workspace CI — runs on the self-hosted Windows runner (home-windows-1, host mode;
# label windows-amd64). Part of the Windows-host rewrite (docs/windows-host-rewrite.md, M0).
# label windows-amd64). Part of the Windows-host rewrite (design/windows-host-rewrite.md, M0).
#
# Stage 1 (this file): PROBE the runner's driver toolchain (WDK / EWDK / cargo-make / LLVM / the
# inf2cat/stampinf/devgen/signtool tools) so we know what's provisioned BEFORE writing driver code,
@@ -76,7 +76,7 @@ jobs:
head "EWDK"
Write-Host ("EWDKROOT = " + ($env:EWDKROOT ?? '<unset>'))
head "LLVM / clang (README pins 21.1.2 for wdk-sys bindgen)"
head "LLVM / clang (bindgen 0.72 builds on the runner default clang)"
Write-Host ("LIBCLANG_PATH = " + ($env:LIBCLANG_PATH ?? '<unset>'))
$clang = Get-Command clang -ErrorAction SilentlyContinue
if ($clang) { & clang --version } else { Write-Host "clang: NOT on PATH" }
@@ -119,12 +119,12 @@ jobs:
env:
# wdk-build otherwise picks 10.0.28000.0 (no km/crt) and bindgen fails — pin the WDK SDK version.
Version_Number: '10.0.26100.0'
# wdk-sys bindgen layout tests overflow (E0080) on the runner's default LLVM (ToT/22-dev); point at
# the pinned LLVM 21.1.2 that windows-drivers-rs builds clean against (provisioned to C:\llvm-21).
LIBCLANG_PATH: 'C:\llvm-21\bin'
# No LIBCLANG_PATH pin: the vendored bindgen 0.72 builds clean on the runner's default clang 22
# (the shipping pack proves it). A 0.71-era layout-test overflow once needed LLVM 21; the 0.72 bump
# retired that — see design/windows-build-and-packaging.md.
steps:
- uses: actions/checkout@v4
- name: Ensure WDK + cargo-wdk + LLVM 21.1.2 (idempotent self-provision)
- name: Ensure WDK + cargo-wdk (idempotent self-provision)
# Run the provisioning script here too so driver-build is self-sufficient and never races a
# separate provision run on the single runner. Path is relative to the job working-directory
# (packaging/windows/drivers). Near-noop once the toolchain is present.
+16
View File
@@ -56,6 +56,22 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Locale-safety gate (installer-run scripts must be ASCII)
shell: pwsh
# The installer runs these via powershell.exe (Windows PowerShell 5.1) and cmd.exe on the END
# USER's box. PS 5.1 reads a BOM-less script in the active ANSI codepage, so on a non-UTF-8 locale
# (e.g. German Windows-1252) a stray em-dash mis-decodes into a curly quote and the script aborts
# with "unterminated string" - exactly how the pf-vdisplay driver install silently failed in the
# field. Keep every installer-run script pure ASCII (matches install-gamepad-drivers.ps1).
run: |
$bad = Get-ChildItem packaging/windows/*.ps1, scripts/windows/*.ps1, scripts/windows/*.cmd -ErrorAction SilentlyContinue |
Where-Object { [IO.File]::ReadAllText($_.FullName) -match '[^\x00-\x7F]' }
if ($bad) {
$bad.FullName | ForEach-Object { Write-Output "::error::non-ASCII in installer-run script: $_" }
throw "installer-run scripts must be pure ASCII (PS 5.1 mis-parses them on non-UTF-8 locales)"
}
Write-Output "installer-run scripts are ASCII-clean"
- name: Configure + version
shell: pwsh
run: |
+1
View File
@@ -13,6 +13,7 @@ clients/apple/PunktfunkCore.xcframework/
clients/apple/.swiftpm/
# Generated App Store screenshots (tools/screenshots.sh output; uploaded as a CI artifact)
clients/apple/screenshots/
clients/linux/screenshots/
# Xcode per-user state
xcuserdata/
+41 -10
View File
@@ -2,7 +2,7 @@
Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protocol core
(`punktfunk-core`) exposed over a C ABI and native clients per platform. Full design:
[`docs/implementation-plan.md`](docs/implementation-plan.md). Status table: `README.md`.
[`design/implementation-plan.md`](design/implementation-plan.md). Status table: `README.md`.
## Where the work stands
@@ -27,7 +27,15 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
Input: mouse/keyboard (libei via RemoteDesktop portal on KWin/GNOME, gamescope's own EIS
socket, wlr protocols on Sway) and **gamepads** (uinput X-Box-360 pads + rumble
back-channel; validated live — pad created/destroyed with the session). Management REST API +
checked-in OpenAPI doc (`mgmt.rs`).
checked-in OpenAPI doc (`mgmt.rs`). **Web-console performance capture** (`stats_recorder.rs`,
design: [`design/stats-capture-plan.md`](design/stats-capture-plan.md)): the operator arms stats
recording from the web console, plays, stops, and reviews the run as graphs (per-stage latency
breakdown · fps new/repeat · goodput · loss/FEC). A shared `Arc<StatsRecorder>` ring (the hot-path
gate is a runtime `AtomicBool`, replacing the startup-only `PUNKTFUNK_PERF`) is fed by **both** the
native `virtual_stream` and the GameStream encode loop at their existing ~2 s/~1 s aggregation
boundary, and finished captures are saved as on-disk recordings
(`~/.config/punktfunk/captures/*.json`) browsable/exportable from the console's **Performance** page
(recharts). Endpoints `/api/v1/stats/*` (bearer-only). *Implemented; not yet on-glass validated.*
- **Native protocol (`punktfunk/1`): full session planes, validated live.** QUIC
control plane (`punktfunk-core` `quic` feature: Hello{mode}/Welcome{full Config}/Start), data
plane = the hardened core `Session` over raw UDP with **GF(2¹⁶) Leopard FEC + AES-GCM**
@@ -104,9 +112,16 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
captures the HDR desktop as FP16/Rgb10a2 (DDA FP16 for the secure desktop), the encoder forces HEVC
Main10 + BT.2020 PQ (NVENC ABGR10/P010; AMF/QSV P010 + a swscale Rgb10a2→P010 fallback), the client
auto-detects PQ from the HEVC VUI — gated by `PUNKTFUNK_10BIT` + client `VIDEO_CAP_10BIT`; **Windows
host only** (the Linux host stays 8-bit, blocked upstream). **AMF/QSV is CI-green but not yet
on-glass validated** (no AMD/Intel Windows box in the lab); NVENC is live-validated. Newer/less
battle-tested than the Linux host. Packaging: `packaging/windows/`.
host only** (the Linux host stays 8-bit, blocked upstream). **Vulkan-game HDR over the virtual
display**: NVIDIA/AMD Vulkan ICDs refuse to *advertise* an HDR color space for a surface on an IddCx
indirect display (so Vulkan games — Doom: The Dark Ages, id Tech, etc. — say "device does not support
HDR"), even though the ICD happily *accepts + presents* a forced HDR swapchain there. A tiny always-on
Vulkan **implicit layer** (`packaging/windows/pf-vkhdr-layer/`, `VK_LAYER_PUNKTFUNK_hdr_inject`)
injects the `HDR10_ST2084`/scRGB surface formats into `vkGetPhysicalDeviceSurfaceFormats[2]KHR`,
self-gated on the display's actual advanced-color state (no-op on SDR / real monitors); bundled +
HKLM-registered by the installer. **Live-validated: Doom: The Dark Ages enables HDR over the virtual
display.** **AMF/QSV is CI-green but not yet on-glass validated** (no AMD/Intel Windows box in the
lab); NVENC is live-validated. Newer/less battle-tested than the Linux host. Packaging: `packaging/windows/`.
## What's left
@@ -245,8 +260,8 @@ bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip
```
Generated artifacts are **checked in** and CI fails on drift: `include/punktfunk_core.h`
(cbindgen from `punktfunk-core/src/abi.rs`) and `docs/api/openapi.json` (regenerate with
`cargo run -p punktfunk-host -- openapi > docs/api/openapi.json`; spec lives in `mgmt.rs`).
(cbindgen from `punktfunk-core/src/abi.rs`) and `api/openapi.json` (regenerate with
`cargo run -p punktfunk-host -- openapi > api/openapi.json`; spec lives in `mgmt.rs`).
CI is Gitea Actions (`.gitea/workflows/`, guide: docs-site `ci.md`): `ci.yml` runs the
workspace checks inside the `git.unom.io/unom/punktfunk-rust-ci` image plus web/docs-site
@@ -268,7 +283,7 @@ crates/punktfunk-host/
zerocopy/{egl,cuda,vulkan}.rs dmabuf → CUDA → NVENC (tiled via EGL/GL, LINEAR via Vulkan)
inject/{libei,wlr,gamepad,dualsense}.rs input backends (uinput xpad + UHID DualSense)
encode/{nvenc,linux,vaapi,ffmpeg_win,sw}.rs per-GPU encoders (NVENC · Linux NVENC/CUDA · VAAPI · AMF/QSV · openh264)
capture.rs · encode.rs · audio.rs · spike.rs · punktfunk1.rs · mgmt.rs · native_pairing.rs
capture.rs · encode.rs · audio.rs · spike.rs · punktfunk1.rs · mgmt.rs · native_pairing.rs · stats_recorder.rs
clients/probe/ punktfunk/1 reference/probe client (headless test/measurement tool)
clients/linux/ native Linux client (GTK4/libadwaita · FFmpeg · PipeWire · SDL3)
clients/windows/ native Windows client (WinUI 3 via windows-reactor · D3D11 · WASAPI · SDL3)
@@ -276,7 +291,7 @@ clients/apple/ native macOS/iOS/tvOS client (Swift · VideoToolbox · GameCon
clients/android/ native Android client (Kotlin app + native/ Rust JNI core over punktfunk-core)
clients/decky/ Steam Deck Decky plugin
crates/punktfunk-host/src/{capture/dxgi,vdisplay/sudovda,encode/ffmpeg_win,inject/gamepad_windows,audio/wasapi_*,service}.rs Windows host backends
web/ TanStack web console over the mgmt API (status · devices · pairing)
web/ TanStack web console over the mgmt API (status · devices · pairing · performance graphs)
packaging/ apt(deb) · RPM/COPR · Arch/sysext · Flatpak · Bazzite bootc · Windows host installer (per-dir READMEs)
tools/{loss-harness,latency-probe}/ measurement (plan §10)
scripts/ 60-punktfunk.rules · punktfunk-host.service · host.env.example · headless/
@@ -331,7 +346,23 @@ FFI also link-needs `libGL`/`libgbm`/`libcuda` at build time). Env knobs: `PUNKT
`PUNKTFUNK_COMPOSITOR=kwin|gamescope|mutter`, `PUNKTFUNK_ZEROCOPY=1`, `PUNKTFUNK_GAMESCOPE_APP=...`,
`PUNKTFUNK_INPUT_BACKEND=...`, `PUNKTFUNK_PERF=1` (per-stage timing), `PUNKTFUNK_VIDEO_DROP=N` (FEC
test), `PUNKTFUNK_FEC_PCT=N`, `PUNKTFUNK_DSCP=1` (opt-in DSCP/SO_PRIORITY media QoS on the data +
GameStream video/audio sockets; no-op on the wire on Windows without a qWAVE policy).
GameStream video/audio sockets; no-op on the wire on Windows without a qWAVE policy),
`PUNKTFUNK_444=1` (full-chroma HEVC 4:4:4, see below).
**HEVC 4:4:4 (full chroma, Range Extensions)**: opt-in via `PUNKTFUNK_444`, negotiated like 10-bit —
the host emits 4:4:4 only when the client advertised `VIDEO_CAP_444` (wire bit `0x04` + ABI
`PUNKTFUNK_VIDEO_CAP_444`), the codec is HEVC, the session is single-process, **and** a GPU probe
(`encode::can_encode_444`, run before the Welcome) confirms support — else it resolves to 4:2:0 and
`Welcome::chroma_format` reflects the real value (honest downgrade; the client reads it via
`punktfunk_connection_chroma_format`). **punktfunk/1-native only** — GameStream/Moonlight stays 4:2:0
(stock clients can't decode 4:4:4). **NVENC is the implemented path**: Linux `hevc_nvenc` feeds a
swscale'd `yuv444p` (RGB-in is always 4:2:0 — verified on the RTX 5070 Ti — so the session forces CPU
RGB capture for 4:4:4); Windows NVENC keeps ARGB input + FREXT profile + `chromaFormatIDC=3` and the
DDA capturer delivers RGB. VAAPI / AMF / QSV **decline** (probe returns false — no validated 4:4:4
hardware in the lab; they'd produce 4:2:0). Software (openh264) is 4:2:0-only. Test with
`PUNKTFUNK_CLIENT_444=1 punktfunk-probe --out x.h265` then `ffprobe x.h265` (expect `pix_fmt yuv444p`).
*Linux NVENC mechanism validated on the RTX 5070 Ti (ffmpeg CLI); Windows NVENC + 10-bit-4:4:4 not yet
on-glass validated.*
## Conventions
Generated
+1 -1
View File
@@ -2828,6 +2828,7 @@ dependencies = [
"fec-rs",
"hmac",
"libc",
"opus",
"proptest",
"quinn",
"rand 0.9.4",
@@ -2855,7 +2856,6 @@ dependencies = [
"anyhow",
"ash",
"ashpd",
"audiopus_sys",
"axum",
"axum-server",
"base64",
+21 -9
View File
@@ -1,13 +1,20 @@
# punktfunk
<p align="center">
<img src="assets/punktfunk-logo.svg" alt="punktfunk" width="320" />
</p>
**Low-latency desktop and game streaming, Linux-first.** Run the host on a Linux machine — or a
Windows PC — with an NVIDIA GPU, connect from a Mac, PC, phone, tablet, or TV, and stream your desktop
or games — each device at its **own native resolution and refresh rate**, over your local network.
<p align="center"><b>Low-latency desktop and game streaming with first-class Linux and Windows hosts.</b></p>
Run the host on a Linux machine or a Windows PC, connect from a Mac, PC, phone, tablet, or TV, and
stream your desktop or games — each device at its **own native resolution and refresh rate**, over
your local network.
📖 **Documentation: [docs.punktfunk.unom.io](https://docs.punktfunk.unom.io)** — start with
[How It Works](https://docs.punktfunk.unom.io/docs/how-it-works) or the
[Quick Start](https://docs.punktfunk.unom.io/docs/quickstart).
💬 **Community: [Discord](https://discord.gg/kaPNvzMuGU)** — chat, support, and **Android beta
access** · **[r/Punktfunk](https://www.reddit.com/r/Punktfunk/)**.
punktfunk pairs a **virtual-display streaming host** with native clients on every platform. It speaks
the existing **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/) client works
day one — and adds its own faster **`punktfunk/1`** protocol that breaks the ~1 Gbps FEC wall with a
@@ -19,6 +26,11 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
- **Your device's exact mode.** For each client that connects, the host spins up a virtual display
sized to that device — 1080p60 to a laptop, 1440p120 to a desktop, 4K to a TV, all at once. No
letterboxing, no scaling, no rearranging your real monitors.
- **A real virtual display on Windows, too.** On Linux the host uses per-compositor virtual outputs;
on Windows you get the same on-the-fly virtual display — at the client's exact mode, no physical
monitor or dummy HDMI plug, even on the secure desktop (UAC / lock screen). It also has **its own
indirect display driver (IDD)** the host pushes finished frames straight into, rather than scraping
a screen — tight, push-based integration that's unusual for a Windows streaming host.
- **Low latency, GPU end to end.** Frames go straight from the compositor to the NVENC encoder with
zero CPU copies (dmabuf → CUDA/Vulkan → NVENC), over a transport tuned for responsiveness rather
than throughput. Stable 240 fps at 5120×1440; sub-millisecond capture-to-reassembly on a LAN.
@@ -35,7 +47,7 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
| **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened |
| **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
| **Native protocol**`punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
| **Windows host** (NVIDIA, x64) | 🟡 Implemented & shipping as a signed installer (DXGI capture · SudoVDA virtual display · NVENC · WASAPI · ViGEm); NVIDIA-only, newer than the Linux host |
| **Windows host** (x64) | 🟡 Implemented & shipping as a signed installer: DXGI/WGC capture · its own all-Rust IddCx **virtual display** (secure-desktop capable) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test |
| **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch |
| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, Oboe audio, controllers, discovery, pairing |
@@ -61,14 +73,14 @@ roadmap: **[/docs/roadmap](https://docs.punktfunk.unom.io/docs/roadmap)**.
Pick your platform and install from its package registry — the per-platform guide covers adding the
repo, first run, and the web console. The Linux host is the primary, most battle-tested path; a
Windows host (NVIDIA-only) also ships as a signed installer.
Windows host also ships as a signed installer (all-vendor: NVIDIA, AMD, Intel).
| Platform | Install | Guide |
|--------|---------|-------|
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [Ubuntu — GNOME](https://docs.punktfunk.unom.io/docs/ubuntu-gnome) · [KDE](https://docs.punktfunk.unom.io/docs/ubuntu-kde) |
| **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(or the bootc image)* | [Fedora — KDE](https://docs.punktfunk.unom.io/docs/fedora-kde) · [Bazzite](https://docs.punktfunk.unom.io/docs/bazzite) |
| **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS)* | [packaging/arch](packaging/arch/README.md) |
| **Windows** (NVIDIA, x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) |
| **Windows** (x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) |
`punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status).
After install, run `punktfunk-host serve` inside your desktop session (the secure native default;
@@ -113,7 +125,7 @@ and the [docs site](https://docs.punktfunk.unom.io).
```
crates/
punktfunk-core/ protocol · FEC · pacing · crypto · QUIC control plane — the C ABI (lib + cdylib + staticlib)
punktfunk-host/ Linux host: virtual displays · capture · encode · input · GameStream · punktfunk/1 · mgmt
punktfunk-host/ the host (Linux + Windows): virtual displays · capture · encode · input · GameStream · punktfunk/1 · mgmt
clients/
apple/ macOS / iOS / tvOS app (Swift · VideoToolbox · Metal · GameController)
linux/ Linux desktop app (Rust · GTK4/libadwaita · FFmpeg/VAAPI · PipeWire · SDL3)
@@ -124,7 +136,7 @@ clients/
web/ web console (TanStack) over the management API — status · devices · pairing
packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite bootc image
docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io
docs/ design notes & deep-dive plans
design/ design notes & deep-dive plans (index: design/README.md)
include/punktfunk_core.h cbindgen-generated C header (checked in)
tools/ latency-probe · loss-harness (measurement)
```
+528
View File
@@ -978,6 +978,309 @@
}
}
},
"/api/v1/stats/capture/live": {
"get": {
"tags": [
"stats"
],
"summary": "Live in-progress capture",
"description": "The full sample time-series of the capture currently recording, for live graphing. `404` when\nnothing is armed.",
"operationId": "statsCaptureLive",
"responses": {
"200": {
"description": "The in-progress capture (meta + samples so far)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Capture"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No capture is currently recording",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/capture/start": {
"post": {
"tags": [
"stats"
],
"summary": "Start a stats capture",
"description": "Arms a new performance-stats capture. Idempotent: if a capture is already running this returns\nthe current status unchanged. While armed, the streaming loops emit aggregated samples (~ every\n12 s) into the in-progress capture, readable live via `GET /stats/capture/live`.",
"operationId": "statsCaptureStart",
"responses": {
"200": {
"description": "Capture armed (or already running)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/StatsStatus"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/capture/status": {
"get": {
"tags": [
"stats"
],
"summary": "Stats capture status",
"description": "Whether a capture is armed, its sample count, and start time. Poll this (e.g. every 2 s) to\ndrive the capture-control UI.",
"operationId": "statsCaptureStatus",
"responses": {
"200": {
"description": "In-progress capture status (idle when not armed)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/StatsStatus"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/capture/stop": {
"post": {
"tags": [
"stats"
],
"summary": "Stop the stats capture",
"description": "Disarms the in-progress capture and writes it to disk atomically, returning its summary. If\nnothing was recording, returns `204 No Content`.",
"operationId": "statsCaptureStop",
"responses": {
"200": {
"description": "Capture stopped and saved",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CaptureMeta"
}
}
}
},
"204": {
"description": "Nothing was recording"
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not write the recording to disk",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/recordings": {
"get": {
"tags": [
"stats"
],
"summary": "List saved recordings",
"description": "Every saved capture's summary (the `meta` head only — not the sample body), newest first.",
"operationId": "statsRecordingsList",
"responses": {
"200": {
"description": "Saved capture summaries, newest first",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/CaptureMeta"
}
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/stats/recordings/{id}": {
"get": {
"tags": [
"stats"
],
"summary": "Get a saved recording",
"description": "The full capture (meta + samples) for `id`, for graphing or download.",
"operationId": "statsRecordingGet",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The recording id (its filename stem)",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "The full capture",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Capture"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No recording with that id",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "The recording file is unreadable",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
},
"delete": {
"tags": [
"stats"
],
"summary": "Delete a saved recording",
"description": "Removes the recording `id` from disk. `404` if there is no such recording.",
"operationId": "statsRecordingDelete",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The recording id (its filename stem)",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"204": {
"description": "Recording deleted"
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No recording with that id",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not delete the recording",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/status": {
"get": {
"tags": [
@@ -1125,6 +1428,89 @@
}
}
},
"Capture": {
"type": "object",
"description": "A full capture: summary + the sample time-series. The wire + on-disk shape.",
"required": [
"meta",
"samples"
],
"properties": {
"meta": {
"$ref": "#/components/schemas/CaptureMeta"
},
"samples": {
"type": "array",
"items": {
"$ref": "#/components/schemas/StatsSample"
}
}
}
},
"CaptureMeta": {
"type": "object",
"description": "Capture summary — the filename stem plus the negotiated mode/codec/client. Stored at the head\nof each on-disk recording and listed standalone (without the sample body) by\n[`StatsRecorder::list`].",
"required": [
"id",
"started_unix_ms",
"duration_ms",
"kind",
"width",
"height",
"fps",
"codec",
"client",
"sample_count"
],
"properties": {
"client": {
"type": "string",
"description": "Short label / fingerprint prefix, or `\"\"` if unknown."
},
"codec": {
"type": "string",
"description": "`\"h264\" | \"hevc\" | \"av1\"`."
},
"duration_ms": {
"type": "integer",
"format": "int64",
"minimum": 0
},
"fps": {
"type": "integer",
"format": "int32",
"minimum": 0
},
"height": {
"type": "integer",
"format": "int32",
"minimum": 0
},
"id": {
"type": "string",
"description": "e.g. `\"2026-06-26T20-14-03Z_5120x1440\"` — also the filename stem."
},
"kind": {
"type": "string",
"description": "`\"native\" | \"gamestream\"`."
},
"sample_count": {
"type": "integer",
"format": "int32",
"minimum": 0
},
"started_unix_ms": {
"type": "integer",
"format": "int64",
"minimum": 0
},
"width": {
"type": "integer",
"format": "int32",
"minimum": 0
}
}
},
"CustomEntry": {
"type": "object",
"description": "A user-added title, persisted in `~/.config/punktfunk/library.json`. Same shape the API\nreturns and the web console edits.",
@@ -1595,6 +1981,144 @@
}
}
},
"StageTiming": {
"type": "object",
"description": "One pipeline stage's latency in an aggregation window (microseconds).",
"required": [
"name",
"p50_us",
"p99_us"
],
"properties": {
"name": {
"type": "string",
"description": "`\"capture\" | \"submit\" | \"encode\" | \"packetize\" | \"send\"` (path-dependent)."
},
"p50_us": {
"type": "number",
"format": "float"
},
"p99_us": {
"type": "number",
"format": "float"
}
}
},
"StatsSample": {
"type": "object",
"description": "One aggregated sample (~ every 2 s native, ~ every 1 s GameStream).",
"required": [
"t_ms",
"session_id",
"stages",
"fps",
"repeat_fps",
"mbps",
"bitrate_kbps",
"frames_dropped",
"packets_dropped",
"send_dropped",
"fec_recovered"
],
"properties": {
"bitrate_kbps": {
"type": "integer",
"format": "int32",
"description": "Configured target bitrate.",
"minimum": 0
},
"fec_recovered": {
"type": "integer",
"format": "int32",
"description": "FEC shards recovered this window (delta).",
"minimum": 0
},
"fps": {
"type": "number",
"format": "float",
"description": "Genuine NEW frames/s from the source."
},
"frames_dropped": {
"type": "integer",
"format": "int32",
"description": "Frames dropped this window (delta).",
"minimum": 0
},
"mbps": {
"type": "number",
"format": "float",
"description": "Transmit goodput (Mb/s)."
},
"packets_dropped": {
"type": "integer",
"format": "int32",
"description": "Packets dropped this window (receiver-side / reassembler, where known).",
"minimum": 0
},
"repeat_fps": {
"type": "number",
"format": "float",
"description": "Re-encoded holds/s (source-starvation indicator)."
},
"send_dropped": {
"type": "integer",
"format": "int32",
"description": "Host send-buffer overflow / EAGAIN this window (delta).",
"minimum": 0
},
"session_id": {
"type": "integer",
"format": "int32",
"description": "Disambiguates concurrent sessions (usually constant).",
"minimum": 0
},
"stages": {
"type": "array",
"items": {
"$ref": "#/components/schemas/StageTiming"
},
"description": "Ordered pipeline stages for this path."
},
"t_ms": {
"type": "integer",
"format": "int64",
"description": "Milliseconds since capture start (monotonic; stamped by [`StatsRecorder::push_sample`]).",
"minimum": 0
}
}
},
"StatsStatus": {
"type": "object",
"description": "Snapshot of the in-progress capture for the management API.",
"required": [
"armed",
"sample_count",
"started_unix_ms",
"kind"
],
"properties": {
"armed": {
"type": "boolean",
"description": "Capture currently running."
},
"kind": {
"type": "string",
"description": "Path of the in-progress capture (`\"\"` if idle)."
},
"sample_count": {
"type": "integer",
"format": "int32",
"description": "Samples in the in-progress capture.",
"minimum": 0
},
"started_unix_ms": {
"type": "integer",
"format": "int64",
"description": "Unix start time of the in-progress capture (`0` if idle).",
"minimum": 0
}
}
},
"StreamInfo": {
"type": "object",
"description": "RTSP-negotiated stream parameters.",
@@ -1696,6 +2220,10 @@
{
"name": "library",
"description": "Game library: installed-store titles (Steam) plus user-curated custom entries"
},
{
"name": "stats",
"description": "Streaming performance-stats capture: arm/stop a recording, read the live + saved time-series for graphing"
}
]
}
+33
View File
@@ -0,0 +1,33 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg width="100%" height="100%" viewBox="0 0 579 298" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:2;">
<style>
/* Theme-adaptive so the logo stays readable on both light and dark README
backgrounds: deep violet (the brand-mark palette) on light, the original
light violet on dark. Evaluated by the viewer's color scheme. */
.pf-wm { fill: #6c5bf3; }
.pf-back { fill: #a79ff8; }
.pf-deep { fill: #6c5bf3; }
@media (prefers-color-scheme: dark) {
.pf-wm { fill: #cec9fb; }
.pf-back { fill: #f2f1fe; }
.pf-deep { fill: #8c7ef5; }
}
</style>
<g>
<g>
<path class="pf-wm" style="fill-rule:nonzero;" d="M21.144,176.635l0,102.687l31.253,0l0,-35.563l73.436,0l0,-23.555l-73.436,0l0,-19.398l77.285,0l0,-24.171l-108.537,0Z"/>
<path class="pf-wm" style="fill-rule:nonzero;" d="M136.148,176.635l0,47.264c0.154,16.627 0.154,16.627 0.308,20.014c0.77,15.087 2.463,21.4 7.544,26.634c7.698,8.16 20.014,10.315 59.272,10.315c23.863,0 34.178,-0.616 43.415,-2.463c11.7,-2.463 19.552,-10.623 21.246,-22.323c0.924,-7.236 1.078,-8.929 1.54,-32.176l0,-47.264l-31.253,0l0,47.264c0,2.155 -0.154,7.082 -0.308,10.623c-0.462,9.699 -1.232,12.47 -3.695,15.087c-3.387,3.695 -9.853,4.619 -31.407,4.619c-26.634,0 -32.638,-1.693 -34.332,-9.853c-0.77,-4.157 -0.77,-4.311 -1.078,-20.476l0,-47.264l-31.253,0Z"/>
<path class="pf-wm" style="fill-rule:nonzero;" d="M275.938,176.527l0,102.687l31.868,0l-0.77,-76.669l3.387,0l54.038,76.669l54.346,0l0,-102.687l-31.868,0l0.77,76.515l-3.233,0l-53.73,-76.515l-54.808,0Z"/>
<path class="pf-wm" style="fill-rule:nonzero;" d="M425.273,176.527l0,102.687l31.253,0l0,-39.258l17.089,0l46.032,39.258l47.418,0l-64.353,-52.344l59.426,-50.959l-47.88,0l-40.644,37.873l-17.089,0l0,-37.257l-31.253,0Z"/>
</g>
<path class="pf-back" style="fill-rule:nonzero;" d="M65.442,150.143c24.514,0 44.298,-19.784 44.298,-44.298c0,-24.514 -19.784,-44.298 -44.298,-44.298c-24.514,0 -44.298,19.784 -44.298,44.298c0,24.514 19.784,44.298 44.298,44.298Z"/>
<path class="pf-deep" style="fill-rule:nonzero;" d="M141.063,92.871c17.334,-17.334 17.334,-45.312 0,-62.647c-17.334,-17.334 -45.312,-17.334 -62.647,-0c-17.334,17.334 -17.334,45.312 0,62.647c17.334,17.334 45.312,17.334 62.647,-0Z"/>
<path style="fill:url(#_Linear1);" d="M121.228,104.359c-14.777,3.965 -31.187,0.136 -42.811,-11.488c-11.624,-11.624 -15.453,-28.034 -11.488,-42.811c14.777,-3.965 31.187,-0.136 42.811,11.488c11.624,11.624 15.453,28.034 11.488,42.811Z"/>
</g>
<defs>
<linearGradient id="_Linear1" x1="0" y1="0" x2="1" y2="0" gradientUnits="userSpaceOnUse" gradientTransform="matrix(31.323323,-31.323323,31.323323,31.323323,78.416832,92.870811)">
<stop offset="0" style="stop-color:#cec9fb;stop-opacity:0"/>
<stop offset="1" style="stop-color:#fcfcff;stop-opacity:1"/>
</linearGradient>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 3.0 KiB

+21
View File
@@ -62,6 +62,10 @@ android {
buildFeatures { compose = true }
// Roborazzi/Robolectric render Compose on the host JVM (the CI screenshot harness) and need the
// merged Android resources + the app's manifest/theme available to the unit tests.
testOptions { unitTests { isIncludeAndroidResources = true } }
compileOptions {
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
@@ -99,4 +103,21 @@ dependencies {
// Android TV components (we target phone + TV) land in the TV-UI milestone:
// implementation("androidx.tv:tv-material:1.1.0")
// The manifest already declares leanback so the scaffold installs on TV.
// --- CI screenshot harness (Roborazzi on the JVM via Robolectric — no emulator/GPU). The
// screenshot tests render the real Compose UI with mock state; never load the JNI core, so the
// job runs `:app:testDebugUnitTest -PskipRustBuild` (see kit/build.gradle.kts). ---
testImplementation(composeBom)
testImplementation("androidx.compose.ui:ui-test-junit4")
debugImplementation("androidx.compose.ui:ui-test-manifest") // the ComponentActivity test host
testImplementation("junit:junit:4.13.2")
testImplementation("org.robolectric:robolectric:4.16.1")
testImplementation("io.github.takahirom.roborazzi:roborazzi:1.64.0")
testImplementation("io.github.takahirom.roborazzi:roborazzi-compose:1.64.0")
}
// Record (write) the screenshots when the unit tests run. These tests exist to GENERATE marketing
// images, not to diff goldens, so always capture rather than verify.
tasks.withType<Test>().configureEach {
systemProperty("roborazzi.test.record", "true")
}
@@ -163,7 +163,7 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
targetHost, targetPort, w, h, hz,
id.certPem, id.privateKeyPem, pinHex ?: "",
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled,
hdrEnabled, settings.audioChannels,
)
}
connecting = false
@@ -16,9 +16,18 @@ data class Settings(
val bitrateKbps: Int = 0,
val compositor: Int = 0,
val gamepad: Int = 0,
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
* can capture; the resolved count drives the decoder + AAudio layout. */
val audioChannels: Int = 2,
val micEnabled: Boolean = false,
/** Show the live stats overlay (FPS / throughput / latency) during a stream. */
val statsHudEnabled: Boolean = true,
/**
* Touch input model. `true` (default) = trackpad: the cursor stays put on touch-down and moves
* by the finger's relative delta (swipe to nudge, lift and re-swipe to walk it across), tap to
* click where it is. `false` = direct pointing: the cursor jumps to the finger (the old behaviour).
*/
val trackpadMode: Boolean = true,
)
/** Loads/saves [Settings] in the app-private `punktfunk_settings` prefs. */
@@ -33,8 +42,10 @@ class SettingsStore(context: Context) {
bitrateKbps = prefs.getInt(K_BITRATE, 0),
compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0),
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
micEnabled = prefs.getBoolean(K_MIC, false),
statsHudEnabled = prefs.getBoolean(K_HUD, true),
trackpadMode = prefs.getBoolean(K_TRACKPAD, true),
)
fun save(s: Settings) {
@@ -45,8 +56,10 @@ class SettingsStore(context: Context) {
.putInt(K_BITRATE, s.bitrateKbps)
.putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad)
.putInt(K_AUDIO_CH, s.audioChannels)
.putBoolean(K_MIC, s.micEnabled)
.putBoolean(K_HUD, s.statsHudEnabled)
.putBoolean(K_TRACKPAD, s.trackpadMode)
.apply()
}
@@ -57,8 +70,10 @@ class SettingsStore(context: Context) {
const val K_BITRATE = "bitrate_kbps"
const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad"
const val K_AUDIO_CH = "audio_channels"
const val K_MIC = "mic_enabled"
const val K_HUD = "stats_hud_enabled"
const val K_TRACKPAD = "trackpad_mode"
}
}
@@ -124,6 +139,13 @@ val REFRESH_OPTIONS = listOf(
240 to "240 Hz",
)
/** (channel count, label). 2 = stereo (default), 6 = 5.1, 8 = 7.1. */
val AUDIO_CHANNEL_OPTIONS = listOf(
2 to "Stereo",
6 to "5.1 Surround",
8 to "7.1 Surround",
)
/** (kbps, label). `0` = host default. */
val BITRATE_OPTIONS = listOf(
0 to "Automatic",
@@ -104,6 +104,12 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
}
SettingsGroup("Audio") {
SettingDropdown(
label = "Audio channels",
options = AUDIO_CHANNEL_OPTIONS,
selected = s.audioChannels,
) { ch -> update(s.copy(audioChannels = ch)) }
ToggleRow(
title = "Microphone",
subtitle = "Send your mic to the host's virtual microphone",
@@ -119,6 +125,16 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
)
}
SettingsGroup("Pointer") {
ToggleRow(
title = "Trackpad mode",
subtitle = "Relative cursor like a laptop touchpad — swipe to nudge, tap to click. " +
"Off = the cursor jumps to your finger.",
checked = s.trackpadMode,
onCheckedChange = { on -> update(s.copy(trackpadMode = on)) },
)
}
SettingsGroup("Overlay") {
ToggleRow(
title = "Stats overlay",
@@ -41,6 +41,7 @@ import io.unom.punktfunk.kit.NativeBridge
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.delay
import kotlin.math.abs
import kotlin.math.hypot
import kotlin.math.roundToInt
// Touch-gesture tuning (px / ms). TAP_SLOP: movement under this still counts as a tap, not a drag.
@@ -50,6 +51,15 @@ private const val TAP_SLOP = 12f
private const val TAP_DRAG_MS = 250L
private const val SCROLL_DIV = 4f
// Trackpad-mode pointer ballistics (relative one-finger motion). POINTER_SENS: base finger-px →
// host-px gain (~1:1, never twitchy). The rest is mild acceleration so a flick crosses the screen
// while a slow drag stays precise: above ACCEL_SPEED_FLOOR px/ms the gain ramps by ACCEL_GAIN per
// px/ms, capped at ACCEL_MAX (so a fast swipe can't fling the cursor uncontrollably).
private const val POINTER_SENS = 1.3f
private const val ACCEL_GAIN = 0.6f
private const val ACCEL_SPEED_FLOOR = 0.3f
private const val ACCEL_MAX = 3.0f
@Composable
fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val context = LocalContext.current
@@ -68,8 +78,11 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Live decode stats for the HUD. Poll once a second for the whole stream (cheap, and each call
// drains+resets the native window so it never grows unbounded even while the overlay is hidden);
// `showStats` only gates rendering. A 3-finger tap toggles it live; the default comes from Settings.
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var showStats by remember { mutableStateOf(SettingsStore(context).load().statsHudEnabled) }
var showStats by remember { mutableStateOf(initialSettings.statsHudEnabled) }
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
val trackpad = initialSettings.trackpadMode
LaunchedEffect(handle) {
while (true) {
delay(1000)
@@ -145,13 +158,18 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (showStats) {
stats?.let { StatsOverlay(it, Modifier.align(Alignment.TopStart).padding(12.dp)) }
}
// Touch → mouse, absolute "direct pointing" like the Apple client: the host cursor follows
// your finger (MouseMoveAbs, host-normalized against the overlay size — which fills the video,
// so finger position maps straight onto the remote screen). Gestures: tap = left click;
// two-finger tap = right click; two-finger drag = scroll; tap-then-press-and-drag = left-drag
// (text selection / moving windows); three-finger tap = toggle the stats HUD.
// Touch → mouse. Two models, chosen by the Trackpad-mode setting:
// • trackpad (default): the cursor STAYS where it is on touch-down and moves by the finger's
// relative delta (MouseMove) with mild pointer acceleration — swipe to nudge, lift and
// re-swipe to walk it across, tap to click where it is. This is what makes the cursor
// reachable on a small screen.
// • direct (opt-out): the cursor jumps to the finger and follows it (MouseMoveAbs,
// host-normalized against the overlay size), the old "direct pointing" behaviour.
// Both share the same gesture vocabulary: tap = left click; two-finger tap = right click;
// two-finger drag = scroll; tap-then-press-and-drag = left-drag (text selection / moving
// windows); three-finger tap = toggle the stats HUD.
Box(
Modifier.fillMaxSize().pointerInput(handle) {
Modifier.fillMaxSize().pointerInput(handle, trackpad) {
var lastTapUp = 0L
var lastTapX = 0f
var lastTapY = 0f
@@ -176,7 +194,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val isDrag = down.uptimeMillis - lastTapUp < TAP_DRAG_MS &&
abs(startX - lastTapX) < TAP_SLOP && abs(startY - lastTapY) < TAP_SLOP
lastTapUp = 0L // consume the arming either way
moveAbs(startX, startY) // cursor jumps to the finger immediately
// Direct mode jumps the cursor to the finger; trackpad mode leaves it put (the
// whole point — you nudge it with swipes instead).
if (!trackpad) moveAbs(startX, startY)
if (isDrag) NativeBridge.nativeSendPointerButton(handle, 1, true)
var moved = false
@@ -185,6 +205,14 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
var prevCx = startX
var prevCy = startY
var upTime = down.uptimeMillis
// Trackpad relative-motion state: the tracked finger, its last position/time, and
// the sub-pixel remainder so a slow drag isn't lost to Int truncation.
var trackId = down.id
var prevX = startX
var prevY = startY
var prevT = down.uptimeMillis
var accX = 0f
var accY = 0f
while (true) {
val ev = awaitPointerEvent()
@@ -217,15 +245,46 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
moved = true
}
} else if (!scrolling) {
// One finger → the cursor follows it (skipped once a gesture turned into
// a scroll, so dropping back to one finger doesn't jerk the cursor).
// One finger (skipped once a gesture turned into a scroll, so dropping
// back to one finger doesn't jerk the cursor).
val p = pressed.firstOrNull { it.id == down.id } ?: pressed.first()
if (abs(p.position.x - startX) > TAP_SLOP ||
abs(p.position.y - startY) > TAP_SLOP
) {
moved = true
}
moveAbs(p.position.x, p.position.y)
if (trackpad) {
// Relative: move by the finger delta × (sensitivity × acceleration),
// carrying the sub-pixel remainder. Re-anchor (zero delta this frame)
// if the tracked finger changed, so lifting one of several fingers
// never jumps the cursor.
if (p.id != trackId) {
trackId = p.id
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
}
val dx = p.position.x - prevX
val dy = p.position.y - prevY
val dt = (p.uptimeMillis - prevT).coerceAtLeast(1L)
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
val speed = hypot(dx, dy) / dt // finger px per ms
val accel = (1f + ACCEL_GAIN * (speed - ACCEL_SPEED_FLOOR).coerceAtLeast(0f))
.coerceAtMost(ACCEL_MAX)
accX += dx * POINTER_SENS * accel
accY += dy * POINTER_SENS * accel
val outX = accX.toInt() // truncates toward zero → remainder kept w/ sign
val outY = accY.toInt()
if (outX != 0 || outY != 0) {
NativeBridge.nativeSendPointerMove(handle, outX, outY)
accX -= outX
accY -= outY
}
} else {
moveAbs(p.position.x, p.position.y) // direct: cursor follows the finger
}
}
ev.changes.forEach { it.consume() }
}
@@ -239,7 +298,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
NativeBridge.nativeSendPointerButton(handle, 3, true)
NativeBridge.nativeSendPointerButton(handle, 3, false)
}
else -> { // tap → left click, and arm tap-and-drag
else -> { // tap → left click (at the cursor's current spot), arm tap-drag
NativeBridge.nativeSendPointerButton(handle, 1, true)
NativeBridge.nativeSendPointerButton(handle, 1, false)
lastTapUp = upTime
@@ -260,7 +319,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped]`.
*/
@Composable
private fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
if (s.size < 10) return
val w = s[6].toInt()
val h = s[7].toInt()
@@ -10,7 +10,9 @@ import androidx.compose.ui.platform.LocalContext
// punktfunk brand violets (from the app icon: #6C5BF3 / #A79FF8 / #D2C9FB on a #16132A indigo).
// Used as the fallback dark scheme on pre-Android-12 devices; on 12+ we defer to Material You.
private val BrandDark = darkColorScheme(
// `internal` (not private) so the CI screenshot tests can force the deterministic brand palette —
// Material You dynamic colour has no wallpaper to seed from under the Robolectric JVM renderer.
internal val BrandDark = darkColorScheme(
primary = Color(0xFFA79FF8),
onPrimary = Color(0xFF1B1442),
primaryContainer = Color(0xFF4C3FB3),
@@ -0,0 +1,74 @@
package io.unom.punktfunk.screenshots
import androidx.activity.ComponentActivity
import androidx.compose.ui.test.junit4.createAndroidComposeRule
import androidx.compose.ui.test.onRoot
import com.github.takahirom.roborazzi.captureRoboImage
import com.github.takahirom.roborazzi.captureScreenRoboImage
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
/**
* App-store / marketing screenshots of the native Android client, rendered on the JVM by Roborazzi
* (Robolectric Native Graphics) — no emulator, GPU, host, or JNI core. The scenes (ShotScenes.kt)
* render the REAL Compose UI with mock state.
*
* `sdk = [36]` is mandatory: Robolectric ships android-all jars only up to API 36 (Android 16), and
* the app's compileSdk is 37. PNGs land in build/outputs/roborazzi/.
*/
@RunWith(RobolectricTestRunner::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(sdk = [36], qualifiers = "w360dp-h800dp-xxhdpi")
class ScreenshotTest {
@get:Rule
val compose = createAndroidComposeRule<ComponentActivity>()
private val out = "build/outputs/roborazzi"
// Pausing the animation clock before composing (then advancing once past the entrance animation
// and freezing) is what makes a text-field-bearing scene capturable: a focused field blinks its
// cursor via an infinite animation that otherwise keeps Compose perpetually "busy", so
// setContent's wait-for-idle never returns. Frozen, the capture is also deterministic.
/** Full-screen content scenes: the compose root fills the device, so a root capture is the shot. */
private fun shootRoot(name: String, content: @androidx.compose.runtime.Composable () -> Unit) {
compose.mainClock.autoAdvance = false
compose.setContent { ShotTheme(content) }
compose.mainClock.advanceTimeBy(800)
compose.onRoot().captureRoboImage("$out/phone-$name.png")
}
/** Dialog scenes: the AlertDialog is a separate window, so capture the whole screen (all windows). */
private fun shootScreen(name: String, content: @androidx.compose.runtime.Composable () -> Unit) {
compose.mainClock.autoAdvance = false
compose.setContent { ShotTheme(content) }
compose.mainClock.advanceTimeBy(800)
captureScreenRoboImage("$out/phone-$name.png")
}
@Test
fun hosts() = shootRoot("hosts") { HostsScene() }
@Test
fun settings() = shootRoot("settings") { SettingsScene() }
@Test
@Config(sdk = [36], qualifiers = "w800dp-h360dp-xxhdpi") // landscape — the stream is immersive
fun stream() = shootRoot("stream") { StreamScene() }
@Test
fun trust() = shootScreen("trust") {
HostsScene()
TrustDialog()
}
@Test
fun pair() = shootScreen("pair") {
HostsScene()
PairDialog()
}
}
@@ -0,0 +1,195 @@
package io.unom.punktfunk.screenshots
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.grid.GridCells
import androidx.compose.foundation.lazy.grid.GridItemSpan
import androidx.compose.foundation.lazy.grid.LazyVerticalGrid
import androidx.compose.foundation.lazy.grid.items
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import io.unom.punktfunk.BrandDark
import io.unom.punktfunk.Settings
import io.unom.punktfunk.SettingsScreen
import io.unom.punktfunk.StatsOverlay
import io.unom.punktfunk.components.HostCard
import io.unom.punktfunk.components.SectionLabel
import io.unom.punktfunk.models.HostStatus
// The CI screenshot scenes: the REAL app composables, fed embedded mock state, under the forced
// brand palette (Material You has no wallpaper to seed from on the JVM). The stream-video surface
// and ConnectScreen/App are intentionally absent — they require the live JNI core / a session.
/** Forces the deterministic punktfunk brand scheme (see Theme.kt) instead of dynamic colour. */
@Composable
internal fun ShotTheme(content: @Composable () -> Unit) {
MaterialTheme(colorScheme = BrandDark, content = content)
}
private data class MockHost(val name: String, val address: String, val status: HostStatus)
private val SAVED = listOf(
MockHost("Living Room PC", "192.168.1.42:9777", HostStatus.PAIRED),
MockHost("Office", "192.168.1.50:9777", HostStatus.TOFU),
)
private val DISCOVERED = listOf(
MockHost("studio-deck", "192.168.1.61:9777", HostStatus.PAIRING),
MockHost("HTPC", "192.168.1.70:9777", HostStatus.TOFU),
)
/** The connect screen's host grid, reconstructed from the real HostCard/SectionLabel components. */
@Composable
internal fun HostsScene() {
Surface(Modifier.fillMaxSize(), color = MaterialTheme.colorScheme.background) {
LazyVerticalGrid(
columns = GridCells.Adaptive(minSize = 160.dp),
modifier = Modifier.fillMaxSize(),
contentPadding = androidx.compose.foundation.layout.PaddingValues(16.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
item(span = { GridItemSpan(maxLineSpan) }) {
Column(
horizontalAlignment = Alignment.CenterHorizontally,
modifier = Modifier.fillMaxWidth(),
) {
Spacer(Modifier.height(8.dp))
Text("Punktfunk", style = MaterialTheme.typography.headlineLarge)
Text(
"stream a remote desktop",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.height(24.dp))
}
}
item(span = { GridItemSpan(maxLineSpan) }) { SectionLabel("Saved hosts") }
items(SAVED) { h ->
HostCard(h.name, h.address, h.status, enabled = true, onConnect = {}, onForget = {}, onRename = {})
}
item(span = { GridItemSpan(maxLineSpan) }) {
Spacer(Modifier.height(12.dp))
SectionLabel("Discovered on the network")
}
items(DISCOVERED) { h ->
HostCard(h.name, h.address, h.status, enabled = true, onConnect = {}, onForget = null)
}
}
}
}
/** The real SettingsScreen, fed a representative non-default Settings. */
@Composable
internal fun SettingsScene() {
Surface(Modifier.fillMaxSize(), color = MaterialTheme.colorScheme.background) {
SettingsScreen(
initial = Settings(
width = 1920,
height = 1080,
hz = 120,
bitrateKbps = 50_000,
compositor = 1,
gamepad = 2,
micEnabled = true,
statsHudEnabled = true,
trackpadMode = true,
),
onChange = {},
onBack = {},
)
}
}
/** The real TOFU AlertDialog (mirrors ConnectScreen's PendingTrust.Kind.TRUST_NEW), shown over the host grid. */
@Composable
internal fun TrustDialog() {
AlertDialog(
onDismissRequest = {},
title = { Text("Trust this host?") },
text = {
Column {
Text("First connection to 192.168.1.61:9777.")
Text("Fingerprint 9f8e7d6c5b4a3928…")
Text(
"This host allows trust-on-first-use, but that can't tell an impostor " +
"from the real host. Pairing with a PIN is stronger — it proves both sides.",
)
}
},
confirmButton = { TextButton({}) { Text("Trust (TOFU)") } },
dismissButton = { TextButton({}) { Text("Pair with PIN…") } },
)
}
/** The PIN-pairing AlertDialog (mirrors ConnectScreen's PendingTrust.Kind.PAIR). The live screen
* uses OutlinedTextFields, but a TextField inside a Dialog window never reaches idle under
* Robolectric (its focus/cursor machinery animates forever) — so the PIN is shown as a static
* display here, which also reads better in a marketing shot. */
@Composable
internal fun PairDialog() {
AlertDialog(
onDismissRequest = {},
title = { Text("Pair with PIN") },
text = {
Column {
Text("Enter the 4-digit PIN shown on the host.")
Spacer(Modifier.height(16.dp))
Surface(
color = MaterialTheme.colorScheme.surfaceVariant,
shape = MaterialTheme.shapes.medium,
modifier = Modifier.fillMaxWidth(),
) {
Text(
"4 8 2 7",
style = MaterialTheme.typography.headlineMedium,
textAlign = TextAlign.Center,
modifier = Modifier.fillMaxWidth().padding(vertical = 16.dp),
)
}
Spacer(Modifier.height(12.dp))
Text(
"This device: Pixel 9 Pro",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
},
confirmButton = { TextButton({}) { Text("Pair") } },
dismissButton = { TextButton({}) { Text("Cancel") } },
)
}
/** The live stats HUD (the real StatsOverlay) over a synthetic "streamed frame" gradient. */
@Composable
internal fun StreamScene() {
Box(
Modifier
.fillMaxSize()
.background(
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
),
) {
// [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped]
StatsOverlay(
doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0),
Modifier.align(Alignment.TopStart).padding(12.dp),
)
}
}
+8 -2
View File
@@ -99,6 +99,12 @@ val cargoNdkDebug = registerCargoNdk("cargoNdkDebug", release = false)
val cargoNdkRelease = registerCargoNdk("cargoNdkRelease", release = true)
afterEvaluate {
tasks.named("preDebugBuild").configure { dependsOn(cargoNdkDebug) }
tasks.named("preReleaseBuild").configure { dependsOn(cargoNdkRelease) }
// `-PskipRustBuild` skips the cargo-ndk native build — for JVM-only tasks (the Roborazzi
// screenshot unit tests render Compose on the JVM and never load libpunktfunk_android.so), so
// CI/local screenshot runs don't need the Rust toolchain or NDK. The native build stays wired
// for every normal APK/AAR build.
if (!project.hasProperty("skipRustBuild")) {
tasks.named("preDebugBuild").configure { dependsOn(cargoNdkDebug) }
tasks.named("preReleaseBuild").configure { dependsOn(cargoNdkRelease) }
}
}
@@ -45,6 +45,7 @@ object NativeBridge {
compositorPref: Int,
gamepadPref: Int,
hdrEnabled: Boolean,
audioChannels: Int,
): Long
/** 64-hex SHA-256 of the cert the host presented on [handle]; valid after a successful connect. */
+189 -27
View File
@@ -1,8 +1,21 @@
//! Android audio playback (android-only): pull Opus packets from the connector, decode to
//! interleaved f32 stereo, and feed AAudio (LowLatency) via its realtime data callback through a
//! jitter ring. Mirrors [`crate::decode`]: one thread we own (the Opus decode producer) plus a
//! shutdown flag; the realtime callback thread is owned by AAudio. Ring logic ported from
//! `punktfunk-client-linux/src/audio.rs` (prime ~3 quanta, drop-oldest cap, re-prime on drain).
//! interleaved f32 (stereo or 5.1/7.1 surround), and feed AAudio (LowLatency) via its realtime data
//! callback through a jitter ring. Mirrors [`crate::decode`]: one thread we own (the Opus decode
//! producer) plus a shutdown flag; the realtime callback thread is owned by AAudio.
//!
//! The layout is the host-RESOLVED channel count (`NativeClient::audio_channels`, negotiated at
//! connect), so an older/clamping host that can only capture stereo is decoded + played as stereo.
//! 2 = stereo / 6 = 5.1 / 8 = 7.1, in the canonical wire order FL FR FC LFE RL RR SL SR.
//!
//! The ring started as a port of `punktfunk-client-linux/src/audio.rs`, but AAudio — unlike
//! PipeWire, which adaptively rate-matches the stream and absorbs a shallow buffer — hands us a raw
//! realtime callback and makes us own the buffer. So this client diverges deliberately to stop the
//! Android-only crackle: (1) the callback is allocation/free-free — decoded buffers are recycled to
//! the producer via a free-list instead of being freed on the audio thread (Android's Scudo `free`
//! has unbounded tail latency); (2) the jitter ring is deeper (~40 ms prime / ~150 ms hard cap) and
//! decoupled from the tiny LowLatency burst size, with de-prime hysteresis so a transient drain
//! doesn't manufacture a silence; (3) the AAudio HW buffer is primed above its 2-burst default and
//! grown on XRuns (Google's anti-glitch technique).
use ndk::audio::{
AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode,
@@ -13,16 +26,75 @@ use punktfunk_core::error::PunktfunkError;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::mpsc::{sync_channel, SyncSender, TrySendError};
use std::sync::mpsc::{sync_channel, Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
const CHANNELS: usize = 2;
const SAMPLE_RATE: i32 = 48_000;
/// Decoded-chunk hand-off depth: 64 × 5 ms = 320 ms slack (matches the core's AUDIO_QUEUE).
const RING_CHUNKS: usize = 64;
/// Opus decode scratch: worst-case 120 ms stereo frame (5760 samples/ch × 2 ch).
const PCM_SCRATCH: usize = 5760 * CHANNELS;
// --- Jitter-ring depths, in MILLISECONDS (scaled to interleaved-f32 samples at runtime). --------
// The channel count is negotiated, not a compile-time const, so these are kept in ms and multiplied
// by `ms` (interleaved-f32 samples per millisecond at the resolved layout) inside `start`.
// Unlike the Linux client (PipeWire adaptively rate-matches the stream to the graph clock, masking
// host↔DAC drift + a shallow ring), AAudio hands us a raw callback and we own the buffer: drift and
// WiFi power-save bunching land as underruns/overflows = crackle. So Android runs a deliberately
// deeper, smoothly-managed ring than Linux — keep the two clients' depths intentionally divergent.
/// Prime/target floor: fill to ~40 ms before playing (and after a sustained drain). Deep enough to
/// ride out WiFi arrival jitter + clock drift; the dominant Android-only anti-crackle lever.
const PRIME_FLOOR_MS: usize = 40;
/// Ceiling for the burst-scaled target (so a large quantum can't push the prime depth too high).
const PRIME_CEIL_MS: usize = 80;
/// Drop-oldest headroom above the target before trimming — a ~80 ms band swallows an arrival burst
/// without overflowing.
const JITTER_HEADROOM_MS: usize = 80;
/// Hard latency bound: never let the ring exceed ~150 ms (the only thing that caps added latency).
const HARD_CAP_MS: usize = 150;
/// Re-prime (go silent to refill) only after this many CONSECUTIVE empty callbacks, so one transient
/// drain doesn't manufacture a fresh 40 ms silence (the old `if ring.is_empty()` re-primed instantly).
const DEPRIME_AFTER_CALLBACKS: u32 = 5;
/// Throttle the AAudio XRun-driven HW-buffer grow check (cheap, but no need to poll every quantum).
const XRUN_CHECK_EVERY: u32 = 128;
/// Opus decoder for the audio plane: a plain stereo decoder (the validated path) or a multistream
/// decoder for 5.1/7.1, both behind one `decode_float`. Built from the host-RESOLVED channel count
/// via the shared layout table. Mirrors the Linux client's `AudioDec`.
enum AudioDec {
Stereo(opus::Decoder),
Surround(opus::MSDecoder),
}
impl AudioDec {
fn new(channels: u8) -> Result<AudioDec, opus::Error> {
if channels == 2 {
Ok(AudioDec::Stereo(opus::Decoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
)?))
} else {
let l = punktfunk_core::audio::layout_for(channels, false);
Ok(AudioDec::Surround(opus::MSDecoder::new(
SAMPLE_RATE as u32,
l.streams,
l.coupled,
l.mapping,
)?))
}
}
fn decode_float(
&mut self,
input: &[u8],
out: &mut [f32],
fec: bool,
) -> Result<usize, opus::Error> {
match self {
AudioDec::Stereo(d) => d.decode_float(input, out, fec),
AudioDec::Surround(d) => d.decode_float(input, out, fec),
}
}
}
/// Diagnostics — written by the decode thread + the realtime callback, logged periodically. The
/// audio analogue of the video `fed`/`rendered` counters (we can't "screenshot" sound).
@@ -42,27 +114,57 @@ pub struct AudioPlayback {
}
impl AudioPlayback {
/// Open AAudio (LowLatency, 48 kHz/stereo/f32) with a realtime callback draining a jitter ring,
/// then spawn the Opus decode thread. `None` on failure (the caller leaves video streaming).
/// Open AAudio (LowLatency, 48 kHz/f32, the host-resolved channel layout) with a realtime
/// callback draining a jitter ring, then spawn the Opus decode thread. `None` on failure (the
/// caller leaves video streaming).
pub fn start(client: Arc<NativeClient>) -> Option<AudioPlayback> {
// Build playback from the host-RESOLVED channel count (never the request): 2 = stereo /
// 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = punktfunk_core::audio::normalize_channels(client.audio_channels) as usize;
// Interleaved f32 samples per millisecond at this layout (48 kHz × channels); the ms-
// denominated jitter-ring depths scale by it.
let ms = (SAMPLE_RATE as usize / 1000) * channels;
let prime_floor = PRIME_FLOOR_MS * ms;
let prime_ceil = PRIME_CEIL_MS * ms;
let jitter_headroom = JITTER_HEADROOM_MS * ms;
let hard_cap_max = HARD_CAP_MS * ms;
let counters = Arc::new(Counters::default());
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list: drained PCM buffers go BACK to the decode thread to be refilled, so the
// realtime callback never frees heap (Android's Scudo allocator has unbounded free() tail
// latency — a free on the audio thread is an XRun = a click) and the decode thread rarely
// allocates. Same depth as the data channel.
let (free_tx, free_rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Realtime consumer state, owned by the callback (FnMut) — no lock: AAudio calls it from a
// single high-priority thread, and the decode thread only touches `tx`.
// single high-priority thread, and the decode thread only touches `tx`/`free_rx`.
let cb_counters = counters.clone();
let mut ring: VecDeque<f32> = VecDeque::with_capacity(PCM_SCRATCH);
// Pre-reserve the ring so `extend` never reallocates on the realtime thread. Worst transient
// before the trim below = the hard cap plus one full channel of 5 ms (480-f32) frames — the
// punktfunk protocol always sends 5 ms Opus frames (host `audio_thread`); a larger frame
// would force a one-time realloc, asserted (not silently corrupted) in `decode_loop`.
let mut ring: VecDeque<f32> = VecDeque::with_capacity(hard_cap_max + RING_CHUNKS * 5 * ms);
let mut primed = false;
let callback = move |_s: &AudioStream, data: *mut c_void, num_frames: i32| {
let want = num_frames as usize * CHANNELS;
let mut empties: u32 = 0; // consecutive empty callbacks (de-prime hysteresis)
let mut cb_count: u32 = 0; // callbacks since open (throttles the XRun grow check)
let mut last_xrun: i32 = 0; // last AAudio XRun count we grew the buffer for
let callback = move |s: &AudioStream, data: *mut c_void, num_frames: i32| {
let want = num_frames as usize * channels;
// SAFETY: AAudio provides `num_frames * channel_count` F32 slots at `data`.
let out = unsafe { std::slice::from_raw_parts_mut(data as *mut f32, want) };
while let Ok(chunk) = rx.try_recv() {
ring.extend(chunk);
// Drain decoded chunks into the ring WITHOUT freeing on the RT thread: `drain(..)` empties
// each Vec but keeps its capacity, then the empty buffer is handed back for reuse. The
// only RT-thread free is the rare case where the recycle channel is momentarily full.
while let Ok(mut chunk) = rx.try_recv() {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
// Prime to ~3 quanta (15 ms; floor 15 ms / ceiling 200 ms); drop OLDEST above the cap.
let target = (3 * want).clamp(720 * CHANNELS, 9600 * CHANNELS);
while ring.len() > target.max(want) + want {
// Jitter buffer: prime to ~40 ms (prime_floor) before playing and after a sustained drain;
// drop-oldest only above a wide ~120 ms band. Decoupled from the AAudio burst `want` (tiny
// on the LowLatency MMAP path) so the depth doesn't collapse to a single quantum.
let target = (3 * want).clamp(prime_floor, prime_ceil);
let hard_cap = (target + jitter_headroom).min(hard_cap_max);
while ring.len() > hard_cap {
ring.pop_front();
}
if !primed && ring.len() >= target {
@@ -79,12 +181,34 @@ impl AudioPlayback {
out.fill(0.0);
cb_counters.underruns.fetch_add(1, Ordering::Relaxed);
}
// Re-prime only after a RUN of empty callbacks, not a single transient one — otherwise
// every momentary drain costs a fresh 40 ms silence (the old behaviour, self-inflicted
// crackle on any jitter spike).
if ring.is_empty() {
primed = false; // re-prime after a genuine drain (avoids sustained crackle on loss)
empties += 1;
if empties >= DEPRIME_AFTER_CALLBACKS {
primed = false;
}
} else {
empties = 0;
}
cb_counters
.ring_depth
.store(ring.len() as u64, Ordering::Relaxed);
// Google's AAudio anti-glitch technique: when the device reports new XRuns, grow the HW
// buffer by one burst (up to capacity). getXRunCount + setBufferSizeInFrames are both
// callback-safe / non-blocking, and set clamps to capacity so it self-limits. Throttled.
cb_count = cb_count.wrapping_add(1);
if cb_count % XRUN_CHECK_EVERY == 0 {
let xr = s.x_run_count();
if xr > last_xrun {
last_xrun = xr;
let burst = s.frames_per_burst().max(1);
let grown =
(s.buffer_size_in_frames() + burst).min(s.buffer_capacity_in_frames());
let _ = s.set_buffer_size_in_frames(grown);
}
}
AudioCallbackResult::Continue
};
@@ -93,7 +217,11 @@ impl AudioPlayback {
.ok()?
.direction(AudioDirection::Output)
.sample_rate(SAMPLE_RATE)
.channel_count(CHANNELS as i32)
// The wire order (FL FR FC LFE RL RR SL SR) is the standard AAudio/Android channel
// order, so this is an IDENTITY mapping — no permute. AAudio infers the 5.1/7.1 mask
// from `channel_count` (the ndk crate's builder exposes no setChannelMask); the host
// captures + Opus-encodes in exactly this order.
.channel_count(channels as i32)
.format(AudioFormat::PCM_Float)
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(AudioSharingMode::Shared)
@@ -109,19 +237,31 @@ impl AudioPlayback {
log::error!("audio: request_start: {e}");
return None;
}
// Lift the AAudio HW buffer off its brittle ~2-burst LowLatency default so a single late
// callback doesn't immediately underrun; the in-callback XRun loop grows it further if the
// device still glitches. set_buffer_size_in_frames clamps to capacity.
let burst = stream.frames_per_burst().max(1);
let _ =
stream.set_buffer_size_in_frames((burst * 3).min(stream.buffer_capacity_in_frames()));
// perf != LowLatency or rate != 48000 means AAudio silently fell to a resampled legacy path
// (different burst behaviour) — surface it so the field can tell that apart from plain jitter.
log::info!(
"audio: AAudio started rate={} ch={} fmt={:?} burst={}",
"audio: AAudio started rate={} ch={} fmt={:?} perf={:?} share={:?} burst={} buf={}/{}",
stream.sample_rate(),
stream.channel_count(),
stream.format(),
stream.performance_mode(),
stream.sharing_mode(),
stream.frames_per_burst(),
stream.buffer_size_in_frames(),
stream.buffer_capacity_in_frames(),
);
let shutdown = Arc::new(AtomicBool::new(false));
let sd = shutdown.clone();
let join = std::thread::Builder::new()
.name("pf-audio".into())
.spawn(move || decode_loop(client, tx, sd, counters))
.spawn(move || decode_loop(client, tx, free_rx, sd, counters, channels))
.ok();
Some(AudioPlayback {
@@ -143,31 +283,53 @@ impl Drop for AudioPlayback {
}
/// Producer: `next_audio` → Opus `decode_float` → push interleaved f32 into the ring channel.
/// Buffers come from (and return to) the realtime callback's recycle free-list so the steady state
/// is allocation-free on both threads.
fn decode_loop(
client: Arc<NativeClient>,
tx: SyncSender<Vec<f32>>,
free_rx: Receiver<Vec<f32>>,
shutdown: Arc<AtomicBool>,
counters: Arc<Counters>,
channels: usize,
) {
let mut dec = match opus::Decoder::new(SAMPLE_RATE as u32, opus::Channels::Stereo) {
// Interleaved f32 samples per millisecond at this layout — the ring's 5 ms reserve check below.
let ms = (SAMPLE_RATE as usize / 1000) * channels;
// Opus decode scratch: worst-case 120 ms frame (5760 samples/ch) × channels.
let pcm_scratch = 5760 * channels;
let mut dec = match AudioDec::new(channels as u8) {
Ok(d) => d,
Err(e) => {
log::error!("audio: opus decoder init: {e} — audio disabled");
return;
}
};
let mut pcm = vec![0f32; PCM_SCRATCH];
let mut pcm = vec![0f32; pcm_scratch];
let mut window_peak = 0f32; // loudest |sample| since the last log — tells a tone from silence
while !shutdown.load(Ordering::Relaxed) {
match client.next_audio(Duration::from_millis(5)) {
Ok(pkt) => match dec.decode_float(&pkt.data, &mut pcm, false) {
Ok(samples) => {
let n = samples * CHANNELS;
let n = samples * channels;
for &s in &pcm[..n] {
window_peak = window_peak.max(s.abs());
}
// The ring's pre-reservation in `start` assumes the protocol's 5 ms (≤480-f32/ch)
// frames; a larger frame would force a one-time realloc on the RT thread. Catch a
// future host frame-size change here in debug, not as a silent audio glitch.
debug_assert!(
n <= 5 * ms,
"audio frame {n} f32 exceeds the 5 ms ring reserve"
);
let count = counters.opus_decoded.fetch_add(1, Ordering::Relaxed) + 1;
match tx.try_send(pcm[..n].to_vec()) {
// Reuse a recycled buffer if the callback handed one back; only allocate when the
// free-list is momentarily empty (startup / after a backpressure drop).
let mut buf = free_rx
.try_recv()
.unwrap_or_else(|_| Vec::with_capacity(pcm_scratch));
buf.clear();
buf.extend_from_slice(&pcm[..n]);
match tx.try_send(buf) {
Ok(()) | Err(TrySendError::Full(_)) => {} // drop-newest under backpressure
Err(TrySendError::Disconnected(_)) => break,
}
+12 -4
View File
@@ -140,10 +140,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeGenerateIde
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref): Long`. `certPem`/`keyPem` empty = anonymous, else presented as the
/// persistent identity. `pinHex` empty = TOFU (read `nativeHostFingerprint` after), else 64-hex
/// SHA-256 to pin the host (mismatch → 0). `bitrateKbps` 0 = host default. `compositorPref`/
/// `gamepadPref` are `CompositorPref`/`GamepadPref` wire bytes (0 = Auto; unknown → Auto).
/// compositorPref, gamepadPref, hdrEnabled, audioChannels): Long`. `certPem`/`keyPem` empty =
/// anonymous, else presented as the persistent identity. `pinHex` empty = TOFU (read
/// `nativeHostFingerprint` after), else 64-hex SHA-256 to pin the host (mismatch → 0). `bitrateKbps`
/// 0 = host default. `compositorPref`/`gamepadPref` are `CompositorPref`/`GamepadPref` wire bytes
/// (0 = Auto; unknown → Auto). `audioChannels` is the requested surround layout (2/6/8; normalized,
/// anything else → stereo) — the host clamps it and the resolved count drives playback.
/// Returns an opaque handle, or 0 on failure (logged).
#[no_mangle]
#[allow(clippy::too_many_arguments)]
@@ -162,6 +164,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
compositor_pref: jint,
gamepad_pref: jint,
hdr_enabled: jboolean,
audio_channels: jint,
) -> jlong {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
@@ -213,6 +216,11 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
} else {
0
},
// Requested surround layout (2 = stereo / 6 = 5.1 / 8 = 7.1). The host clamps to what it can
// capture and echoes the resolved count in `connector.audio_channels`, which drives the
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
// normalizes to stereo here.
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
None, // launch: default app
pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous)
+1 -1
View File
@@ -361,4 +361,4 @@ ever switched to a logged-in GUI session, re-adding macOS to the job's capture s
- Mid-stream renegotiation (resolution change without reconnect) is designed-for but not
implemented (the Welcome is one-shot today).
- Host-side gamepad injection needs `/dev/uinput` access on the box (udev rule from
`docs/linux-setup.md`).
`design/linux-setup.md`).
@@ -25,6 +25,7 @@ struct ContentView: View {
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@@ -252,6 +253,7 @@ struct ContentView: View {
setting: PunktfunkConnection.GamepadType(
rawValue: UInt32(clamping: gamepadType)) ?? .auto),
bitrateKbps: UInt32(clamping: bitrateKbps),
audioChannels: UInt8(clamping: audioChannels),
launchID: launchID,
allowTofu: host.pinnedSHA256 == nil)
}
@@ -351,6 +353,7 @@ struct ContentView: View {
compositor: pref,
gamepad: pad,
bitrateKbps: bitrate,
audioChannels: UInt8(clamping: audioChannels),
autoTrust: true)
}
}
@@ -99,6 +99,7 @@ final class SessionModel: ObservableObject {
compositor: PunktfunkConnection.Compositor = .auto,
gamepad: PunktfunkConnection.GamepadType = .auto,
bitrateKbps: UInt32 = 0,
audioChannels: UInt8 = 2,
hdrEnabled: Bool = true,
launchID: String? = nil,
allowTofu: Bool = false,
@@ -137,7 +138,7 @@ final class SessionModel: ObservableObject {
width: width, height: height, refreshHz: hz,
pinSHA256: pin, identity: identity, compositor: compositor,
gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps,
launchID: launchID) }
audioChannels: audioChannels, launchID: launchID) }
await MainActor.run { [weak self] in
guard let self else { return }
// The user may have abandoned this attempt (window closed, another host
@@ -25,6 +25,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@ObservedObject private var gamepads = GamepadManager.shared
@@ -173,6 +174,10 @@ struct SettingsView: View {
TVSelectionRow(title: "Stream mode", options: options, selection: modeTag)
TVSelectionRow(
title: "Bitrate", options: bitrateOptions, selection: $bitrateKbps)
TVSelectionRow(
title: "Audio channels",
options: [("Stereo", 2), ("5.1 Surround", 6), ("7.1 Surround", 8)],
selection: $audioChannels)
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.caption)
@@ -271,6 +276,11 @@ struct SettingsView: View {
@ViewBuilder private var audioSection: some View {
Section {
Picker("Audio channels", selection: $audioChannels) {
Text("Stereo").tag(2)
Text("5.1 Surround").tag(6)
Text("7.1 Surround").tag(8)
}
#if os(macOS)
Picker("Speaker", selection: $speakerUID) {
Text("System default").tag("")
@@ -15,6 +15,9 @@ public enum DefaultsKey {
public static let gamepadType = "punktfunk.gamepadType"
public static let gamepadID = "punktfunk.gamepadID"
public static let bitrateKbps = "punktfunk.bitrateKbps"
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the in-core decode + AVAudioEngine layout.
public static let audioChannels = "punktfunk.audioChannels"
public static let micEnabled = "punktfunk.micEnabled"
public static let speakerUID = "punktfunk.speakerUID"
public static let micUID = "punktfunk.micUID"
@@ -235,6 +235,12 @@ public final class PunktfunkConnection {
/// drain `nextHdrMeta`.
public var isHDR: Bool { colorTransfer == 16 || colorTransfer == 18 }
/// The audio channel count the host resolved for this session (the Welcome's echo of the
/// requested `audioChannels`, clamped to what the host can capture): `2` (stereo), `6` (5.1)
/// or `8` (7.1). Build the playback layout from THIS, never the request. `2` for an older host.
/// PCM from `nextAudioPcm` is interleaved in the canonical wire order FL FR FC LFE RL RR SL SR.
public private(set) var resolvedAudioChannels: UInt8 = 2
/// Connect and start a session at the requested mode (the host creates a native virtual
/// output at exactly this size/refresh). Blocks up to `timeoutMs`.
///
@@ -264,6 +270,7 @@ public final class PunktfunkConnection {
gamepad: GamepadType = .auto,
bitrateKbps: UInt32 = 0,
videoCaps: UInt8 = 0,
audioChannels: UInt8 = 2,
launchID: String? = nil,
timeoutMs: UInt32 = 10_000
) throws {
@@ -279,16 +286,16 @@ public final class PunktfunkConnection {
withOptionalCString(launchID) { launch in
if let pin = pinSHA256 {
return pin.withUnsafeBytes { p in
punktfunk_connect_ex5(
punktfunk_connect_ex6(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, launch,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, launch,
p.bindMemory(to: UInt8.self).baseAddress, &observed,
cert, key, timeoutMs)
}
}
return punktfunk_connect_ex5(
return punktfunk_connect_ex6(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, launch,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, launch,
nil, &observed, cert, key, timeoutMs)
}
}
@@ -320,6 +327,9 @@ public final class PunktfunkConnection {
colorMatrix = mtx
colorFullRange = fullRange != 0
bitDepth = depth
var ac: UInt8 = 2
_ = punktfunk_connection_audio_channels(handle, &ac)
resolvedAudioChannels = ac
}
/// A bandwidth speed-test measurement (see `startSpeedTest`). Partial until `done`.
@@ -468,6 +478,50 @@ public final class PunktfunkConnection {
}
}
/// One decoded audio frame from `nextAudioPcm`: interleaved 32-bit float at 48 kHz, in the
/// canonical wire channel order FL FR FC LFE RL RR SL SR (the first `channels`).
public struct AudioPCM: Sendable {
/// Interleaved f32 samples (`frameCount * channels` long), wire channel order.
public let samples: [Float]
/// Samples per channel.
public let frameCount: Int
/// Channel count (2/6/8) `resolvedAudioChannels`.
public let channels: Int
public let ptsNs: UInt64
public let seq: UInt32
}
/// Pull the next audio frame, **decoded in-core** to interleaved f32 PCM Apple's AudioToolbox
/// Opus path is stereo-only, so surround (and, for uniformity, stereo too) is decoded by the
/// Rust core (libopus multistream) and handed back as PCM. nil on timeout, throws `.closed` once
/// the session ended. Drain from a dedicated audio thread (do NOT also call `nextAudio` they
/// share the underlying queue). The returned `samples` are copied out, so the buffer is owned.
public func nextAudioPcm(timeoutMs: UInt32 = 100) throws -> AudioPCM? {
audioLock.lock()
defer { audioLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkAudioPcm()
let rc = punktfunk_connection_next_audio_pcm(h, &out, timeoutMs)
switch rc {
case statusOK:
let channels = Int(out.channels)
let total = Int(out.frame_count) * channels
guard let base = out.samples, total > 0 else { return nil }
// Copy: the pointer borrows connection memory only until the next PCM call.
let samples = Array(UnsafeBufferPointer(start: base, count: total))
return AudioPCM(
samples: samples, frameCount: Int(out.frame_count),
channels: channels, ptsNs: out.pts_ns, seq: out.seq)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Pull the next force-feedback update for the GCController haptics engine:
/// `(pad, lowFrequency, highFrequency)` with 0...0xFFFF amplitudes, (0, 0) = stop.
/// Drain from the (single) feedback thread, alongside `nextHidOutput`.
@@ -19,13 +19,13 @@ import os
private let log = Logger(subsystem: "io.unom.punktfunk", category: "audio")
/// SPSC-ish jitter ring (interleaved stereo float), drain thread render callback.
/// The unfair lock is held for microseconds; fine at render-callback rates. Priming:
/// SPSC-ish jitter ring (interleaved float, `channels` per frame), drain thread render
/// callback. The unfair lock is held for microseconds; fine at render-callback rates. Priming:
/// reads return silence until enough is buffered (at least `prefill`, and at least one
/// packet more than the device's render quantum large-buffer devices would otherwise
/// chronically out-demand the prefill and oscillate prime dropout re-prime), and an
/// underrun re-primes, concealing jitter as one short dip instead of sustained crackle.
/// All counts stay even (whole stereo frames), so L/R interleave can never flip.
/// All counts stay whole frames (multiples of `channels`), so the interleave can never slip.
final class AudioRing: @unchecked Sendable {
private var buf: [Float]
private var readIdx = 0
@@ -34,12 +34,14 @@ final class AudioRing: @unchecked Sendable {
private var renderQuantum = 0
private let prefill: Int
private let highWater: Int
private let channels: Int
private let lock = OSAllocatedUnfairLock()
/// `capacity`/`prefill` in samples (interleaved 2 per frame, both must be even).
init(capacity: Int, prefill: Int) {
/// `capacity`/`prefill` in samples (interleaved `channels` per frame, both whole frames).
init(capacity: Int, prefill: Int, channels: Int) {
buf = [Float](repeating: 0, count: capacity)
self.prefill = prefill
self.channels = channels
highWater = prefill * 4
}
@@ -74,8 +76,8 @@ final class AudioRing: @unchecked Sendable {
renderQuantum = max(renderQuantum, count)
let available = writeIdx - readIdx
if !primed {
// 480 samples = one 5 ms host packet of slack beyond the device's demand.
if available >= max(prefill, renderQuantum + 480) {
// One 5 ms host packet (240 frames × channels) of slack beyond the device's demand.
if available >= max(prefill, renderQuantum + 240 * channels) {
primed = true
} else {
for i in 0..<count { out[i] = 0 }
@@ -113,10 +115,55 @@ private final class StopFlag: @unchecked Sendable {
/// Render-block-owned scratch storage: freed exactly when the closure (and thus the
/// last possible render call) is released never racing CoreAudio.
private final class ScratchBuffer {
let ptr = UnsafeMutablePointer<Float>.allocate(capacity: 8192 * 2)
// 8192 frames × up to 8 channels (7.1) the render block caps `frames` at 8192.
let ptr = UnsafeMutablePointer<Float>.allocate(capacity: 8192 * 8)
deinit { ptr.deallocate() }
}
/// CoreAudio channel layout for the canonical wire order FL FR FC LFE RL RR [SL SR]. nil for
/// stereo (the standard layout is correct). For 5.1/7.1 we list explicit channel labels via
/// `kAudioChannelLayoutTag_UseChannelDescriptions` preset tags (DTS_5_1 etc.) don't reliably
/// match Moonlight's order. NB the 7.1 mapping (verified against the WASAPI 0x63F + SPA orderings):
/// wire idx 4-5 = RL/RR = the WAVE *back* pair LeftSurround/RightSurround; idx 6-7 = SL/SR = the
/// WAVE *side* pair LeftSurroundDirect/RightSurroundDirect. (Using RearSurround* for 6-7 would
/// swap side/back vs the Windows/Linux clients.)
private func wireChannelLayout(channels: Int) -> AVAudioChannelLayout? {
let labels: [AudioChannelLabel]
switch channels {
case 6:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen, kAudioChannelLabel_LeftSurround,
kAudioChannelLabel_RightSurround,
]
case 8:
labels = [
kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center,
kAudioChannelLabel_LFEScreen,
kAudioChannelLabel_LeftSurround, kAudioChannelLabel_RightSurround, // wire RL/RR (back)
kAudioChannelLabel_LeftSurroundDirect, kAudioChannelLabel_RightSurroundDirect, // wire SL/SR (side)
]
default:
return nil
}
let size = MemoryLayout<AudioChannelLayout>.size
+ (labels.count - 1) * MemoryLayout<AudioChannelDescription>.stride
let raw = UnsafeMutableRawPointer.allocate(byteCount: size, alignment: 16)
defer { raw.deallocate() }
let layout = raw.bindMemory(to: AudioChannelLayout.self, capacity: 1)
layout.pointee.mChannelLayoutTag = kAudioChannelLayoutTag_UseChannelDescriptions
layout.pointee.mChannelBitmap = AudioChannelBitmap(rawValue: 0)
layout.pointee.mNumberChannelDescriptions = UInt32(labels.count)
let descs = UnsafeMutableBufferPointer(
start: &layout.pointee.mChannelDescriptions, count: labels.count)
for (i, lbl) in labels.enumerated() {
descs[i] = AudioChannelDescription(
mChannelLabel: lbl, mChannelFlags: AudioChannelFlags(rawValue: 0),
mCoordinates: (0, 0, 0))
}
return AVAudioChannelLayout(layout: layout)
}
public final class SessionAudio {
private let connection: PunktfunkConnection
private let flag = StopFlag()
@@ -229,9 +276,13 @@ public final class SessionAudio {
// MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) {
// 1 s of interleaved stereo capacity, ~20 ms prefill: four 5 ms host packets of
// jitter absorption before the first sample plays.
let ring = AudioRing(capacity: 96_000, prefill: 1920)
// Build the playback layout from the host-RESOLVED channel count (never the request):
// 2 = stereo / 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = Int(connection.resolvedAudioChannels)
// 1 s interleaved capacity, ~20 ms prefill (four 5 ms host packets of jitter absorption
// before the first sample plays), both scaled by the channel count.
let ring = AudioRing(
capacity: 48_000 * channels, prefill: 960 * channels, channels: channels)
let engine = AVAudioEngine()
#if os(macOS)
@@ -247,21 +298,32 @@ public final class SessionAudio {
}
#endif
// Engine-native deinterleaved float; the render block deinterleaves from the ring.
guard let format = AVAudioFormat(standardFormatWithSampleRate: 48_000, channels: 2)
else { return }
// Engine-native deinterleaved float; the render block deinterleaves from the ring. Surround
// uses an explicit wire-order channel layout; the mixer downmixes to the output device when
// it has fewer speakers (e.g. an iPhone's stereo built-ins). (Explicit if/else rather than
// map/flatMap so it's correct whether the channelLayout initializer is failable or not.)
var format: AVAudioFormat?
if channels == 2 {
format = AVAudioFormat(standardFormatWithSampleRate: 48_000, channels: 2)
} else if let layout = wireChannelLayout(channels: channels) {
format = AVAudioFormat(standardFormatWithSampleRate: 48_000, channelLayout: layout)
}
guard let format else {
log.error("could not build \(channels)-channel audio format — audio disabled")
return
}
let scratch = ScratchBuffer() // block-owned; freed with the closure
let source = AVAudioSourceNode(format: format) { _, _, frameCount, abl -> OSStatus in
let frames = Int(frameCount)
guard frames <= 8192 else { return kAudioUnitErr_TooManyFramesToProcess }
ring.read(into: scratch.ptr, count: frames * 2)
ring.read(into: scratch.ptr, count: frames * channels)
let buffers = UnsafeMutableAudioBufferListPointer(abl)
if buffers.count >= 2,
let left = buffers[0].mData?.assumingMemoryBound(to: Float.self),
let right = buffers[1].mData?.assumingMemoryBound(to: Float.self) {
for f in 0..<frames {
left[f] = scratch.ptr[f * 2]
right[f] = scratch.ptr[f * 2 + 1]
// Deinterleave the wire-order interleaved ring into the engine's per-channel buses.
if buffers.count >= channels {
for ch in 0..<channels {
if let dst = buffers[ch].mData?.assumingMemoryBound(to: Float.self) {
for f in 0..<frames { dst[f] = scratch.ptr[f * channels + ch] }
}
}
}
return noErr
@@ -292,29 +354,20 @@ public final class SessionAudio {
stateLock.unlock()
let thread = Thread { [connection, flag, drainDone] in
defer { drainDone.signal() }
guard let decoder = try? OpusDecoder(framesPerPacket: 240),
let pcm = AVAudioPCMBuffer(
pcmFormat: decoder.pcmFormat, frameCapacity: 5760)
else {
log.error("Opus decoder unavailable — audio playback disabled")
return
}
// Decode happens IN-CORE (libopus multistream) AudioToolbox's Opus path is
// stereo-only and is handed back as interleaved f32 PCM in wire channel order.
while !flag.isStopped {
let packet: AudioPacket?
let pcm: PunktfunkConnection.AudioPCM?
do {
packet = try connection.nextAudio(timeoutMs: 100)
pcm = try connection.nextAudioPcm(timeoutMs: 100)
} catch {
break // session closed
}
guard let packet else { continue }
do {
let frames = try decoder.decode(packet.data, into: pcm)
if frames > 0, let p = pcm.floatChannelData?[0] {
ring.write(p, count: Int(frames) * 2)
guard let pcm, pcm.frameCount > 0 else { continue }
pcm.samples.withUnsafeBufferPointer { p in
if let base = p.baseAddress {
ring.write(base, count: pcm.frameCount * pcm.channels)
}
} catch {
// One corrupt packet a dead stream; skip it.
log.warning("audio decode failed: \(error.localizedDescription)")
}
}
}
+36 -1
View File
@@ -45,8 +45,9 @@ Gaming Mode automatically.
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. |
| `src/backend.ts` | Typed `callable` bridges to `main.py`. |
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut targets (so the window is focusable). |
| `main.py` | Backend: `discover` / `pair` / `runner_info` / `get_settings` / `set_settings` / `kill_stream`. |
| `main.py` | Backend: `discover` / `pair` / `runner_info` / `get_settings` / `set_settings` / `kill_stream` / `check_update`. |
| `plugin.json` | Decky plugin manifest. |
| `update.json` | CI-baked `{channel, manifest}` — where `check_update()` polls (absent on dev builds). |
| `decky.pyi` | Type stub for the injected `decky` module (vendored from the template). |
### Discovery (`discover()`)
@@ -140,6 +141,40 @@ shows up in the Quick Access Menu.
> [`../../packaging/flatpak/README.md`](../../packaging/flatpak/README.md)) — install that on
> the Deck too, or the panel's Connect surfaces a `client-not-found` error.
## Updating (self-update, no store)
The plugin updates itself without the official Decky store. CI (`decky.yml`) publishes a tiny
per-channel `manifest.json` next to the zip in the Gitea registry:
```json
{"version":"0.3.123","artifact":".../punktfunk-decky/0.3.123/punktfunk.zip","sha256":"…"}
```
and bakes an `update.json` (`{channel, manifest}`) into the plugin so it knows which channel it was
installed from. The backend `check_update()` reads the **installed** version from `package.json`
the value Decky itself reports (it does **not** read `plugin.json`) — fetches the channel manifest,
and compares. When a newer build exists the frontend shows an **Update to vX** button that drives
Decky Loader's own install RPC:
```ts
window.DeckyBackend.callable("utilities/install_plugin")(artifact, "punktfunk", version, hash, /*UPDATE=*/2)
```
The loader (root) downloads the immutable per-version zip, **SHA-256-verifies** it against `hash`,
replaces `~/homebrew/plugins/punktfunk`, and hot-reloads — the unprivileged backend never writes the
root-owned plugins dir itself. `window.DeckyBackend` / `utilities/install_plugin` are loader
internals (not `@decky/api`), so every access is guarded; missing them, the button falls back to a
toast pointing at **Install Plugin from URL**.
> CI stamps a **plain numeric** semver per channel (`0.3.<run>` canary, `X.Y.Z` stable) into
> `package.json`. Decky's `compare-versions` orders pre-release identifiers lexically (so `ci10 < ci9`)
> — a `-ciN` suffix would mis-detect updates.
**Optional — native Updates tab:** Decky's store is single-source (a custom store URL *replaces* the
official catalog), so punktfunk doesn't ship one by default. A user who wants the native update badge
can point Decky → Settings → **Custom store** at a punktfunk-only store JSON — not recommended if you
use other plugins, since it hides the official catalog.
## Limitations / next steps
- **Needs on-Deck validation in Gaming Mode**: the Steam-shortcut launch (`AddShortcut` /
+3 -1
View File
@@ -31,4 +31,6 @@ fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
# exec so the flatpak client IS the game process — when it exits, Steam ends the "game" and
# Gaming Mode reclaims focus automatically (no manual refocus needed).
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST"
# --fullscreen: present the stream chrome-less and fullscreen (the client also auto-detects the
# Deck/gamescope env, and ignores the flag harmlessly on older builds that predate it).
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen
+141 -4
View File
@@ -17,6 +17,8 @@ The backend's jobs are the things Steam can't do:
* **get_settings() / set_settings()** — read/write the flatpak client's stream settings JSON
(resolution / bitrate / gamepad), so the Deck UI configures the stream the client reads.
* **kill_stream()** — force-stop a wedged stream (``flatpak kill``).
* **check_update()** — poll the registry's per-channel ``manifest.json`` and report whether a
newer build is available (the frontend then drives Decky's own install RPC to apply it).
The TXT-record keys parsed (``proto`` / ``fp`` / ``pair`` / ``id``) are defined by the host
advert in ``crates/punktfunk-host/src/discovery.rs``.
@@ -26,7 +28,10 @@ import asyncio
import json
import os
import shutil
import ssl
import stat
import time
import urllib.request
from pathlib import Path
import decky
@@ -37,22 +42,99 @@ APP_ID = "io.unom.Punktfunk"
# Service type advertised by punktfunk/1 hosts (matches NATIVE_SERVICE in the Rust host).
SERVICE_TYPE = "_punktfunk._udp"
# The flatpak client persists identity / known-hosts / settings under HOME/.config/punktfunk;
# inside the flatpak sandbox HOME is ~/.var/app/<APP_ID>, so the real on-disk location is this.
# The backend writes settings here so the (sandboxed) client reads them.
# The flatpak client persists identity / known-hosts / settings under HOME/.config/punktfunk.
# The sandbox HOME resolves to the REAL user home (== DECKY_USER_HOME), NOT the per-app
# ~/.var/app/<APP_ID> dir — verified on-device (`flatpak run … sh -c 'echo $HOME'` prints
# /home/deck, and the manifest's `--filesystem=~/.config/punktfunk` grants exactly that path;
# we also pass HOME=DECKY_USER_HOME into `flatpak run`, see _flatpak_env). Pointing here is what
# lets plugin settings actually reach the client AND lets us read the client's known-hosts to
# tell whether THIS device is already paired with a given host.
def _client_config_dir() -> Path:
return Path(decky.DECKY_USER_HOME) / ".var" / "app" / APP_ID / ".config" / "punktfunk"
return Path(decky.DECKY_USER_HOME) / ".config" / "punktfunk"
def _settings_path() -> Path:
return _client_config_dir() / "client-gtk-settings.json"
def _paired_fingerprints() -> set[str]:
"""Host cert fingerprints (lowercase hex) this client has PIN-paired, from the client's
known-hosts store. Keyed by fingerprint so it survives a host changing IP address."""
try:
data = json.loads((_client_config_dir() / "client-known-hosts.json").read_text())
except (OSError, json.JSONDecodeError):
return set()
hosts = data.get("hosts", []) if isinstance(data, dict) else []
return {
h["fp_hex"].lower()
for h in hosts
if isinstance(h, dict) and h.get("paired") and isinstance(h.get("fp_hex"), str)
}
def _runner_path() -> str:
"""Absolute path to the launch wrapper shipped with the plugin (bin/punktfunkrun.sh)."""
return str(Path(decky.DECKY_PLUGIN_DIR) / "bin" / "punktfunkrun.sh")
# ----------------------------------------------------------------------------------------
# Self-update check (no Decky store). The plugin is distributed via "Install Plugin from
# URL" pointing at our Gitea generic registry, so the official store never sees it and
# can't offer updates. Instead the backend polls a tiny per-channel ``manifest.json`` the
# CI publishes next to the zip, compares it to the installed version, and the frontend
# offers a one-tap update that drives Decky's own (root, privileged) install RPC. The
# channel + manifest URL are baked into ``update.json`` by CI (.gitea/workflows/decky.yml);
# a dev/sideload build has no ``update.json`` and update checks are simply disabled.
_UPDATE_TTL_S = 1800.0 # cache a successful check for 30 min (the QAM remounts often)
_update_cache: dict = {"at": 0.0, "data": None}
def _update_config() -> dict:
"""The CI-baked ``{channel, manifest}`` next to the plugin (absent on dev builds)."""
try:
return json.loads((Path(decky.DECKY_PLUGIN_DIR) / "update.json").read_text())
except (OSError, json.JSONDecodeError):
return {}
def _installed_version() -> str:
"""The version Decky itself reports for this plugin — it reads ``package.json`` (NOT
plugin.json), so the CI stamps the build version there."""
try:
pkg = json.loads((Path(decky.DECKY_PLUGIN_DIR) / "package.json").read_text())
return str(pkg.get("version", "0.0.0"))
except (OSError, json.JSONDecodeError):
return "0.0.0"
def _semver_tuple(v: str) -> tuple[int, int, int]:
"""A tolerant (major, minor, patch) tuple for ``>`` comparison. We control the version
format (plain numeric ``X.Y.Z`` on both channels), so leading-int-per-component is
enough; any pre-release suffix is dropped before comparing."""
parts: list[int] = []
for comp in str(v).split("-", 1)[0].split(".")[:3]:
digits = ""
for ch in comp:
if ch.isdigit():
digits += ch
else:
break
parts.append(int(digits) if digits else 0)
while len(parts) < 3:
parts.append(0)
return (parts[0], parts[1], parts[2])
def _fetch_json(url: str, timeout: float = 8.0) -> dict:
"""Blocking HTTPS GET of a small JSON document (run in an executor)."""
req = urllib.request.Request(
url, headers={"Accept": "application/json", "User-Agent": "punktfunk-decky"}
)
ctx = ssl.create_default_context()
with urllib.request.urlopen(req, timeout=timeout, context=ctx) as resp:
return json.loads(resp.read().decode("utf-8", errors="replace"))
def _flatpak() -> str | None:
return shutil.which("flatpak") or (
"/usr/bin/flatpak" if Path("/usr/bin/flatpak").exists() else None
@@ -179,6 +261,13 @@ class Plugin:
if stderr:
decky.logger.debug("avahi-browse stderr: %s", stderr.decode(errors="replace"))
hosts = _parse_avahi_browse(stdout.decode(errors="replace"))
# Mark which hosts THIS device has already paired (by cert fingerprint), so the UI can
# show "Stream" instead of "Pair" — the mDNS `pair` field is the host's policy, not our
# per-device pairing state.
paired = _paired_fingerprints()
for h in hosts:
fp = h.get("fp") or ""
h["paired"] = bool(fp) and fp.lower() in paired
decky.logger.info("discovered %d punktfunk host(s)", len(hosts))
return hosts
@@ -279,6 +368,54 @@ class Plugin:
return {"ok": False}
return {"ok": True}
async def check_update(self, force: bool = False) -> dict:
"""Is a newer build available in our registry? Compares the installed version
(``package.json``) against the per-channel ``manifest.json`` the CI publishes, and
returns everything the frontend needs to drive Decky's install RPC. Non-fatal: any
failure (no channel baked in, network down) returns ``update_available: False``.
"""
current = _installed_version()
cfg = _update_config()
result = {
"current": current,
"latest": current,
"artifact": "",
"hash": "",
"channel": str(cfg.get("channel", "")),
"update_available": False,
}
manifest_url = cfg.get("manifest")
if not manifest_url:
result["error"] = "update-channel-unknown" # dev / sideloaded build
return result
now = time.monotonic()
cached = _update_cache["data"]
if not force and cached and (now - _update_cache["at"]) < _UPDATE_TTL_S:
return cached
try:
loop = asyncio.get_running_loop()
manifest = await loop.run_in_executor(None, _fetch_json, manifest_url)
except Exception as exc: # noqa: BLE001
decky.logger.warning("update check failed: %s", exc)
result["error"] = "fetch-failed"
return result # transient — don't cache, retry next open
latest = str(manifest.get("version", current))
result["latest"] = latest
result["artifact"] = str(manifest.get("artifact", ""))
result["hash"] = str(manifest.get("sha256", ""))
result["update_available"] = bool(result["artifact"]) and (
_semver_tuple(latest) > _semver_tuple(current)
)
if result["update_available"]:
decky.logger.info("update available: %s -> %s (%s)", current, latest, result["channel"])
_update_cache["at"] = now
_update_cache["data"] = result
return result
# ---- Decky lifecycle ----
async def _main(self):
+13 -1
View File
@@ -5,8 +5,9 @@ export interface Host {
name: string;
host: string;
port: number;
pair: string; // "required" | "optional"
pair: string; // "required" | "optional" — the HOST's policy
fp: string;
paired: boolean; // whether THIS device has already PIN-paired this host (by fingerprint)
}
export interface PairResult {
@@ -32,6 +33,16 @@ export interface StreamSettings {
mic_enabled: boolean;
}
export interface UpdateInfo {
current: string; // installed version (package.json)
latest: string; // newest version in our registry for this channel
artifact: string; // immutable zip URL Decky should install
hash: string; // sha256 of that zip (Decky verifies it)
channel: string; // "latest" (stable) | "canary"
update_available: boolean;
error?: string; // "update-channel-unknown" (dev build) | "fetch-failed"
}
export const discover = callable<[], Host[]>("discover");
export const pair = callable<
[host: string, port: number, pin: string, name: string],
@@ -43,3 +54,4 @@ export const setSettings = callable<[settings: StreamSettings], { ok: boolean }>
"set_settings",
);
export const killStream = callable<[], { ok: boolean }>("kill_stream");
export const checkUpdate = callable<[force: boolean], UpdateInfo>("check_update");
+269 -38
View File
@@ -10,12 +10,22 @@ import {
PanelSectionRow,
SliderField,
Spinner,
Tabs,
ToggleField,
showModal,
staticClasses,
} from "@decky/ui";
import { definePlugin, routerHook, toaster } from "@decky/api";
import { FC, useCallback, useEffect, useState } from "react";
import {
Component,
CSSProperties,
ErrorInfo,
FC,
ReactNode,
useCallback,
useEffect,
useState,
} from "react";
import {
FaTv,
FaSyncAlt,
@@ -23,19 +33,130 @@ import {
FaLockOpen,
FaPlay,
FaArrowLeft,
FaDownload,
} from "react-icons/fa";
import {
discover,
getSettings,
pair,
setSettings,
checkUpdate,
Host,
StreamSettings,
UpdateInfo,
} from "./backend";
import { launchStream } from "./steam";
const ROUTE = "/punktfunk";
// Decky Loader exposes its already-authenticated WSRouter as a global. This is NOT part of
// @decky/api (it's a loader internal), so we treat it as optional and guard every use — on a
// loader without it we fall back to manual "Install Plugin from URL". We use it to drive
// Decky's own privileged install path (the root loader does the download + SHA-256 verify +
// extract + hot-reload), which is the only way a plugin can update itself: ~/homebrew/plugins
// is root-owned, so our unprivileged backend can't swap its own files.
declare global {
interface Window {
DeckyBackend?: {
callable: (route: string) => (...args: unknown[]) => Promise<unknown>;
};
}
}
// PluginInstallType.UPDATE in decky-loader's browser.py (INSTALL=0/REINSTALL=1/UPDATE=2/…).
const INSTALL_TYPE_UPDATE = 2;
// ----------------------------------------------------------------------------------------
// Error boundary — contains ANY render failure in our UI so a single bad render can never take
// down the whole Quick Access "Decky" section (Decky's tab-level boundary shows the generic
// "Something went wrong while displaying this content" for the entire tab when one plugin
// throws). The realistic trigger is a future Steam client update that makes a @decky/ui
// component resolve to `undefined` (React then throws "Element type is invalid"). The fallback
// is built from ONLY plain DOM elements + inline styles, so it cannot itself depend on a
// (possibly broken) Steam-internal component — it is guaranteed to render.
// ----------------------------------------------------------------------------------------
class PluginErrorBoundary extends Component<
{ children: ReactNode },
{ error: Error | null }
> {
state: { error: Error | null } = { error: null };
static getDerivedStateFromError(error: Error) {
return { error };
}
componentDidCatch(error: Error, info: ErrorInfo) {
// Surface it for diagnosis, but never rethrow — containment is the whole point.
// eslint-disable-next-line no-console
console.error("[punktfunk] contained UI render error:", error, info?.componentStack);
}
render() {
const { error } = this.state;
if (!error) return this.props.children;
return (
<div style={{ padding: "1em", lineHeight: 1.45 }}>
<div style={{ fontWeight: "bold", marginBottom: "0.4em" }}>
punktfunk couldnt draw this view
</div>
<div style={{ opacity: 0.8, marginBottom: "0.6em" }}>
The plugin hit a display error your Steam Deck is fine. Reload punktfunk from
Decky&apos;s plugin list, or update the plugin.
</div>
<div
style={{
opacity: 0.55,
fontFamily: "monospace",
fontSize: "0.8em",
wordBreak: "break-word",
}}
>
{String(error?.message ?? error)}
</div>
</div>
);
}
}
// Checks our registry for a newer build on mount (the backend caches + is non-fatal offline).
function useUpdate() {
const [info, setInfo] = useState<UpdateInfo | null>(null);
useEffect(() => {
void checkUpdate(false)
.then(setInfo)
.catch(() => {});
}, []);
return info;
}
async function applyUpdate(info: UpdateInfo) {
try {
const backend = window.DeckyBackend;
if (backend?.callable) {
// Fire-and-forget: the loader reinstalls + reloads THIS plugin, tearing the panel down
// before any result could arrive — so never await it. Decky shows its own confirm prompt.
void backend.callable("utilities/install_plugin")(
info.artifact,
"punktfunk",
info.latest,
info.hash,
INSTALL_TYPE_UPDATE,
);
toaster.toast({
title: "punktfunk",
body: `Updating to v${info.latest}… confirm the Decky prompt.`,
});
return;
}
} catch {
// fall through to the manual path
}
toaster.toast({
title: "punktfunk",
body: "Update from Decky → Developer → Install Plugin from URL.",
});
}
// ----------------------------------------------------------------------------------------
// Discovery hook — shared by the QAM panel and the full page.
// ----------------------------------------------------------------------------------------
@@ -255,20 +376,24 @@ const SettingsSection: FC = () => {
// One host row on the full page.
// ----------------------------------------------------------------------------------------
const HostRow: FC<{ host: Host }> = ({ host }) => {
const pairRequired = host.pair === "required";
// The host's policy is `pair=required`, but if THIS device is already paired we don't need to
// pair again — show it as trusted and go straight to Stream.
const needsPair = host.pair === "required" && !host.paired;
return (
<Field
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{pairRequired ? <FaLock /> : <FaLockOpen />}
{needsPair ? <FaLock /> : <FaLockOpen />}
{host.name}
</span>
}
description={`${host.host}:${host.port}${pairRequired ? " · pairing required" : ""}`}
description={`${host.host}:${host.port}${
needsPair ? " · pairing required" : host.paired ? " · paired" : ""
}`}
childrenContainerWidth="max"
>
<Focusable style={{ display: "flex", gap: "0.5em" }}>
{pairRequired && (
{needsPair && (
<DialogButton
style={{ minWidth: "5em" }}
onClick={() =>
@@ -288,52 +413,129 @@ const HostRow: FC<{ host: Host }> = ({ host }) => {
};
// ----------------------------------------------------------------------------------------
// The fullscreen page (registered as the /punktfunk route).
// The fullscreen page (registered as the /punktfunk route) — a tabbed Hosts / Settings view.
// ----------------------------------------------------------------------------------------
// Bottom inset so the last control clears Gaming Mode's footer hint bar. Routed pages render
// *under* that bar otherwise — that's why the last Stream-settings row was getting hidden. The
// value is generous on purpose (and harmless where the tab area already insets); tune to taste.
const SAFE_BOTTOM = "80px";
// Each tab is its own scroll area so long content is always reachable above the footer.
const tabScroll: CSSProperties = {
height: "100%",
overflowY: "auto",
padding: "0.5em 2.5em",
paddingBottom: SAFE_BOTTOM,
boxSizing: "border-box",
};
const HostsTab: FC<{
hosts: Host[];
scanning: boolean;
refresh: () => void;
}> = ({ hosts, scanning, refresh }) => (
<div style={tabScroll}>
<Field
label="Discover"
description={
scanning
? "Scanning the LAN…"
: `${hosts.length} host${hosts.length === 1 ? "" : "s"} on your network`
}
childrenContainerWidth="max"
bottomSeparator={hosts.length ? "standard" : "none"}
>
<DialogButton style={{ minWidth: "8em" }} disabled={scanning} onClick={refresh}>
{scanning ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
) : (
<FaSyncAlt style={{ marginRight: "0.5em" }} />
)}
{scanning ? "Scanning…" : "Refresh"}
</DialogButton>
</Field>
{hosts.length === 0 && !scanning && (
<Field
focusable={false}
description="No punktfunk hosts found. Make sure a host is running on the same network."
>
No hosts found
</Field>
)}
{hosts.map((h) => (
<HostRow key={h.fp || `${h.host}:${h.port}`} host={h} />
))}
</div>
);
const SettingsTab: FC = () => (
<div style={tabScroll}>
<SettingsSection />
</div>
);
const PunktfunkPage: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const update = useUpdate();
const [tab, setTab] = useState("hosts");
return (
<div
style={{
marginTop: "40px",
height: "calc(100% - 40px)",
overflowY: "auto",
padding: "0 2.5em 2.5em",
display: "flex",
flexDirection: "column",
}}
>
<Focusable style={{ display: "flex", alignItems: "center", gap: "1em", marginBottom: "1em" }}>
<Focusable
style={{
display: "flex",
alignItems: "center",
gap: "1em",
padding: "0 2.5em",
marginBottom: "0.4em",
flexShrink: 0,
}}
>
<DialogButton
style={{ width: "3em", minWidth: "3em" }}
style={{ width: "3em", minWidth: "3em", padding: 0 }}
onClick={() => Navigation.NavigateBack()}
>
<FaArrowLeft />
</DialogButton>
<div className={staticClasses.Title} style={{ flex: 1 }}>
<div className={staticClasses?.Title} style={{ flex: 1, margin: 0 }}>
punktfunk
</div>
<DialogButton style={{ width: "10em" }} disabled={scanning} onClick={refresh}>
{scanning ? (
<Spinner style={{ height: "1em", marginRight: "0.5em" }} />
) : (
<FaSyncAlt style={{ marginRight: "0.5em" }} />
)}
{scanning ? "Scanning…" : "Refresh"}
</DialogButton>
{update?.update_available && (
<DialogButton style={{ minWidth: "9em" }} onClick={() => applyUpdate(update)}>
<FaDownload style={{ marginRight: "0.4em" }} />
Update v{update.latest}
</DialogButton>
)}
</Focusable>
<div style={{ fontSize: "1.1em", fontWeight: "bold", margin: "0.5em 0" }}>Hosts</div>
{hosts.length === 0 && !scanning && (
<Field focusable={false}>No hosts discovered on the LAN.</Field>
)}
{hosts.map((h) => (
<HostRow key={h.fp || `${h.host}:${h.port}`} host={h} />
))}
<div style={{ fontSize: "1.1em", fontWeight: "bold", margin: "1.5em 0 0.5em" }}>
Stream settings
<div style={{ flex: 1, minHeight: 0 }}>
<Tabs
activeTab={tab}
onShowTab={(id: string) => setTab(id)}
autoFocusContents
tabs={[
{
id: "hosts",
title: "Hosts",
content: <HostsTab hosts={hosts} scanning={scanning} refresh={refresh} />,
},
{
id: "settings",
title: "Settings",
content: <SettingsTab />,
},
]}
/>
</div>
<SettingsSection />
</div>
);
};
@@ -343,9 +545,25 @@ const PunktfunkPage: FC = () => {
// ----------------------------------------------------------------------------------------
const QamPanel: FC = () => {
const { hosts, scanning, refresh } = useHosts();
const update = useUpdate();
return (
<>
{update?.update_available && (
<PanelSection title="Update">
<PanelSectionRow>
<ButtonItem
layout="below"
onClick={() => applyUpdate(update)}
label={`v${update.current} → v${update.latest}`}
>
<FaDownload style={{ marginRight: "0.5em" }} />
Update punktfunk
</ButtonItem>
</PanelSectionRow>
</PanelSection>
)}
<PanelSection title="punktfunk">
<PanelSectionRow>
<ButtonItem
@@ -378,25 +596,25 @@ const QamPanel: FC = () => {
</PanelSectionRow>
)}
{hosts.map((h) => {
const pairRequired = h.pair === "required";
const needsPair = h.pair === "required" && !h.paired;
return (
<PanelSectionRow key={h.fp || `${h.host}:${h.port}`}>
<ButtonItem
layout="below"
onClick={() =>
pairRequired
needsPair
? showModal(<PairModal host={h} onPaired={() => startStream(h)} />)
: startStream(h)
}
label={
<span style={{ display: "inline-flex", alignItems: "center", gap: "0.4em" }}>
{pairRequired ? <FaLock /> : <FaLockOpen />}
{needsPair ? <FaLock /> : <FaLockOpen />}
{h.name}
</span>
}
description={`${h.host}:${h.port}`}
description={`${h.host}:${h.port}${h.paired ? " · paired" : ""}`}
>
{pairRequired ? "Pair & Stream" : "Stream"}
{needsPair ? "Pair & Stream" : "Stream"}
</ButtonItem>
</PanelSectionRow>
);
@@ -406,12 +624,25 @@ const QamPanel: FC = () => {
);
};
// Full page behind the boundary — registered as the /punktfunk route.
const PunktfunkRoute: FC = () => (
<PluginErrorBoundary>
<PunktfunkPage />
</PluginErrorBoundary>
);
export default definePlugin(() => {
routerHook.addRoute(ROUTE, PunktfunkPage, { exact: true });
routerHook.addRoute(ROUTE, PunktfunkRoute, { exact: true });
return {
name: "punktfunk",
titleView: <div className={staticClasses.Title}>punktfunk</div>,
content: <QamPanel />,
// `staticClasses?.Title` is guarded so a future client that drops the export can't throw
// at plugin-load time (an error boundary only catches render-time, not load-time, errors).
titleView: <div className={staticClasses?.Title}>punktfunk</div>,
content: (
<PluginErrorBoundary>
<QamPanel />
</PluginErrorBoundary>
),
icon: <FaTv />,
onDismount() {
routerHook.removeRoute(ROUTE);
+22 -2
View File
@@ -24,12 +24,31 @@ declare const SteamClient: {
SetShortcutExe(appId: number, exe: string): void;
SetShortcutStartDir(appId: number, dir: string): void;
SetAppLaunchOptions(appId: number, options: string): void;
SetAppHidden(appId: number, hidden: boolean): void;
RunGame(gameId: string, _unused: string, _i: number, _j: number): void;
TerminateApp(gameId: string, _b: boolean): void;
};
};
// Steam removed `SteamClient.Apps.SetAppHidden`. Hiding a non-Steam shortcut now goes through
// `collectionStore.SetAppsAsHidden([appId], true)` — but that looks the app up in appStore, which
// only registers a freshly-created shortcut a moment later (calling it immediately throws on a
// null overview). So hiding is BEST-EFFORT + DEFERRED and must NEVER block the launch.
declare const collectionStore:
| { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void }
| undefined;
function hideShortcut(appId: number): void {
const attempt = () => {
try {
collectionStore?.SetAppsAsHidden?.([appId], true);
} catch {
/* overview not registered yet, or the API changed — cosmetic, ignore */
}
};
attempt(); // succeeds immediately for an already-registered (reused) shortcut
setTimeout(attempt, 2500); // fresh shortcut: retry once its app overview lands
}
const SHORTCUT_NAME = "punktfunk";
// The 64-bit "gameid" RunGame wants, derived from a 32-bit non-Steam shortcut appId: the
@@ -88,7 +107,8 @@ async function ensureShortcut(): Promise<number> {
);
SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME);
// Hide it from the library — it's an implementation detail, launched programmatically.
SteamClient.Apps.SetAppHidden(appId, true);
// Best-effort + deferred (see hideShortcut); never let it block the launch.
hideShortcut(appId);
rememberAppId(appId);
return appId;
}
+95 -2
View File
@@ -22,6 +22,8 @@ struct App {
gamepad: crate::gamepad::GamepadService,
/// One session at a time — ignore connects while one is starting/running.
busy: std::cell::Cell<bool>,
/// Steam Deck / Gaming-Mode launch: fullscreen the window (chrome-less) when a stream starts.
fullscreen: bool,
}
impl App {
@@ -41,7 +43,13 @@ pub fn run() -> glib::ExitCode {
if let Some(pin) = arg_value("--pair") {
return headless_pair(&pin);
}
let app = adw::Application::builder().application_id(APP_ID).build();
let mut builder = adw::Application::builder().application_id(APP_ID);
// Screenshot mode launches the app once per scene back-to-back; NON_UNIQUE keeps each
// launch its own primary instance instead of forwarding to a still-registered name.
if shot_scene().is_some() {
builder = builder.flags(gtk::gio::ApplicationFlags::NON_UNIQUE);
}
let app = builder.build();
app.connect_activate(build_ui);
// GTK doesn't see our argv (`--connect` is handled in `build_ui`); an empty argv also
// keeps GApplication from rejecting unknown options.
@@ -56,6 +64,20 @@ fn arg_value(flag: &str) -> Option<String> {
.filter(|v| !v.starts_with("--"))
}
/// True if argv contains `flag` (a valueless switch).
fn arg_flag(flag: &str) -> bool {
std::env::args().any(|a| a == flag)
}
/// Run the stream fullscreen with no window chrome — the Steam Deck / Gaming-Mode launch path.
/// The Decky wrapper passes `--fullscreen`; we also honor the Deck/gamescope env as a fallback
/// so a manual launch under Gaming Mode does the right thing too.
fn fullscreen_mode() -> bool {
arg_flag("--fullscreen")
|| std::env::var_os("SteamDeck").is_some()
|| std::env::var_os("GAMESCOPE_WAYLAND_DISPLAY").is_some()
}
/// Run the SPAKE2 PIN ceremony without a GTK window and persist the verified host to the
/// known-hosts store as paired, so a later `--connect` connects silently. Same identity
/// store the streaming path uses (same binary), so pairing here makes the stream work.
@@ -161,6 +183,7 @@ fn build_ui(gtk_app: &adw::Application) {
identity,
gamepad: crate::gamepad::GamepadService::start(),
busy: std::cell::Cell::new(false),
fullscreen: fullscreen_mode(),
});
let hosts_page = crate::ui_hosts::new(
@@ -182,11 +205,65 @@ fn build_ui(gtk_app: &adw::Application) {
nav.add(&hosts_page);
window.present();
// CI screenshot mode: render one scripted, host-free scene and signal readiness
// (clients/linux/tools/screenshots.sh). Mutually exclusive with a real connect.
if let Some(scene) = shot_scene() {
run_shot(app, &scene);
return;
}
if let Some(req) = cli_connect_request() {
initiate_connect(app, req);
}
}
/// `PUNKTFUNK_SHOT_SCENE`, when set, selects a scripted host-free scene for CI screenshots.
fn shot_scene() -> Option<String> {
std::env::var("PUNKTFUNK_SHOT_SCENE")
.ok()
.filter(|s| !s.is_empty())
}
/// Render one mock-populated, host-free scene over the already-presented window, then print
/// `PF_SHOT_READY` once it has had a moment to map + settle so the driver knows when to capture.
/// No `NativeClient` or session is created. The stream scene is deliberately absent — its page
/// requires a live connector (`ui_stream::new` takes an `Arc<NativeClient>`).
fn run_shot(app: Rc<App>, scene: &str) {
// A plausible host for the trust/pair dialogs (fp_hex is 64 hex chars, like a real SHA-256).
let mock_req = || ConnectRequest {
name: "Living Room PC".to_string(),
addr: "192.168.1.42".to_string(),
port: 9777,
fp_hex: Some(
"9f8e7d6c5b4a39281706f5e4d3c2b1a0998877665544332211ffeeddccbbaa00".to_string(),
),
pair_optional: true,
};
match scene {
// The saved-hosts grid reads ~/.config/punktfunk/client-known-hosts.json, which the
// driver seeds — so the already-shown hosts page is the scene; nothing to do here.
"hosts" | "02-hosts" => {}
"settings" | "03-settings" => {
crate::ui_settings::show(&app.window, app.settings.clone(), &app.gamepad);
}
"trust" | "04-trust" => tofu_dialog(app.clone(), mock_req()),
"pair" | "05-pair" => pin_dialog(app.clone(), mock_req()),
other => tracing::warn!("unknown PUNKTFUNK_SHOT_SCENE={other:?}; showing hosts only"),
}
let settle_ms = std::env::var("PUNKTFUNK_SHOT_SETTLE_MS")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(900);
let scene = scene.to_string();
glib::timeout_add_local_once(std::time::Duration::from_millis(settle_ms), move || {
use std::io::Write as _;
println!("PF_SHOT_READY scene={scene}");
let _ = std::io::stdout().flush();
});
}
/// The trust gate in front of every connect. The host is the policy authority (it
/// advertises `pair=optional` only when it accepts unpaired clients); the client renders
/// its trust UI from that:
@@ -375,6 +452,7 @@ fn speed_test(app: Rc<App>, req: ConnectRequest) {
GamepadPref::Auto,
0, // bitrate_kbps (host default)
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
2, // audio_channels: speed-test probe, stereo
None, // launch: speed-test probe connect, no game
pin,
Some(identity),
@@ -443,11 +521,19 @@ fn resolve_mode(app: &App) -> punktfunk_core::config::Mode {
refresh_hz: s.refresh_hz,
};
if mode.width == 0 || mode.refresh_hz == 0 {
// Prefer the monitor the window is on; fall back to the display's first monitor. On a
// `--connect` launch the window may not be mapped yet when this runs, and without the
// fallback we'd drop to the 1920×1080 floor below — wrong on the Deck (1280×800).
let monitor = app
.window
.surface()
.zip(gdk::Display::default())
.and_then(|(surf, d)| d.monitor_at_surface(&surf));
.and_then(|(surf, d)| d.monitor_at_surface(&surf))
.or_else(|| {
gdk::Display::default()
.and_then(|d| d.monitors().item(0))
.and_then(|o| o.downcast::<gdk::Monitor>().ok())
});
if let Some(m) = monitor {
let geo = m.geometry();
let scale = m.scale_factor().max(1);
@@ -488,6 +574,7 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
},
bitrate_kbps: s.bitrate_kbps,
mic_enabled: s.mic_enabled,
audio_channels: s.audio_channels,
pin,
identity: app.identity.clone(),
};
@@ -540,6 +627,12 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
&title,
);
app.nav.push(&p.page);
// Steam Deck / Gaming Mode: gamescope fullscreens the window but GTK doesn't
// know it, so its header bar stays drawn. Enter GTK fullscreen explicitly —
// the stream page's `connect_fullscreened_notify` then hides all chrome.
if app.fullscreen {
app.window.fullscreen();
}
page = Some(p);
}
SessionEvent::Stats(s) => {
+21 -10
View File
@@ -27,16 +27,17 @@ pub struct AudioPlayer {
}
impl AudioPlayer {
/// Spawn the PipeWire playback thread. Failure (no PipeWire in the session) is
/// survivable — the caller streams video-only.
pub fn spawn() -> Result<AudioPlayer> {
/// Spawn the PipeWire playback thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no PipeWire in the session) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u32) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the PipeWire loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = pw_thread(pcm_rx, quit_rx) {
if let Err(e) = pw_thread(pcm_rx, quit_rx, channels as usize) {
tracing::warn!(error = %e, "audio playback thread ended");
}
})
@@ -48,8 +49,8 @@ impl AudioPlayer {
})
}
/// Queue one interleaved-stereo f32 chunk. Drops the chunk if the PipeWire side is
/// wedged (the renderer conceals the gap; never block the session pump).
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// PipeWire side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
@@ -71,11 +72,14 @@ struct PlayerData {
rx: Receiver<Vec<f32>>,
ring: VecDeque<f32>,
primed: bool,
/// Interleaved channel count this stream was opened with (2/6/8).
channels: usize,
}
fn pw_thread(
pcm_rx: Receiver<Vec<f32>>,
quit_rx: pipewire::channel::Receiver<Terminate>,
channels: usize,
) -> Result<()> {
use pipewire as pw;
use pw::{properties::properties, spa};
@@ -115,6 +119,7 @@ fn pw_thread(
rx: pcm_rx,
ring: VecDeque::new(),
primed: false,
channels,
};
let _listener = stream
@@ -130,19 +135,19 @@ fn pw_thread(
while let Ok(chunk) = ud.rx.try_recv() {
ud.ring.extend(chunk);
}
let stride = 4 * CHANNELS; // F32LE interleaved
let stride = 4 * ud.channels; // F32LE interleaved
let datas = buffer.datas_mut();
if datas.is_empty() {
return;
}
let data = &mut datas[0];
let want_frames = data.data().map(|s| s.len() / stride).unwrap_or(0);
let want = want_frames * CHANNELS;
let want = want_frames * ud.channels;
// Adaptive jitter buffer (same shape as the host's virtual mic): prime to
// ~3 quanta, cap at ~1 quantum of slack beyond that, re-prime after a
// genuine drain.
let target = (3 * want).clamp(720 * CHANNELS, 9600 * CHANNELS);
let target = (3 * want).clamp(720 * ud.channels, 9600 * ud.channels);
while ud.ring.len() > target.max(want) + want {
ud.ring.pop_front();
}
@@ -182,7 +187,13 @@ fn pw_thread(
let mut info = AudioInfoRaw::new();
info.set_format(AudioFormat::F32LE);
info.set_rate(SAMPLE_RATE);
info.set_channels(CHANNELS as u32);
info.set_channels(channels as u32);
// Channel positions in canonical wire order (FL FR FC LFE RL RR SL SR) so PipeWire routes each
// slot to the matching speaker (and downmixes when the sink has fewer). Identity, no permute.
let order = punktfunk_core::audio::spa_positions(channels as u8);
let mut positions = [0u32; 64];
positions[..order.len()].copy_from_slice(order);
info.set_position(positions);
let obj = pw::spa::pod::Object {
type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(),
id: pw::spa::param::ParamType::EnumFormat.as_raw(),
+50 -7
View File
@@ -20,6 +20,8 @@ pub struct SessionParams {
pub compositor: CompositorPref,
pub gamepad: GamepadPref,
pub bitrate_kbps: u32,
/// Requested audio channel count (2/6/8); the host echoes the resolved value.
pub audio_channels: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Pinned host fingerprint; `None` = trust on first use (caller persists the observed one).
@@ -83,6 +85,42 @@ fn now_ns() -> u64 {
.unwrap_or(0)
}
/// Opus decoder for the audio plane: a plain stereo decoder (the validated path) or a multistream
/// decoder for 5.1/7.1, both behind one `decode_float`. Built from the host-RESOLVED channel count
/// via the shared layout table.
enum AudioDec {
Stereo(opus::Decoder),
Surround(opus::MSDecoder),
}
impl AudioDec {
fn new(channels: u8) -> Result<AudioDec, opus::Error> {
if channels == 2 {
Ok(AudioDec::Stereo(opus::Decoder::new(
48_000,
opus::Channels::Stereo,
)?))
} else {
let l = punktfunk_core::audio::layout_for(channels, false);
Ok(AudioDec::Surround(opus::MSDecoder::new(
48_000, l.streams, l.coupled, l.mapping,
)?))
}
}
fn decode_float(
&mut self,
input: &[u8],
out: &mut [f32],
fec: bool,
) -> Result<usize, opus::Error> {
match self {
AudioDec::Stereo(d) => d.decode_float(input, out, fec),
AudioDec::Surround(d) => d.decode_float(input, out, fec),
}
}
}
fn pump(
params: SessionParams,
ev_tx: async_channel::Sender<SessionEvent>,
@@ -96,7 +134,8 @@ fn pump(
params.compositor,
params.gamepad,
params.bitrate_kbps,
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
params.audio_channels,
None, // launch: the Linux client has no library picker yet
params.pin,
Some(params.identity),
@@ -134,11 +173,14 @@ fn pump(
}
};
// Audio is best-effort: a session without it still streams. Gamepads are the
// app-lifetime service's job (the UI attaches it on Connected).
let player = audio::AudioPlayer::spawn()
// app-lifetime service's job (the UI attaches it on Connected). Build the decoder + playback
// from the host-RESOLVED channel count (never the request), so an older/clamping host that
// resolves stereo is decoded as stereo.
let channels = connector.audio_channels;
let player = audio::AudioPlayer::spawn(channels as u32)
.map_err(|e| tracing::warn!(error = %e, "audio disabled"))
.ok();
let mut opus_dec = opus::Decoder::new(48_000, opus::Channels::Stereo)
let mut opus_dec = AudioDec::new(channels)
.map_err(|e| tracing::warn!(error = %e, "opus decoder failed — audio disabled"))
.ok();
let _mic = params
@@ -157,8 +199,8 @@ fn pump(
let mut bytes_n = 0u64;
let mut decode_us_sum = 0u64;
let mut lat_us: Vec<u64> = Vec::with_capacity(256);
let mut pcm = vec![0f32; 5760 * 2]; // decode scratch: max Opus frame (120 ms stereo)
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut pcm = vec![0f32; 5760 * channels as usize]; // scratch: max Opus frame (120 ms) × channels
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
@@ -221,7 +263,8 @@ fn pump(
while let Ok(pkt) = connector.next_audio(Duration::ZERO) {
if let (Some(player), Some(dec)) = (&player, opus_dec.as_mut()) {
match dec.decode_float(&pkt.data, &mut pcm, false) {
Ok(samples) => player.push(pcm[..samples * 2].to_vec()),
// `samples` is per-channel; the interleaved frame is `samples * channels`.
Ok(samples) => player.push(pcm[..samples * channels as usize].to_vec()),
Err(e) => tracing::debug!(error = %e, "opus decode"),
}
}
+12
View File
@@ -90,6 +90,14 @@ impl KnownHosts {
self.hosts.iter().find(|h| h.addr == addr && h.port == port)
}
/// Forget the entry with this fingerprint. Returns true if one was removed (the user
/// will have to pair/trust again to reconnect).
pub fn remove_by_fp(&mut self, fp_hex: &str) -> bool {
let before = self.hosts.len();
self.hosts.retain(|h| h.fp_hex != fp_hex);
self.hosts.len() != before
}
/// Insert or refresh an entry, keyed by fingerprint. `paired` only ever upgrades
/// (a later TOFU connect must not demote a PIN-paired host).
pub fn upsert(&mut self, entry: KnownHost) {
@@ -124,6 +132,9 @@ pub struct Settings {
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + playback layout.
pub audio_channels: u8,
}
impl Default for Settings {
@@ -137,6 +148,7 @@ impl Default for Settings {
compositor: "auto".into(),
inhibit_shortcuts: true,
mic_enabled: false,
audio_channels: 2,
}
}
}
+46
View File
@@ -181,6 +181,52 @@ pub fn new(
// pinned connect; TOFU eligibility is irrelevant.
pair_optional: false,
};
// Forget this host (drops the pinned fingerprint — a later connect re-pairs).
// Confirmed first, since it's destructive and a misclick on the Deck is easy.
let remove_btn = gtk::Button::from_icon_name("user-trash-symbolic");
remove_btn.set_tooltip_text(Some("Remove saved host"));
remove_btn.set_valign(gtk::Align::Center);
remove_btn.add_css_class("flat");
{
let fp = k.fp_hex.clone();
let name = k.name.clone();
let saved_list = saved_list.clone();
let saved_label = saved_label.clone();
let row = row.clone();
remove_btn.connect_clicked(move |_| {
let dialog = adw::AlertDialog::new(
Some("Remove saved host?"),
Some(&format!(
"Forget “{name}”? You'll need to pair (or trust) it again to reconnect."
)),
);
dialog.add_responses(&[("cancel", "Cancel"), ("remove", "Remove")]);
dialog.set_response_appearance(
"remove",
adw::ResponseAppearance::Destructive,
);
dialog.set_default_response(Some("cancel"));
dialog.set_close_response("cancel");
{
// Scoped clones for the response handler so `row` survives for present().
let fp = fp.clone();
let saved_list = saved_list.clone();
let saved_label = saved_label.clone();
let row = row.clone();
dialog.connect_response(Some("remove"), move |_, _| {
let mut known = KnownHosts::load();
known.remove_by_fp(&fp);
let _ = known.save();
saved_list.remove(&row);
let empty = known.hosts.is_empty();
saved_list.set_visible(!empty);
saved_label.set_visible(!empty);
});
}
dialog.present(Some(&row));
});
}
row.add_suffix(&remove_btn);
let speed_btn = gtk::Button::from_icon_name("network-transmit-receive-symbolic");
speed_btn.set_tooltip_text(Some("Test network speed"));
speed_btn.set_valign(gtk::Align::Center);
+20
View File
@@ -140,6 +140,16 @@ pub fn show(
input.add(&inhibit_row);
let audio = adw::PreferencesGroup::builder().title("Audio").build();
let surround_row = adw::ComboRow::builder()
.title("Audio channels")
.subtitle("Request stereo or surround (the host downmixes if its output has fewer)")
.model(&gtk::StringList::new(&[
"Stereo",
"5.1 Surround",
"7.1 Surround",
]))
.build();
audio.add(&surround_row);
let mic_row = adw::SwitchRow::builder()
.title("Stream microphone")
.subtitle("Send the default input device to the host's virtual microphone")
@@ -170,6 +180,11 @@ pub fn show(
compositor_row.set_selected(comp_i as u32);
inhibit_row.set_active(s.inhibit_shortcuts);
mic_row.set_active(s.mic_enabled);
surround_row.set_selected(match s.audio_channels {
6 => 1,
8 => 2,
_ => 0,
});
}
let dialog = adw::PreferencesDialog::new();
@@ -186,6 +201,11 @@ pub fn show(
.to_string();
s.inhibit_shortcuts = inhibit_row.is_active();
s.mic_enabled = mic_row.is_active();
s.audio_channels = match surround_row.selected() {
1 => 6,
2 => 8,
_ => 2,
};
s.save();
});
dialog.present(Some(parent));
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env bash
# Capture host-free UI screenshots of the native Linux client under a virtual X
# display. Mirrors the iOS harness (clients/apple/tools/screenshots.sh): one app
# launch per scene (PUNKTFUNK_SHOT_SCENE), the app renders a mock-populated REAL
# view and prints `PF_SHOT_READY`, then we grab the X root window. No host, GPU, or
# live stream — only the chrome scenes (the stream page needs a live connector).
#
# cargo build --release -p punktfunk-client-linux
# bash clients/linux/tools/screenshots.sh # → clients/linux/screenshots/<scene>.png
# bash clients/linux/tools/screenshots.sh hosts pair # a subset
#
# Env knobs: BIN (client binary), OUT (output dir), GEOMETRY (Xvfb WxHxDepth),
# SETTLE (extra seconds after PF_SHOT_READY), SHOT_DISPLAY (X display), GSK_RENDERER
# (gl|ngl|cairo — gl/llvmpipe by default for full libadwaita fidelity).
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" # clients/linux
BIN="${BIN:-$here/../../target/release/punktfunk-client}"
OUT="${OUT:-$here/screenshots}"
# The client window maps at its 1100x720 default; with no WM under Xvfb it lands at the
# top-left, so keep the root just larger so the full window (incl. its CSD shadow) is
# captured by a root grab with only a thin margin to crop.
GEOMETRY="${GEOMETRY:-1280x800x24}"
SETTLE="${SETTLE:-1.2}"
SHOT_DISPLAY="${SHOT_DISPLAY:-:99}"
if [ "$#" -gt 0 ]; then SCENES=("$@"); else SCENES=(hosts settings trust pair); fi
[ -x "$BIN" ] || {
echo "client binary not found: $BIN (build it first: cargo build --release -p punktfunk-client-linux)" >&2
exit 1
}
# Isolated scratch HOME: the client generates its identity here on first run, and the
# saved-hosts grid is read from client-known-hosts.json, so seed mock hosts for the
# `hosts` scene (the dialogs/settings build their own mock state in-app).
WORK="$(mktemp -d)"
export HOME="$WORK"
mkdir -p "$HOME/.config/punktfunk"
cat >"$HOME/.config/punktfunk/client-known-hosts.json" <<'JSON'
{
"hosts": [
{ "name": "Living Room PC", "addr": "192.168.1.42", "port": 9777,
"fp_hex": "9f8e7d6c5b4a39281706f5e4d3c2b1a0998877665544332211ffeeddccbbaa00",
"paired": true },
{ "name": "Office", "addr": "192.168.1.50", "port": 9777,
"fp_hex": "a1b2c3d4e5f60718293a4b5c6d7e8f90112233445566778899aabbccddeeff00",
"paired": false }
]
}
JSON
# Software-rendered X session — no GPU/Wayland. GL/llvmpipe runs the real NGL renderer
# (cairo is documented-incomplete for 3D-transformed content / libadwaita transitions).
unset WAYLAND_DISPLAY
export DISPLAY="$SHOT_DISPLAY"
export GDK_BACKEND=x11
export LIBGL_ALWAYS_SOFTWARE=1
export GALLIUM_DRIVER="${GALLIUM_DRIVER:-llvmpipe}"
export GSK_RENDERER="${GSK_RENDERER:-gl}"
Xvfb "$SHOT_DISPLAY" -screen 0 "$GEOMETRY" -nolisten tcp >"$WORK/xvfb.log" 2>&1 &
XVFB_PID=$!
cleanup() {
kill "$XVFB_PID" 2>/dev/null || true
rm -rf "$WORK"
}
trap cleanup EXIT
# Wait for the display to accept connections.
for _ in $(seq 1 50); do
if command -v xdpyinfo >/dev/null 2>&1; then
xdpyinfo -display "$SHOT_DISPLAY" >/dev/null 2>&1 && break
else
[ -e "/tmp/.X11-unix/X${SHOT_DISPLAY#:}" ] && break
fi
sleep 0.1
done
capture() {
local out="$1"
if command -v import >/dev/null 2>&1; then
import -silent -window root "$out"
elif command -v scrot >/dev/null 2>&1; then
scrot -o "$out"
else
echo "no screenshot tool — install imagemagick or scrot" >&2
return 1
fi
}
mkdir -p "$OUT"
rc=0
for scene in "${SCENES[@]}"; do
: >"$WORK/log"
PUNKTFUNK_SHOT_SCENE="$scene" "$BIN" >"$WORK/log" 2>&1 &
pid=$!
ready=0
for _ in $(seq 1 200); do # up to ~20s
if grep -q "PF_SHOT_READY" "$WORK/log"; then
ready=1
break
fi
if ! kill -0 "$pid" 2>/dev/null; then break; fi
sleep 0.1
done
if [ "$ready" = 1 ]; then
sleep "$SETTLE"
if capture "$OUT/$scene.png"; then
echo "$scene$OUT/$scene.png"
else
rc=1
fi
else
echo "$scene: client never signalled PF_SHOT_READY" >&2
sed 's/^/ /' "$WORK/log" >&2 || true
rc=1
fi
kill "$pid" 2>/dev/null || true
wait "$pid" 2>/dev/null || true
done
exit "$rc"
+3 -4
View File
@@ -18,8 +18,7 @@ tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# LAN host discovery (`--discover`): browse the native `_punktfunk._udp` mDNS service the host
# advertises (same crate/version the host advertises with).
mdns-sd = "0.20"
# Linux-only: --mic-test's Opus encoder (libopus). The mic UPLINK itself is portable —
# only this synthetic-tone test rig needs the encoder.
[target.'cfg(target_os = "linux")'.dependencies]
# Opus: multistream DECODE of the host's audio plane (the surround validator) + `--mic-test`'s
# encoder. libopus is already in the graph via `punktfunk-core`'s quic feature; this exposes the
# name directly. Cross-platform (cmake-vendored), so the probe builds + validates everywhere.
opus = "0.3"
+51 -6
View File
@@ -78,6 +78,10 @@ struct Args {
gamepad: GamepadPref,
/// `--bitrate KBPS` — request this encoder bitrate (kilobits/s); 0 = host default.
bitrate_kbps: u32,
/// `--audio-channels N` — request stereo (2), 5.1 (6) or 7.1 (8) audio; default 2. The probe
/// multistream-decodes the host's frames and asserts the per-channel sample count, so it's the
/// headless validator for the surround encode path.
audio_channels: u8,
/// `--launch ID` — ask the host to launch a library title in this session (a store-qualified
/// id from the host's `GET /api/v1/library`, e.g. `steam:570`). Host resolves it; `None` = none.
launch: Option<String>,
@@ -201,6 +205,11 @@ fn parse_args() -> Args {
compositor,
gamepad,
bitrate_kbps: get("--bitrate").and_then(|s| s.parse().ok()).unwrap_or(0),
audio_channels: punktfunk_core::audio::normalize_channels(
get("--audio-channels")
.and_then(|s| s.parse().ok())
.unwrap_or(2),
),
launch: get("--launch").map(str::to_string),
speed_test: get("--speed-test").and_then(|s| {
let (kbps, ms) = s.split_once(':')?;
@@ -385,13 +394,23 @@ async fn session(args: Args) -> Result<()> {
// `--launch ID` — host resolves it against its own library and runs it this session.
launch: args.launch.clone(),
// This headless tool just dumps the bitstream (no decode), so it can always claim
// 10-bit support. Gated by env so latency runs stay on the 8-bit baseline:
// PUNKTFUNK_CLIENT_10BIT=1 advertises VIDEO_CAP_10BIT to exercise the host Main10 path.
video_caps: if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() {
punktfunk_core::quic::VIDEO_CAP_10BIT
} else {
0
// 10-bit / 4:4:4 support. Gated by env so latency runs stay on the 8-bit 4:2:0 baseline:
// PUNKTFUNK_CLIENT_10BIT=1 advertises VIDEO_CAP_10BIT (host Main10 path);
// PUNKTFUNK_CLIENT_444=1 advertises VIDEO_CAP_444 (host HEVC 4:4:4 path) — verify the
// resulting chroma with `ffprobe` on the `--out` .h265.
video_caps: {
let mut caps = 0u8;
if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_10BIT;
}
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444;
}
caps
},
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the
// host's frames to exercise the surround encode path headlessly.
audio_channels: args.audio_channels,
}
.encode(),
)
@@ -408,6 +427,8 @@ async fn session(args: Args) -> Result<()> {
bit_depth = welcome.bit_depth,
color = ?welcome.color,
hdr = welcome.color.is_hdr(),
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format,
"session offer"
);
@@ -830,13 +851,37 @@ async fn session(args: Args) -> Result<()> {
hidout_pkts.clone(),
);
let conn2 = conn.clone();
// Build a multistream decoder for the host-RESOLVED layout so the probe actually decodes
// the surround stream (not just counts bytes) — the headless validator for the encode path.
let audio_channels = welcome.audio_channels;
tokio::spawn(async move {
use std::sync::atomic::Ordering::Relaxed;
let mut hdr_logged = false;
let layout = punktfunk_core::audio::layout_for(audio_channels, false);
let mut audio_dec =
opus::MSDecoder::new(48_000, layout.streams, layout.coupled, layout.mapping).ok();
let mut pcm = vec![0f32; 5760 * audio_channels as usize];
let mut audio_decoded_logged = false;
while let Ok(d) = conn2.read_datagram().await {
if let Some((_, _, opus)) = punktfunk_core::quic::decode_audio_datagram(&d) {
a.fetch_add(1, Relaxed);
ab.fetch_add(opus.len() as u64, Relaxed);
// Decode + validate: the per-channel sample count must be a legal Opus frame
// size; log the first success so a loopback test can assert surround decoded.
if let Some(dec) = audio_dec.as_mut() {
match dec.decode_float(opus, &mut pcm, false) {
Ok(samples) if !audio_decoded_logged => {
audio_decoded_logged = true;
tracing::info!(
channels = audio_channels,
samples_per_channel = samples,
"audio decoded (Opus multistream)"
);
}
Ok(_) => {}
Err(e) => tracing::debug!(error = %e, "probe audio decode"),
}
}
} else if punktfunk_core::quic::decode_rumble_datagram(&d).is_some() {
r.fetch_add(1, Relaxed);
} else if let Some(meta) = punktfunk_core::quic::decode_hdr_meta_datagram(&d) {
+32 -2
View File
@@ -39,6 +39,9 @@ const DECODERS: &[(&str, &str)] = &[
];
/// Bitrate presets in Mb/s; `0` = host default.
const BITRATES_MBPS: &[u32] = &[0, 10, 20, 30, 50, 80, 150];
/// Audio channel presets: `(channel count, display label)`. The host clamps to what it can
/// capture; the resolved count drives the decoder + WASAPI render layout.
const AUDIO_CHANNELS: &[(u8, &str)] = &[(2, "Stereo"), (6, "5.1 Surround"), (8, "7.1 Surround")];
#[derive(Clone, PartialEq)]
enum Screen {
@@ -598,6 +601,7 @@ fn connect(
compositor: CompositorPref::Auto,
gamepad: gamepad_pref,
bitrate_kbps: s.bitrate_kbps,
audio_channels: s.audio_channels,
mic_enabled: s.mic_enabled,
hdr_enabled: s.hdr_enabled,
decoder: DecoderPref::from_name(&s.decoder),
@@ -886,6 +890,23 @@ fn settings_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Eleme
s.save();
})
};
let ac_i = AUDIO_CHANNELS
.iter()
.position(|&(v, _)| v == s.audio_channels)
.unwrap_or(0) as i32;
let ac_names: Vec<String> = AUDIO_CHANNELS.iter().map(|&(_, l)| l.to_string()).collect();
let channels_combo = {
let ctx = ctx.clone();
ComboBox::new(ac_names)
.header("Audio channels")
.selected_index(ac_i)
.on_selection_changed(move |i: i32| {
let (v, _) = AUDIO_CHANNELS[(i.max(0) as usize).min(AUDIO_CHANNELS.len() - 1)];
let mut s = ctx.settings.lock().unwrap();
s.audio_channels = v;
s.save();
})
};
let header = grid((
text_block("Settings")
@@ -934,8 +955,17 @@ fn settings_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Eleme
.spacing(10.0),
);
let audio_card =
card(vstack((text_block("Audio").font_size(15.0).semibold(), mic_toggle)).spacing(10.0));
let audio_card = card(
vstack((
text_block("Audio").font_size(15.0).semibold(),
text_block("Request stereo or surround — the host downmixes if its output has fewer.")
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
channels_combo,
mic_toggle,
))
.spacing(10.0),
);
page(vec![
header.into(),
+28 -12
View File
@@ -21,9 +21,9 @@ use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
const SAMPLE_RATE: usize = 48_000;
/// The microphone uplink stays stereo (the host's virtual mic is stereo). The render path is
/// multichannel — its channel count + block align are runtime, driven by the host-resolved layout.
const CHANNELS: usize = 2;
/// 48 kHz stereo f32: 2 channels * 4 bytes = 8 bytes per frame.
const BLOCK_ALIGN: usize = CHANNELS * 4;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
@@ -34,9 +34,10 @@ pub struct AudioPlayer {
}
impl AudioPlayer {
/// Spawn the WASAPI render thread. Failure (no render endpoint on this box) is
/// survivable — the caller streams video-only.
pub fn spawn() -> Result<AudioPlayer> {
/// Spawn the WASAPI render thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no render endpoint on this box) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u8) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the WASAPI loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
@@ -45,14 +46,14 @@ impl AudioPlayer {
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, stop_t, ready_tx) {
if let Err(e) = render_thread(pcm_rx, stop_t, ready_tx, channels) {
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!("WASAPI render: 48 kHz stereo f32 (default endpoint)");
tracing::info!(channels, "WASAPI render: 48 kHz f32 (default endpoint)");
Ok(AudioPlayer {
pcm_tx,
stop,
@@ -66,8 +67,8 @@ impl AudioPlayer {
}
}
/// Queue one interleaved-stereo f32 chunk. Drops the chunk if the WASAPI side is wedged
/// (the renderer conceals the gap; never block the session pump).
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// WASAPI side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
@@ -88,6 +89,7 @@ fn render_thread(
pcm_rx: Receiver<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
channels: u8,
) -> Result<()> {
if let Err(e) = wasapi::initialize_mta()
.ok()
@@ -97,12 +99,26 @@ fn render_thread(
return Ok(());
}
let res = (|| -> Result<()> {
// F32LE interleaved: channels × 4 bytes/sample. Stereo (channels == 2) is byte-identical
// to the old fixed path (mask 0x3, block align 8).
let block_align = channels as usize * 4;
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
// The explicit dwChannelMask is the wire order (FL FR FC LFE RL RR SL SR); 5.1 = 0x3F,
// 7.1 = 0x63F. WASAPI delivers channels in ascending mask-bit order, which equals the wire
// order, so the render mapping is the identity — no permute. `autoconvert` (below) lets the
// audio engine downmix when the endpoint has fewer speakers.
let desired = WaveFormat::new(
32,
32,
&SampleType::Float,
SAMPLE_RATE,
channels as usize,
Some(punktfunk_core::audio::wasapi_channel_mask(channels)),
);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
@@ -139,10 +155,10 @@ fn render_thread(
if avail_frames == 0 {
continue;
}
let want_bytes = avail_frames * BLOCK_ALIGN;
let want_bytes = avail_frames * block_align;
// Prime to ~3 quanta; cap at ~1 quantum of slack beyond that; re-prime on drain.
let target = (3 * want_bytes).clamp(720 * BLOCK_ALIGN, 9600 * BLOCK_ALIGN);
let target = (3 * want_bytes).clamp(720 * block_align, 9600 * block_align);
while ring.len() > target.max(want_bytes) + want_bytes {
ring.pop_front();
}
+2
View File
@@ -177,6 +177,8 @@ fn run_headless_cli(args: &[String], identity: (String, String)) {
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps,
// Headless CLI path (test/scripting) — stereo baseline; the GUI sources this from settings.
audio_channels: 2,
mic_enabled: flag("--mic"),
hdr_enabled: !flag("--no-hdr"),
decoder,
+49 -6
View File
@@ -23,6 +23,8 @@ pub struct SessionParams {
pub compositor: CompositorPref,
pub gamepad: GamepadPref,
pub bitrate_kbps: u32,
/// Requested audio channel count (2/6/8); the host echoes the resolved value.
pub audio_channels: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Advertise 10-bit + HDR10 so the host may upgrade HDR content to a Main10/PQ stream.
@@ -94,6 +96,42 @@ fn now_ns() -> u64 {
.unwrap_or(0)
}
/// Opus decoder for the audio plane: a plain stereo decoder (the validated path) or a multistream
/// decoder for 5.1/7.1, both behind one `decode_float`. Built from the host-RESOLVED channel count
/// via the shared layout table.
enum AudioDec {
Stereo(opus::Decoder),
Surround(opus::MSDecoder),
}
impl AudioDec {
fn new(channels: u8) -> Result<AudioDec, opus::Error> {
if channels == 2 {
Ok(AudioDec::Stereo(opus::Decoder::new(
48_000,
opus::Channels::Stereo,
)?))
} else {
let l = punktfunk_core::audio::layout_for(channels, false);
Ok(AudioDec::Surround(opus::MSDecoder::new(
48_000, l.streams, l.coupled, l.mapping,
)?))
}
}
fn decode_float(
&mut self,
input: &[u8],
out: &mut [f32],
fec: bool,
) -> Result<usize, opus::Error> {
match self {
AudioDec::Stereo(d) => d.decode_float(input, out, fec),
AudioDec::Surround(d) => d.decode_float(input, out, fec),
}
}
}
fn pump(
params: SessionParams,
ev_tx: async_channel::Sender<SessionEvent>,
@@ -122,6 +160,7 @@ fn pump(
}
0
},
params.audio_channels,
None, // launch: the Windows client has no library picker yet
params.pin,
Some(params.identity),
@@ -161,11 +200,14 @@ fn pump(
let mut hardware = decoder.is_hardware();
let mut hdr = false;
// Audio is best-effort: a session without it still streams. Gamepads are the
// app-lifetime service's job (the UI attaches it on Connected).
let player = audio::AudioPlayer::spawn()
// app-lifetime service's job (the UI attaches it on Connected). Build the decoder + playback
// from the host-RESOLVED channel count (never the request), so an older/clamping host that
// resolves stereo is decoded as stereo.
let channels = connector.audio_channels;
let player = audio::AudioPlayer::spawn(channels)
.map_err(|e| tracing::warn!(error = %e, "audio disabled"))
.ok();
let mut opus_dec = opus::Decoder::new(48_000, opus::Channels::Stereo)
let mut opus_dec = AudioDec::new(channels)
.map_err(|e| tracing::warn!(error = %e, "opus decoder failed — audio disabled"))
.ok();
let _mic = params
@@ -184,8 +226,8 @@ fn pump(
let mut bytes_n = 0u64;
let mut decode_us_sum = 0u64;
let mut lat_us: Vec<u64> = Vec::with_capacity(256);
let mut pcm = vec![0f32; 5760 * 2]; // decode scratch: max Opus frame (120 ms stereo)
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut pcm = vec![0f32; 5760 * channels as usize]; // scratch: max Opus frame (120 ms) × channels
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
@@ -253,7 +295,8 @@ fn pump(
while let Ok(pkt) = connector.next_audio(Duration::ZERO) {
if let (Some(player), Some(dec)) = (&player, opus_dec.as_mut()) {
match dec.decode_float(&pkt.data, &mut pcm, false) {
Ok(samples) => player.push(pcm[..samples * 2].to_vec()),
// `samples` is per-channel; the interleaved frame is `samples * channels`.
Ok(samples) => player.push(pcm[..samples * channels as usize].to_vec()),
Err(e) => tracing::debug!(error = %e, "opus decode"),
}
}
+4
View File
@@ -130,6 +130,9 @@ pub struct Settings {
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + WASAPI render layout.
pub audio_channels: u8,
/// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream (the client
/// presents it on a 10-bit ST.2084 swapchain). No effect on SDR content.
pub hdr_enabled: bool,
@@ -148,6 +151,7 @@ impl Default for Settings {
compositor: "auto".into(),
inhibit_shortcuts: true,
mic_enabled: false,
audio_channels: 2,
hdr_enabled: true,
decoder: "auto".into(),
}
+1 -1
View File
@@ -276,7 +276,7 @@ pub mod frame {
/// These were hand-duplicated as `OFF_*`/`SHM_*` constants in `inject/{gamepad,dualsense}_windows.rs`
/// and (as bare literals — `*view.add(140)`) in the standalone `xusb-driver`/`dualsense-driver`
/// workspaces, guarded only by "must match" comments — the top ABI-drift hazard the audit flagged
/// (`docs/windows-host-rewrite.md` §2.7). Owning them here with `Pod` derives + `offset_of!`
/// (`design/windows-host-rewrite.md` §2.7). Owning them here with `Pod` derives + `offset_of!`
/// asserts makes a one-sided edit a compile error.
///
/// The host creates the section (privileged, permissive DACL so the restricted WUDFHost token can
+7 -1
View File
@@ -19,7 +19,7 @@ crate-type = ["lib", "cdylib", "staticlib"]
default = []
# Control-plane QUIC (pairing, config, reverse audio). tokio is permitted ONLY here,
# never on the per-frame hot path. Off by default so the core stays runtime-free.
quic = ["dep:quinn", "dep:tokio", "dep:rustls", "dep:rcgen", "dep:rustls-pki-types", "dep:sha2", "dep:hmac", "dep:spake2"]
quic = ["dep:quinn", "dep:tokio", "dep:rustls", "dep:rcgen", "dep:rustls-pki-types", "dep:sha2", "dep:hmac", "dep:spake2", "dep:opus"]
[dependencies]
reed-solomon-simd = "3.1" # GF(2^16) Leopard-RS, SIMD, O(n log n) — the wall-breaker (P2)
@@ -51,6 +51,12 @@ sha2 = { version = "0.10", optional = true }
hmac = { version = "0.12", optional = true }
spake2 = { version = "0.4", optional = true }
tokio = { version = "1", optional = true, features = ["rt-multi-thread", "net", "sync", "macros"] }
# In-core Opus (multistream) DECODE for the C-ABI `punktfunk_connection_next_audio_pcm` path —
# used by embedders without a multistream-capable Opus decoder (Apple's AudioToolbox is
# stereo-only). The Rust clients link `opus` themselves and decode the raw `next_audio` frames,
# so this only matters when the connection API (quic) is built. Same libopus the host vendors;
# cargo unifies the build. Multistream API: `opus::MSDecoder` (lib.rs:1187).
opus = { version = "0.3", optional = true }
# `libc` for batched UDP syscalls: `sendmmsg`/`recvmmsg` on Linux (the 1 Gbps+ lever) and the
# `recv(MSG_DONTWAIT)` drain on the other unix (Apple/BSD) targets, which have no `recvmmsg`
+219
View File
@@ -467,6 +467,23 @@ pub struct PunktfunkConnection {
last: std::sync::Mutex<Option<crate::session::Frame>>,
/// Same, for `punktfunk_connection_next_audio` (independent of the video slot).
last_audio: std::sync::Mutex<Option<crate::client::AudioPacket>>,
/// Decode-in-core state for `punktfunk_connection_next_audio_pcm` (Apple / any embedder
/// without a multistream Opus decoder). The decoder is built lazily from the negotiated
/// `inner.audio_channels`; `pcm` is a fixed-capacity reusable buffer the returned pointer
/// borrows until the next PCM call (same contract as `last_audio`).
audio_pcm: std::sync::Mutex<AudioPcmState>,
}
/// Lazily-initialized in-core Opus decode state. A coupled-1-stream multistream decoder is
/// equivalent to a plain stereo decoder, so one [`opus::MSDecoder`] handles 2/6/8 channels.
#[cfg(feature = "quic")]
#[derive(Default)]
struct AudioPcmState {
decoder: Option<opus::MSDecoder>,
/// Interleaved f32 PCM, wire channel order. Pre-sized to the largest legal Opus frame
/// (120 ms @ 48 kHz = 5760 samples/ch) × 8 channels so decode never reallocates (which would
/// dangle the pointer handed to the embedder).
pcm: Vec<f32>,
}
/// `PunktfunkHidOutput::kind` — lightbar RGB (`r`/`g`/`b` valid).
@@ -708,12 +725,18 @@ pub const PUNKTFUNK_VIDEO_CAP_10BIT: u8 = 0x01;
/// Video-capability bit for [`punktfunk_connect_ex5`] (`video_caps`): the client can present
/// BT.2020 PQ HDR10 (implies 10-bit). (Mirrors `quic::VIDEO_CAP_HDR`.)
pub const PUNKTFUNK_VIDEO_CAP_HDR: u8 = 0x02;
/// Video-capability bit for [`punktfunk_connect_ex5`] (`video_caps`): the client can decode a
/// full-chroma 4:4:4 HEVC stream (Range Extensions). The host emits 4:4:4 only when this is set,
/// the host opted in, the codec is HEVC, and the GPU supports it — else the stream stays 4:2:0 and
/// [`punktfunk_connection_chroma_format`] reports the real value. (Mirrors `quic::VIDEO_CAP_444`.)
pub const PUNKTFUNK_VIDEO_CAP_444: u8 = 0x04;
// Keep the ABI cap bits in lockstep with the wire constants (compile-time guard against drift).
#[cfg(feature = "quic")]
const _: () = {
assert!(PUNKTFUNK_VIDEO_CAP_10BIT == crate::quic::VIDEO_CAP_10BIT);
assert!(PUNKTFUNK_VIDEO_CAP_HDR == crate::quic::VIDEO_CAP_HDR);
assert!(PUNKTFUNK_VIDEO_CAP_444 == crate::quic::VIDEO_CAP_444);
};
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
@@ -980,6 +1003,58 @@ pub unsafe extern "C" fn punktfunk_connect_ex5(
client_cert_pem: *const std::os::raw::c_char,
client_key_pem: *const std::os::raw::c_char,
timeout_ms: u32,
) -> *mut PunktfunkConnection {
// Delegate to the surround-aware variant requesting stereo (the pre-surround behaviour).
unsafe {
punktfunk_connect_ex6(
host,
port,
width,
height,
refresh_hz,
compositor,
gamepad,
bitrate_kbps,
video_caps,
2, // audio_channels = stereo
launch_id,
pin_sha256,
observed_sha256_out,
client_cert_pem,
client_key_pem,
timeout_ms,
)
}
}
/// Like [`punktfunk_connect_ex5`], but additionally requests the audio channel count:
/// `2` (stereo, the default behaviour of every earlier variant), `6` (5.1) or `8` (7.1). The host
/// clamps the request to what it can actually capture and echoes the resolved count via
/// [`punktfunk_connection_audio_channels`]; the `0xC9` audio frames are Opus-(multi)stream encoded
/// for that layout. A client that wants surround calls this; everything else inherits stereo.
///
/// # Safety
/// Same as [`punktfunk_connect`].
#[cfg(feature = "quic")]
#[no_mangle]
#[allow(clippy::too_many_arguments)]
pub unsafe extern "C" fn punktfunk_connect_ex6(
host: *const std::os::raw::c_char,
port: u16,
width: u32,
height: u32,
refresh_hz: u32,
compositor: u32,
gamepad: u32,
bitrate_kbps: u32,
video_caps: u8,
audio_channels: u8,
launch_id: *const std::os::raw::c_char,
pin_sha256: *const u8,
observed_sha256_out: *mut u8,
client_cert_pem: *const std::os::raw::c_char,
client_key_pem: *const std::os::raw::c_char,
timeout_ms: u32,
) -> *mut PunktfunkConnection {
let r = std::panic::catch_unwind(AssertUnwindSafe(|| {
if host.is_null() {
@@ -1029,6 +1104,7 @@ pub unsafe extern "C" fn punktfunk_connect_ex5(
gamepad,
bitrate_kbps,
video_caps,
crate::audio::normalize_channels(audio_channels),
launch,
pin,
identity,
@@ -1045,6 +1121,7 @@ pub unsafe extern "C" fn punktfunk_connect_ex5(
inner: c,
last: std::sync::Mutex::new(None),
last_audio: std::sync::Mutex::new(None),
audio_pcm: std::sync::Mutex::new(AudioPcmState::default()),
}))
}
Err(_) => std::ptr::null_mut(),
@@ -1250,6 +1327,121 @@ pub unsafe extern "C" fn punktfunk_connection_next_audio(
})
}
/// Read the audio channel count the host resolved for this session (from its Welcome): `2`
/// (stereo), `6` (5.1) or `8` (7.1). `*out` is filled when non-NULL. The `0xC9` Opus frames are
/// (multistream-)encoded for this layout; an embedder decoding raw frames itself must build its
/// decoder from THIS value (see [`crate::audio::layout_for`]) — or use
/// [`punktfunk_connection_next_audio_pcm`], which decodes in-core. Available immediately after a
/// successful connect (it doesn't change without a reconfigure).
///
/// # Safety
/// `c` is a valid connection handle; `out` is NULL or writable for one `u8`.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_audio_channels(
c: *mut PunktfunkConnection,
out: *mut u8,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if !out.is_null() {
// SAFETY: `out` is non-null and the caller guarantees it is writable for one `u8`.
unsafe { *out = c.inner.audio_channels };
}
PunktfunkStatus::Ok
})
}
/// One decoded audio frame from [`punktfunk_connection_next_audio_pcm`]: interleaved 32-bit
/// float PCM at 48 kHz, in the canonical wire channel order `FL FR FC LFE RL RR SL SR` (the
/// first `channels` of it). `samples` points at `frame_count * channels` floats and borrows
/// connection memory **until the next PCM call** on this handle.
#[cfg(feature = "quic")]
#[repr(C)]
pub struct PunktfunkAudioPcm {
/// Interleaved f32 samples (wire channel order), `frame_count * channels` long.
pub samples: *const f32,
/// Samples per channel in this frame.
pub frame_count: u32,
/// Channel count (2/6/8) — the negotiated [`punktfunk_connection_audio_channels`].
pub channels: u8,
/// Source packet sequence number.
pub seq: u32,
/// Capture presentation timestamp (ns).
pub pts_ns: u64,
}
/// Pull the next audio frame and **decode it in-core** to interleaved f32 PCM — for embedders
/// without a multistream-capable Opus decoder (e.g. Apple, whose AudioToolbox Opus path is
/// stereo-only). The decoder is built once from the negotiated channel count and handles 2/6/8
/// channels (a 1-coupled-stream multistream decoder is exactly a stereo decoder). Same
/// timeout/closed semantics as [`punktfunk_connection_next_audio`]; `out->samples` borrows
/// connection memory until the next PCM call on this handle. Use EITHER this or
/// [`punktfunk_connection_next_audio`] on a given connection, from one dedicated audio thread —
/// not both (they share the underlying queue).
///
/// # Safety
/// `c` is a valid connection handle; `out` is writable. At most one thread pulls audio.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_next_audio_pcm(
c: *mut PunktfunkConnection,
out: *mut PunktfunkAudioPcm,
timeout_ms: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if out.is_null() {
return PunktfunkStatus::NullPointer;
}
let channels = crate::audio::normalize_channels(c.inner.audio_channels);
let pkt = match c
.inner
.next_audio(std::time::Duration::from_millis(timeout_ms as u64))
{
Ok(pkt) => pkt,
Err(e) => return e.status(),
};
let mut state = c.audio_pcm.lock().unwrap();
if state.decoder.is_none() {
let layout = crate::audio::layout_for(channels, false);
match opus::MSDecoder::new(48_000, layout.streams, layout.coupled, layout.mapping) {
Ok(d) => {
// Largest legal Opus frame is 120 ms = 5760 samples/ch.
state.pcm = vec![0f32; 5760 * channels as usize];
state.decoder = Some(d);
}
Err(_) => return PunktfunkStatus::Unsupported,
}
}
let AudioPcmState { decoder, pcm } = &mut *state;
let dec = decoder.as_mut().unwrap();
// `decode_float` divides the output buffer length by the channel count to get the
// per-channel capacity; an empty payload requests packet-loss concealment.
match dec.decode_float(&pkt.data, pcm, false) {
Ok(frame_count) => {
unsafe {
*out = PunktfunkAudioPcm {
samples: pcm.as_ptr(),
frame_count: frame_count as u32,
channels,
seq: pkt.seq,
pts_ns: pkt.pts_ns,
};
}
PunktfunkStatus::Ok
}
Err(_) => PunktfunkStatus::BadPacket,
}
})
}
/// Pull the next rumble (force-feedback) update, waiting up to `timeout_ms`. Amplitudes
/// are 0..0xFFFF (`low` = low-frequency motor, `high` = high-frequency), `(0, 0)` = stop.
/// Same timeout/closed semantics as [`punktfunk_connection_next_audio`].
@@ -1414,6 +1606,33 @@ pub unsafe extern "C" fn punktfunk_connection_color_info(
})
}
/// Read the session's resolved chroma subsampling (from the host's Welcome) as the HEVC
/// `chroma_format_idc`: `1` = 4:2:0 (the default every pre-4:4:4 host produced), `3` = full-chroma
/// 4:4:4. `*out` is filled when non-NULL. The in-band SPS is authoritative; this lets the embedder
/// pre-size its decoder / pick a 4:4:4 pixel format up front. Available immediately after a
/// successful connect (it doesn't change without a reconfigure).
///
/// # Safety
/// `c` is a valid connection handle; `out` is NULL or writable for one `u8`.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_chroma_format(
c: *mut PunktfunkConnection,
out: *mut u8,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if !out.is_null() {
// SAFETY: `out` is non-null and the caller guarantees it is writable for one `u8`.
unsafe { *out = c.inner.chroma_format };
}
PunktfunkStatus::Ok
})
}
/// Send one input event to the host as a QUIC datagram (non-blocking enqueue).
///
/// # Safety
+298
View File
@@ -0,0 +1,298 @@
//! Shared audio layout: the single source of truth for Opus (multi)stream surround across the
//! host, the GameStream compatibility path, and every client decoder.
//!
//! **Canonical wire channel order** is `FL FR FC LFE RL RR SL SR` (the GameStream/Moonlight
//! order, and the PipeWire/PulseAudio default map for 6/8 channels). Every host capturer
//! delivers PCM in this order and every client decodes into it, so the Opus multistream
//! `mapping` is the **identity** (`[0, 1, …, channels-1]`) on both ends — punktfunk owns the
//! encoder and every decoder, so the GFE-style pre-rotation Moonlight needs over SDP
//! (`gamestream::audio::surround_params`) is a GameStream-only concern and never touches the
//! native `punktfunk/1` path.
//!
//! Channel counts the protocol negotiates: `2` (stereo), `6` (5.1) and `8` (7.1). Anything
//! else clamps to stereo ([`normalize_channels`]).
/// Canonical wire channel positions; the index is the channel's slot in the interleaved PCM
/// frame. A count of N uses positions `0..N` (always a prefix of this 8-channel order).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
#[repr(u8)]
pub enum WirePos {
FrontLeft = 0,
FrontRight = 1,
FrontCenter = 2,
Lfe = 3,
RearLeft = 4,
RearRight = 5,
SideLeft = 6,
SideRight = 7,
}
/// The full 8-channel wire order; the N-channel order is its first N entries.
pub const WIRE_ORDER_8: [WirePos; 8] = {
use WirePos::*;
[
FrontLeft,
FrontRight,
FrontCenter,
Lfe,
RearLeft,
RearRight,
SideLeft,
SideRight,
]
};
/// One Opus (multi)stream layout. `mapping` is the libopus multistream mapping we encode AND
/// decode with — identity, since punktfunk owns both ends. `streams`/`coupled` give the
/// normal-quality coupling (FL,FR)+(FC,LFE) [+(RL,RR) on 7.1] with the remaining channels as
/// mono streams; high quality is one mono stream per channel. Bitrates match Sunshine's
/// per-config values (stereo keeps punktfunk's live-validated 128 kbps).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct OpusLayout {
/// Interleaved channel count (2, 6 or 8).
pub channels: u8,
/// Number of Opus streams in the multistream packet.
pub streams: u8,
/// How many of those streams are coupled (stereo) pairs.
pub coupled: u8,
/// libopus multistream channel mapping — identity `[0, 1, …, channels-1]`.
pub mapping: &'static [u8],
/// Target Opus bitrate in bits/sec (hard CBR; constant packet size, which GameStream's
/// audio FEC relies on).
pub bitrate: i32,
}
/// Stereo: a plain coupled pair. The 128 kbps live-validated config.
pub const LAYOUT_STEREO: OpusLayout = OpusLayout {
channels: 2,
streams: 1,
coupled: 1,
mapping: &[0, 1],
bitrate: 128_000,
};
/// 5.1 normal quality: (FL,FR)+(FC,LFE) coupled, RL+RR mono.
pub const LAYOUT_51: OpusLayout = OpusLayout {
channels: 6,
streams: 4,
coupled: 2,
mapping: &[0, 1, 2, 3, 4, 5],
bitrate: 256_000,
};
/// 5.1 high quality: one mono stream per channel.
pub const LAYOUT_51_HQ: OpusLayout = OpusLayout {
channels: 6,
streams: 6,
coupled: 0,
mapping: &[0, 1, 2, 3, 4, 5],
bitrate: 1_536_000,
};
/// 7.1 normal quality: (FL,FR)+(FC,LFE)+(RL,RR) coupled, SL+SR mono.
pub const LAYOUT_71: OpusLayout = OpusLayout {
channels: 8,
streams: 5,
coupled: 3,
mapping: &[0, 1, 2, 3, 4, 5, 6, 7],
bitrate: 450_000,
};
/// 7.1 high quality: one mono stream per channel.
pub const LAYOUT_71_HQ: OpusLayout = OpusLayout {
channels: 8,
streams: 8,
coupled: 0,
mapping: &[0, 1, 2, 3, 4, 5, 6, 7],
bitrate: 2_048_000,
};
/// Pick the layout for a negotiated channel count. Unknown counts fall back to stereo (clients
/// only ever request 2/6/8). `high_quality` selects the uncoupled high-bitrate config.
pub fn layout_for(channels: u8, high_quality: bool) -> &'static OpusLayout {
match (channels, high_quality) {
(6, false) => &LAYOUT_51,
(6, true) => &LAYOUT_51_HQ,
(8, false) => &LAYOUT_71,
(8, true) => &LAYOUT_71_HQ,
_ => &LAYOUT_STEREO,
}
}
/// Clamp an arbitrary (wire / requested) channel count to one the protocol negotiates. `0`,
/// absent, or any unsupported value becomes stereo.
pub fn normalize_channels(requested: u8) -> u8 {
match requested {
6 => 6,
8 => 8,
_ => 2,
}
}
// ---- per-platform channel-layout helpers (pure data; no platform deps) --------------------
/// Windows `WAVEFORMATEXTENSIBLE.dwChannelMask` for the wire layout.
///
/// NB 7.1 == `0x63F` (FL FR FC LFE **BL BR SL SR**), NOT `0xFF` — `0xFF` selects the
/// front-of-center pair FLC/FRC, the wrong speakers. WASAPI delivers channels in ascending
/// mask-bit order, which equals the wire order, so the decoded PCM needs no permutation.
pub const fn wasapi_channel_mask(channels: u8) -> u32 {
const FL: u32 = 0x1;
const FR: u32 = 0x2;
const FC: u32 = 0x4;
const LFE: u32 = 0x8;
const BL: u32 = 0x10; // back left (wire RL)
const BR: u32 = 0x20; // back right (wire RR)
const SL: u32 = 0x200; // side left
const SR: u32 = 0x400; // side right
match channels {
6 => FL | FR | FC | LFE | BL | BR, // 0x3F
8 => FL | FR | FC | LFE | BL | BR | SL | SR, // 0x63F
_ => FL | FR, // 0x3 (stereo)
}
}
/// PipeWire / SPA `enum spa_audio_channel` positions in wire order — identical to the host
/// capture side (`punktfunk-host` `audio::linux::spa_positions`): FL=3 FR=4 FC=5 LFE=6 SL=7
/// SR=8 RL=12 RR=13. Identity routing: the client sets these on its playback node so PipeWire
/// maps each wire slot to the matching speaker (and downmixes when the sink has fewer).
pub fn spa_positions(channels: u8) -> &'static [u32] {
const STEREO: [u32; 2] = [3, 4]; // FL FR
const C51: [u32; 6] = [3, 4, 5, 6, 12, 13]; // FL FR FC LFE RL RR
const C71: [u32; 8] = [3, 4, 5, 6, 12, 13, 7, 8]; // FL FR FC LFE RL RR SL SR
match channels {
6 => &C51,
8 => &C71,
_ => &STEREO,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn layout_table_is_consistent() {
for l in [
&LAYOUT_STEREO,
&LAYOUT_51,
&LAYOUT_51_HQ,
&LAYOUT_71,
&LAYOUT_71_HQ,
] {
// Mapping is identity and exactly `channels` entries long.
assert_eq!(l.mapping.len(), l.channels as usize);
for (i, &m) in l.mapping.iter().enumerate() {
assert_eq!(m as usize, i, "mapping must be identity for {l:?}");
}
// libopus invariant: total channels == coupled*2 + (streams - coupled).
assert_eq!(
l.coupled * 2 + (l.streams - l.coupled),
l.channels,
"stream/coupled accounting for {l:?}"
);
assert!(l.coupled <= l.streams);
assert!(l.bitrate > 0);
}
}
#[test]
fn layout_for_picks_expected() {
assert_eq!(layout_for(2, false), &LAYOUT_STEREO);
assert_eq!(layout_for(6, false), &LAYOUT_51);
assert_eq!(layout_for(6, true), &LAYOUT_51_HQ);
assert_eq!(layout_for(8, false), &LAYOUT_71);
assert_eq!(layout_for(8, true), &LAYOUT_71_HQ);
// Unknown / 0 → stereo.
assert_eq!(layout_for(0, false), &LAYOUT_STEREO);
assert_eq!(layout_for(3, false), &LAYOUT_STEREO);
assert_eq!(layout_for(7, true), &LAYOUT_STEREO);
}
#[test]
fn normalize_clamps_to_negotiable() {
assert_eq!(normalize_channels(2), 2);
assert_eq!(normalize_channels(6), 6);
assert_eq!(normalize_channels(8), 8);
for bad in [0u8, 1, 3, 4, 5, 7, 9, 255] {
assert_eq!(normalize_channels(bad), 2, "{bad} must clamp to stereo");
}
}
#[test]
fn wasapi_masks_are_correct() {
assert_eq!(wasapi_channel_mask(2), 0x3);
assert_eq!(wasapi_channel_mask(6), 0x3F);
assert_eq!(wasapi_channel_mask(8), 0x63F); // NOT 0xFF
// Bit count must equal the channel count.
assert_eq!(wasapi_channel_mask(2).count_ones(), 2);
assert_eq!(wasapi_channel_mask(6).count_ones(), 6);
assert_eq!(wasapi_channel_mask(8).count_ones(), 8);
}
#[test]
fn spa_positions_match_wire_order() {
assert_eq!(spa_positions(2), &[3, 4]);
assert_eq!(spa_positions(6), &[3, 4, 5, 6, 12, 13]);
assert_eq!(spa_positions(8), &[3, 4, 5, 6, 12, 13, 7, 8]);
assert_eq!(spa_positions(2).len(), 2);
assert_eq!(spa_positions(6).len(), 6);
assert_eq!(spa_positions(8).len(), 8);
}
/// Real-libopus proof that the shared layout round-trips with channel identity: a tone fed
/// into wire channel N (host `opus::MSEncoder`) comes back out on channel N (client
/// `opus::MSDecoder`), for stereo / 5.1 / 7.1. This is the single guarantee the whole
/// feature rests on — encoder layout == decoder layout == identity mapping — so if a layout
/// constant is ever wrong, this fails. Gated on `quic` (where `opus` is a dependency).
#[cfg(feature = "quic")]
#[test]
fn multistream_layout_roundtrips_with_channel_identity() {
const SR: u32 = 48_000;
const SAMPLES: usize = 240; // 5 ms @ 48 kHz
for &channels in &[2u8, 6, 8] {
let l = layout_for(channels, false);
let ch = l.channels as usize;
let mut enc = opus::MSEncoder::new(
SR,
l.streams,
l.coupled,
l.mapping,
opus::Application::LowDelay,
)
.expect("MSEncoder");
enc.set_bitrate(opus::Bitrate::Bits(l.bitrate)).unwrap();
enc.set_vbr(false).unwrap();
let mut dec =
opus::MSDecoder::new(SR, l.streams, l.coupled, l.mapping).expect("MSDecoder");
for tone_ch in 0..ch {
let mut out = vec![0u8; 4000];
let mut energy = vec![0f64; ch];
// A few frames to clear the codec startup transient before measuring.
for f in 0..8 {
let mut frame = vec![0f32; SAMPLES * ch];
for t in 0..SAMPLES {
let phase = (f * SAMPLES + t) as f32 * 440.0 * 2.0 * std::f32::consts::PI
/ SR as f32;
frame[t * ch + tone_ch] = 0.5 * phase.sin();
}
let n = enc.encode_float(&frame, &mut out).unwrap();
let mut decoded = vec![0f32; SAMPLES * ch];
let got = dec.decode_float(&out[..n], &mut decoded, false).unwrap();
assert_eq!(got, SAMPLES, "{channels}ch frame size");
if f >= 4 {
for t in 0..SAMPLES {
for (c, e) in energy.iter_mut().enumerate() {
*e += (decoded[t * ch + c] as f64).powi(2);
}
}
}
}
let loudest = (0..ch)
.max_by(|&a, &b| energy[a].total_cmp(&energy[b]))
.unwrap();
assert_eq!(
loudest, tone_ch,
"{channels}ch: tone in channel {tone_ch} must come out on {tone_ch} (energies {energy:?})"
);
}
}
}
}
+34 -2
View File
@@ -40,8 +40,9 @@ enum CtrlRequest {
/// mode, the host-resolved compositor backend, the host-resolved gamepad backend, the host's
/// certificate fingerprint, the resolved encoder bitrate (kbps), and the host↔client clock offset
/// (ns, host minus client; 0 = no skew correction / an old host that didn't answer the handshake).
/// The trailing `u8` is the resolved encode bit depth (8/10) and [`ColorInfo`] the resolved colour
/// signalling, both from the [`Welcome`].
/// The trailing `u8`s are the resolved encode bit depth (8/10), the chroma `chroma_format_idc`
/// (1 = 4:2:0, 3 = 4:4:4), and the resolved audio channel count (2/6/8), with [`ColorInfo`] the
/// resolved colour signalling — all from the [`Welcome`].
type Negotiated = (
Mode,
CompositorPref,
@@ -51,6 +52,8 @@ type Negotiated = (
i64,
u8,
ColorInfo,
u8,
u8,
);
/// Accumulated state of an in-flight / finished speed test. The data-plane pump mirrors the
@@ -202,6 +205,17 @@ pub struct NativeClient {
/// decoder/presenter from this. [`ColorInfo::SDR_BT709`] for an older host. The static HDR
/// mastering metadata (when [`ColorInfo::is_hdr`]) arrives via [`NativeClient::next_hdr_meta`].
pub color: ColorInfo,
/// The chroma subsampling the host resolved for this session ([`Welcome::chroma_format`]), as the
/// HEVC `chroma_format_idc`: [`quic::CHROMA_IDC_420`] (4:2:0, the default / older host) or
/// [`quic::CHROMA_IDC_444`] (full-chroma 4:4:4). The in-band SPS is authoritative; this lets the
/// client pre-size its decoder. `CHROMA_IDC_420` for an older host that didn't report it.
pub chroma_format: u8,
/// The audio channel count the host resolved for this session ([`Welcome::audio_channels`]):
/// `2` (stereo), `6` (5.1) or `8` (7.1). The client MUST build its Opus (multistream) decoder
/// from this value (via [`crate::audio::layout_for`]) — never from its own request — so an older
/// host that omits it (→ `2`) yields working stereo. The `0xC9` audio frames are encoded with the
/// matching layout.
pub audio_channels: u8,
}
/// Pin the calling thread to the user-interactive QoS class on Apple targets.
@@ -246,6 +260,9 @@ impl NativeClient {
// VIDEO_CAP_HDR) — the host upgrades to a 10-bit / HDR encode only when the matching bit is
// set. 0 = the 8-bit BT.709 stream every client understands.
video_caps: u8,
// Requested audio channel count (2 = stereo / 6 = 5.1 / 8 = 7.1); the host clamps to what it
// can capture and echoes the result in [`NativeClient::audio_channels`].
audio_channels: u8,
launch: Option<String>,
pin: Option<[u8; 32]>,
identity: Option<(String, String)>,
@@ -298,6 +315,7 @@ impl NativeClient {
gamepad,
bitrate_kbps,
video_caps,
audio_channels,
launch,
pin,
identity,
@@ -329,6 +347,8 @@ impl NativeClient {
clock_offset_ns,
bit_depth,
color,
chroma_format,
audio_channels,
) = match ready_rx.recv_timeout(timeout) {
Ok(Ok(t)) => t,
Ok(Err(e)) => return Err(e),
@@ -360,6 +380,8 @@ impl NativeClient {
clock_offset_ns,
bit_depth,
color,
chroma_format,
audio_channels,
})
}
@@ -666,6 +688,7 @@ struct WorkerArgs {
gamepad: GamepadPref,
bitrate_kbps: u32,
video_caps: u8,
audio_channels: u8,
launch: Option<String>,
pin: Option<[u8; 32]>,
identity: Option<(String, String)>,
@@ -697,6 +720,7 @@ async fn worker_main(args: WorkerArgs) {
gamepad,
bitrate_kbps,
video_caps,
audio_channels,
launch,
pin,
identity,
@@ -763,6 +787,8 @@ async fn worker_main(args: WorkerArgs) {
// VIDEO_CAP_10BIT | VIDEO_CAP_HDR). The host only upgrades to a 10-bit / HDR encode
// when the matching bit is set, so `0` stays an 8-bit BT.709 stream.
video_caps,
// Requested surround channel count; the host echoes the resolved value in Welcome.
audio_channels,
}
.encode(),
)
@@ -834,6 +860,8 @@ async fn worker_main(args: WorkerArgs) {
clock_offset_ns,
welcome.bit_depth,
welcome.color,
welcome.chroma_format,
welcome.audio_channels,
))
};
@@ -850,6 +878,8 @@ async fn worker_main(args: WorkerArgs) {
clock_offset_ns,
bit_depth,
color,
chroma_format,
audio_channels,
) = match setup.await {
Ok(t) => t,
Err(e) => {
@@ -866,6 +896,8 @@ async fn worker_main(args: WorkerArgs) {
clock_offset_ns,
bit_depth,
color,
chroma_format,
audio_channels,
)));
// Input task: embedder events → QUIC datagrams.
+1
View File
@@ -25,6 +25,7 @@
#![forbid(unsafe_op_in_unsafe_fn)]
pub mod abi;
pub mod audio;
#[cfg(feature = "quic")]
pub mod client;
pub mod config;
+98 -7
View File
@@ -78,12 +78,33 @@ pub struct Hello {
/// zero-length name/launch placeholder precedes it when those are absent so the offset stays
/// deterministic. Omitted by older clients (decodes to `0`).
pub video_caps: u8,
/// Requested audio channel count: `2` (stereo, default), `6` (5.1) or `8` (7.1). The host
/// resolves it against what it can capture and echoes the final count in
/// [`Welcome::audio_channels`], which is what both ends build their Opus (multistream)
/// codec from. Appended after `video_caps` as a single trailing byte; when it differs from
/// the stereo default the name/launch/video_caps placeholders are forced (0) so it lands at a
/// deterministic offset. Omitted by older clients / when `2` (decodes to `2`, i.e. stereo) so
/// the stereo wire form stays byte-identical to the pre-surround build.
pub audio_channels: u8,
}
/// [`Hello::video_caps`] bit: the client can decode a 10-bit (Main10) HEVC stream.
pub const VIDEO_CAP_10BIT: u8 = 0x01;
/// [`Hello::video_caps`] bit: the client can present BT.2020 PQ HDR10 (implies 10-bit).
pub const VIDEO_CAP_HDR: u8 = 0x02;
/// [`Hello::video_caps`] bit: the client can decode a full-chroma **4:4:4** HEVC stream (HEVC
/// Range Extensions / Rec.ITU-T H.265 `chroma_format_idc = 3`). The host emits 4:4:4 ONLY when this
/// bit is set, the host opted in (`PUNKTFUNK_444`), the codec is HEVC, **and** the GPU/driver
/// actually supports a 4:4:4 encode (probed) — otherwise the session stays 4:2:0 and
/// [`Welcome::chroma_format`] reflects the real resolved value. Independent of 10-bit/HDR (4:4:4 is a
/// chroma decision, bit depth is a depth decision; the two may combine where the hardware allows).
pub const VIDEO_CAP_444: u8 = 0x04;
/// HEVC `chroma_format_idc` for 4:2:0 — what every pre-4:4:4 build produced and the back-compat
/// default when a peer omits [`Welcome::chroma_format`].
pub const CHROMA_IDC_420: u8 = 1;
/// HEVC `chroma_format_idc` for full-chroma 4:4:4 (Range Extensions).
pub const CHROMA_IDC_444: u8 = 3;
/// Per-session colour signalling (CICP / ITU-T H.273 code points) the host resolved for the
/// encoded video, carried on [`Welcome`]. A client configures its decoder/presenter from these
@@ -198,6 +219,22 @@ pub struct Welcome {
/// [`ColorInfo::SDR_BT709`]. The client configures its decoder/presenter from this instead of
/// guessing from the bitstream; the mastering metadata arrives separately on [`HDR_META_MAGIC`].
pub color: ColorInfo,
/// The chroma subsampling the host actually encodes at, as the HEVC `chroma_format_idc`:
/// [`CHROMA_IDC_420`] (4:2:0, default / older host) or [`CHROMA_IDC_444`] (full-chroma 4:4:4,
/// enabled only when the client advertised [`VIDEO_CAP_444`] *and* the host could open a real
/// 4:4:4 encode). The client sizes its decoder/surface pool from this; the in-band SPS carries
/// the authoritative value, so this is a hint (and the honest-downgrade channel — if the host
/// requested 4:4:4 but the GPU declined, this reads `CHROMA_IDC_420`). Appended after the colour
/// bytes as a single trailing byte; an older host that omits it decodes to [`CHROMA_IDC_420`].
pub chroma_format: u8,
/// The audio channel count the host actually resolved and **will** send on the `0xC9` plane:
/// `2` (stereo, default), `6` (5.1) or `8` (7.1). Echoes [`Hello::audio_channels`] clamped to
/// what the host can capture (Linux PipeWire always synthesizes the count; Windows WASAPI
/// loopback is clamped to the render endpoint's mix-format channels). The client builds its Opus
/// (multistream) decoder from THIS value via [`crate::audio::layout_for`] — never from its own
/// request — so an older host that omits the byte (→ `2`) always yields working stereo. Appended
/// after `chroma_format` as a single trailing byte.
pub audio_channels: u8,
}
/// `client → host`: data plane is bound, begin streaming.
@@ -630,10 +667,11 @@ impl Hello {
// so a Hello with neither name nor launch stays byte-identical to the bitrate-era form
// (26 bytes). When `launch` is present we must still emit name's length byte (0 for None)
// so `launch` lands at a deterministic offset.
// `video_caps` is the last trailing field, after `launch`; when it's present (non-zero)
// the name/launch length bytes must still be emitted (0 for absent) so it lands at a
// `video_caps`/`audio_channels` are the trailing fields, after `launch`; when either is
// present (video_caps non-zero / audio_channels not stereo) the name/launch length bytes
// AND the video_caps byte must still be emitted (0 / 0) so the later byte lands at a
// deterministic offset — the same discipline `launch` already imposes on `name`.
let need_placeholders = self.video_caps != 0;
let need_placeholders = self.video_caps != 0 || self.audio_channels != 2;
match (&self.name, &self.launch) {
(None, None) if !need_placeholders => {}
(name, _) => {
@@ -648,10 +686,15 @@ impl Hello {
b.push(l.len() as u8);
b.extend_from_slice(l.as_bytes());
}
// video_caps: single trailing byte. Last field.
if self.video_caps != 0 {
// video_caps: single trailing byte. Emitted when non-zero OR when audio_channels follows
// (so audio_channels lands at a deterministic offset right after it).
if self.video_caps != 0 || self.audio_channels != 2 {
b.push(self.video_caps);
}
// audio_channels: single trailing byte. Last field; omitted when stereo (default).
if self.audio_channels != 2 {
b.push(self.audio_channels);
}
b
}
@@ -714,6 +757,15 @@ impl Hello {
let launch_len = b.get(launch_off).copied().unwrap_or(0) as usize;
b.get(launch_off + 1 + launch_len).copied().unwrap_or(0)
},
// Optional trailing audio-channel byte, one past video_caps. Absent on an older client
// → stereo. Normalized so a corrupt/unsupported value can't build a bad decoder.
audio_channels: {
let name_len = b.get(26).copied().unwrap_or(0) as usize;
let launch_off = 27 + name_len;
let launch_len = b.get(launch_off).copied().unwrap_or(0) as usize;
let video_caps_off = launch_off + 1 + launch_len;
crate::audio::normalize_channels(b.get(video_caps_off + 1).copied().unwrap_or(2))
},
})
}
}
@@ -747,6 +799,10 @@ impl Welcome {
b.push(self.color.transfer);
b.push(self.color.matrix);
b.push(self.color.full_range);
// Chroma subsampling at offset 64 — older clients stop before this → 4:2:0 (CHROMA_IDC_420).
b.push(self.chroma_format);
// Audio channel count at offset 65 — older clients stop before this → stereo (2).
b.push(self.audio_channels);
b
}
@@ -755,7 +811,8 @@ impl Welcome {
// scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45]
// salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59]
// bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63]
// (everything from compositor on is an optional trailing byte; an older host stops earlier).
// chroma_format[64] audio_channels[65] (everything from compositor on is an optional
// trailing byte; an older host stops earlier).
if b.len() < 53 || &b[0..4] != MAGIC {
return Err(PunktfunkError::InvalidArg("bad Welcome"));
}
@@ -812,6 +869,15 @@ impl Welcome {
matrix: b.get(62).copied().unwrap_or(ColorInfo::MC_BT709),
full_range: b.get(63).copied().unwrap_or(0),
},
// Optional trailing chroma byte — absent on an older host (or an explicit 0 / unknown
// value) → 4:2:0. Only `CHROMA_IDC_444` flips the client to a 4:4:4 decode.
chroma_format: match b.get(64).copied() {
Some(CHROMA_IDC_444) => CHROMA_IDC_444,
_ => CHROMA_IDC_420,
},
// Optional trailing audio-channel byte — absent on an older host → stereo. Any
// non-{6,8} value normalizes to stereo so a corrupt byte never builds a bad decoder.
audio_channels: crate::audio::normalize_channels(b.get(65).copied().unwrap_or(2)),
})
}
@@ -1809,6 +1875,8 @@ mod tests {
bitrate_kbps: 50_000,
bit_depth: 10,
color: ColorInfo::HDR10_BT2020_PQ,
chroma_format: CHROMA_IDC_444,
audio_channels: 2,
};
assert_eq!(Welcome::decode(&w.encode()).unwrap(), w);
}
@@ -1851,6 +1919,7 @@ mod tests {
name: Some("Test Device".into()),
launch: Some("steam:570".into()),
video_caps: VIDEO_CAP_10BIT,
audio_channels: 2,
};
assert_eq!(Hello::decode(&h.encode()).unwrap(), h);
let s = Start {
@@ -1930,6 +1999,7 @@ mod tests {
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
};
let enc = h.encode();
assert_eq!(enc.len(), 26);
@@ -1969,9 +2039,11 @@ mod tests {
bitrate_kbps: 120_000,
bit_depth: 10,
color: ColorInfo::HDR10_BT2020_PQ,
chroma_format: CHROMA_IDC_444,
audio_channels: 6, // 5.1 — exercises the non-default trailing byte
};
let wenc = w.encode();
assert_eq!(wenc.len(), 64); // 60 base + 4 colour bytes
assert_eq!(wenc.len(), 66); // 60 base + 4 colour + 1 chroma + 1 audio-channels byte
let legacy_w = Welcome::decode(&wenc[..53]).unwrap();
assert_eq!(legacy_w.compositor, CompositorPref::Auto);
assert_eq!(legacy_w.gamepad, GamepadPref::Auto);
@@ -1991,13 +2063,29 @@ mod tests {
let pre_color_w = Welcome::decode(&wenc[..60]).unwrap();
assert_eq!(pre_color_w.bit_depth, 10);
assert_eq!(pre_color_w.color, ColorInfo::SDR_BT709);
assert_eq!(pre_color_w.chroma_format, CHROMA_IDC_420); // pre-chroma host → 4:2:0
assert_eq!(legacy_w.color, ColorInfo::SDR_BT709);
assert_eq!(legacy_w.chroma_format, CHROMA_IDC_420);
// A pre-chroma (64-byte) Welcome carries colour but no chroma/audio bytes → 4:2:0 + stereo.
let pre_chroma_w = Welcome::decode(&wenc[..64]).unwrap();
assert_eq!(pre_chroma_w.color, ColorInfo::HDR10_BT2020_PQ);
assert_eq!(pre_chroma_w.chroma_format, CHROMA_IDC_420);
assert_eq!(pre_chroma_w.audio_channels, 2); // audio byte (offset 65) absent → stereo
// A pre-audio (65-byte) Welcome carries chroma but no audio byte → 4:4:4 + stereo.
let pre_audio_w = Welcome::decode(&wenc[..65]).unwrap();
assert_eq!(pre_audio_w.chroma_format, CHROMA_IDC_444);
assert_eq!(pre_audio_w.audio_channels, 2);
assert_eq!(Welcome::decode(&wenc).unwrap().bitrate_kbps, 120_000);
assert_eq!(Welcome::decode(&wenc).unwrap().bit_depth, 10); // full form carries it
assert_eq!(
Welcome::decode(&wenc).unwrap().color,
ColorInfo::HDR10_BT2020_PQ
);
assert_eq!(
Welcome::decode(&wenc).unwrap().chroma_format,
CHROMA_IDC_444
); // full form carries 4:4:4
assert_eq!(Welcome::decode(&wenc).unwrap().audio_channels, 6); // ...and 5.1
}
#[test]
@@ -2015,6 +2103,7 @@ mod tests {
name: Some("Enrico's MacBook".into()),
launch: None,
video_caps: 0,
audio_channels: 2,
};
let enc = base.encode();
assert_eq!(
@@ -2062,6 +2151,7 @@ mod tests {
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
};
// launch alone (no name): a zero-length name placeholder keeps the offset deterministic.
let with_launch = Hello {
@@ -2268,6 +2358,7 @@ mod tests {
name: None,
launch: None,
video_caps: 0,
audio_channels: 2,
}
.encode();
assert!(PairRequest::decode(&h).is_err(), "abi {abi} parsed as pair");
+14 -2
View File
@@ -13,8 +13,10 @@ use std::process::Command;
fn native_libs() -> &'static [&'static str] {
if cfg!(target_os = "macos") {
// The workspace build unifies features into the staticlib, and `quic` pulls
// rustls's platform verifier → Security/CoreFoundation.
// rustls's platform verifier → Security/CoreFoundation, plus libopus (the in-core
// `next_audio_pcm` decode path) which the `abi.rs` object references.
&[
"-lopus",
"-liconv",
"-lm",
"-framework",
@@ -23,7 +25,17 @@ fn native_libs() -> &'static [&'static str] {
"CoreFoundation",
]
} else if cfg!(target_os = "linux") {
&["-lgcc_s", "-lutil", "-lrt", "-lpthread", "-lm", "-ldl"]
// `-lopus`: the `quic` feature pulls in-core Opus decode (`next_audio_pcm`), whose
// symbols the linked `abi.rs` object references. Before `-lm` (opus needs libm).
&[
"-lopus",
"-lgcc_s",
"-lutil",
"-lrt",
"-lpthread",
"-lm",
"-ldl",
]
} else {
&[]
}
+4 -7
View File
@@ -61,9 +61,10 @@ utoipa-scalar = { version = "0.3", features = ["axum"] }
tower = { version = "0.5", features = ["util"] }
http-body-util = "0.1"
# Opus stereo encode for the host->client audio plane. The `opus` crate vendors libopus via
# `audiopus_sys` (cmake-built from source — no system lib, no vcpkg), so it builds on Windows MSVC
# too (needs CMake + NASM, both on the box). Both platforms that have an audio-capture backend.
# Opus encode for the host->client audio plane — stereo (`opus::Encoder`) AND 5.1/7.1 surround
# (`opus::MSEncoder`, the safe multistream API the crate exposes; no `audiopus_sys` needed). The
# crate vendors libopus (cmake-built from source — no system lib, no vcpkg), so it builds on Windows
# MSVC too (needs CMake + NASM, both on the box). Both platforms that have an audio-capture backend.
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
opus = "0.3"
@@ -99,10 +100,6 @@ serde_json = "1"
rusqlite = { version = "0.40", features = ["bundled"] }
# Builds/validates the xkb keymap uploaded to the virtual keyboard + tracks modifier state.
xkbcommon = "0.8"
# The safe `opus` crate is stereo-only; surround (5.1/7.1) needs the libopus *multistream*
# encoder (`opus_multistream_encoder_*`). `audiopus_sys` is the sys layer `opus` already
# vendors (same libopus link), so this adds bindings, not a second copy of the library.
audiopus_sys = "0.2"
# libei (EI sender) for the portable input path on KWin/GNOME (RemoteDesktop portal).
# The `tokio` feature wires reis's event stream into tokio's reactor.
reis = { version = "0.6.1", features = ["tokio"] }
@@ -0,0 +1,73 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="fake_input">
<copyright>
SPDX-FileCopyrightText: 2015 Martin Gräßlin
SPDX-License-Identifier: LGPL-2.1-or-later
</copyright>
<interface name="org_kde_kwin_fake_input" version="4">
<description summary="Fake input manager">
This interface allows other processes to provide fake input events.
Purpose is on the one hand side to provide testing facilities like XTest
on X11, but also to support use cases like remote control (a remote
desktop server). The compositor gates the interface: it is only exposed
to clients authorized through their .desktop X-KDE-Wayland-Interfaces, so
binding it is the authorization — no per-event confirmation dialog.
</description>
<request name="authenticate">
<description summary="Information about the application requesting fake input">
A FakeInput is required to authenticate itself by providing the
application name and the reason for fake input. The compositor may use
this information to decide whether to allow or deny the request.
</description>
<arg name="application" type="string" summary="user visible name of the application requesting fake input"/>
<arg name="reason" type="string" summary="reason of why fake input is requested"/>
</request>
<request name="pointer_motion">
<description summary="pointer motion event"/>
<arg name="delta_x" type="fixed" summary="X delta of the relative pointer motion"/>
<arg name="delta_y" type="fixed" summary="Y delta of the relative pointer motion"/>
</request>
<request name="button">
<description summary="pointer button event"/>
<arg name="button" type="uint" summary="evdev button code"/>
<arg name="state" type="uint" summary="button state, 0 released, 1 pressed"/>
</request>
<request name="axis">
<description summary="pointer axis (scroll) event"/>
<arg name="axis" type="uint" summary="wl_pointer.axis (0 vertical, 1 horizontal)"/>
<arg name="value" type="fixed" summary="axis value"/>
</request>
<request name="touch_down" since="2">
<description summary="touch down event"/>
<arg name="id" type="uint" summary="unique id of this touch point; must not be reused until up"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="touch_motion" since="2">
<description summary="touch motion event"/>
<arg name="id" type="uint" summary="unique id of an existing touch point"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="touch_up" since="2">
<description summary="touch up event"/>
<arg name="id" type="uint" summary="unique id of an existing touch point"/>
</request>
<request name="touch_cancel" since="2">
<description summary="cancel all current touch points"/>
</request>
<request name="touch_frame" since="2">
<description summary="end a set of touch events (atomic frame)"/>
</request>
<request name="pointer_motion_absolute" since="3">
<description summary="absolute pointer motion event"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="keyboard_key" since="4">
<description summary="keyboard key event"/>
<arg name="button" type="uint" summary="evdev key code"/>
<arg name="state" type="uint" summary="key state, 0 released, 1 pressed"/>
</request>
</interface>
</protocol>
+8 -1
View File
@@ -320,11 +320,18 @@ fn mic_pw_thread(
.into_inner();
let mut params = [Pod::from_bytes(&values).context("mic pod from bytes")?];
// RT_PROCESS: run the producer callback on PipeWire's realtime data loop, so the source is a
// *synchronous* graph node that joins its consumer's driver group and is actually driven. Without
// it the node is async/main-loop and, in the host's busy multi-stream graph (desktop-audio +
// video capture + the session), never acquires a driver — it stays suspended and its process()
// never fires, so every recorder hears pure silence (the long-standing "Linux host mic broken").
stream
.connect(
spa::utils::Direction::Output, // we PRODUCE samples (a source)
None,
pw::stream::StreamFlags::AUTOCONNECT | pw::stream::StreamFlags::MAP_BUFFERS,
pw::stream::StreamFlags::AUTOCONNECT
| pw::stream::StreamFlags::MAP_BUFFERS
| pw::stream::StreamFlags::RT_PROCESS,
&mut params,
)
.context("pw mic stream connect")?;
@@ -1,7 +1,9 @@
//! WASAPI loopback capture of the default render endpoint (system output) — the Windows analogue
//! of the PipeWire sink-monitor backend. Delivers interleaved f32 PCM at 48 kHz stereo, ready for
//! the existing Opus path with NO resampling (WASAPI shared-mode autoconvert does any SRC). WASAPI
//! objects are COM-apartment-bound and not `Send`, so they live on a dedicated thread (mirrors
//! of the PipeWire sink-monitor backend. Delivers interleaved f32 PCM at 48 kHz in the requested
//! channel count (stereo / 5.1 / 7.1, canonical wire order FL FR FC LFE RL RR SL SR via the
//! explicit `dwChannelMask`), ready for the Opus path with NO resampling (WASAPI shared-mode
//! autoconvert does any SRC + up/downmix to the requested layout). WASAPI objects are
//! COM-apartment-bound and not `Send`, so they live on a dedicated thread (mirrors
//! `linux::PwAudioCapturer`); only the channel + stop flag + join handle are in the struct.
use super::{AudioCapturer, SAMPLE_RATE};
@@ -14,9 +16,6 @@ use std::thread::{self, JoinHandle};
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
// 48 kHz stereo 32-bit float: 2 channels * 4 bytes = 8 bytes per frame.
const BLOCK_ALIGN: usize = 2 * 4;
pub struct WasapiLoopbackCapturer {
chunks: Receiver<Vec<f32>>,
channels: u32,
@@ -27,8 +26,8 @@ pub struct WasapiLoopbackCapturer {
impl WasapiLoopbackCapturer {
pub fn open(channels: u32) -> Result<WasapiLoopbackCapturer> {
anyhow::ensure!(
channels == 2,
"WASAPI loopback backend is stereo-only (got {channels})"
matches!(channels, 2 | 6 | 8),
"WASAPI loopback backend supports 2/6/8 channels (got {channels})"
);
let (tx, rx) = sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
@@ -39,7 +38,7 @@ impl WasapiLoopbackCapturer {
let join = thread::Builder::new()
.name("punktfunk-wasapi-audio".into())
.spawn(move || {
if let Err(e) = capture_thread(tx, stop_t, ready_tx) {
if let Err(e) = capture_thread(tx, stop_t, ready_tx, channels) {
tracing::error!(error = format!("{e:#}"), "wasapi loopback thread failed");
}
})
@@ -47,7 +46,8 @@ impl WasapiLoopbackCapturer {
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!(
"WASAPI loopback capture: 48 kHz stereo f32 (default render endpoint)"
channels,
"WASAPI loopback capture: 48 kHz f32 (default render endpoint)"
);
Ok(WasapiLoopbackCapturer {
chunks: rx,
@@ -95,7 +95,10 @@ fn capture_thread(
tx: SyncSender<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
channels: u32,
) -> Result<()> {
// Interleaved f32: channels * 4 bytes per frame.
let block_align = channels as usize * 4;
// COM must be initialized on THIS thread (MTA), before any device call.
if let Err(e) = wasapi::initialize_mta()
.ok()
@@ -106,16 +109,29 @@ fn capture_thread(
}
let res = (|| -> Result<()> {
// Loopback = capture the RENDER endpoint: get the default render device, but open a CAPTURE
// client with loopback=true over it.
// client with loopback=true over it. NOTE: the virtual mic (`super::wasapi_mic`) is guarded
// to NEVER target this same endpoint — otherwise the client's injected mic would be captured
// here and streamed back to the client (infinite echo). Keep that guard in sync if this
// device selection ever changes.
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint (loopback needs a render device)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// 48 kHz stereo f32 interleaved; autoconvert lets WASAPI's shared-mode SRC match the engine
// mix format to ours, so we never resample in Rust. Loopback is implied by capturing a
// RENDER device with Direction::Capture in shared mode (wasapi sets STREAMFLAGS_LOOPBACK).
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE as usize, 2, None);
// 48 kHz f32 interleaved in the requested channel layout; autoconvert lets WASAPI's
// shared-mode SRC match the engine mix format to ours (incl. up/downmix to the requested
// channel count), so we never resample/remix in Rust. The explicit dwChannelMask pins the
// wire order (FL FR FC LFE RL RR SL SR; 7.1 = 0x63F, not 0xFF). Loopback is implied by
// capturing a RENDER device with Direction::Capture in shared mode (STREAMFLAGS_LOOPBACK).
let mask = punktfunk_core::audio::wasapi_channel_mask(channels as u8);
let desired = WaveFormat::new(
32,
32,
&SampleType::Float,
SAMPLE_RATE as usize,
channels as usize,
Some(mask),
);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
@@ -151,7 +167,7 @@ fn capture_thread(
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / BLOCK_ALIGN) * BLOCK_ALIGN;
let whole = (bytes.len() / block_align) * block_align;
if whole == 0 {
continue;
}
@@ -5,8 +5,18 @@
//!
//! Target device, by friendly-name substring (first match wins; override with `PUNKTFUNK_MIC_DEVICE`):
//! "Steam Streaming Microphone" (ships with Steam Remote Play — exactly this purpose), VB-Audio
//! "CABLE Input", VoiceMeeter, or anything with "virtual" in the name. If none is present we return an
//! error with install guidance and the host runs without mic passthrough.
//! "CABLE Input", VoiceMeeter, or anything with "virtual" in the name. If none is present we
//! auto-install the Steam Streaming audio pair (see [`install_steam_audio_pair`]); failing that we
//! return an error with install guidance and the host runs without mic passthrough.
//!
//! **Anti-echo guard (the whole point of this being non-trivial).** The desktop-audio plane
//! ([`super::wasapi_cap`]) loopback-captures the **default render endpoint**. WASAPI loopback
//! captures the *mixed* output of an endpoint — i.e. everything any app renders to it, including
//! what THIS module writes. So if the virtual-mic target is the same device the loopback captures,
//! the client's uplinked mic is captured straight back into the host→client audio stream: an
//! infinite echo. [`find_device`] therefore **excludes the default render endpoint** from the
//! candidates — the mic is guaranteed to land on a different device. (Linux gets this for free: its
//! mic is a dedicated `Audio/Source` node, structurally separate from the monitored sink.)
//!
//! `push` enqueues decoded interleaved-f32 PCM into a bounded ring (drop-oldest beyond ~80 ms so mic
//! latency stays bounded); a dedicated COM-apartment thread renders it event-driven, filling silence
@@ -113,8 +123,23 @@ impl VirtualMic for WasapiVirtualMic {
}
}
/// Resolve the virtual-mic target among render endpoints by friendly-name. Logs all candidates so a
/// missing device is diagnosable.
/// The endpoint ID of the device the desktop-audio loopback records (the **default render
/// endpoint**, see [`super::wasapi_cap`]). The virtual mic must never target this device — injecting
/// there echoes the client's mic back into the host→client audio stream. `None` if it can't be
/// resolved (then [`find_device`] can't prove a candidate is safe and falls back to name-only
/// matching — no worse than before the guard existed).
fn default_render_id() -> Option<String> {
wasapi::DeviceEnumerator::new()
.ok()?
.get_default_device(&Direction::Render)
.ok()?
.get_id()
.ok()
}
/// Resolve the virtual-mic target among render endpoints by friendly-name, **excluding the endpoint
/// the loopback captures** (the [`default_render_id`] anti-echo guard). Logs all candidates so a
/// missing/skipped device is diagnosable.
fn find_device() -> Result<wasapi::Device> {
let enumerator = wasapi::DeviceEnumerator::new().context("DeviceEnumerator")?;
let collection = enumerator
@@ -124,8 +149,11 @@ fn find_device() -> Result<wasapi::Device> {
let want = std::env::var("PUNKTFUNK_MIC_DEVICE")
.ok()
.map(|s| s.to_lowercase());
// The device the loopback captures — a name match on it is rejected below (would echo).
let loopback_id = default_render_id();
let mut names = Vec::new();
let mut found = None;
let mut skipped_loopback = false;
for i in 0..n {
let Ok(dev) = collection.get_device_at_index(i) else {
continue;
@@ -137,16 +165,37 @@ fn find_device() -> Result<wasapi::Device> {
None => CANDIDATES.iter().any(|c| lname.contains(c)),
};
if hit && found.is_none() {
found = Some(dev);
// Anti-echo guard: never inject into the endpoint the loopback captures.
let is_loopback = match (dev.get_id().ok(), loopback_id.as_deref()) {
(Some(id), Some(lb)) => id == lb,
_ => false,
};
if is_loopback {
skipped_loopback = true;
tracing::warn!(device = %name,
"virtual-mic candidate is the loopback (default render) endpoint — skipping; \
injecting there would echo the client's mic into the desktop-audio stream");
} else {
found = Some(dev);
}
}
names.push(name);
}
found.ok_or_else(|| {
anyhow!(
"no virtual-mic device among render endpoints {names:?}. Install VB-Audio Virtual Cable \
or enable Steam Remote Play's microphone (Steam Streaming Microphone), or set \
PUNKTFUNK_MIC_DEVICE=<friendly-name substring>."
)
if skipped_loopback {
anyhow!(
"the only virtual-mic candidate among render endpoints {names:?} is the default \
playback device the host loopback-captures — injecting there would echo the mic \
back to the client. Add a SEPARATE virtual audio device for the mic (e.g. the Steam \
Streaming Microphone) or set a different default playback device, then reconnect."
)
} else {
anyhow!(
"no virtual-mic device among render endpoints {names:?}. Install VB-Audio Virtual \
Cable or enable Steam Remote Play's microphone (Steam Streaming Microphone), or set \
PUNKTFUNK_MIC_DEVICE=<friendly-name substring>."
)
}
})
}
@@ -156,15 +205,15 @@ fn find_or_install_device() -> Result<wasapi::Device> {
match find_device() {
Ok(d) => Ok(d),
Err(e) => {
tracing::info!("no virtual mic device present — attempting auto-install");
// SAFETY: `try_install_virtual_mic` is `unsafe` only because it `LoadLibraryExW`s
tracing::info!("no usable virtual mic device present — attempting auto-install");
// SAFETY: `install_steam_audio_pair` is `unsafe` only because it `LoadLibraryExW`s
// `newdev.dll` and calls `DiInstallDriverW` through a `transmute`d function pointer;
// calling it imposes no extra precondition here (it takes no args and aliases nothing).
// Its internal contract holds: the `DiInstall` type matches the documented
// `BOOL DiInstallDriverW(HWND, PCWSTR, DWORD, PBOOL)` ABI, and it passes a
// NUL-terminated UTF-16 INF path with null/zero optional args. Invoked once on the
// dedicated mic thread.
if unsafe { try_install_virtual_mic() } {
if unsafe { install_steam_audio_pair() } {
find_device()
} else {
Err(e)
@@ -173,13 +222,26 @@ fn find_or_install_device() -> Result<wasapi::Device> {
}
}
/// Best-effort: install a virtual mic device so one exists without the user installing anything.
/// Mirrors Apollo's Steam Streaming Speakers install — Steam Remote Play ships
/// `SteamStreamingMicrophone.inf` next to the speakers INF, so install it via `DiInstallDriverW`
/// (loaded from `newdev.dll`, like Apollo, to avoid an extra windows-crate feature). Needs admin (the
/// host runs as SYSTEM). Returns true on success; false (no-op) if Steam isn't installed (INF absent),
/// the install is denied, or `PUNKTFUNK_NO_MIC_INSTALL` is set.
unsafe fn try_install_virtual_mic() -> bool {
/// Best-effort: install BOTH Steam Streaming audio devices (the "Steam pair") so mic passthrough
/// works out of the box and the host has a desktop-audio sink distinct from the mic. Steam Remote
/// Play ships `SteamStreamingMicrophone.inf` + `SteamStreamingSpeakers.inf`: the microphone gives the
/// virtual mic a target whose **capture** endpoint apps record from, and the speakers give a
/// **render** endpoint a headless box can loopback-capture that is NOT the mic — so the loopback and
/// the mic land on different devices and never echo (see [`find_device`]). Returns true if either
/// installed. No-op when Steam isn't installed (INFs absent), the install is denied (needs admin —
/// the host runs as SYSTEM), or `PUNKTFUNK_NO_MIC_INSTALL` is set.
unsafe fn install_steam_audio_pair() -> bool {
// Microphone first (the mic's actual target); speakers second (the distinct desktop-audio sink).
let mic = try_install_steam_audio("SteamStreamingMicrophone.inf");
let spk = try_install_steam_audio("SteamStreamingSpeakers.inf");
mic || spk
}
/// Install one Steam Streaming driver INF by filename via `DiInstallDriverW` (loaded from
/// `newdev.dll`, like Apollo, to avoid an extra windows-crate feature). See
/// [`install_steam_audio_pair`] for the contract; `inf_name` is a bare filename under Steam's
/// per-arch `drivers\Windows10\{arch}\` directory.
unsafe fn try_install_steam_audio(inf_name: &str) -> bool {
use windows::core::{s, w, PCWSTR};
use windows::Win32::Foundation::HWND;
use windows::Win32::System::Environment::ExpandEnvironmentStringsW;
@@ -197,12 +259,11 @@ unsafe fn try_install_virtual_mic() -> bool {
let subdir = "arm64";
#[cfg(not(any(target_arch = "x86_64", target_arch = "aarch64")))]
let subdir = "x86";
let template: Vec<u16> = format!(
"%CommonProgramFiles(x86)%\\Steam\\drivers\\Windows10\\{subdir}\\SteamStreamingMicrophone.inf"
)
.encode_utf16()
.chain(std::iter::once(0))
.collect();
let template: Vec<u16> =
format!("%CommonProgramFiles(x86)%\\Steam\\drivers\\Windows10\\{subdir}\\{inf_name}")
.encode_utf16()
.chain(std::iter::once(0))
.collect();
let mut path = vec![0u16; 1024];
let n = ExpandEnvironmentStringsW(PCWSTR(template.as_ptr()), Some(path.as_mut_slice()));
if n == 0 || n as usize > path.len() {
@@ -210,7 +271,7 @@ unsafe fn try_install_virtual_mic() -> bool {
}
let Ok(newdev) = LoadLibraryExW(w!("newdev.dll"), None, LOAD_LIBRARY_SEARCH_SYSTEM32) else {
tracing::warn!("could not load newdev.dll — virtual-mic auto-install unavailable");
tracing::warn!("could not load newdev.dll — Steam-audio auto-install unavailable");
return false;
};
let Some(addr) = GetProcAddress(newdev, s!("DiInstallDriverW")) else {
@@ -226,13 +287,17 @@ unsafe fn try_install_virtual_mic() -> bool {
std::ptr::null_mut(),
) != 0;
if ok {
tracing::info!("installed the Steam Streaming Microphone virtual device");
tracing::info!(
inf = inf_name,
"installed a Steam Streaming virtual audio device"
);
std::thread::sleep(Duration::from_secs(5)); // let the audio subsystem register the endpoint
} else {
let err = windows::Win32::Foundation::GetLastError();
tracing::info!(
inf = inf_name,
?err,
"no virtual mic auto-installed (Steam absent / not admin) — see manual-install guidance"
"Steam-audio device not auto-installed (Steam absent / not admin) — see install guidance"
);
}
ok
+35 -10
View File
@@ -62,6 +62,11 @@ pub struct OutputFormat {
/// HDR: the capturer converts to 10-bit (IDD-push FP16 → `Rgb10a2`; the DDA secure-desktop HDR hint).
/// `false` = 8-bit SDR.
pub hdr: bool,
/// Full-chroma 4:4:4 session: the capturer must keep full chroma — deliver packed **RGB**
/// (`Bgra` / `Rgb10a2`), NOT the subsampled `Nv12`/`P010` the Windows video-engine path produces by
/// default — because 4:4:4 can only be recovered from a full-chroma source. NVENC then does the
/// RGB→YUV444 CSC at encode (chroma_format_idc=3). `false` on every 4:2:0 session.
pub chroma_444: bool,
}
impl OutputFormat {
@@ -73,6 +78,8 @@ impl OutputFormat {
OutputFormat {
gpu: gpu_encode(),
hdr,
// The GameStream + spike paths are always 4:2:0 (4:4:4 is punktfunk/1-native only).
chroma_444: false,
}
}
}
@@ -361,13 +368,16 @@ pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
#[cfg(target_os = "linux")]
pub fn capture_virtual_output(
vout: crate::vdisplay::VirtualOutput,
_want: OutputFormat,
want: OutputFormat,
_capture: crate::session_plan::CaptureBackend,
) -> Result<Box<dyn Capturer>> {
// The Linux host stays 8-bit (HDR is blocked upstream) and the portal negotiates its own format, so
// the `OutputFormat` is unused here; the capture backend is always the portal (the `CaptureBackend`
// arg is a Windows-only dispatch — ignored here).
linux::PortalCapturer::from_virtual_output(vout).map(|c| Box::new(c) as Box<dyn Capturer>)
// The Linux host stays 8-bit (HDR is blocked upstream) and the portal negotiates its own pixel
// format, so only `want.gpu` is honored here: it gates GPU zero-copy capture (the capture backend
// is always the portal — the `CaptureBackend` arg is a Windows-only dispatch). `gpu = false`
// (a 4:4:4 NVENC session) forces the CPU mmap path so the encoder gets CPU-resident RGB to swscale
// into YUV444P — otherwise it would receive CUDA frames and bail.
linux::PortalCapturer::from_virtual_output(vout, want.gpu)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// `PUNKTFUNK_NO_WGC=1` forces the pure single-process DDA (Desktop Duplication) path everywhere: it
@@ -394,6 +404,14 @@ pub fn capture_virtual_output(
})?;
let pref = vout.preferred_mode;
let keep = vout.keepalive;
// Full-chroma 4:4:4 needs a full-chroma RGB source. The IDD-push and WGC paths emit subsampled
// NV12/P010 by default, which can't reconstruct 4:4:4; route a 4:4:4 session to DDA, which delivers
// RGB (Bgra) when its `chroma_444` flag is set. (IDD-push/WGC 4:4:4 capture is a follow-up.)
if want.chroma_444 && capture != CaptureBackend::Dda {
tracing::info!("4:4:4 session — using DDA capture (RGB source) instead of {capture:?}");
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>);
}
// P2 direct frame push (kill DDA): consume frames straight from the pf-vdisplay driver's shared
// ring — no Desktop Duplication, no win32u reparenting hook. Resolved once in the `SessionPlan`
// (was re-derived from `config().idd_push` here); `IddPush` takes the keepalive (owns the virtual
@@ -414,8 +432,15 @@ pub fn capture_virtual_output(
error = %format!("{e:#}"),
"IDD-push open/attach failed — falling back to DDA"
);
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false)
.map(|c| Box::new(c) as Box<dyn Capturer>);
return dxgi::DuplCapturer::open(
target,
pref,
keep,
want.gpu,
false,
want.chroma_444,
)
.map(|c| Box::new(c) as Box<dyn Capturer>);
}
}
}
@@ -426,7 +451,7 @@ pub fn capture_virtual_output(
// chosen backend (it owns the SudoVDA keepalive), so there's no open-time auto-fallback. The
// backend choice (`dda`/`dxgi`/`PUNKTFUNK_NO_WGC` → DDA, else WGC) is now resolved once in the plan.
if capture == CaptureBackend::Dda {
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false)
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>);
}
// WGC default, with a watchdog'd DDA fallback. WGC's Direct3D11CaptureFramePool::CreateFreeThreaded
@@ -461,12 +486,12 @@ pub fn capture_virtual_output(
}
Ok(Err(e)) => {
tracing::warn!(error = %format!("{e:#}"), "WGC open failed — falling back to DDA");
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false)
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
Err(_) => {
tracing::warn!("WGC open timed out (CreateFreeThreaded hang on the virtual display) — falling back to DDA");
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false)
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
}
+70 -5
View File
@@ -40,6 +40,13 @@ pub struct PortalCapturer {
/// branch to tell "format never negotiated" (modifier/format mismatch) apart from "negotiated
/// but no buffers arrived" (compositor idle/unmapped) — the two black-screen root causes.
negotiated: Arc<AtomicBool>,
/// True only while the PipeWire stream is `Streaming`. [`try_latest`](Self::try_latest) reads it
/// to distinguish a static desktop (alive, no new buffers) from a dead source (left `Streaming`).
streaming: Arc<AtomicBool>,
/// When the stream first dropped out of `Streaming` with no new frame; used to grace a transient
/// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream
/// is `Streaming`.
stall_since: Option<std::time::Instant>,
/// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis.
node_id: u32,
/// Stops the PipeWire loop on teardown (sent in `Drop`). Without it a dropped or failed
@@ -82,21 +89,29 @@ impl PortalCapturer {
node_id,
"ScreenCast portal session started; connecting PipeWire"
);
Ok(spawn_pipewire(Some(fd), node_id, None)?.into_capturer(node_id, None))
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok(spawn_pipewire(Some(fd), node_id, None, true)?.into_capturer(node_id, None))
}
/// Build a capturer from an already-created virtual output ([`crate::vdisplay::VirtualOutput`]):
/// connect PipeWire to its node (`remote_fd` selects portal-remote vs. default-daemon) and
/// take ownership of its keepalive so the output lives exactly as long as this capturer. This
/// is how the client's requested resolution becomes the captured resolution without scaling.
pub fn from_virtual_output(vout: crate::vdisplay::VirtualOutput) -> Result<PortalCapturer> {
/// `allow_zerocopy` mirrors [`OutputFormat::gpu`](crate::capture::OutputFormat): `false` forces the
/// CPU mmap path (a 4:4:4 NVENC session needs CPU-resident RGB), `true` keeps the GPU zero-copy
/// path subject to `PUNKTFUNK_ZEROCOPY`.
pub fn from_virtual_output(
vout: crate::vdisplay::VirtualOutput,
allow_zerocopy: bool,
) -> Result<PortalCapturer> {
tracing::info!(
node_id = vout.node_id,
allow_zerocopy,
"connecting PipeWire to virtual output"
);
let node_id = vout.node_id;
Ok(
spawn_pipewire(vout.remote_fd, node_id, vout.preferred_mode)?
spawn_pipewire(vout.remote_fd, node_id, vout.preferred_mode, allow_zerocopy)?
.into_capturer(node_id, Some(vout.keepalive)),
)
}
@@ -109,6 +124,7 @@ struct PwHandles {
frames: Receiver<CapturedFrame>,
active: Arc<AtomicBool>,
negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>,
}
@@ -121,6 +137,8 @@ impl PwHandles {
frames: self.frames,
active: self.active,
negotiated: self.negotiated,
streaming: self.streaming,
stall_since: None,
node_id,
quit: Some(self.quit),
join: Some(self.join),
@@ -136,6 +154,12 @@ fn spawn_pipewire(
fd: Option<OwnedFd>,
node_id: u32,
preferred: Option<(u32, u32, u32)>,
// Allow GPU zero-copy capture (dmabuf→CUDA/VA). `false` forces the CPU mmap path even when
// `PUNKTFUNK_ZEROCOPY` is set — a 4:4:4 NVENC session needs CPU-resident RGB (the encoder
// swscales RGB→YUV444P; `hevc_nvenc` can't 4:4:4 from a CUDA RGB surface), so the session plan
// passes `gpu = false` for it. Without this, a 4:4:4 session under `PUNKTFUNK_ZEROCOPY=1` would
// get CUDA frames and the encoder would bail (`want_444 && cuda`).
allow_zerocopy: bool,
) -> Result<PwHandles> {
// Frames flow from the pipewire thread over a small bounded channel.
let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8);
@@ -143,11 +167,13 @@ fn spawn_pipewire(
let active_cb = active.clone();
let negotiated = Arc::new(AtomicBool::new(false));
let negotiated_cb = negotiated.clone();
let streaming = Arc::new(AtomicBool::new(false));
let streaming_cb = streaming.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope.
let (quit_tx, quit_rx) = ::pipewire::channel::channel::<()>();
let zerocopy = crate::zerocopy::enabled();
let zerocopy = allow_zerocopy && crate::zerocopy::enabled();
let join = thread::Builder::new()
.name("punktfunk-pipewire".into())
.spawn(move || {
@@ -157,6 +183,7 @@ fn spawn_pipewire(
frame_tx,
active_cb,
negotiated_cb,
streaming_cb,
zerocopy,
preferred,
quit_rx,
@@ -169,6 +196,7 @@ fn spawn_pipewire(
frames: frame_rx,
active,
negotiated,
streaming,
quit: quit_tx,
join,
})
@@ -219,6 +247,28 @@ impl Capturer for PortalCapturer {
}
}
}
if latest.is_some() || self.streaming.load(Ordering::Relaxed) {
// A frame arrived, or the source is alive but idle (static desktop) — normal. Clear any
// stall and repeat the last frame on `None`, exactly as before.
self.stall_since = None;
return Ok(latest);
}
// No new frame AND the stream has left `Streaming` (Paused/Unconnected/Error). The source
// went away — a compositor torn down on a Gaming↔Desktop switch, a removed virtual output.
// Grace a brief window (a transient mid-stream renegotiation can blip out of Streaming and
// back) before declaring it lost so the encode loop rebuilds in place rather than freezing
// on the last frame forever.
const STALL_GRACE: Duration = Duration::from_millis(1500);
let since = *self.stall_since.get_or_insert_with(std::time::Instant::now);
if since.elapsed() >= STALL_GRACE {
self.stall_since = None;
return Err(anyhow!(
"PipeWire source stalled (node {}): stream left Streaming for >{}ms with no frames \
— the compositor/virtual output went away (session switch?)",
self.node_id,
STALL_GRACE.as_millis()
));
}
Ok(latest)
}
@@ -467,6 +517,10 @@ mod pipewire {
/// Set once a video format is agreed (`param_changed`), so a first-frame timeout can tell
/// "format never negotiated" apart from "negotiated but no buffers arrived".
negotiated: Arc<AtomicBool>,
/// True only while the PipeWire stream is in `Streaming` (the source is alive). Goes false on
/// `Paused`/`Unconnected`/`Error` — the source vanished (compositor torn down on a session
/// switch). Read by [`PortalCapturer::try_latest`] to surface a sustained drop as a loss.
streaming: Arc<AtomicBool>,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer.
importer: Option<crate::zerocopy::EglImporter>,
/// VAAPI zero-copy: hand the raw dmabuf to the encoder (which imports + GPU-CSCs it) instead
@@ -1056,6 +1110,7 @@ mod pipewire {
tx: SyncSender<CapturedFrame>,
active: Arc<AtomicBool>,
negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
zerocopy: bool,
preferred: Option<(u32, u32, u32)>,
quit_rx: pw::channel::Receiver<()>,
@@ -1150,6 +1205,7 @@ mod pipewire {
tx,
active,
negotiated,
streaming,
importer,
vaapi_passthrough,
nv12: crate::zerocopy::nv12_enabled(),
@@ -1174,8 +1230,17 @@ mod pipewire {
let _listener = stream
.add_local_listener_with_user_data(data)
.state_changed(|_stream, _ud, old, new| {
.state_changed(|_stream, ud, old, new| {
tracing::info!(?old, ?new, "pipewire stream state");
// Track whether the node is actively producing. A live source sits in `Streaming`
// (a static desktop just sends no buffers); anything else — `Paused`/`Unconnected`/
// `Error` — means the source went away (compositor died, virtual output removed on a
// Gaming↔Desktop switch). `try_latest` turns a sustained non-Streaming state into a
// capture-loss so the encode loop rebuilds instead of freezing on the last frame.
ud.streaming.store(
matches!(new, pw::stream::StreamState::Streaming),
Ordering::Relaxed,
);
})
.param_changed(|_stream, ud, id, param| {
let Some(param) = param else { return };
@@ -1,5 +1,5 @@
//! Input-desktop watcher (Windows) — the authoritative "normal vs secure desktop" signal for the
//! two-process secure-desktop design (docs/windows-secure-desktop.md).
//! two-process secure-desktop design (design/archive/windows-secure-desktop.md).
//!
//! Windows switches the *input desktop* to "Winlogon" (the secure desktop) for UAC elevation, the
//! lock screen and the login screen, and back to "Default" for the normal session. WGC captures only
@@ -2010,6 +2010,10 @@ pub struct DuplCapturer {
/// first, retried (legacy DuplicateOutput can't capture HDR). Set for the secure-desktop DDA leg
/// when the SudoVDA is in HDR; threaded into every (re)duplication incl. ACCESS_LOST recovery.
want_hdr: bool,
/// Full-chroma 4:4:4 session: deliver packed RGB (`Bgra` SDR / `Rgb10a2` HDR) and SKIP the
/// video-engine RGB→YUV (NV12/P010) conversion — NVENC reconstructs 4:4:4 only from a full-chroma
/// source, so we hand it the RGB texture and it CSCs to YUV444 at encode (chroma_format_idc=3).
chroma_444: bool,
/// HDR (scRGB FP16) capture state. Set when the duplication surface is `R16G16B16A16_FLOAT`
/// (the desktop has HDR on). The frame can't be `CopyResource`d into a BGRA target, so the HDR
/// path copies it into an FP16 SRV texture, composites the cursor, then runs [`HdrConverter`] to
@@ -2087,6 +2091,8 @@ impl DuplCapturer {
// stage 5) so the capturer never re-derives the encode backend itself.
gpu: bool,
want_hdr: bool,
// 4:4:4 session → deliver RGB, skip the NV12/P010 video-engine conversion (see the field doc).
chroma_444: bool,
) -> Result<Self> {
// SAFETY: runs on the capture thread that will own this `DuplCapturer`. `install_gpu_pref_hook()`
// and the DPI-context calls take by-value handles / no args and touch only thread/process state;
@@ -2311,6 +2317,7 @@ impl DuplCapturer {
gpu_copy: None,
last_present: None,
want_hdr,
chroma_444,
hdr_fp16: is_hdr_init,
hdr_meta: hdr_meta_init,
fp16_src: None,
@@ -3088,7 +3095,10 @@ impl DuplCapturer {
// Video-engine path: scRGB FP16 → BT.2020 PQ P010 on the VIDEO engine (no 3D shader, and
// NVENC encodes P010 natively). Fall back to the HdrConverter pixel shader (3D) only if the
// video processor is unavailable.
if let Some(p010) = self.convert_to_yuv(&src, true) {
if let Some(p010) = (!self.chroma_444)
.then(|| self.convert_to_yuv(&src, true))
.flatten()
{
self.last_present = Some((p010.clone(), PixelFormat::P010));
return Ok(CapturedFrame {
width: self.width,
@@ -3148,7 +3158,10 @@ impl DuplCapturer {
// conversion AND NVENC's encode stay OFF the 3D engine — the only way to keep up when a
// game pins the 3D engine at ~100%. Fall back to handing NVENC the BGRA texture (it then
// does RGB→YUV internally on the 3D/compute engine).
if let Some(nv12) = self.convert_to_yuv(&gpu, false) {
if let Some(nv12) = (!self.chroma_444)
.then(|| self.convert_to_yuv(&gpu, false))
.flatten()
{
self.last_present = Some((nv12.clone(), PixelFormat::Nv12));
return Ok(CapturedFrame {
width: self.width,
@@ -562,7 +562,7 @@ impl IddPushCapturer {
/// Block (bounded) until the driver has ATTACHED to the host ring (`DRV_STATUS_OPENED`) **and published
/// a first frame**, else fail so the caller can fall back to DDA (audit §5.1 +
/// `docs/windows-host-rewrite.md` §2.5 — the GB1 game-capture fix).
/// `design/windows-host-rewrite.md` §2.5 — the GB1 game-capture fix).
///
/// Requiring the first frame — not just the attach — catches the *reconnect-into-a-broken-state* case:
/// a fullscreen game can leave the virtual display in a format/size that the driver's `publish()` guard
@@ -1,5 +1,5 @@
//! Host-side WGC helper relay (Windows two-process secure-desktop design,
//! docs/windows-secure-desktop.md — step 4).
//! design/archive/windows-secure-desktop.md — step 4).
//!
//! WGC won't activate under the SYSTEM account, so the SYSTEM host can't capture the normal desktop
//! itself. Instead it spawns `punktfunk-host wgc-helper` in the **interactive user session** (so WGC works)
+7 -2
View File
@@ -4,10 +4,10 @@
//! environment before the host starts, and **for the knobs captured here the environment is constant for the
//! process lifetime**, so a lazily-parsed global is equivalent to "parsed once at startup".
//!
//! **Goal-1 stages 12** (`docs/windows-host-rewrite.md` §2.2): stage 1 stood this up; stage 2 migrated the
//! **Goal-1 stages 12** (`design/windows-host-rewrite.md` §2.2): stage 1 stood this up; stage 2 migrated the
//! genuinely-constant operator/dispatch knobs onto it (the dispatch-disagreement bug class: `idd_push`,
//! `capture_backend`, `encoder_pref`, `render_adapter`, `no_wgc`, the vdisplay backend select — plus the
//! plan-named `secure_dda`/`idd_depth`/`zerocopy`/`ten_bit` and the multi-site `perf`/`compositor`/
//! plan-named `secure_dda`/`idd_depth`/`zerocopy`/`ten_bit`/`four_four_four` and the multi-site `perf`/`compositor`/
//! `video_source`/`gamepad`). `SessionPlan` (stage 3) consumes it as the single owner of the
//! capture/topology/encoder decision.
//!
@@ -63,6 +63,10 @@ pub struct HostConfig {
pub zerocopy: bool,
/// `PUNKTFUNK_10BIT` — host policy gate for HEVC Main10 (only honored when the client also advertised 10-bit).
pub ten_bit: bool,
/// `PUNKTFUNK_444` — host policy gate for full-chroma HEVC 4:4:4 (Range Extensions). Honored only
/// when the client also advertised 4:4:4, the codec is HEVC, and the GPU/driver supports a 4:4:4
/// encode (probed) — otherwise the session stays 4:2:0. Independent of `ten_bit` (chroma vs depth).
pub four_four_four: bool,
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
pub perf: bool,
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic).
@@ -112,6 +116,7 @@ impl HostConfig {
.unwrap_or(2),
zerocopy: flag("PUNKTFUNK_ZEROCOPY"),
ten_bit: flag("PUNKTFUNK_10BIT"),
four_four_four: flag("PUNKTFUNK_444"),
perf: flag("PUNKTFUNK_PERF"),
video_source: val("PUNKTFUNK_VIDEO_SOURCE"),
compositor: val("PUNKTFUNK_COMPOSITOR"),
+134 -3
View File
@@ -29,6 +29,33 @@ pub enum Codec {
Av1,
}
/// Chroma subsampling the encoder emits, negotiated with the client (the `PUNKTFUNK_444` gate + the
/// client's `VIDEO_CAP_444` + a GPU probe). `Yuv420` is the universal default; `Yuv444` is HEVC-only,
/// native-protocol-only (GameStream stays 4:2:0), and the host only ever passes it after
/// [`can_encode_444`] confirmed the active backend supports it.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum ChromaFormat {
#[default]
Yuv420,
Yuv444,
}
impl ChromaFormat {
/// The HEVC `chroma_format_idc` this maps to: `1` (4:2:0) or `3` (4:4:4). Also the wire value
/// echoed in [`punktfunk_core::quic::Welcome::chroma_format`].
pub fn idc(self) -> u8 {
match self {
ChromaFormat::Yuv420 => punktfunk_core::quic::CHROMA_IDC_420,
ChromaFormat::Yuv444 => punktfunk_core::quic::CHROMA_IDC_444,
}
}
/// True for full-chroma 4:4:4.
pub fn is_444(self) -> bool {
matches!(self, ChromaFormat::Yuv444)
}
}
impl Codec {
/// The FFmpeg NVENC encoder name (selected by name, not codec id — the latter would
/// pick the software encoder).
@@ -89,6 +116,13 @@ pub struct EncoderCaps {
/// When `false`, `set_hdr_meta` is a no-op and no in-band grade reaches the client. Only the
/// Windows direct-NVENC path attaches it today.
pub supports_hdr_metadata: bool,
/// The opened encoder is actually producing a full-chroma 4:4:4 (`chroma_format_idc = 3`) stream.
/// `false` on every 4:2:0 session (the default) and on a backend that declined 4:4:4. Set by the
/// NVENC backends (Linux + Windows). The chroma is committed to the wire (`Welcome::chroma_format`)
/// from the pre-open probe, so this is a *post-open cross-check*: the session glue logs loudly if
/// the encoder's real chroma disagrees with what was negotiated (the in-band SPS is authoritative
/// for the decoder either way).
pub chroma_444: bool,
}
/// A hardware encoder. One per session; runs on the encode thread.
@@ -193,8 +227,29 @@ pub fn open_video(
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Box<dyn Encoder>> {
validate_dimensions(codec, width, height)?;
// Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`)
// and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero.
// The mid-stream Reconfigure path already guards `refresh_hz > 0`; enforcing it at this single
// open chokepoint makes EVERY path (initial Hello, GameStream ANNOUNCE, Reconfigure) safe
// regardless of which backend opens (security-review 2026-06-28 S5).
if fps == 0 || fps > 1000 {
anyhow::bail!("invalid refresh/fps {fps}: must be 1..=1000 Hz");
}
// 4:4:4 is HEVC-only. The negotiator should never pass `Yuv444` for another codec (it gates on
// `codec == H265`), but defend the contract here so a future caller can't silently emit a stream
// no decoder expects: a non-HEVC 4:4:4 request degrades to 4:2:0 with a warning.
let chroma = if chroma.is_444() && codec != Codec::H265 {
tracing::warn!(
?codec,
"4:4:4 requested for a non-HEVC codec — encoding 4:2:0"
);
ChromaFormat::Yuv420
} else {
chroma
};
#[cfg(target_os = "linux")]
{
// Pick the GPU encode backend. NVIDIA → NVENC/CUDA (the original path, unchanged);
@@ -203,8 +258,17 @@ pub fn open_video(
// its errors crisply instead of silently trying the other).
let pref = crate::config::config().encoder_pref.as_str();
let open_vaapi = || -> Result<Box<dyn Encoder>> {
vaapi::VaapiEncoder::open(codec, format, width, height, fps, bitrate_bps, bit_depth)
.map(|e| Box::new(e) as Box<dyn Encoder>)
vaapi::VaapiEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| Box::new(e) as Box<dyn Encoder>)
};
match pref {
"nvenc" | "nvidia" | "cuda" => open_nvenc_probed(
@@ -216,6 +280,7 @@ pub fn open_video(
bitrate_bps,
cuda,
bit_depth,
chroma,
),
"vaapi" | "amd" | "intel" => open_vaapi(),
"auto" | "" => {
@@ -231,6 +296,7 @@ pub fn open_video(
bitrate_bps,
cuda,
bit_depth,
chroma,
)
} else {
open_vaapi()
@@ -260,6 +326,7 @@ pub fn open_video(
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| Box::new(e) as Box<dyn Encoder>)
}
@@ -289,6 +356,7 @@ pub fn open_video(
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| Box::new(e) as Box<dyn Encoder>)
}
@@ -333,6 +401,7 @@ pub fn open_video(
bitrate_bps,
cuda,
bit_depth,
chroma,
);
anyhow::bail!("video encode requires Linux or Windows")
}
@@ -355,6 +424,7 @@ fn open_nvenc_probed(
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Box<dyn Encoder>> {
const MIN_PROBE_BPS: u64 = 50_000_000;
let mut candidates = vec![bitrate_bps];
@@ -369,7 +439,9 @@ fn open_nvenc_probed(
}
let mut last: Option<anyhow::Error> = None;
for (i, &b) in candidates.iter().enumerate() {
match linux::NvencEncoder::open(codec, format, width, height, fps, b, cuda, bit_depth) {
match linux::NvencEncoder::open(
codec, format, width, height, fps, b, cuda, bit_depth, chroma,
) {
Ok(enc) => {
if i > 0 {
tracing::warn!(
@@ -446,6 +518,65 @@ pub fn vaapi_codec_support() -> CodecSupport {
})
}
/// Whether the active GPU encode backend can actually produce a full-chroma **4:4:4** HEVC stream.
/// Resolved (and cached, once) *before* the Welcome so the host advertises the chroma it will really
/// encode — the honest-downgrade channel. 4:4:4 is HEVC-only; the probe opens a tiny encoder on the
/// active backend (NVENC FREXT is broad on NVIDIA, but VAAPI / AMF / QSV 4:4:4 is hardware-specific,
/// so it must be probed, never assumed). Non-HEVC codecs are always `false`.
#[cfg(any(target_os = "linux", target_os = "windows"))]
pub fn can_encode_444(codec: Codec) -> bool {
use std::sync::OnceLock;
if codec != Codec::H265 {
return false;
}
static CACHE: OnceLock<bool> = OnceLock::new();
*CACHE.get_or_init(|| {
let supported = {
#[cfg(target_os = "linux")]
{
// Mirror open_video's backend dispatch: VAAPI (AMD/Intel) vs NVENC (NVIDIA).
if linux_zero_copy_is_vaapi() {
vaapi::probe_can_encode_444(codec)
} else {
linux::probe_can_encode_444(codec)
}
}
#[cfg(target_os = "windows")]
{
match windows_resolved_backend() {
WindowsBackend::Nvenc => {
#[cfg(feature = "nvenc")]
{
nvenc::probe_can_encode_444(codec)
}
#[cfg(not(feature = "nvenc"))]
{
false
}
}
WindowsBackend::Amf | WindowsBackend::Qsv => {
#[cfg(feature = "amf-qsv")]
{
let vendor = match windows_resolved_backend() {
WindowsBackend::Qsv => ffmpeg_win::WinVendor::Qsv,
_ => ffmpeg_win::WinVendor::Amf,
};
ffmpeg_win::probe_can_encode_444(vendor, codec)
}
#[cfg(not(feature = "amf-qsv"))]
{
false
}
}
WindowsBackend::Software => false,
}
}
};
tracing::info!(supported, "HEVC 4:4:4 encode capability probed");
supported
})
}
// ---------------------------------------------------------------------------------------------
// Windows backend selection (the analogue of the Linux nvidia_present / linux_zero_copy_is_vaapi
// logic). NVIDIA → NVENC, AMD → AMF, Intel → QSV; `auto` (default) reads the DXGI adapter vendor.
+205 -11
View File
@@ -11,7 +11,7 @@
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{Codec, EncodedFrame, Encoder};
use super::{ChromaFormat, Codec, EncodedFrame, Encoder};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use ffmpeg::format::Pixel;
@@ -19,9 +19,33 @@ use ffmpeg::util::frame::Video as VideoFrame;
use ffmpeg::{codec, encoder, Dictionary, Packet, Rational};
use ffmpeg_next as ffmpeg;
use std::os::raw::c_int;
use std::ptr;
use ffmpeg::ffi; // = ffmpeg_sys_next
/// swscale: nearest-neighbour scaler flag (`SWS_POINT`). We never rescale (src dims == dst dims), so
/// the resampler choice only governs the colour-conversion path; POINT is the cheapest.
const SWS_POINT: c_int = 0x10;
/// swscale colorspace id for ITU-R BT.709 (`SWS_CS_ITU709`) — the CSC coefficients for our RGB→YUV.
const SWS_CS_ITU709: c_int = 1;
/// The swscale *source* pixel format for a captured packed RGB/BGR layout (the real byte order, not
/// the NVENC-padded `*0` form). Used by the 4:4:4 RGB→YUV444P conversion path. Mirrors the VAAPI
/// CPU-input mapping; YUV/10-bit inputs can't feed this path (the 4:4:4 session forces packed RGB).
fn sws_src_pixel(format: PixelFormat) -> Result<Pixel> {
Ok(match format {
PixelFormat::Bgrx => Pixel::BGRZ, // bgr0
PixelFormat::Rgbx => Pixel::RGBZ, // rgb0
PixelFormat::Bgra => Pixel::BGRA,
PixelFormat::Rgba => Pixel::RGBA,
PixelFormat::Rgb => Pixel::RGB24,
PixelFormat::Bgr => Pixel::BGR24,
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 => {
bail!("NVENC 4:4:4 CPU-input path supports packed RGB/BGR only; got {format:?}")
}
})
}
/// `AVCUDADeviceContext` (libavutil/hwcontext_cuda.h) — not in the ffmpeg-sys bindings (the
/// crate doesn't allowlist that header), so mirror its stable 3-pointer layout. We set the
/// first field to *our* `CUcontext` so NVENC shares the context the EGL importer maps into.
@@ -131,6 +155,10 @@ pub struct NvencEncoder {
frame: Option<VideoFrame>,
/// Zero-copy path: CUDA hwdevice/hwframes contexts (the encoder takes `AV_PIX_FMT_CUDA`).
cuda: Option<CudaHw>,
/// 4:4:4 path only: swscale context converting the captured packed RGB/BGR → planar YUV444P
/// (BT.709 limited) into [`Self::frame`], because `hevc_nvenc` only emits 4:4:4 from a YUV444
/// *input* (RGB-in is always 4:2:0). `None` on the ordinary 4:2:0 RGB path. Freed in `Drop`.
sws_444: Option<*mut ffi::SwsContext>,
src_format: PixelFormat,
expand: bool,
width: u32,
@@ -142,10 +170,12 @@ pub struct NvencEncoder {
force_kf: bool,
}
// `CudaHw` holds raw `AVBufferRef`s; the encoder lives on a single thread. The CPU encoder is
// already `Send` via ffmpeg-next; assert it for the CUDA fields too.
// `CudaHw` holds raw `AVBufferRef`s and `sws_444` a raw `SwsContext`; the encoder lives on a single
// thread. The CPU encoder is already `Send` via ffmpeg-next; assert it for the raw fields too.
// SAFETY: `NvencEncoder` owns an ffmpeg-next `Encoder`/`VideoFrame` (already `Send`) plus a `CudaHw`
// holding raw `AVBufferRef`s, which are not `Send` by default. The encoder is owned and driven by
// holding raw `AVBufferRef`s and an optional raw `SwsContext`, none of which are `Send` by default.
// The `SwsContext` is a self-contained swscale state object with no thread affinity, touched only
// through `&mut self` on the one encode thread. The encoder is owned and driven by
// exactly ONE thread — the per-session encode thread it is moved to — and is only touched through
// `&mut self` methods, so it is never aliased or accessed concurrently. The wrapped libav contexts
// (and the shared `CUcontext` the `CudaHw` references) have no thread affinity, so transferring
@@ -164,6 +194,7 @@ impl NvencEncoder {
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Self> {
// TODO(hdr): Linux 10-bit parity. Unlike the Windows raw-SDK path (which upconverts 8-bit
// ARGB → Main10 via pixelBitDepthMinus8), libavcodec hevc_nvenc needs a 10-bit input pixel
@@ -175,6 +206,18 @@ impl NvencEncoder {
"Linux NVENC 10-bit not yet wired — encoding 8-bit"
);
}
// Full-chroma 4:4:4 (HEVC Range Extensions). `hevc_nvenc` only emits 4:4:4 from a YUV444
// *input* frame — feeding RGB always subsamples to 4:2:0 regardless of profile (verified on
// the RTX 5070 Ti). So a 4:4:4 session swscales the captured RGB → YUV444P (BT.709 limited)
// and feeds that with `profile=rext`. The negotiator gates this to HEVC + the single-process
// CPU-capture topology, so `cuda` must be false here; defend the contract.
let want_444 = chroma.is_444() && codec == Codec::H265;
if want_444 && cuda {
bail!(
"NVENC 4:4:4 needs CPU RGB frames (the session forces non-zero-copy capture for \
4:4:4); got a CUDA frame capture/encoder negotiation mismatch"
);
}
ffmpeg::init().context("ffmpeg init")?;
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
// SAFETY: `av_log_set_level` sets libav's global integer log level; `48` (= AV_LOG_DEBUG)
@@ -185,7 +228,14 @@ impl NvencEncoder {
let name = codec.nvenc_name();
let av_codec = encoder::find_by_name(name)
.ok_or_else(|| anyhow!("{name} not built into libavcodec"))?;
let (nvenc_pixel, expand) = nvenc_input(format);
let (rgb_pixel, rgb_expand) = nvenc_input(format);
// 4:4:4 feeds NVENC a planar YUV444P frame we produce by swscale; the ordinary path feeds the
// captured RGB straight in and lets NVENC's internal CSC subsample to 4:2:0.
let (nvenc_pixel, expand) = if want_444 {
(Pixel::YUV444P, false)
} else {
(rgb_pixel, rgb_expand)
};
let mut video = codec::context::Context::new_with_codec(av_codec)
.encoder()
@@ -234,12 +284,12 @@ impl NvencEncoder {
(*video.as_mut_ptr()).gop_size = -1;
}
// NV12 path: we did the RGB→YUV conversion ourselves as BT.709 *limited* range, so signal
// that in the bitstream VUI (colorspace/range/primaries/transfer) — otherwise the client
// decoder assumes a default and the picture comes out washed-out / wrong-contrast. The
// RGB-input paths leave these unset (NVENC's internal CSC writes its own VUI). Matches the
// Windows NV12 path's BT.709 limited-range signalling.
if matches!(format, PixelFormat::Nv12) {
// NV12 / 4:4:4 paths: we do the RGB→YUV conversion ourselves as BT.709 *limited* range
// (swscale), so signal that in the bitstream VUI (colorspace/range/primaries/transfer) —
// otherwise the client decoder assumes a default and the picture comes out washed-out /
// wrong-contrast. The RGB-input 4:2:0 path leaves these unset (NVENC's internal CSC writes
// its own VUI). Matches the Windows NV12 path's BT.709 limited-range signalling.
if matches!(format, PixelFormat::Nv12) || want_444 {
// SAFETY: same `video` builder — `raw = video.as_mut_ptr()` is the non-null, properly-
// aligned, sole-owned, not-yet-opened `AVCodecContext`. We set its four VUI colour enum
// fields to valid `AVColorSpace`/`AVColorRange`/`AVColorPrimaries`/`AVColorTransfer-
@@ -280,6 +330,45 @@ impl NvencEncoder {
None
};
// 4:4:4: build the RGB→YUV444P swscale (BT.709 limited, no rescale). Mirrors the VAAPI CPU
// path's RGB→NV12 scaler, but the dst is full-chroma planar 4:4:4.
let sws_444 = if want_444 {
let src_av = pixel_to_av(sws_src_pixel(format)?);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated, packed-RGB-only source), the
// dst is YUV444P. The trailing filter/param pointers are null = "use defaults" (documented
// as accepted). No Rust memory is borrowed; the returned pointer is null-checked below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
height as c_int,
src_av,
width as c_int,
height as c_int,
ffi::AVPixelFormat::AV_PIX_FMT_YUV444P,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
ptr::null(),
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→YUV444P) failed");
}
// SAFETY: `sws` is the non-null context from the call above (null-checked). The ITU-709
// coefficient table from `sws_getCoefficients` is a process-lifetime libswscale static,
// reused for src+dst matrices; `sws_setColorspaceDetails` only reads it and writes scalar
// CSC settings into `sws` (limited-range dst: dstRange = 0). No Rust memory is passed.
unsafe {
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, 0, 0, 1 << 16, 1 << 16);
}
Some(sws)
} else {
None
};
// Low-latency NVENC tuning (plan §7 / linux-setup doc).
let mut opts = Dictionary::new();
opts.set("preset", "p1"); // fastest
@@ -288,6 +377,12 @@ impl NvencEncoder {
opts.set("bf", "0");
opts.set("delay", "0");
opts.set("forced-idr", "1"); // RFI/request_keyframe → real IDR under the infinite GOP
if want_444 {
// HEVC Range Extensions — the profile that carries chroma_format_idc=3. With a YUV444P
// input `hevc_nvenc` auto-selects it, but pin it explicitly so the chroma is never silently
// dropped on a future libavcodec.
opts.set("profile", "rext");
}
// Split-frame encode across both NVENC engines (GB203 has 2) when the pixel rate exceeds
// a single engine's HEVC capacity (~1 Gpix/s); e.g. 5120x1440@240 = 1.77 Gpix/s needs it,
@@ -321,6 +416,7 @@ impl NvencEncoder {
enc,
frame,
cuda: cuda_hw,
sws_444,
src_format: format,
expand,
width,
@@ -333,6 +429,15 @@ impl NvencEncoder {
}
impl Encoder for NvencEncoder {
fn caps(&self) -> super::EncoderCaps {
super::EncoderCaps {
// 4:4:4 iff this session opened the RGB→YUV444P swscale path (FREXT). RFI/HDR-SEI stay
// unsupported on libavcodec NVENC (the trait defaults).
chroma_444: self.sws_444.is_some(),
..super::EncoderCaps::default()
}
}
fn submit(&mut self, captured: &CapturedFrame) -> Result<()> {
anyhow::ensure!(
captured.width == self.width && captured.height == self.height,
@@ -411,6 +516,47 @@ impl NvencEncoder {
bytes.len(),
src_row * h
);
// 4:4:4: swscale the packed RGB straight into the planar YUV444P input frame (BT.709 limited),
// then send it — no byte-expand. The 4:2:0 RGB path (below) feeds NVENC packed RGB directly.
if let Some(sws) = self.sws_444 {
let frame = self
.frame
.as_mut()
.context("CPU frame missing (encoder opened in CUDA mode)")?;
// SAFETY: `format == self.src_format` and `bytes.len() >= src_row * h` (the `ensure!`s
// above), so `sws_scale` reads `h` rows of `src_row` bytes from `src_data[0] = bytes`
// (packed RGB is single-plane; the other src planes are null/0) — all in bounds. `sws` is
// the non-null context built in `open`. The dst is `frame`'s underlying `AVFrame`: its
// `data`/`linesize` in-struct arrays were sized for YUV444P by `VideoFrame::new`, and the
// 3 planes are each `width`×`height`. All pointers are live locals for this synchronous
// call; the encoder runs only on this thread (`unsafe impl Send`), so no aliasing/race.
unsafe {
let dst_av = frame.as_mut_ptr();
let src_data: [*const u8; 4] =
[bytes.as_ptr(), ptr::null(), ptr::null(), ptr::null()];
let src_stride: [c_int; 4] = [src_row as c_int, 0, 0, 0];
let r = ffi::sws_scale(
sws,
src_data.as_ptr(),
src_stride.as_ptr(),
0,
h as c_int,
(*dst_av).data.as_ptr(),
(*dst_av).linesize.as_ptr(),
);
if r < 0 {
bail!("sws_scale(RGB→YUV444P) failed ({r})");
}
}
frame.set_pts(Some(pts));
frame.set_kind(if idr {
ffmpeg::picture::Type::I
} else {
ffmpeg::picture::Type::None
});
self.enc.send_frame(frame).context("send_frame(444)")?;
return Ok(());
}
let frame = self
.frame
.as_mut()
@@ -526,3 +672,51 @@ impl NvencEncoder {
Ok(())
}
}
impl Drop for NvencEncoder {
fn drop(&mut self) {
if let Some(sws) = self.sws_444.take() {
// SAFETY: `sws` is the non-null `SwsContext` allocated by `sws_getContext` in `open` and
// owned exclusively by this encoder (taken out of the field so it can't be freed twice).
// `sws_freeContext` frees it; nothing else references it after this single-threaded drop.
unsafe { ffi::sws_freeContext(sws) };
}
}
}
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode HEVC **4:4:4** (Range
/// Extensions). Opens a tiny real `hevc_nvenc` 4:4:4 session — the exact path [`NvencEncoder::open`]
/// takes for a live 4:4:4 stream — and reports whether it succeeded. HEVC-only; the result is cached
/// by the caller ([`crate::encode::can_encode_444`]). A GPU/driver/ffmpeg without RExt 4:4:4 fails
/// the open here, so the host resolves the session to 4:2:0 before the Welcome (honest downgrade).
pub fn probe_can_encode_444(codec: Codec) -> bool {
if codec != Codec::H265 {
return false;
}
if ffmpeg::init().is_err() {
return false;
}
// Quiet ffmpeg's open error on a GPU that lacks 4:4:4 — the probe failing is an expected outcome.
// SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
// (no pointer args) and are always sound post-init.
let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
let ok = NvencEncoder::open(
codec,
PixelFormat::Bgra,
640,
480,
30,
2_000_000,
false, // CPU input (the 4:4:4 path never uses CUDA)
8,
ChromaFormat::Yuv444,
)
.is_ok();
// SAFETY: restore the saved global log level (scalar arg, no pointers).
unsafe { ffi::av_log_set_level(prev) };
ok
}
@@ -160,6 +160,18 @@ pub fn probe_can_encode(codec: Codec) -> bool {
}
}
/// Whether the active VAAPI GPU can encode HEVC **4:4:4** (Range Extensions). **Deferred in v1 —
/// always `false`.** VAAPI HEVC 4:4:4 encode is narrow and vendor-specific (the lab's AMD Phoenix1 /
/// RDNA3 exposes only `VAProfileHEVCMain`/`Main10` `EncSlice`, no `Main444`), and there is no
/// validated hardware to build + verify the 4:4:4 surface/profile path against. Returning `false`
/// keeps the negotiation honest: a VAAPI host resolves every session to 4:2:0 before the Welcome, so
/// the client never builds a 4:4:4 decoder it would only get 4:2:0 frames for. (Follow-up: implement
/// and validate on an Intel Arc / RDNA4-class box that advertises a HEVC 4:4:4 encode entrypoint.)
pub fn probe_can_encode_444(_codec: Codec) -> bool {
tracing::info!("VAAPI HEVC 4:4:4 encode is not implemented yet — declining (encoding 4:2:0)");
false
}
/// Drain the encoder for one packet (shared poll logic).
fn poll_encoder(enc: &mut encoder::video::Encoder, fps: u32) -> Result<Option<EncodedFrame>> {
let mut pkt = Packet::empty();
@@ -848,6 +860,7 @@ pub struct VaapiEncoder {
unsafe impl Send for VaapiEncoder {}
impl VaapiEncoder {
#[allow(clippy::too_many_arguments)]
pub fn open(
codec: Codec,
format: PixelFormat,
@@ -856,10 +869,18 @@ impl VaapiEncoder {
fps: u32,
bitrate_bps: u64,
bit_depth: u8,
chroma: super::ChromaFormat,
) -> Result<Self> {
if bit_depth != 8 {
tracing::warn!(bit_depth, "VAAPI 10-bit not yet wired — encoding 8-bit");
}
// VAAPI 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel hardware in the
// lab exposes a HEVC 4:4:4 encode entrypoint, and the probe returns false so the host never
// negotiates 4:4:4 for a VAAPI session. If a request slips through, fall back to 4:2:0 rather
// than emit an unverified stream — the host signalled 4:2:0 in the Welcome anyway.
if chroma.is_444() {
tracing::warn!("VAAPI 4:4:4 encode not implemented — encoding 4:2:0");
}
ffmpeg::init().context("ffmpeg init")?;
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
// SAFETY: `av_log_set_level` sets libav's global integer log level; `48` (= AV_LOG_DEBUG)
@@ -31,7 +31,7 @@
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{Codec, EncodedFrame, Encoder};
use super::{ChromaFormat, Codec, EncodedFrame, Encoder};
use crate::capture::{dxgi::D3d11Frame, CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use ffmpeg::format::Pixel;
@@ -241,6 +241,18 @@ unsafe fn open_win_encoder(
/// driver/runtime rejects codecs the video engine can't do (AV1 on pre-RDNA3 AMD / pre-Arc Intel,
/// or HEVC on a very old part). Used to build the GameStream codec advertisement so a client never
/// negotiates a codec the encoder can't open. Torn down immediately.
/// Whether the active AMD (AMF) / Intel (QSV) GPU can encode HEVC **4:4:4**. **Deferred in v1 —
/// always `false`.** AMF/QSV HEVC 4:4:4 encode is narrow (AMD RDNA3+, Intel Arc/Xe2+) and the
/// libavcodec profile/pixel-format incantation is vendor- and driver-specific — a wrong profile
/// `avcodec_open2` *silently* falls back to 4:2:0, so a positive probe would need a verify-by-frame,
/// and there is no AMD/Intel Windows box in the lab to build + validate that against. Returning
/// `false` keeps the negotiation honest: an AMF/QSV host resolves every session to 4:2:0 before the
/// Welcome. (Follow-up: implement + validate on an RDNA3+/Arc Windows box.)
pub fn probe_can_encode_444(_vendor: WinVendor, _codec: Codec) -> bool {
tracing::info!("AMF/QSV HEVC 4:4:4 encode is not implemented yet — declining (encoding 4:2:0)");
false
}
pub fn probe_can_encode(vendor: WinVendor, codec: Codec) -> bool {
if ffmpeg::init().is_err() {
return false;
@@ -1096,6 +1108,7 @@ pub struct FfmpegWinEncoder {
unsafe impl Send for FfmpegWinEncoder {}
impl FfmpegWinEncoder {
#[allow(clippy::too_many_arguments)]
#[allow(clippy::too_many_arguments)]
pub fn open(
vendor: WinVendor,
@@ -1106,7 +1119,15 @@ impl FfmpegWinEncoder {
fps: u32,
bitrate_bps: u64,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Self> {
// AMF/QSV 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel Windows
// hardware in the lab, and the AMF/QSV HEVC 4:4:4 profile/format incantations are vendor- and
// driver-specific (a wrong profile silently encodes 4:2:0). The probe returns false so the host
// never negotiates 4:4:4 for an AMF/QSV session; if a request slips through, fall back to 4:2:0.
if chroma.is_444() {
tracing::warn!("AMF/QSV 4:4:4 encode not implemented — encoding 4:2:0");
}
ffmpeg::init().context("ffmpeg init")?;
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
// SAFETY: `ffmpeg::init()` ran on the line above, so libav is initialised; `av_log_set_level`
@@ -16,7 +16,7 @@
// Every `unsafe` block / impl in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{Codec, EncodedFrame, Encoder, EncoderCaps};
use super::{ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use std::collections::{HashMap, VecDeque};
@@ -57,6 +57,15 @@ pub struct NvencD3d11Encoder {
buffer_fmt: nv::NV_ENC_BUFFER_FORMAT,
/// Encoded bit depth (8 or 10). 10 → HEVC Main10 (NVENC upconverts the 8-bit ARGB input).
bit_depth: u8,
/// Full-chroma 4:4:4 (HEVC Range Extensions, `chroma_format_idc = 3`) requested for this session.
/// NVENC ingests the RGB (ARGB/ABGR10) input and CSCs it to YUV444 internally; the `FREXT` profile
/// and `chromaFormatIDC = 3` in the encode config carry the chroma. Gated on the GPU's
/// `NV_ENC_CAPS_SUPPORT_YUV444_ENCODE` (cleared in `query_caps` on a card that lacks it) and on an
/// RGB input format (NV12/P010 capture can't reconstruct 4:4:4). HEVC-only.
chroma_444: bool,
/// `NV_ENC_CAPS_SUPPORT_YUV444_ENCODE` from the caps probe — whether this GPU can 4:4:4 encode at
/// all. `chroma_444` is forced off when this is false (graceful downgrade to 4:2:0).
yuv444_supported: bool,
/// HDR: the capturer is delivering BT.2020 PQ 10-bit (`PixelFormat::Rgb10a2`) frames. Sets the
/// `ABGR10` input format + the BT.2020/PQ colour VUI. Derived per-frame from the capture format
/// (HDR can toggle mid-session); a change re-inits the session.
@@ -103,6 +112,7 @@ pub struct NvencD3d11Encoder {
unsafe impl Send for NvencD3d11Encoder {}
impl NvencD3d11Encoder {
#[allow(clippy::too_many_arguments)]
pub fn open(
codec: Codec,
_format: PixelFormat,
@@ -111,6 +121,7 @@ impl NvencD3d11Encoder {
fps: u32,
bitrate_bps: u64,
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Self> {
Ok(Self {
encoder: ptr::null_mut(),
@@ -122,6 +133,9 @@ impl NvencD3d11Encoder {
bitrate_bps,
buffer_fmt: nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ARGB,
bit_depth,
// 4:4:4 is HEVC-only; the GPU-support gate is applied in `query_caps`.
chroma_444: chroma.is_444() && codec == Codec::H265,
yuv444_supported: false,
hdr: false,
hdr_meta: None,
regs: HashMap::new(),
@@ -209,6 +223,7 @@ impl NvencD3d11Encoder {
let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX);
let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX);
let ten_bit = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_10BIT_ENCODE);
let yuv444 = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE);
let rfi = self.get_cap(
enc,
nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_REF_PIC_INVALIDATION,
@@ -235,6 +250,13 @@ impl NvencD3d11Encoder {
self.bit_depth = 8;
self.hdr = false;
}
// Same for 4:4:4: a card without YUV444 encode falls back to 4:2:0. (The host already probed
// this via `probe_can_encode_444` before the Welcome, so this is a belt-and-braces guard.)
self.yuv444_supported = yuv444 != 0;
if self.chroma_444 && !self.yuv444_supported {
tracing::warn!("NVENC: this GPU can't 4:4:4 encode — falling back to 4:2:0");
self.chroma_444 = false;
}
self.rfi_supported = rfi != 0;
self.custom_vbv = custom_vbv != 0;
tracing::info!(
@@ -313,9 +335,31 @@ impl NvencD3d11Encoder {
cfg.encodeCodecConfig.hevcConfig.tier = 1;
cfg.encodeCodecConfig.hevcConfig.level = 0;
// 10-bit HEVC Main10 (HDR foundation): NVENC upconverts the 8-bit input; 8-bit leaves the
// preset default (Main) untouched.
if self.bit_depth == 10 {
// Chroma + bit depth. Full-chroma 4:4:4 (HEVC Range Extensions) takes precedence and composes
// with 10-bit (Main 4:4:4 10): NVENC ingests the RGB input (ARGB / ABGR10) and CSCs it to
// YUV444 internally when `chromaFormatIDC = 3` under the FREXT profile. Only valid on an RGB
// input — a subsampled NV12/P010 source can't reconstruct full chroma (so the capturer is
// forced to RGB for a 4:4:4 session, and we guard on the input format here too).
//
// ON-GLASS TODO (RTX box): confirm ARGB + chromaFormatIDC=3 + FREXT yields a *true* 4:4:4
// stream. NVENC's RGB→YUV CSC is documented to honor chromaFormatIDC (unlike libavcodec's
// wrapper, which always subsamples RGB to 4:2:0 — hence the Linux path feeds planar YUV444
// instead). If on-glass shows 4:2:0, the follow-up is a BGRA→AYUV shader feeding the native
// `NV_ENC_BUFFER_FORMAT_AYUV` 4:4:4 input format.
let rgb_input = matches!(
self.buffer_fmt,
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ARGB
| nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_ABGR10
);
if self.chroma_444 && rgb_input {
cfg.profileGUID = nv::NV_ENC_HEVC_PROFILE_FREXT_GUID;
cfg.encodeCodecConfig.hevcConfig.set_chromaFormatIDC(3);
if self.bit_depth == 10 {
cfg.encodeCodecConfig.hevcConfig.set_pixelBitDepthMinus8(2); // Main 4:4:4 10
}
} else if self.bit_depth == 10 {
// 10-bit HEVC Main10 (HDR foundation): NVENC upconverts the 8-bit input; 8-bit leaves the
// preset default (Main) untouched.
cfg.profileGUID = nv::NV_ENC_HEVC_PROFILE_MAIN10_GUID;
cfg.encodeCodecConfig.hevcConfig.set_pixelBitDepthMinus8(2); // 10 - 8
}
@@ -787,6 +831,9 @@ impl Encoder for NvencD3d11Encoder {
EncoderCaps {
supports_rfi: self.rfi_supported,
supports_hdr_metadata: self.hdr,
// Reflects what the session actually configured (cleared in `query_caps` if the GPU lacks
// YUV444 encode), so the glue can confirm 4:4:4 vs the negotiated request.
chroma_444: self.chroma_444,
}
}
@@ -904,3 +951,69 @@ impl Drop for NvencD3d11Encoder {
unsafe { self.teardown() };
}
}
/// Probe whether the active NVIDIA GPU can encode HEVC **4:4:4** (`NV_ENC_CAPS_SUPPORT_YUV444_ENCODE`).
/// Creates a throwaway hardware D3D11 device + NVENC session, queries the cap, and tears down. HEVC-only;
/// the result is cached by the caller ([`crate::encode::can_encode_444`]) and read *before* the Welcome
/// so the host advertises the chroma it can really encode (honest downgrade to 4:2:0 on a card without it).
pub fn probe_can_encode_444(codec: Codec) -> bool {
use windows::Win32::Foundation::HMODULE;
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_FEATURE_LEVEL_11_0};
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_SDK_VERSION,
};
if codec != Codec::H265 {
return false;
}
// SAFETY: a self-contained probe owning every handle it creates. `D3D11CreateDevice` (HARDWARE
// driver, NULL adapter) fills `device` or returns Err (→ false). `open_encode_session_ex` opens an
// NVENC session against that device's raw pointer (valid while `device` is held) or errors (→ false,
// tearing nothing down). `get_encode_caps` reads one scalar cap into `val` via the loaded API table.
// `destroy_encoder` frees the session exactly once; `device`/its context drop with the COM wrappers.
// No handle escapes this call and nothing runs concurrently.
unsafe {
let mut device: Option<ID3D11Device> = None;
if D3D11CreateDevice(
None,
D3D_DRIVER_TYPE_HARDWARE,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
None,
)
.is_err()
{
return false;
}
let Some(device) = device else { return false };
let mut params = nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS {
version: nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER,
deviceType: nv::NV_ENC_DEVICE_TYPE::NV_ENC_DEVICE_TYPE_DIRECTX,
device: device.as_raw(),
apiVersion: nv::NVENCAPI_VERSION,
..Default::default()
};
let mut enc: *mut c_void = ptr::null_mut();
if (API.open_encode_session_ex)(&mut params, &mut enc)
.result_without_string()
.is_err()
{
return false;
}
let mut param = nv::NV_ENC_CAPS_PARAM {
version: nv::NV_ENC_CAPS_PARAM_VER,
capsToQuery: nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE,
reserved: [0; 62],
};
let mut val: i32 = 0;
let ok = (API.get_encode_caps)(enc, nv::NV_ENC_CODEC_HEVC_GUID, &mut param, &mut val)
.result_without_string()
.is_ok()
&& val != 0;
let _ = (API.destroy_encoder)(enc);
ok
}
}
+71 -267
View File
@@ -41,8 +41,6 @@ type Aes128CbcEnc = cbc::Encryptor<aes::Aes128>;
/// `RTP_PAYLOAD_TYPE_FEC 127`).
const AUDIO_PACKET_TYPE: u8 = 97;
const AUDIO_FEC_PACKET_TYPE: u8 = 127;
/// Stereo Opus bitrate (unchanged from the live-validated stereo path).
const OPUS_BITRATE: i32 = 128_000;
/// Audio FEC geometry (moonlight-common-c `RtpAudioQueue.h`: `RTPA_DATA_SHARDS 4`,
/// `RTPA_FEC_SHARDS 2`). Blocks are aligned: the client synthesizes the block base as
@@ -82,67 +80,20 @@ impl Default for AudioParams {
}
}
/// One Opus (multi)stream layout. Channel order is the GameStream/Moonlight order
/// FL FR FC LFE RL RR [SL SR]; `mapping` is the libopus multistream mapping we *encode*
/// with — identical to Sunshine's `audio.cpp stream_configs` (verified verbatim 2026-06-10):
/// identity mapping, so normal quality couples (FL,FR) and (FC,LFE) [+ (RL,RR) on 7.1] with
/// the remaining channels as mono streams; high quality is one mono stream per channel.
/// Bitrates are Sunshine's per-config values (stereo keeps punktfunk's existing 128 kbps).
pub struct OpusLayout {
pub channels: u8,
pub streams: u8,
pub coupled: u8,
pub mapping: &'static [u8],
pub bitrate: i32,
}
pub const LAYOUT_STEREO: OpusLayout = OpusLayout {
channels: 2,
streams: 1,
coupled: 1,
mapping: &[0, 1],
bitrate: OPUS_BITRATE,
};
pub const LAYOUT_51: OpusLayout = OpusLayout {
channels: 6,
streams: 4,
coupled: 2,
mapping: &[0, 1, 2, 3, 4, 5],
bitrate: 256_000,
};
pub const LAYOUT_51_HQ: OpusLayout = OpusLayout {
channels: 6,
streams: 6,
coupled: 0,
mapping: &[0, 1, 2, 3, 4, 5],
bitrate: 1_536_000,
};
pub const LAYOUT_71: OpusLayout = OpusLayout {
channels: 8,
streams: 5,
coupled: 3,
mapping: &[0, 1, 2, 3, 4, 5, 6, 7],
bitrate: 450_000,
};
pub const LAYOUT_71_HQ: OpusLayout = OpusLayout {
channels: 8,
streams: 8,
coupled: 0,
mapping: &[0, 1, 2, 3, 4, 5, 6, 7],
bitrate: 2_048_000,
// The Opus surround layout table (channel order FL FR FC LFE RL RR [SL SR], identity mapping,
// Sunshine's per-config bitrates) now lives in `punktfunk_core::audio`, shared with the native
// `punktfunk/1` path and every client decoder. Re-export the pieces the GameStream module + its
// RTSP SDP (`rtsp.rs`) reference; the GFE-specific `surround_params` SDP rotation stays below.
pub use punktfunk_core::audio::{
OpusLayout, LAYOUT_51, LAYOUT_51_HQ, LAYOUT_71, LAYOUT_71_HQ, LAYOUT_STEREO,
};
/// Pick the encoder layout for the negotiated session parameters. Unknown channel counts
/// fall back to stereo (the client can only request 2/6/8 — `AUDIO_CONFIGURATION_*` in
/// Pick the encoder layout for the negotiated session parameters. Thin wrapper over the shared
/// [`punktfunk_core::audio::layout_for`] keyed on this module's [`AudioParams`] (unknown channel
/// counts fall back to stereo; the client can only request 2/6/8 — `AUDIO_CONFIGURATION_*` in
/// Limelight.h).
pub fn layout_for(params: &AudioParams) -> &'static OpusLayout {
match (params.channels, params.high_quality) {
(6, false) => &LAYOUT_51,
(6, true) => &LAYOUT_51_HQ,
(8, false) => &LAYOUT_71,
(8, true) => &LAYOUT_71_HQ,
_ => &LAYOUT_STEREO,
}
punktfunk_core::audio::layout_for(params.channels, params.high_quality)
}
/// The `a=fmtp:97 surround-params=` digit string for a layout: channelCount, streams,
@@ -345,21 +296,21 @@ fn run(
}
/// Opus encoder for one session: the plain stereo encoder (the live-validated path, byte
/// identical) or a libopus multistream encoder for 5.1/7.1.
/// identical) or the safe `opus::MSEncoder` multistream encoder for 5.1/7.1. Both are
/// cross-platform (Linux + Windows) — surround no longer needs `audiopus_sys`.
#[cfg(any(target_os = "linux", target_os = "windows"))]
enum SessionEncoder {
Stereo(opus::Encoder),
// Surround needs the libopus *multistream* encoder via `audiopus_sys` (Linux-only dep).
#[cfg(target_os = "linux")]
Surround(MsEncoder),
Surround(opus::MSEncoder),
}
#[cfg(any(target_os = "linux", target_os = "windows"))]
impl SessionEncoder {
fn new(layout: &'static OpusLayout) -> Result<SessionEncoder> {
// RESTRICTED_LOWDELAY (`opus::Application::LowDelay`) + hard CBR, matching Sunshine — CBR
// keeps the Opus packet size constant, which the GameStream audio FEC (equal-length shards)
// relies on, and the client asserts a constant per-stream TOC.
if layout.channels == 2 {
// RESTRICTED_LOWDELAY + CBR, matching Sunshine — CBR keeps the Opus TOC byte
// constant, which the client asserts per stream.
let mut enc = opus::Encoder::new(
SAMPLE_RATE,
opus::Channels::Stereo,
@@ -370,138 +321,32 @@ impl SessionEncoder {
enc.set_vbr(false).ok();
Ok(SessionEncoder::Stereo(enc))
} else {
#[cfg(target_os = "linux")]
{
Ok(SessionEncoder::Surround(MsEncoder::new(layout)?))
}
#[cfg(not(target_os = "linux"))]
{
anyhow::bail!(
"surround audio ({} ch) needs the libopus multistream encoder (Linux only) — \
use a stereo session",
layout.channels
)
}
let mut enc = opus::MSEncoder::new(
SAMPLE_RATE,
layout.streams,
layout.coupled,
layout.mapping,
opus::Application::LowDelay,
)
.map_err(|e| anyhow::anyhow!("create Opus multistream encoder: {e}"))?;
enc.set_bitrate(opus::Bitrate::Bits(layout.bitrate)).ok();
enc.set_vbr(false).ok();
Ok(SessionEncoder::Surround(enc))
}
}
/// Encode one interleaved frame (`samples_per_channel * channels` f32s) into `out`,
/// returning the packet length.
fn encode_float(
&mut self,
frame: &[f32],
samples_per_channel: usize,
out: &mut [u8],
) -> Result<usize> {
// `samples_per_channel` only feeds the multistream (surround) encoder; stereo infers it.
#[cfg(not(target_os = "linux"))]
let _ = samples_per_channel;
/// Encode one interleaved frame into `out`, returning the packet length. Both encoders infer
/// the per-channel sample count from `frame.len()` and their channel count.
fn encode_float(&mut self, frame: &[f32], out: &mut [u8]) -> Result<usize> {
match self {
SessionEncoder::Stereo(enc) => enc.encode_float(frame, out).context("opus encode"),
#[cfg(target_os = "linux")]
SessionEncoder::Surround(enc) => enc.encode_float(frame, samples_per_channel, out),
SessionEncoder::Surround(enc) => enc
.encode_float(frame, out)
.context("opus multistream encode"),
}
}
}
/// RAII wrapper for `OpusMSEncoder` (the safe `opus` crate is stereo-only; the multistream
/// API comes from `audiopus_sys`, the same libopus the crate already links). Configured like
/// the stereo path: RESTRICTED_LOWDELAY, hard CBR, per-layout bitrate.
#[cfg(target_os = "linux")]
struct MsEncoder {
st: std::ptr::NonNull<audiopus_sys::OpusMSEncoder>,
}
// SAFETY: `MsEncoder` owns a unique `OpusMSEncoder` via `NonNull` (it is neither `Clone` nor
// `Sync`, so the pointer is never aliased). libopus's multistream encoder state is a self-contained
// heap allocation with no thread-local or thread-affine state, so moving ownership to another thread
// is sound; every method takes `&mut self`, keeping access single-threaded at any instant.
#[cfg(target_os = "linux")]
unsafe impl Send for MsEncoder {}
#[cfg(target_os = "linux")]
impl MsEncoder {
fn new(layout: &OpusLayout) -> Result<MsEncoder> {
use std::os::raw::c_int;
let mut err: c_int = 0;
// SAFETY: every scalar arg is a valid libopus input (sample rate, channel/stream/coupled
// counts, the RESTRICTED_LOWDELAY application constant). `layout.mapping.as_ptr()` addresses
// a 'static slice of exactly `layout.channels` bytes (every `OpusLayout` constant upholds
// that), which is the element count `opus_multistream_encoder_create` reads through it, and
// `&mut err` is a live local the call writes its status into. libopus copies the mapping into
// its own allocation, so the pointer need only be valid for the call; the returned pointer is
// null/`OPUS_OK`-checked below before any use.
let st = unsafe {
audiopus_sys::opus_multistream_encoder_create(
SAMPLE_RATE as i32,
layout.channels as c_int,
layout.streams as c_int,
layout.coupled as c_int,
layout.mapping.as_ptr(),
audiopus_sys::OPUS_APPLICATION_RESTRICTED_LOWDELAY,
&mut err,
)
};
let st = std::ptr::NonNull::new(st)
.filter(|_| err == audiopus_sys::OPUS_OK)
.ok_or_else(|| anyhow::anyhow!("opus_multistream_encoder_create failed ({err})"))?;
// SAFETY: `st` is the non-null encoder `opus_multistream_encoder_create` just returned, owned
// exclusively here. Each `opus_multistream_encoder_ctl` call passes a valid request constant
// with the single by-value `c_int` argument that request's variadic ABI expects
// (`OPUS_SET_BITRATE_REQUEST` → bitrate, `OPUS_SET_VBR_REQUEST` → 0). No pointer escapes the
// call and the encoder outlives it.
unsafe {
audiopus_sys::opus_multistream_encoder_ctl(
st.as_ptr(),
audiopus_sys::OPUS_SET_BITRATE_REQUEST,
layout.bitrate as c_int,
);
audiopus_sys::opus_multistream_encoder_ctl(
st.as_ptr(),
audiopus_sys::OPUS_SET_VBR_REQUEST,
0 as c_int, // hard CBR (constant packet size — also what audio FEC relies on)
);
}
Ok(MsEncoder { st })
}
fn encode_float(
&mut self,
frame: &[f32],
samples_per_channel: usize,
out: &mut [u8],
) -> Result<usize> {
// SAFETY: `self.st` is the live encoder from `new`. libopus reads `samples_per_channel *
// channels` f32s through `frame.as_ptr()`; every caller passes a `frame` of exactly that
// length together with the matching `samples_per_channel` (`audio_body`'s `frame_len =
// samples_per_channel * layout.channels`; the round-trip tests size identically), so the read
// stays in bounds. `out.as_mut_ptr()` is written for at most `out.len()` bytes, which is
// passed as the capacity bound. Both buffers are live locals outliving this synchronous call;
// the return value is range-checked before being used as a length.
let n = unsafe {
audiopus_sys::opus_multistream_encode_float(
self.st.as_ptr(),
frame.as_ptr(),
samples_per_channel as std::os::raw::c_int,
out.as_mut_ptr(),
out.len() as i32,
)
};
anyhow::ensure!(n > 0, "opus_multistream_encode_float failed ({n})");
Ok(n as usize)
}
}
#[cfg(target_os = "linux")]
impl Drop for MsEncoder {
fn drop(&mut self) {
// SAFETY: `self.st` is the encoder `opus_multistream_encoder_create` returned; this
// `MsEncoder` owns it uniquely and `drop` runs exactly once, so the destroy frees it once
// with no subsequent use.
unsafe { audiopus_sys::opus_multistream_encoder_destroy(self.st.as_ptr()) }
}
}
#[cfg(any(target_os = "linux", target_os = "windows"))]
fn audio_body(
cap: &mut dyn AudioCapturer,
@@ -565,7 +410,7 @@ fn audio_body(
*s = (*s * gain).clamp(-1.0, 1.0);
}
}
let n = enc.encode_float(&frame, samples_per_channel, &mut out)?;
let n = enc.encode_float(&frame, &mut out)?;
// AES-128-CBC the Opus payload (RTP header stays plaintext). Per-packet IV =
// BE32(rikeyid + seq) in [0..4], zero elsewhere; PKCS7 padding.
let iv_seq = (rikeyid as u32).wrapping_add(seq as u32);
@@ -775,41 +620,33 @@ mod tests {
/// Real-codec proof of the 5.1 mapping math: encode with our encoder layout, decode with
/// the mapping a stock Moonlight client derives from our advertised surround-params
/// (parse → GFE swap), and verify a tone fed into each input channel comes out on the
/// same output channel.
#[cfg(target_os = "linux")]
/// same output channel. Cross-platform via the safe `opus` crate — this also guards the
/// (now un-gated) Windows GameStream surround build.
#[test]
fn multistream_51_roundtrip_channel_identity() {
let layout = &LAYOUT_51;
let samples = 240; // 5 ms
let ch = layout.channels as usize;
// Client-side decoder mapping derived exactly as moonlight-common-c does.
// Client-side decoder mapping derived exactly as moonlight-common-c does (GFE swap).
let s = surround_params(layout, false);
let digits: Vec<u8> = s.bytes().map(|b| b - b'0').collect();
let client_mapping = client_swap(&digits[3..]);
let mut err = 0i32;
// SAFETY: scalar args are valid libopus inputs. `client_mapping.as_ptr()` addresses a
// `Vec<u8>` of exactly `ch` entries (derived from the advertised surround-params), which is
// the element count the decoder reads through it, and `&mut err` is a live local the call
// writes. The returned pointer is `OPUS_OK`/non-null-checked immediately below before use.
let dec = unsafe {
audiopus_sys::opus_multistream_decoder_create(
SAMPLE_RATE as i32,
ch as i32,
layout.streams as i32,
layout.coupled as i32,
client_mapping.as_ptr(),
&mut err,
)
};
assert_eq!(err, audiopus_sys::OPUS_OK);
assert!(!dec.is_null());
let mut dec =
opus::MSDecoder::new(SAMPLE_RATE, layout.streams, layout.coupled, &client_mapping)
.expect("multistream decoder");
for tone_ch in 0..ch {
let mut enc = MsEncoder::new(layout).unwrap();
let mut enc = opus::MSEncoder::new(
SAMPLE_RATE,
layout.streams,
layout.coupled,
layout.mapping,
opus::Application::LowDelay,
)
.expect("multistream encoder");
let mut out = vec![0u8; 1400];
let mut decoded = vec![0f32; samples * ch];
let mut energy = vec![0f64; ch];
// A few frames so the codec converges past its startup transient.
for f in 0..8 {
@@ -819,28 +656,15 @@ mod tests {
/ SAMPLE_RATE as f32;
frame[t * ch + tone_ch] = 0.5 * phase.sin();
}
let n = enc.encode_float(&frame, samples, &mut out).unwrap();
let n = enc.encode_float(&frame, &mut out).unwrap();
assert!(n > 0);
// SAFETY: `dec` is the non-null decoder asserted above. `out.as_ptr()` is read for
// the `n` encoded bytes just produced by `encode_float`; `decoded.as_mut_ptr()` is
// written for up to `samples * ch` f32s and `decoded` is exactly that long; `samples`
// is the per-channel frame size. All buffers are live locals outliving the call; the
// return is checked to equal `samples`.
let got = unsafe {
audiopus_sys::opus_multistream_decode_float(
dec,
out.as_ptr(),
n as i32,
decoded.as_mut_ptr(),
samples as i32,
0,
)
};
assert_eq!(got as usize, samples);
let mut decoded = vec![0f32; samples * ch];
let got = dec.decode_float(&out[..n], &mut decoded, false).unwrap();
assert_eq!(got, samples);
if f >= 4 {
for t in 0..samples {
for c in 0..ch {
energy[c] += (decoded[t * ch + c] as f64).powi(2);
for (c, e) in energy.iter_mut().enumerate() {
*e += (decoded[t * ch + c] as f64).powi(2);
}
}
}
@@ -854,9 +678,6 @@ mod tests {
(energies: {energy:?})"
);
}
// SAFETY: `dec` is the decoder `opus_multistream_decoder_create` returned; the test owns it
// and destroys it exactly once here, after the final decode — no later use, no double free.
unsafe { audiopus_sys::opus_multistream_decoder_destroy(dec) };
}
/// Live 5.1 capture → multistream encode → decode, against a real PipeWire session.
@@ -869,7 +690,15 @@ mod tests {
fn surround_capture_live() {
let mut cap = crate::audio::open_audio_capture(6).expect("open 6ch capture");
let layout = &LAYOUT_51;
let mut enc = MsEncoder::new(layout).unwrap();
let mut enc = opus::MSEncoder::new(
SAMPLE_RATE,
layout.streams,
layout.coupled,
layout.mapping,
opus::Application::LowDelay,
)
.unwrap();
enc.set_vbr(false).ok(); // hard CBR so packet sizes are constant (audio FEC relies on it)
let mut out = vec![0u8; 1400];
let mut acc: Vec<f32> = Vec::new();
let frame_len = 240 * 6;
@@ -880,49 +709,24 @@ mod tests {
acc.extend_from_slice(&chunk);
while acc.len() >= frame_len && packets < 100 {
let frame: Vec<f32> = acc.drain(..frame_len).collect();
let n = enc.encode_float(&frame, 240, &mut out).unwrap();
let n = enc.encode_float(&frame, &mut out).unwrap();
sizes.insert(n);
packets += 1;
}
}
// Hard CBR: every multistream packet must be the same size (audio FEC relies on it).
assert_eq!(sizes.len(), 1, "CBR sizes: {sizes:?}");
// And a stock client's decoder must accept them.
// And a stock client's GFE-derived decoder must accept them.
let s = surround_params(layout, false);
let digits: Vec<u8> = s.bytes().map(|b| b - b'0').collect();
let client_mapping = client_swap(&digits[3..]);
let mut err = 0i32;
// SAFETY: scalar args are valid; `client_mapping.as_ptr()` addresses a 6-entry `Vec<u8>`
// (matches the 6-channel layout the decoder reads through it), alive past the call, and
// `&mut err` is a live local. The pointer is `OPUS_OK`-checked before use.
let dec = unsafe {
audiopus_sys::opus_multistream_decoder_create(
48000,
6,
layout.streams as i32,
layout.coupled as i32,
client_mapping.as_ptr(),
&mut err,
)
};
assert_eq!(err, audiopus_sys::OPUS_OK);
let mut dec =
opus::MSDecoder::new(SAMPLE_RATE, layout.streams, layout.coupled, &client_mapping)
.unwrap();
let mut pcm = vec![0f32; 240 * 6];
// SAFETY: `dec` is the non-null decoder from create. `out.as_ptr()` is read for the CBR
// packet length passed in (`*sizes.first()`, a real encoded packet size in `out`);
// `pcm.as_mut_ptr()` is written for up to `240 * 6` f32s and `pcm` is exactly that long;
// `240` is the per-channel frame size. All buffers are live locals outliving the call.
let got = unsafe {
audiopus_sys::opus_multistream_decode_float(
dec,
out.as_ptr(),
*sizes.first().unwrap() as i32,
pcm.as_mut_ptr(),
240,
0,
)
};
// SAFETY: `dec` is owned by the test; destroyed exactly once here after the final decode.
unsafe { audiopus_sys::opus_multistream_decoder_destroy(dec) };
let got = dec
.decode_float(&out[..*sizes.first().unwrap()], &mut pcm, false)
.unwrap();
assert_eq!(got, 240);
}
}
@@ -56,6 +56,9 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
.spawn(move || {
// GCM scheme detected from the first authenticating packet; reused thereafter.
let mut detected: Option<Scheme> = None;
// Consecutive control-decrypt failures for this peer — throttles the warn log so a
// junk-packet flood can't spam unbounded lines (security-review 2026-06-28 #10).
let mut decrypt_fails: u64 = 0;
// Decoded keyboard/mouse is forwarded to a dedicated host-lifetime injector thread —
// NEVER injected inline, so a slow Wayland/libei/SendInput call can't head-block ENet
// keepalive/retransmit servicing on this thread. The injector owns non-Send compositor
@@ -77,6 +80,7 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
Event::Disconnect { .. } => {
tracing::info!("control: client disconnected");
detected = None;
decrypt_fails = 0;
peer = None;
// Unplug the session's virtual pads.
pads = GamepadManager::new();
@@ -89,6 +93,7 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
channel_id,
packet.data(),
&mut detected,
&mut decrypt_fails,
&inj_tx,
&mut pads,
);
@@ -163,6 +168,7 @@ fn on_receive(
_channel_id: u8,
d: &[u8],
detected: &mut Option<Scheme>,
decrypt_fails: &mut u64,
inj_tx: &Sender<InputEvent>,
pads: &mut GamepadManager,
) {
@@ -180,10 +186,20 @@ fn on_receive(
tracing::info!(?scheme, "control: GCM scheme locked in");
}
*detected = Some(scheme);
*decrypt_fails = 0;
pt
}
None => {
tracing::warn!(len = d.len(), "control: GCM decrypt failed");
// Throttle: a junk-packet flood must not spam one warn line per packet. Log the first
// failure, then only at exponentially-spaced counts (1, 2, 4, 8, …).
*decrypt_fails += 1;
if decrypt_fails.is_power_of_two() {
tracing::warn!(
len = d.len(),
fails = *decrypt_fails,
"control: GCM decrypt failed"
);
}
return;
}
};
@@ -1,7 +1,7 @@
//! Pairing crypto primitives (control plane only — distinct from `punktfunk_core`'s AES-GCM
//! data-plane sealing). GameStream pairing uses: AES-128-**ECB** with **no padding**,
//! SHA-256 (host appversion major ≥ 7), and RSA-PKCS1v15-SHA256 signatures. See the
//! `serverinfo + pairing` section of `docs/research/gamestream-protocol-research.json`.
//! `serverinfo + pairing` section of `design/research/gamestream-protocol-research.json`.
use aes::cipher::generic_array::GenericArray;
use aes::cipher::{BlockDecrypt, BlockEncrypt, KeyInit};
+84 -12
View File
@@ -1,7 +1,7 @@
//! GameStream (P1) control plane — what a stock Moonlight/Artemis client talks to around
//! the media streams: mDNS discovery, the nvhttp serverinfo + pairing HTTP(S) API, RTSP,
//! and the ENet control stream. `tokio`/`axum` live here (control plane, I/O-bound — never
//! the per-frame hot path; that is `punktfunk_core`'s P1 wire codec). See `docs/gamestream-host-plan.md`.
//! the per-frame hot path; that is `punktfunk_core`'s P1 wire codec). See `design/gamestream-host-plan.md`.
//!
//! Status: P1.1 — mDNS `_nvstream._tcp` advertisement + `/serverinfo`. Pairing, RTSP, and
//! the media streams follow (see the GameStream host task list / plan).
@@ -90,6 +90,11 @@ pub struct LaunchSession {
pub fps: u32,
/// `/launch?appid=N` — selects the app-catalog entry (session recipe).
pub appid: u32,
/// Source IP of the paired HTTPS client that issued `/launch`. The unauthenticated RTSP/UDP
/// media plane binds to this so only the launching peer can start/own the stream — an
/// unpaired RTSP peer cannot ride a paired client's launch (security-review 2026-06-28 #4).
/// `None` if the address could not be captured (then RTSP falls back to launch-present only).
pub peer_ip: Option<std::net::IpAddr>,
}
/// Shared control-plane state used as the axum app state.
@@ -125,12 +130,21 @@ pub struct AppState {
/// (avoids a PipeWire stream setup per reconnect); drained on reuse so no stale audio is
/// sent, dropped + reopened when a session negotiates a different channel count.
pub audio_cap: std::sync::Arc<std::sync::Mutex<Option<Box<dyn crate::audio::AudioCapturer>>>>,
/// Shared streaming-stats recorder (web-console capture/graph). The GameStream encode loop
/// reads `is_armed()` per frame and emits samples; the same `Arc` is shared with the mgmt API
/// and the native punktfunk/1 loops so one capture spans whichever path is streaming.
pub stats: Arc<crate::stats_recorder::StatsRecorder>,
}
impl AppState {
/// Fresh control-plane state: no active session; the pairing allow-list is loaded from
/// disk (pairings persist across restarts).
pub fn new(host: Host, identity: cert::ServerIdentity) -> AppState {
/// disk (pairings persist across restarts). `stats` is the shared recorder handed to both the
/// mgmt API and the streaming loops.
pub fn new(
host: Host,
identity: cert::ServerIdentity,
stats: Arc<crate::stats_recorder::StatsRecorder>,
) -> AppState {
AppState {
host,
identity,
@@ -145,6 +159,7 @@ impl AppState {
rfi_range: std::sync::Arc::new(std::sync::Mutex::new(None)),
video_cap: std::sync::Arc::new(std::sync::Mutex::new(None)),
audio_cap: std::sync::Arc::new(std::sync::Mutex::new(None)),
stats,
}
}
}
@@ -166,7 +181,10 @@ pub fn serve(
) -> Result<()> {
let host = Host::detect()?;
let identity = cert::ServerIdentity::load_or_create().context("host certificate")?;
let state = Arc::new(AppState::new(host, identity));
// The shared streaming-stats recorder: one handle for the mgmt API, the GameStream encode loop
// (via `AppState`), and the native punktfunk/1 loops (passed to `punktfunk1::serve`).
let stats = crate::stats_recorder::StatsRecorder::new(crate::stats_recorder::default_dir());
let state = Arc::new(AppState::new(host, identity, stats.clone()));
// The native plane always runs, so the shared native-pairing handle (linking the QUIC ceremony
// and the management API) always exists.
let np = Arc::new(
@@ -206,8 +224,8 @@ pub fn serve(
);
tokio::try_join!(
nvhttp::run(state.clone()),
crate::mgmt::run(state.clone(), mgmt, Some(np.clone())),
crate::punktfunk1::serve(native_opts, np),
crate::mgmt::run(state.clone(), mgmt, Some(np.clone()), stats.clone()),
crate::punktfunk1::serve(native_opts, np, stats.clone()),
)?;
} else {
// Secure default: native punktfunk/1 + management API only (no GameStream surface).
@@ -217,8 +235,8 @@ pub fn serve(
(GameStream OFF pass --gamestream for stock-Moonlight compat)"
);
tokio::try_join!(
crate::mgmt::run(state.clone(), mgmt, Some(np.clone())),
crate::punktfunk1::serve(native_opts, np),
crate::mgmt::run(state.clone(), mgmt, Some(np.clone()), stats.clone()),
crate::punktfunk1::serve(native_opts, np, stats.clone()),
)?;
}
Ok(())
@@ -249,9 +267,10 @@ pub(crate) fn config_dir() -> PathBuf {
}
/// Create `dir` (and parents) owner-private — **0700** on Unix (so the host's secrets aren't readable
/// by other local users via a traversable config path). Best-effort on Windows: the dir inherits the
/// (Users-readable) `%ProgramData%` ACL, so secret *files* are individually locked down by
/// [`write_secret_file`]. Tightens an already-existing dir too.
/// by other local users via a traversable config path). On Windows, applies a restrictive DACL
/// ([`restrict_dir_to_system_admins`]) so a local unprivileged user can't pre-create / plant files in
/// the config tree (the default `%ProgramData%` ACL grants Users *create*; security-review
/// 2026-06-28 #3/#11). Tightens (and re-owns) an already-existing dir too.
pub(crate) fn create_private_dir(dir: &std::path::Path) -> std::io::Result<()> {
#[cfg(unix)]
{
@@ -268,7 +287,60 @@ pub(crate) fn create_private_dir(dir: &std::path::Path) -> std::io::Result<()> {
}
#[cfg(not(unix))]
{
std::fs::create_dir_all(dir)
let r = std::fs::create_dir_all(dir);
#[cfg(windows)]
restrict_dir_to_system_admins(dir);
r
}
}
/// Best-effort Windows DACL lockdown of the config *directory* (the companion to
/// [`restrict_to_system_admins`] for files). The default `%ProgramData%` ACL lets `BUILTIN\Users`
/// create subfolders/files (and become `CREATOR OWNER`), so a non-admin could pre-create the
/// `punktfunk` dir or plant a `host.env`/`apps.json` that the privileged SYSTEM service then trusts
/// (LPE; security-review 2026-06-28 #3). This re-owns the dir to Administrators (defeating a
/// pre-creation), strips inheritance, and sets an explicit DACL: SYSTEM/Administrators/OWNER full
/// (object+container inherit so child files/dirs inherit it), and Users **read-only** (so existing
/// reads of non-secret config keep working but a local user can no longer write/plant). Secret files
/// are additionally locked to SYSTEM/Admins by [`write_secret_file`]. Hard-coded SIDs
/// (locale-independent) via the absolute `%SystemRoot%` path; never fatal.
#[cfg(windows)]
fn restrict_dir_to_system_admins(dir: &std::path::Path) {
let icacls = std::env::var("SystemRoot")
.map(|r| format!("{r}\\System32\\icacls.exe"))
.unwrap_or_else(|_| "icacls".to_string());
// Reset ownership of the directory object to Administrators first, so a dir a non-admin may have
// pre-created can't keep OWNER control (an owner can always rewrite the DACL). No `/T` — re-owning
// the dir itself is what defeats the pre-creation; recursing a large captures tree each call is
// needless churn (secret files are individually owner-locked by `write_secret_file`).
let _ = std::process::Command::new(&icacls)
.arg(dir.as_os_str())
.args(["/setowner", "*S-1-5-32-544"]) // BUILTIN\Administrators
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status();
let status = std::process::Command::new(&icacls)
.arg(dir.as_os_str())
.args([
"/inheritance:r",
"/grant:r",
"*S-1-5-18:(OI)(CI)(F)", // NT AUTHORITY\SYSTEM
"/grant:r",
"*S-1-5-32-544:(OI)(CI)(F)", // BUILTIN\Administrators
"/grant:r",
"*S-1-3-4:(OI)(CI)(F)", // OWNER RIGHTS
"/grant:r",
"*S-1-5-32-545:(OI)(CI)(RX)", // BUILTIN\Users — read-only (no create/write → no plant)
])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status();
match status {
Ok(s) if s.success() => {}
_ => tracing::warn!(
dir = %dir.display(),
"config-dir DACL hardening did not fully succeed — a local user may be able to plant config files"
),
}
}
+17 -19
View File
@@ -1,9 +1,14 @@
//! The nvhttp servers: plain HTTP on 47989 and mutual-TLS on 47984. Serves `/serverinfo`,
//! the `/pair` flow, `/applist`, and `/launch`/`/resume`/`/cancel`, plus a punktfunk-only
//! `/pin` endpoint to deliver the Moonlight-displayed PIN. Over HTTPS the client is
//! the `/pair` flow, `/applist`, and `/launch`/`/resume`/`/cancel`. Over HTTPS the client is
//! mutual-TLS-authenticated, so `/serverinfo` reports `PairStatus=1` there.
//!
//! The pairing PIN is delivered out-of-band ONLY through the bearer-authenticated management
//! API (`POST /api/v1/pair/pin`): the operator reads the PIN off the Moonlight client and
//! types it into the host console. There is deliberately NO unauthenticated nvhttp PIN
//! endpoint — one would let a network client submit its own displayed PIN and drive the whole
//! ceremony to a pinned cert with no operator consent (security-review 2026-06-28 #1).
use super::tls::PeerCertFingerprint;
use super::tls::{PeerAddr, PeerCertFingerprint};
use super::{serverinfo, AppState, LaunchSession, HTTPS_PORT, HTTP_PORT, RTSP_PORT};
use anyhow::{anyhow, Context, Result};
use axum::{
@@ -58,7 +63,6 @@ fn router(state: Arc<AppState>, https: bool) -> Router {
Router::new()
.route("/serverinfo", get(h_serverinfo))
.route("/pair", get(h_pair))
.route("/pin", get(h_pin))
.route("/applist", get(h_applist))
.route("/launch", get(h_launch))
.route("/resume", get(h_resume))
@@ -82,19 +86,6 @@ async fn h_serverinfo(
xml(serverinfo::serverinfo_xml(&st.host, https, paired))
}
async fn h_pin(
State(st): State<Arc<AppState>>,
Query(q): Query<HashMap<String, String>>,
) -> impl IntoResponse {
match q.get("pin").filter(|p| !p.is_empty()) {
Some(pin) => {
st.pairing.pin.submit(pin.clone());
"PIN accepted\n".to_string()
}
None => "usage: GET /pin?pin=NNNN\n".to_string(),
}
}
async fn h_applist(
State(st): State<Arc<AppState>>,
peer: Option<Extension<PeerCertFingerprint>>,
@@ -110,6 +101,7 @@ async fn h_applist(
async fn h_launch(
State(st): State<Arc<AppState>>,
peer: Option<Extension<PeerCertFingerprint>>,
addr: Option<Extension<PeerAddr>>,
Query(q): Query<HashMap<String, String>>,
) -> impl IntoResponse {
if !peer_is_paired(&peer, &st) {
@@ -117,7 +109,9 @@ async fn h_launch(
return xml(error_xml());
}
match launch(&st, &q) {
Ok(session) => {
Ok(mut session) => {
// Bind the (unauthenticated) RTSP/UDP media plane to this paired client's source IP.
session.peer_ip = addr.map(|Extension(PeerAddr(a))| a.ip());
*st.launch.lock().unwrap() = Some(session);
tracing::info!(
w = session.width,
@@ -193,6 +187,7 @@ fn launch(_st: &AppState, q: &HashMap<String, String>) -> Result<LaunchSession>
height,
fps,
appid,
peer_ip: None, // set by `h_launch` from the verified HTTPS peer address
})
}
@@ -291,7 +286,10 @@ mod tests {
https_port: HTTPS_PORT,
};
let identity = super::super::cert::ServerIdentity::ephemeral().expect("ephemeral identity");
Arc::new(AppState::new(host, identity))
let stats = crate::stats_recorder::StatsRecorder::new(
std::env::temp_dir().join(format!("pf-nvhttp-stats-{}", std::process::id())),
);
Arc::new(AppState::new(host, identity, stats))
}
fn fp_of(der: &[u8]) -> String {
@@ -1,7 +1,7 @@
//! The 4-phase GameStream pairing state machine (over HTTP), keyed by `uniqueid`. Proves
//! both sides know the PIN (via the SHA-256(salt||pin) AES-ECB key) and own their certs
//! (RSA signatures), then pins the client cert. The final `pairchallenge` happens over
//! HTTPS (handled in `nvhttp`). Byte-exact spec: `docs/research/…-research.json`.
//! HTTPS (handled in `nvhttp`). Byte-exact spec: `design/research/…-research.json`.
use super::cert::ServerIdentity;
use super::crypto;
@@ -17,9 +17,14 @@ use std::sync::Mutex;
use std::time::Duration;
use tokio::sync::Notify;
/// Out-of-band PIN delivery. Moonlight generates + displays a PIN; the user submits it
/// (via the management API's `POST /api/v1/pair/pin` or nvhttp's `GET /pin?pin=NNNN`).
/// `getservercert` parks until a PIN arrives.
/// Out-of-band PIN delivery. Moonlight generates + displays a PIN; the operator submits it
/// via the bearer-authenticated management API (`POST /api/v1/pair/pin`) only — there is no
/// unauthenticated nvhttp delivery path (a network client must never be able to submit its
/// own PIN; security-review 2026-06-28 #1). `getservercert` parks until a PIN arrives.
/// Max pairing handshakes parked in [`PinGate::take`] at once (each holds a slot for up to
/// 300s), bounding a pre-auth waiter flood. Real pairing is one operator-driven client at a time.
const MAX_PARKED_WAITERS: usize = 4;
pub struct PinGate {
pin: Mutex<Option<String>>,
notify: Notify,
@@ -48,7 +53,20 @@ impl PinGate {
}
async fn take(&self, timeout: Duration) -> Option<String> {
self.waiters.fetch_add(1, Ordering::SeqCst);
// Bound the number of pairing handshakes parked at once: each `getservercert` is
// pre-auth and parks for up to 300s, so without a cap an unpaired LAN peer could pin
// unbounded tasks + keep `awaiting_pin` asserted (security-review 2026-06-28 #12).
// Reserve a slot atomically; refuse (treated as "no PIN") once the cap is reached.
if self
.waiters
.fetch_update(Ordering::SeqCst, Ordering::SeqCst, |n| {
(n < MAX_PARKED_WAITERS).then_some(n + 1)
})
.is_err()
{
tracing::warn!("pairing: too many handshakes awaiting a PIN — refusing");
return None;
}
// Decrement on every exit path (PIN delivered, timeout, or future cancellation).
struct WaiterGuard<'a>(&'a AtomicUsize);
impl Drop for WaiterGuard<'_> {
@@ -117,7 +135,8 @@ impl Pairing {
tracing::info!(
uniqueid,
"pairing phase 1 (getservercert) — awaiting PIN: submit `GET /pin?pin=NNNN`"
"pairing phase 1 (getservercert) — awaiting PIN: deliver it via the management \
API `POST /api/v1/pair/pin` (operator reads the PIN off the Moonlight client)"
);
let pin = self
.pin
@@ -304,4 +323,28 @@ mod tests {
assert_eq!(pairing.pin.take(Duration::from_millis(10)).await, None);
assert!(!pairing.pin.awaiting_pin());
}
/// A pre-auth peer flood can park at most `MAX_PARKED_WAITERS` pairing handshakes; the next
/// `take` is refused immediately (returns `None` without parking), bounding the 300s-waiter DoS
/// (security-review 2026-06-28 #12).
#[tokio::test]
async fn pin_gate_caps_parked_waiters() {
let pairing = Arc::new(Pairing::new());
let mut handles = Vec::new();
for _ in 0..MAX_PARKED_WAITERS {
let p = pairing.clone();
handles.push(tokio::spawn(async move {
p.pin.take(Duration::from_secs(5)).await
}));
}
// Wait until all the slots are taken.
while pairing.pin.waiters.load(Ordering::SeqCst) < MAX_PARKED_WAITERS {
tokio::time::sleep(Duration::from_millis(2)).await;
}
// One more is refused right away (no parking), even with a long timeout.
assert_eq!(pairing.pin.take(Duration::from_secs(5)).await, None);
for h in handles {
h.abort();
}
}
}
+33 -22
View File
@@ -14,7 +14,7 @@ use crate::encode::Codec;
use anyhow::{Context, Result};
use std::collections::HashMap;
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};
use std::net::{SocketAddr, TcpListener, TcpStream};
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::time::Duration;
@@ -102,13 +102,12 @@ fn handle_conn(mut stream: TcpStream, state: Arc<AppState>) -> Result<()> {
"RTSP {} | {}", req.head.replace("\r\n", " | "),
if req.body.is_empty() { String::new() } else { format!("body: {}", req.body.replace("\r\n", " | ")) }
);
let resp = handle_request(&req, &state);
let resp = handle_request(&req, &state, peer);
stream.write_all(resp.as_bytes()).context("RTSP write")?;
stream.flush().ok();
// Close (FIN after the flushed response) so the client detects end-of-response.
let _ = stream.shutdown(std::net::Shutdown::Both);
}
let _ = peer;
Ok(())
}
@@ -171,7 +170,7 @@ fn parse_request(head: &str, body: String) -> Request {
}
}
fn handle_request(req: &Request, state: &AppState) -> String {
fn handle_request(req: &Request, state: &AppState, peer: Option<SocketAddr>) -> String {
match req.method.as_str() {
"OPTIONS" => response(
&req.cseq,
@@ -216,16 +215,30 @@ fn handle_request(req: &Request, state: &AppState) -> String {
response(&req.cseq, &[], None)
}
"PLAY" => {
// The RTSP/UDP media plane is UNAUTHENTICATED. A stream may start only for the paired
// client that completed the pairing-gated `/launch` (which set `state.launch`), and —
// when the launching IP is known — only from that same source IP. So an unpaired RTSP
// peer can neither start a stream on an idle host nor ride a paired client's active
// launch (security-review 2026-06-28 #4). `nvhttp` gates `/launch` on a pinned cert.
let launch = *state.launch.lock().unwrap();
let Some(ls) = launch else {
tracing::warn!(?peer, "RTSP PLAY — refused: no paired `/launch` session");
return response_status("401 Unauthorized", &req.cseq, &[], None);
};
if let (Some(want), Some(got)) = (ls.peer_ip, peer.map(|p| p.ip())) {
if want != got {
tracing::warn!(
%want, %got,
"RTSP PLAY — refused: peer IP does not match the launching client"
);
return response_status("401 Unauthorized", &req.cseq, &[], None);
}
}
let cfg = *state.stream.lock().unwrap();
match cfg {
Some(cfg) if !state.streaming.swap(true, Ordering::SeqCst) => {
// Resolve the launched catalog entry (session recipe) for the stream.
let app = state
.launch
.lock()
.unwrap()
.map(|l| l.appid)
.and_then(super::apps::by_id);
let app = super::apps::by_id(ls.appid);
tracing::info!(app = ?app.as_ref().map(|a| &a.title), "RTSP PLAY — starting video stream");
stream::start(
cfg,
@@ -234,6 +247,7 @@ fn handle_request(req: &Request, state: &AppState) -> String {
state.force_idr.clone(),
state.rfi_range.clone(),
state.video_cap.clone(),
state.stats.clone(),
);
}
Some(_) => tracing::info!("RTSP PLAY — stream already running"),
@@ -242,18 +256,15 @@ fn handle_request(req: &Request, state: &AppState) -> String {
// Audio runs independently (Opus on UDP 48000, stereo or 5.1/7.1 multistream per
// the ANNOUNCE); it needs the launch key for the AES-CBC payload encryption the
// client expects.
let launch = *state.launch.lock().unwrap();
if let Some(ls) = launch {
if !state.audio_streaming.swap(true, Ordering::SeqCst) {
tracing::info!("RTSP PLAY — starting audio stream");
audio::start(
state.audio_streaming.clone(),
ls.gcm_key,
ls.rikeyid,
*state.audio_params.lock().unwrap(),
state.audio_cap.clone(),
);
}
if !state.audio_streaming.swap(true, Ordering::SeqCst) {
tracing::info!("RTSP PLAY — starting audio stream");
audio::start(
state.audio_streaming.clone(),
ls.gcm_key,
ls.rikeyid,
*state.audio_params.lock().unwrap(),
state.audio_cap.clone(),
);
}
response(&req.cseq, &[("Session", "DEADBEEFCAFE;timeout = 90")], None)
}
+286 -50
View File
@@ -48,6 +48,7 @@ pub fn start(
force_idr: Arc<AtomicBool>,
rfi_range: RfiSlot,
video_cap: CapturerSlot,
stats: Arc<crate::stats_recorder::StatsRecorder>,
) {
let _ = std::thread::Builder::new()
.name("punktfunk-video".into())
@@ -60,6 +61,7 @@ pub fn start(
&force_idr,
&rfi_range,
&video_cap,
&stats,
) {
tracing::error!(error = %format!("{e:#}"), "video stream failed");
}
@@ -68,6 +70,7 @@ pub fn start(
});
}
#[allow(clippy::too_many_arguments)]
fn run(
cfg: StreamConfig,
app: Option<&super::apps::AppEntry>,
@@ -75,6 +78,9 @@ fn run(
force_idr: &AtomicBool,
rfi_range: &std::sync::Mutex<Option<(i64, i64)>>,
video_cap: &std::sync::Mutex<Option<Box<dyn Capturer>>>,
// Shared stats recorder for the web-console capture/graph. Threaded into `stream_body` (the
// encode loop); per-frame sample emission is wired by a later pass.
stats: &Arc<crate::stats_recorder::StatsRecorder>,
) -> Result<()> {
// GameStream capture/encode thread: apply Windows session tuning (no-op off Windows).
crate::session_tuning::on_hot_thread();
@@ -100,18 +106,20 @@ fn run(
sock.connect(client)
.context("connect client video endpoint")?;
tracing::info!(%client, "video: client endpoint learned");
// Short label for web-console stats captures: the client's peer IP.
let client_label = client.ip().to_string();
// Native client-resolution source: create a compositor virtual output sized to the client's
// request and capture it (no scaling). Self-contained — deliberately NOT pooled in
// `video_cap`, since a reconnect at a different resolution needs a freshly-sized output; the
// output is released when this capturer drops at stream end (RAII via its keepalive).
if crate::config::config().video_source.as_deref() == Some("virtual") {
// The launched app picks the compositor (e.g. gamescope for game entries) and the
// nested command.
let compositor = app
.and_then(|a| a.compositor)
.map(Ok)
.unwrap_or_else(|| crate::vdisplay::detect().context("detect compositor"))?;
// Open the virtual-display source: pick the live compositor, normalize the session env
// (apply_session_env/apply_input_env — gamescope ATTACH/resize + KWin/Mutter retargeting,
// exactly like the native plane), create a virtual output at the client mode, and capture it.
// Re-runnable: the encode loop calls it again on a mid-stream capture loss to FOLLOW a
// Desktop<->Game switch.
let (mut capturer, compositor) = open_gs_virtual_source(cfg, app)?;
tracing::info!(
?compositor,
app = ?app.map(|a| &a.title),
@@ -119,31 +127,6 @@ fn run(
h = cfg.height,
"video source: virtual display (native client resolution)"
);
let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?;
// Carry the resolved launch command on the backend instance (per-session) rather than a
// process-global env var, so concurrent sessions can't stomp each other's launch target.
vd.set_launch_command(app.and_then(|a| a.cmd.clone()));
let vout = vd
.create(punktfunk_core::Mode {
width: cfg.width,
height: cfg.height,
refresh_hz: cfg.fps,
})
.context("create virtual output at client resolution")?;
// `want_hdr=false`: the IDD-push backend (opt-in PUNKTFUNK_IDD_PUSH) has no monitor-HDR
// auto-detection — it converts its always-FP16 ring per this flag — and GameStream HDR is not
// negotiated into StreamConfig here, so an IDD-push GameStream session streams SDR even on an
// HDR desktop. (The default WGC backend DOES auto-detect HDR from the output colorspace, but
// IDD-push bypasses WGC.) Acceptable for the experimental IDD-push A/B path; HDR over IDD-push
// is wired only for punktfunk/1 (want_hdr = negotiated bit_depth >= 10). TODO: derive want_hdr
// from a GameStream HDR flag once StreamConfig carries one.
let mut capturer = capture::capture_virtual_output(
vout,
capture::OutputFormat::resolve(false),
crate::session_plan::CaptureBackend::resolve(),
)
.context("capture virtual output")?;
capturer.set_active(true);
// Launch the app's command now that capture is live, for the backends that DON'T nest it via
// set_launch_command above: Windows (no gamescope) and Linux kwin/mutter/wlroots (which stream
// the existing desktop, so the app must be spawned into the session to land on the streamed
@@ -163,7 +146,22 @@ fn run(
}
}
}
return stream_body(&mut *capturer, &sock, cfg, running, force_idr, rfi_range);
// Rebuild closure: re-open the source on a mid-stream capture loss, RE-DETECTING the live
// compositor — so a Desktop<->Game switch (at the client's fixed mode) is FOLLOWED in place
// without a Moonlight reconnect. (A resolution change can't be followed mid-stream on
// GameStream — WxH is locked at ANNOUNCE — but a session toggle keeps the negotiated mode.)
let rebuild = || open_gs_virtual_source(cfg, app).map(|(c, _)| c);
return stream_body(
&mut capturer,
Some(&rebuild),
&sock,
cfg,
running,
force_idr,
rfi_range,
stats,
&client_label,
);
}
// Reuse the persistent capturer (one screencast session → clean reconnect); create it on
@@ -183,12 +181,70 @@ fn run(
}
};
capturer.set_active(true);
let result = stream_body(&mut *capturer, &sock, cfg, running, force_idr, rfi_range);
// Portal/synthetic source: no compositor virtual output to re-detect, so no rebuild closure.
let result = stream_body(
&mut capturer,
None,
&sock,
cfg,
running,
force_idr,
rfi_range,
stats,
&client_label,
);
capturer.set_active(false);
*video_cap.lock().unwrap() = Some(capturer);
result
}
/// Open the virtual-display video source for a GameStream session: pick the LIVE compositor + normalize
/// the session env (apply_session_env/apply_input_env — gamescope ATTACH/resize, KWin/Mutter
/// retargeting) exactly like the native plane (punktfunk1.rs resolve_compositor), create a virtual
/// output at the client's mode, and capture it. Returns the capturer (it owns the output's keepalive;
/// the stateless VirtualDisplay factory is dropped here) plus the resolved compositor. An apps.json
/// entry can PIN a compositor (skips the live detect/retarget). Re-run on a mid-stream capture loss to
/// FOLLOW a Desktop<->Game switch: it re-detects the now-live compositor and re-targets at it. Does NOT
/// launch the app (that happens once at stream start; a rebuild must not re-spawn it).
fn open_gs_virtual_source(
cfg: StreamConfig,
app: Option<&super::apps::AppEntry>,
) -> Result<(Box<dyn Capturer>, crate::vdisplay::Compositor)> {
let compositor = if let Some(c) = app.and_then(|a| a.compositor) {
c
} else {
let active = crate::vdisplay::detect_active_session();
crate::vdisplay::apply_session_env(&active);
let c = crate::vdisplay::compositor_for_kind(active.kind)
.map(Ok)
.unwrap_or_else(crate::vdisplay::detect)
.context("detect compositor")?;
crate::vdisplay::apply_input_env(c);
c
};
let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?;
// Carry the resolved launch command on the backend instance (per-session) rather than a
// process-global env var, so concurrent sessions can't stomp each other's launch target.
vd.set_launch_command(app.and_then(|a| a.cmd.clone()));
let vout = vd
.create(punktfunk_core::Mode {
width: cfg.width,
height: cfg.height,
refresh_hz: cfg.fps,
})
.context("create virtual output at client resolution")?;
// want_hdr=false: GameStream HDR is not negotiated into StreamConfig here (the default WGC backend
// still auto-detects HDR from the output colorspace; only the opt-in IDD-push path streams SDR).
let capturer = capture::capture_virtual_output(
vout,
capture::OutputFormat::resolve(false),
crate::session_plan::CaptureBackend::resolve(),
)
.context("capture virtual output")?;
capturer.set_active(true);
Ok((capturer, compositor))
}
/// One frame's packets, handed from the encode thread to the send thread.
type PacketBatch = Vec<Vec<u8>>;
@@ -326,15 +382,36 @@ fn spawn_sender(
Ok(())
}
/// Percentile of a slice (sorts it in place first). `q` in `0.0..=1.0`. Used for the web-console
/// stats sample's per-stage p50/p99.
fn percentile(v: &mut [u32], q: f64) -> u32 {
if v.is_empty() {
return 0;
}
v.sort_unstable();
let i = ((v.len() as f64 * q) as usize).min(v.len() - 1);
v[i]
}
/// The encode → packetize loop, over a borrowed capturer. Sending runs on a dedicated thread
/// (see [`spawn_sender`]) so a send spike can never stall capture/encode.
#[allow(clippy::too_many_arguments)]
fn stream_body(
capturer: &mut dyn Capturer,
// `&mut Box` (not `&mut dyn`) so a mid-stream capture-loss rebuild can SWAP the capturer in place.
capturer: &mut Box<dyn Capturer>,
// Re-open the video source on capture loss (virtual-display path → follow a Desktop<->Game switch);
// `None` for the portal/synthetic source, which has nothing to re-detect (propagate the error).
rebuild: Option<&dyn Fn() -> Result<Box<dyn Capturer>>>,
sock: &UdpSocket,
cfg: StreamConfig,
running: &Arc<AtomicBool>,
force_idr: &AtomicBool,
rfi_range: &std::sync::Mutex<Option<(i64, i64)>>,
// Shared stats recorder. The encode loop reads `stats.is_armed()` per frame to decide whether
// to accumulate the per-stage split, then emits a `StatsSample` at its 1 s aggregation boundary.
stats: &Arc<crate::stats_recorder::StatsRecorder>,
// Short client label (peer IP) seeded into the capture meta on the first armed registration.
client_label: &str,
) -> Result<()> {
// The first frame establishes the authoritative size/format for the encoder.
let mut frame = capturer.next_frame().context("capture first frame")?;
@@ -354,6 +431,9 @@ fn stream_body(
cfg.bitrate_kbps as u64 * 1000,
frame.is_cuda(),
8, // GameStream/Moonlight path: 8-bit (its own codec negotiation)
// GameStream/Moonlight stays 4:2:0 — stock Moonlight clients can't decode 4:4:4, and the
// protocol has no chroma negotiation. 4:4:4 is punktfunk/1-native only.
encode::ChromaFormat::Yuv420,
)
.context("open video encoder for stream")?;
// FEC overhead percent (Sunshine default 20). Override with PUNKTFUNK_FEC_PCT (0 = data-only).
@@ -398,19 +478,100 @@ fn stream_body(
let perf = crate::config::config().perf;
let (mut mx_cap, mut mx_enc, mut mx_pkt, mut mx_send, mut mx_pkts, mut uniq) =
(0u128, 0u128, 0u128, 0u128, 0usize, 0u32);
// Web-console stats accumulation (active when `perf` OR a capture is armed): per-stage vectors
// for p50/p99, the goodput bytes queued to the sender this window, the previous window's
// dropped-frame count for delta computation, and the registration id cached on the first sample.
let codec_name = match cfg.codec {
Codec::H264 => "h264",
Codec::H265 => "hevc",
Codec::Av1 => "av1",
};
let mut sid: Option<u32> = None;
let (mut v_cap, mut v_enc, mut v_pkt, mut v_send): (Vec<u32>, Vec<u32>, Vec<u32>, Vec<u32>) =
(Vec::new(), Vec::new(), Vec::new(), Vec::new());
let mut bytes_win: u64 = 0;
let mut last_dropped_batches: u64 = 0;
// Absolute next-frame deadline — the single pacing clock for the loop.
let mut next_frame = Instant::now();
// RFI capability is fixed for the session (probed at encoder open). Query it once so the
// recovery path skips the always-`false` invalidate call on encoders without NVENC RFI and
// forces a keyframe directly instead.
let supports_rfi = enc.caps().supports_rfi;
let mut supports_rfi = enc.caps().supports_rfi;
// Bound consecutive capture-loss rebuilds (a delivered frame clears the counter) so a permanently
// dead source can't loop forever — it ends the stream after the cap, falling back to a reconnect.
const MAX_REBUILDS: u32 = 5;
let mut rebuilds: u32 = 0;
while running.load(Ordering::SeqCst) {
let tick = Instant::now();
// Measure per-stage timing when `PUNKTFUNK_PERF` is set OR a web-console stats capture is
// armed (cheap Relaxed atomic, re-read each frame).
let measure = perf || stats.is_armed();
// Advance to the freshest captured frame if one arrived; otherwise reuse the last.
if let Some(f) = capturer.try_latest().context("capture frame")? {
frame = f;
uniq += 1;
match capturer.try_latest() {
Ok(Some(f)) => {
frame = f;
uniq += 1;
rebuilds = 0; // a delivered frame clears the consecutive-loss counter
}
Ok(None) => {} // no new frame — reuse the last (static/idle desktop)
Err(e) => {
// The capture source went away — the compositor was torn down on a Desktop<->Game
// switch, or the virtual output was removed. On the virtual-display path, re-detect the
// now-live compositor and re-attach IN PLACE (the send thread + packetizer + socket +
// RTP clock all survive), then force an IDR so Moonlight resyncs — so the stream FOLLOWS
// the switch with no client reconnect. Build the new source BEFORE dropping the old.
// Bounded by a counter + a ~40s budget; on exhaustion, end the stream (Moonlight
// reconnect). The portal/synthetic path has no rebuild closure → propagate as before.
let Some(rebuild) = rebuild else {
return Err(e).context("capture frame");
};
rebuilds += 1;
if rebuilds > MAX_REBUILDS {
return Err(e).context("capture lost — rebuild attempts exhausted");
}
tracing::warn!(error = %format!("{e:#}"), rebuild = rebuilds,
"gamestream: capture lost — rebuilding source in place (following a session switch)");
let rebuild_deadline = Instant::now() + Duration::from_secs(40);
let new_cap = loop {
match rebuild() {
Ok(c) => break c,
Err(e2) => {
if !running.load(Ordering::SeqCst) || Instant::now() >= rebuild_deadline
{
return Err(e2)
.context("capture lost — no source within the rebuild budget");
}
tracing::warn!(error = %format!("{e2:#}"),
"gamestream: source not up yet — retrying");
std::thread::sleep(Duration::from_millis(500));
}
}
};
*capturer = new_cap;
capturer.set_active(true);
frame = capturer.next_frame().context("first frame after rebuild")?;
// Re-open the encoder for the new source (same negotiated WxH → same SPS profile) and
// force an IDR so Moonlight resyncs on the first emitted AU.
enc = encode::open_video(
cfg.codec,
frame.format,
frame.width,
frame.height,
cfg.fps,
cfg.bitrate_kbps as u64 * 1000,
frame.is_cuda(),
8,
encode::ChromaFormat::Yuv420, // GameStream stays 4:2:0
)
.context("reopen encoder after rebuild")?;
supports_rfi = enc.caps().supports_rfi;
enc.request_keyframe();
next_frame = Instant::now();
tracing::info!("gamestream: source rebuilt — stream continues");
continue;
}
}
let t_cap = tick.elapsed();
// Honor a client recovery request. Prefer reference-frame invalidation (the encoder
@@ -447,9 +608,19 @@ fn stream_body(
// Hand the frame's packets to the send thread; never block here. A full queue means
// the sender is behind — drop this batch (FEC/RFI covers the client) and keep encoding.
let n = batch.len();
// Goodput this window = bytes actually queued to the sender (a dropped batch never reaches
// the wire, so it's excluded). Summed only when measuring, to keep the idle path free.
let batch_bytes: u64 = if measure {
batch.iter().map(|p| p.len() as u64).sum()
} else {
0
};
if n > 0 {
match batch_tx.try_send(batch) {
Ok(()) => sent_batches += 1,
Ok(()) => {
sent_batches += 1;
bytes_win += batch_bytes;
}
Err(std::sync::mpsc::TrySendError::Full(_)) => {
dropped_batches += 1;
if dropped_batches.is_power_of_two() {
@@ -461,17 +632,26 @@ fn stream_body(
}
}
}
if perf {
if measure {
let t_send = tick.elapsed();
mx_cap = mx_cap.max(t_cap.as_micros());
mx_enc = mx_enc.max((t_enc - t_cap).as_micros());
mx_pkt = mx_pkt.max((t_pkt - t_enc).as_micros());
mx_send = mx_send.max((t_send - t_pkt).as_micros());
let cap_us = t_cap.as_micros();
let enc_us = (t_enc - t_cap).as_micros();
let pkt_us = (t_pkt - t_enc).as_micros();
let send_us = (t_send - t_pkt).as_micros();
mx_cap = mx_cap.max(cap_us);
mx_enc = mx_enc.max(enc_us);
mx_pkt = mx_pkt.max(pkt_us);
mx_send = mx_send.max(send_us);
mx_pkts = mx_pkts.max(n);
v_cap.push(cap_us as u32);
v_enc.push(enc_us as u32);
v_pkt.push(pkt_us as u32);
v_send.push(send_us as u32);
}
fps_count += 1;
if fps_t.elapsed() >= Duration::from_secs(1) {
let secs = fps_t.elapsed().as_secs_f64();
if perf {
// Max µs/stage this second: cap=drain channel, enc=submit (zero-copy device
// copy + NVENC), pkt=poll+FEC+packetize, send=paced packet send. `uniq`=new
@@ -486,12 +666,6 @@ fn stream_body(
max_pkts = mx_pkts,
"video: streaming (perf)"
);
mx_cap = 0;
mx_enc = 0;
mx_pkt = 0;
mx_send = 0;
mx_pkts = 0;
uniq = 0;
} else {
tracing::info!(
fps = fps_count,
@@ -500,6 +674,68 @@ fn stream_body(
"video: streaming"
);
}
// Web-console capture: build the aggregated sample. The host send side exposes no
// receiver-side packet loss / FEC-recovery / send-buffer EAGAIN counters, so those stay
// 0 (not fabricated); `frames_dropped` is the per-frame send-queue overflow delta.
if stats.is_armed() {
let session_id = *sid.get_or_insert_with(|| {
stats.register_session(
"gamestream",
cfg.width,
cfg.height,
cfg.fps,
codec_name,
client_label,
)
});
let sample = crate::stats_recorder::StatsSample {
t_ms: 0, // stamped by push_sample from the capture's monotonic start
session_id,
stages: vec![
crate::stats_recorder::StageTiming {
name: "capture".into(),
p50_us: percentile(&mut v_cap, 0.50) as f32,
p99_us: percentile(&mut v_cap, 0.99) as f32,
},
crate::stats_recorder::StageTiming {
name: "encode".into(),
p50_us: percentile(&mut v_enc, 0.50) as f32,
p99_us: percentile(&mut v_enc, 0.99) as f32,
},
crate::stats_recorder::StageTiming {
name: "packetize".into(),
p50_us: percentile(&mut v_pkt, 0.50) as f32,
p99_us: percentile(&mut v_pkt, 0.99) as f32,
},
crate::stats_recorder::StageTiming {
name: "send".into(),
p50_us: percentile(&mut v_send, 0.50) as f32,
p99_us: percentile(&mut v_send, 0.99) as f32,
},
],
fps: (uniq as f64 / secs) as f32,
repeat_fps: (fps_count.saturating_sub(uniq) as f64 / secs) as f32,
mbps: (bytes_win as f64 * 8.0 / secs / 1_000_000.0) as f32,
bitrate_kbps: cfg.bitrate_kbps,
frames_dropped: dropped_batches.saturating_sub(last_dropped_batches) as u32,
packets_dropped: 0,
send_dropped: 0,
fec_recovered: 0,
};
stats.push_sample(session_id, sample);
}
mx_cap = 0;
mx_enc = 0;
mx_pkt = 0;
mx_send = 0;
mx_pkts = 0;
uniq = 0;
v_cap.clear();
v_enc.clear();
v_pkt.clear();
v_send.clear();
bytes_win = 0;
last_dropped_batches = dropped_batches;
fps_count = 0;
fps_t = Instant::now();
}
+12 -4
View File
@@ -24,6 +24,12 @@ use std::sync::Arc;
#[derive(Clone)]
pub(crate) struct PeerCertFingerprint(pub Option<String>);
/// The TCP source address of an HTTPS request, injected per-connection by [`serve_https`]. Used by
/// `/launch` to record which paired client owns the session so the unauthenticated RTSP/UDP media
/// plane can bind to that peer's IP (security-review 2026-06-28 #4).
#[derive(Clone, Copy)]
pub(crate) struct PeerAddr(pub SocketAddr);
/// HTTPS server that surfaces the verified client cert to handlers. `axum_server` can't expose the
/// peer cert, so this runs the rustls handshake itself (tokio-rustls), reads the peer certificate,
/// and serves the axum `Router` over hyper with the peer's fingerprint attached to every request as
@@ -39,7 +45,7 @@ pub(crate) async fn serve_https(
.await
.with_context(|| format!("bind HTTPS {bind}"))?;
loop {
let (tcp, _peer) = match listener.accept().await {
let (tcp, peer) = match listener.accept().await {
Ok(v) => v,
Err(e) => {
tracing::warn!(error = %e, "HTTPS accept failed");
@@ -63,14 +69,16 @@ pub(crate) async fn serve_https(
.peer_certificates()
.and_then(|c| c.first())
.map(|c| hex::encode(punktfunk_core::quic::endpoint::cert_fingerprint(c.as_ref())));
let peer = PeerCertFingerprint(fp);
let fp = PeerCertFingerprint(fp);
let addr = PeerAddr(peer);
let svc =
hyper::service::service_fn(move |req: hyper::Request<hyper::body::Incoming>| {
let app = app.clone();
let peer = peer.clone();
let fp = fp.clone();
async move {
let mut req = req.map(axum::body::Body::new);
req.extensions_mut().insert(peer);
req.extensions_mut().insert(fp);
req.extensions_mut().insert(addr);
app.oneshot(req).await // Router error is Infallible
}
});
@@ -3,7 +3,7 @@
//! `RTP_PACKET(12, big-endian) + reserved[4] + NV_VIDEO_PACKET(16, little-endian) + payload`
//! and the frame's bitstream is prefixed with an 8-byte `video_short_frame_header_t`, then
//! striped into ≤4 FEC blocks of ≤255 shards. Byte-exact spec:
//! `docs/research/gamestream-protocol-research.json` (video plane).
//! `design/research/gamestream-protocol-research.json` (video plane).
//!
//! FEC (P1.5): each block carries `m = ⌈k·pct/100⌉` ReedSolomon parity shards generated by
//! `punktfunk_core::fec::Gf8Coder` (the nanors-compatible Cauchy GF(2⁸) coder). Crucially, RS runs
+42 -12
View File
@@ -24,6 +24,9 @@ pub trait InputInjector {
pub enum Backend {
/// wlroots virtual pointer + keyboard Wayland protocols — the headless-Sway path.
WlrVirtual,
/// KWin `org_kde_kwin_fake_input` — direct injection, no RemoteDesktop portal / approval dialog
/// (authorized by the host's `.desktop`). The headless KDE-Desktop path; what krdpserver uses.
KwinFakeInput,
/// libei via `reis` — Wayland-native (RemoteDesktop portal). Not yet implemented.
Libei,
/// libei directly against gamescope's own EIS socket (no portal): input lands in the
@@ -47,6 +50,16 @@ pub fn open(backend: Backend) -> Result<Box<dyn InputInjector>> {
anyhow::bail!("wlroots virtual input requires Linux + a Wayland compositor")
}
}
Backend::KwinFakeInput => {
#[cfg(target_os = "linux")]
{
Ok(Box::new(kwin_fake_input::KwinFakeInjector::open()?))
}
#[cfg(not(target_os = "linux"))]
{
anyhow::bail!("KWin fake_input requires Linux + a KWin Wayland session")
}
}
Backend::Libei => {
#[cfg(target_os = "linux")]
{
@@ -63,9 +76,7 @@ pub fn open(backend: Backend) -> Result<Box<dyn InputInjector>> {
#[cfg(target_os = "linux")]
{
Ok(Box::new(libei::LibeiInjector::open_with(
libei::EiSource::SocketPathFile(
crate::vdisplay::gamescope_ei_socket_file().into(),
),
libei::EiSource::SocketPathFile(crate::vdisplay::gamescope_ei_socket_file()),
)?))
}
#[cfg(not(target_os = "linux"))]
@@ -90,12 +101,18 @@ pub fn open(backend: Backend) -> Result<Box<dyn InputInjector>> {
/// Pick the injection backend for the current session. gamescope hosts its own EIS server (no
/// portal), so a gamescope session injects directly into it. wlroots/Sway only implements the
/// ScreenCast portal (no RemoteDesktop), so libei can't run there — use the wlr virtual-input
/// protocols. KWin and GNOME implement RemoteDesktop but not the wlr protocols, so use libei.
/// `PUNKTFUNK_INPUT_BACKEND=wlr|libei|gamescope|uinput` overrides the auto-detection.
/// protocols. **KWin** exposes `org_kde_kwin_fake_input` (direct injection, no portal / approval
/// dialog — the only headless-capable path; what krdpserver uses), so prefer it there. **GNOME**
/// has neither fake_input nor the wlr protocols, so it uses libei via the RemoteDesktop portal
/// (which needs a user to approve, or a pre-seeded grant — not truly headless).
/// `PUNKTFUNK_INPUT_BACKEND=wlr|kwin|libei|gamescope|uinput` overrides the auto-detection.
pub fn default_backend() -> Backend {
if let Ok(v) = std::env::var("PUNKTFUNK_INPUT_BACKEND") {
match v.trim().to_ascii_lowercase().as_str() {
"wlr" | "wlroots" | "wlrvirtual" => return Backend::WlrVirtual,
"kwin" | "fakeinput" | "fake_input" | "kwin-fake-input" => {
return Backend::KwinFakeInput
}
"libei" | "ei" | "portal" => return Backend::Libei,
"gamescope" | "gamescope-ei" => return Backend::GamescopeEi,
"uinput" => return Backend::Uinput,
@@ -112,16 +129,26 @@ pub fn default_backend() -> Backend {
}
#[cfg(not(target_os = "windows"))]
{
if crate::config::config()
.compositor
.as_deref()
.is_some_and(|v| v.trim().eq_ignore_ascii_case("gamescope"))
{
return Backend::GamescopeEi;
// An explicit compositor pick (set per connect / mid-stream) is the strongest signal.
let compositor = crate::config::config().compositor.clone();
if let Some(c) = compositor.as_deref() {
let c = c.trim();
if c.eq_ignore_ascii_case("gamescope") {
return Backend::GamescopeEi;
}
if c.eq_ignore_ascii_case("kwin") {
return Backend::KwinFakeInput;
}
if c.eq_ignore_ascii_case("wlroots") || c.eq_ignore_ascii_case("sway") {
return Backend::WlrVirtual;
}
// mutter (GNOME) falls through to the XDG_CURRENT_DESKTOP check below.
}
let desktop = std::env::var("XDG_CURRENT_DESKTOP").unwrap_or_default();
let d = desktop.to_ascii_uppercase();
if d.contains("KDE") || d.contains("GNOME") {
if d.contains("KDE") {
Backend::KwinFakeInput
} else if d.contains("GNOME") {
Backend::Libei
} else {
Backend::WlrVirtual
@@ -478,6 +505,9 @@ pub mod gamepad {
}
}
#[cfg(target_os = "linux")]
#[path = "inject/linux/kwin_fake_input.rs"]
mod kwin_fake_input;
#[cfg(target_os = "linux")]
#[path = "inject/linux/libei.rs"]
mod libei;
#[cfg(target_os = "windows")]
@@ -0,0 +1,209 @@
//! Headless input injection on KWin via the privileged `org_kde_kwin_fake_input` protocol — the
//! exact path KDE's own headless RDP server (`krdpserver`) uses. KWin advertises this restricted
//! global only to a client authorized through its installed `.desktop` `X-KDE-Wayland-Interfaces`
//! (we ship `io.unom.Punktfunk.Host.desktop`, which lists `org_kde_kwin_fake_input` alongside
//! `zkde_screencast_unstable_v1`). Binding the global IS the authorization, so injection needs **no
//! RemoteDesktop portal and no "Allow remote control?" dialog** — it works with no user present,
//! which the libei/portal path cannot. We connect as an ordinary Wayland client on the KWin session's
//! `$WAYLAND_DISPLAY` and translate events into fake-input requests; keyboard keys are raw Linux
//! evdev codes that KWin resolves through the session's own keymap (no keymap upload, unlike the wlr
//! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space — which
//! on a headless box (single per-session virtual output at the origin, scale 1) equals the streamed
//! output's pixels.
#![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{gs_button_to_evdev, vk_to_evdev, InputEvent, InputInjector};
use anyhow::{Context, Result};
use punktfunk_core::input::InputKind;
use wayland_client::protocol::wl_registry::{self, WlRegistry};
use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle};
// Generate the client bindings for the vendored protocol XML inline (no build.rs), exactly like the
// KWin virtual-output backend. Path is relative to CARGO_MANIFEST_DIR.
#[allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
pub mod fake {
use wayland_client;
use wayland_client::protocol::*;
pub mod __interfaces {
use wayland_client::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("protocols/fake-input.xml");
}
use self::__interfaces::*;
wayland_scanner::generate_client_code!("protocols/fake-input.xml");
}
use fake::org_kde_kwin_fake_input::OrgKdeKwinFakeInput as FakeInput;
/// Highest interface version we drive. `keyboard_key` arrived at v4; KWin advertises ≥4.
const MAX_VERSION: u32 = 4;
/// `wl_pointer.axis` values used by `axis`.
const AXIS_VERTICAL: u32 = 0;
const AXIS_HORIZONTAL: u32 = 1;
/// `code` value marking a horizontal scroll event (mirrors `gamestream::input` / the wlr backend).
const SCROLL_HORIZONTAL: u32 = 1;
/// Registry-bound globals (the Wayland dispatch state).
#[derive(Default)]
struct State {
fake: Option<FakeInput>,
}
impl Dispatch<WlRegistry, ()> for State {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wl_registry::Event,
_: &(),
_: &Connection,
qh: &QueueHandle<Self>,
) {
if let wl_registry::Event::Global {
name,
interface,
version,
} = event
{
if interface == "org_kde_kwin_fake_input" {
state.fake = Some(registry.bind(name, version.min(MAX_VERSION), qh, ()));
}
}
}
}
// fake_input emits no events.
impl Dispatch<FakeInput, ()> for State {
fn event(
_: &mut Self,
_: &FakeInput,
_: <FakeInput as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
pub struct KwinFakeInjector {
conn: Connection,
queue: EventQueue<State>,
state: State,
fake: FakeInput,
}
impl KwinFakeInjector {
pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env()
.context("connect to KWin Wayland (is WAYLAND_DISPLAY set to the KWin socket?)")?;
let mut queue = conn.new_event_queue();
let qh = queue.handle();
let _registry = conn.display().get_registry(&qh, ());
let mut state = State::default();
queue
.roundtrip(&mut state)
.context("Wayland registry roundtrip")?;
let fake = state.fake.clone().context(
"KWin does not expose org_kde_kwin_fake_input to this client — install the host's \
.desktop (io.unom.Punktfunk.Host.desktop, X-KDE-Wayland-Interfaces) and re-login so \
KWin authorizes it (the grant is cached per-exe on first connect), or this is not a \
KWin session",
)?;
// Authenticate (the legacy handshake; for an interface-authorized client KWin accepts it
// without a dialog — same as krdpserver/krfb headless).
fake.authenticate("punktfunk".into(), "remote streaming input".into());
queue
.roundtrip(&mut state)
.context("fake_input authenticate roundtrip")?;
conn.flush().ok();
tracing::info!("KWin fake_input ready (headless keyboard/mouse/touch — no portal)");
Ok(Self {
conn,
queue,
state,
fake,
})
}
}
impl InputInjector for KwinFakeInjector {
fn inject(&mut self, event: &InputEvent) -> Result<()> {
match event.kind {
InputKind::MouseMove => {
self.fake.pointer_motion(event.x as f64, event.y as f64);
}
InputKind::MouseMoveAbs => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
self.fake.pointer_motion_absolute(x, y);
}
}
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
if let Some(btn) = gs_button_to_evdev(event.code) {
let st = u32::from(event.kind == InputKind::MouseButtonDown);
self.fake.button(btn, st);
}
}
InputKind::MouseScroll => {
// GameStream sends WHEEL_DELTA(120)-scaled units; a notch ≈ 15px. Vertical flips
// sign on the Wayland axis, horizontal passes through — same as the wlr backend.
let horizontal = event.code == SCROLL_HORIZONTAL;
let axis = if horizontal {
AXIS_HORIZONTAL
} else {
AXIS_VERTICAL
};
let notches = event.x as f64 / 120.0;
let sign = if horizontal { 1.0 } else { -1.0 };
self.fake.axis(axis, sign * notches * 15.0);
}
InputKind::KeyDown | InputKind::KeyUp => {
// Raw evdev keycode; KWin resolves it through the session's own keymap (and tracks
// modifier state itself, so no separate modifiers request is needed).
if let Some(evdev) = vk_to_evdev(event.code as u8) {
let st = u32::from(event.kind == InputKind::KeyDown);
self.fake.keyboard_key(evdev as u32, st);
} else {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
}
}
// Touch: id = event.code, coords in the client surface w×h packed into flags (same
// absolute mapping as MouseMoveAbs). Each event is its own frame.
InputKind::TouchDown | InputKind::TouchMove => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
if event.kind == InputKind::TouchDown {
self.fake.touch_down(event.code, x, y);
} else {
self.fake.touch_motion(event.code, x, y);
}
self.fake.touch_frame();
}
}
InputKind::TouchUp => {
self.fake.touch_up(event.code);
self.fake.touch_frame();
}
// Gamepads are injected through uinput, not the compositor.
InputKind::GamepadButton | InputKind::GamepadAxis => {}
}
// Surface protocol errors / disconnects, then push the batch to the compositor.
self.queue
.dispatch_pending(&mut self.state)
.context("wayland dispatch")?;
self.conn.flush().context("wayland flush")?;
Ok(())
}
}
@@ -305,6 +305,19 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
let deadline = std::time::Instant::now() + Duration::from_secs(15);
let mut logged = String::new();
loop {
// Defense-in-depth: never follow a symlinked relay file. It lives under `$XDG_RUNTIME_DIR`
// (per-user 0700) so a cross-user plant is already blocked, but refuse a symlink outright
// rather than read through one to an attacker-chosen target (a rogue EIS server would
// keylog/deny the session's input; security-review 2026-06-28 #6).
if std::fs::symlink_metadata(file)
.map(|m| m.file_type().is_symlink())
.unwrap_or(false)
{
return Err(anyhow!(
"EIS relay file {} is a symlink — refusing to follow it",
file.display()
));
}
if let Ok(s) = std::fs::read_to_string(file) {
let name = s.trim();
if !name.is_empty() {
@@ -109,7 +109,7 @@ pub(super) struct SwDeviceProfile<'a> {
/// `profile.instance`). The returned `HSWDEVICE` owns it — `SwDeviceClose` removes it on drop, so the
/// pad appears/disappears with the session and nothing persists.
///
/// **Game-detection identity** (see `docs/windows-dualsense-game-detection.md`). `HIDD_ATTRIBUTES`
/// **Game-detection identity** (see `design/windows-dualsense-game-detection.md`). `HIDD_ATTRIBUTES`
/// alone (VID/PID via the IOCTL) satisfies SDL/HIDAPI/RawInput, but a native PS5 path (libScePad-
/// style raw HID) classifies the *connection type* by walking from the HID child to its parent
/// (`CM_Get_Parent`) and string-matching `"USB"`/`"BTHENUM"` in that parent's
+22 -3
View File
@@ -577,10 +577,11 @@ impl LibraryProvider for EpicProvider {
if p.extension().and_then(|e| e.to_str()) != Some("item") {
continue;
}
let Ok(text) = std::fs::read_to_string(&p) else {
// `.item` manifests are small JSON; cap the read so a planted giant can't OOM the host.
let Some(bytes) = read_capped(&p, 1024 * 1024) else {
continue;
};
let Ok(v) = serde_json::from_str::<serde_json::Value>(&text) else {
let Ok(v) = serde_json::from_slice::<serde_json::Value>(&bytes) else {
continue;
};
if let Some(g) = epic_entry(&v, &art) {
@@ -650,6 +651,23 @@ fn epic_entry(
})
}
/// Read a launcher cache/manifest with a hard size cap, so a local unprivileged user can't plant a
/// multi-GB file under the launcher's (Users-writable) data dir that OOMs the privileged host when
/// it's loaded — then base64/JSON-decoded into further copies — during library enumeration
/// (security-review 2026-06-28 S4). Returns `None` if missing, empty, or over `max`. Mirrors the
/// Linux lutris-art reader's 1 MiB cap.
#[cfg(windows)]
fn read_capped(path: &Path, max: u64) -> Option<Vec<u8>> {
let meta = std::fs::metadata(path).ok()?;
if meta.len() == 0 || meta.len() > max {
if meta.len() > max {
tracing::warn!(path = %path.display(), len = meta.len(), max, "launcher cache exceeds size cap — skipping");
}
return None;
}
std::fs::read(path).ok()
}
/// Best-effort parse of `catcache.bin` (base64-encoded JSON array of catalog items) into
/// catalogItemId → [`Artwork`] from each item's `keyImages`. Empty map on any read/decode failure
/// (the format is community-reverse-engineered + can lag a fresh install → titles just show no art).
@@ -657,7 +675,8 @@ fn epic_entry(
fn epic_art_index(catcache: &Path) -> std::collections::HashMap<String, Artwork> {
use base64::Engine as _;
let mut map = std::collections::HashMap::new();
let Ok(raw) = std::fs::read(catcache) else {
// 32 MiB cap: comfortably fits a real catalog cache, blocks a planted giant (S4).
let Some(raw) = read_capped(catcache, 32 * 1024 * 1024) else {
return map;
};
let Ok(decoded) = base64::engine::general_purpose::STANDARD.decode(raw) else {
+12 -2
View File
@@ -35,6 +35,9 @@ mod gamestream;
mod hdr;
mod inject;
#[cfg(target_os = "windows")]
#[path = "windows/install.rs"]
mod install;
#[cfg(target_os = "windows")]
#[path = "windows/interactive.rs"]
mod interactive;
mod library;
@@ -50,6 +53,7 @@ mod service;
mod session_plan;
mod session_tuning;
mod spike;
mod stats_recorder;
mod vdisplay;
#[cfg(target_os = "windows")]
#[path = "windows/wgc_helper.rs"]
@@ -389,7 +393,7 @@ fn real_main() -> Result<()> {
}
// USER-session WGC helper (Windows two-process secure-desktop design): capture the EXISTING
// SudoVDA via WGC + NVENC, stream AUs on stdout to the SYSTEM host. Spawned by the host
// (CreateProcessAsUser), not run by hand. See docs/windows-secure-desktop.md.
// (CreateProcessAsUser), not run by hand. See design/archive/windows-secure-desktop.md.
#[cfg(target_os = "windows")]
Some("wgc-helper") => {
let get = |flag: &str| {
@@ -421,6 +425,12 @@ fn real_main() -> Result<()> {
// that launches the host into the active interactive session.
#[cfg(target_os = "windows")]
Some("service") => service::main(&args[1..]),
// Install-time work the Windows installer delegates to the exe instead of locale-parsed
// PowerShell *files* (the ANSI-codepage parse-break root fix; see windows/install.rs).
#[cfg(target_os = "windows")]
Some("driver") => install::driver_main(&args[1..]),
#[cfg(target_os = "windows")]
Some("web") => install::web_main(&args[1..]),
Some("-h") | Some("--help") | Some("help") | None => {
print_usage();
Ok(())
@@ -704,7 +714,7 @@ SPIKE OPTIONS:
NOTES:
'portal' needs headless Sway + xdg-desktop-portal-wlr running in this session
(see docs/linux-setup.md). 'synthetic' needs no capture session and always runs.
(see design/linux-setup.md). 'synthetic' needs no capture session and always runs.
Encoded AUs are written to a playable file AND (unless --no-loopback) fed through a
punktfunk_core hostclient loopback that reassembles and byte-verifies each one.
Both 'serve' and 'punktfunk1-host' advertise the native service over mDNS
+219 -9
View File
@@ -6,7 +6,7 @@
//! The API is versioned under `/api/v1` and described by an OpenAPI 3.1 document generated
//! at compile time with `utoipa` — `punktfunk-host openapi` prints it for client codegen, the
//! running server serves it at `/api/v1/openapi.json` plus interactive docs at `/api/docs`,
//! and a copy is checked in at `docs/api/openapi.json` (a test fails if it drifts, like the
//! and a copy is checked in at `api/openapi.json` (a test fails if it drifts, like the
//! cbindgen header).
//!
//! Security: binds loopback by default, serves HTTPS with the host's identity cert, and requires
@@ -20,6 +20,7 @@ use crate::gamestream::{
tls::{serve_https, PeerCertFingerprint},
AppState, APP_VERSION, AUDIO_PORT, CONTROL_PORT, GFE_VERSION, RTSP_PORT, VIDEO_PORT,
};
use crate::stats_recorder::{Capture, CaptureMeta, StatsStatus};
use anyhow::{Context, Result};
use axum::{
extract::{Path, Request, State},
@@ -66,6 +67,9 @@ struct MgmtState {
/// Native (punktfunk/1) pairing — shared with the QUIC host when the unified `serve --native`
/// runs it. `None` ⇒ GameStream-only host (the native endpoints report `enabled: false`).
native: Option<Arc<crate::native_pairing::NativePairing>>,
/// Shared streaming-stats recorder — the same handle the streaming loops emit into, so an
/// operator can arm/stop a capture here and review/list/delete saved recordings.
stats: Arc<crate::stats_recorder::StatsRecorder>,
token: Option<String>,
/// The port we serve on, echoed in [`PortMap`] so a client can persist a full endpoint map.
port: u16,
@@ -77,6 +81,7 @@ pub async fn run(
state: Arc<AppState>,
opts: Options,
native: Option<Arc<crate::native_pairing::NativePairing>>,
stats: Arc<crate::stats_recorder::StatsRecorder>,
) -> Result<()> {
// The mgmt API is HTTPS + token-authenticated ALWAYS (even on loopback): `parse_serve`
// guarantees a token (CLI flag / env / persisted ~/.config/punktfunk/mgmt-token / generated).
@@ -100,7 +105,7 @@ pub async fn run(
auth = "mTLS (paired cert) or bearer (required)",
"management API listening over HTTPS (docs at /api/docs, spec at /api/v1/openapi.json)"
);
let app = app(state, Some(token), opts.bind.port(), native);
let app = app(state, Some(token), opts.bind.port(), native, stats);
serve_https(opts.bind, app, tls).await
}
@@ -110,10 +115,12 @@ fn app(
token: Option<String>,
port: u16,
native: Option<Arc<crate::native_pairing::NativePairing>>,
stats: Arc<crate::stats_recorder::StatsRecorder>,
) -> Router {
let shared = Arc::new(MgmtState {
app: state,
native,
stats,
token,
port,
});
@@ -158,13 +165,19 @@ fn api_router_parts() -> (Router<Arc<MgmtState>>, utoipa::openapi::OpenApi) {
.routes(routes!(request_idr))
.routes(routes!(get_library))
.routes(routes!(create_custom_game))
.routes(routes!(update_custom_game, delete_custom_game)),
.routes(routes!(update_custom_game, delete_custom_game))
.routes(routes!(stats_capture_start))
.routes(routes!(stats_capture_stop))
.routes(routes!(stats_capture_status))
.routes(routes!(stats_capture_live))
.routes(routes!(stats_recordings_list))
.routes(routes!(stats_recording_get, stats_recording_delete)),
)
.split_for_parts()
}
/// The OpenAPI document as pretty JSON — what `punktfunk-host openapi` prints and what is
/// checked in at `docs/api/openapi.json` for client codegen.
/// checked in at `api/openapi.json` for client codegen.
pub fn openapi_json() -> String {
let (_, api) = api_router_parts();
let mut json = api.to_pretty_json().expect("serialize OpenAPI document");
@@ -190,6 +203,7 @@ pub fn openapi_json() -> String {
(name = "native", description = "Native punktfunk/1 pairing: arm a window, display the host PIN, manage paired devices"),
(name = "session", description = "Active streaming session control"),
(name = "library", description = "Game library: installed-store titles (Steam) plus user-curated custom entries"),
(name = "stats", description = "Streaming performance-stats capture: arm/stop a recording, read the live + saved time-series for graphing"),
)
)]
struct ApiDoc;
@@ -1218,6 +1232,185 @@ async fn delete_custom_game(Path(id): Path<String>) -> Response {
}
}
// ---------------------------------------------------------------------------------------
// Streaming stats capture (design/stats-capture-plan.md §2)
// ---------------------------------------------------------------------------------------
/// Start a stats capture
///
/// Arms a new performance-stats capture. Idempotent: if a capture is already running this returns
/// the current status unchanged. While armed, the streaming loops emit aggregated samples (~ every
/// 12 s) into the in-progress capture, readable live via `GET /stats/capture/live`.
#[utoipa::path(
post,
path = "/stats/capture/start",
tag = "stats",
operation_id = "statsCaptureStart",
responses(
(status = OK, description = "Capture armed (or already running)", body = StatsStatus),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
async fn stats_capture_start(State(st): State<Arc<MgmtState>>) -> Json<StatsStatus> {
let status = st.stats.start();
tracing::info!(
started_unix_ms = status.started_unix_ms,
"management API: stats capture armed"
);
Json(status)
}
/// Stop the stats capture
///
/// Disarms the in-progress capture and writes it to disk atomically, returning its summary. If
/// nothing was recording, returns `204 No Content`.
#[utoipa::path(
post,
path = "/stats/capture/stop",
tag = "stats",
operation_id = "statsCaptureStop",
responses(
(status = OK, description = "Capture stopped and saved", body = CaptureMeta),
(status = NO_CONTENT, description = "Nothing was recording"),
(status = INTERNAL_SERVER_ERROR, description = "Could not write the recording to disk", body = ApiError),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
async fn stats_capture_stop(State(st): State<Arc<MgmtState>>) -> Response {
match st.stats.stop() {
Ok(Some(meta)) => {
tracing::info!(id = %meta.id, samples = meta.sample_count, "management API: stats capture saved");
(StatusCode::OK, Json(meta)).into_response()
}
Ok(None) => StatusCode::NO_CONTENT.into_response(),
Err(e) => api_error(
StatusCode::INTERNAL_SERVER_ERROR,
&format!("could not save capture: {e}"),
),
}
}
/// Stats capture status
///
/// Whether a capture is armed, its sample count, and start time. Poll this (e.g. every 2 s) to
/// drive the capture-control UI.
#[utoipa::path(
get,
path = "/stats/capture/status",
tag = "stats",
operation_id = "statsCaptureStatus",
responses(
(status = OK, description = "In-progress capture status (idle when not armed)", body = StatsStatus),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
async fn stats_capture_status(State(st): State<Arc<MgmtState>>) -> Json<StatsStatus> {
Json(st.stats.status())
}
/// Live in-progress capture
///
/// The full sample time-series of the capture currently recording, for live graphing. `404` when
/// nothing is armed.
#[utoipa::path(
get,
path = "/stats/capture/live",
tag = "stats",
operation_id = "statsCaptureLive",
responses(
(status = OK, description = "The in-progress capture (meta + samples so far)", body = Capture),
(status = NOT_FOUND, description = "No capture is currently recording", body = ApiError),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
async fn stats_capture_live(State(st): State<Arc<MgmtState>>) -> Response {
match st.stats.live_snapshot() {
Some(capture) => Json(capture).into_response(),
None => api_error(StatusCode::NOT_FOUND, "no capture is currently recording"),
}
}
/// List saved recordings
///
/// Every saved capture's summary (the `meta` head only — not the sample body), newest first.
#[utoipa::path(
get,
path = "/stats/recordings",
tag = "stats",
operation_id = "statsRecordingsList",
responses(
(status = OK, description = "Saved capture summaries, newest first", body = [CaptureMeta]),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
async fn stats_recordings_list(State(st): State<Arc<MgmtState>>) -> Json<Vec<CaptureMeta>> {
Json(st.stats.list())
}
/// Get a saved recording
///
/// The full capture (meta + samples) for `id`, for graphing or download.
#[utoipa::path(
get,
path = "/stats/recordings/{id}",
tag = "stats",
operation_id = "statsRecordingGet",
params(("id" = String, Path, description = "The recording id (its filename stem)")),
responses(
(status = OK, description = "The full capture", body = Capture),
(status = NOT_FOUND, description = "No recording with that id", body = ApiError),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
(status = INTERNAL_SERVER_ERROR, description = "The recording file is unreadable", body = ApiError),
)
)]
async fn stats_recording_get(State(st): State<Arc<MgmtState>>, Path(id): Path<String>) -> Response {
match st.stats.load(&id) {
Ok(capture) => Json(capture).into_response(),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
api_error(StatusCode::NOT_FOUND, "no recording with that id")
}
Err(e) => api_error(
StatusCode::INTERNAL_SERVER_ERROR,
&format!("could not read recording: {e}"),
),
}
}
/// Delete a saved recording
///
/// Removes the recording `id` from disk. `404` if there is no such recording.
#[utoipa::path(
delete,
path = "/stats/recordings/{id}",
tag = "stats",
operation_id = "statsRecordingDelete",
params(("id" = String, Path, description = "The recording id (its filename stem)")),
responses(
(status = NO_CONTENT, description = "Recording deleted"),
(status = NOT_FOUND, description = "No recording with that id", body = ApiError),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
(status = INTERNAL_SERVER_ERROR, description = "Could not delete the recording", body = ApiError),
)
)]
async fn stats_recording_delete(
State(st): State<Arc<MgmtState>>,
Path(id): Path<String>,
) -> Response {
match st.stats.delete(&id) {
Ok(()) => {
tracing::info!(id, "management API: recording deleted");
StatusCode::NO_CONTENT.into_response()
}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
api_error(StatusCode::NOT_FOUND, "no recording with that id")
}
Err(e) => api_error(
StatusCode::INTERNAL_SERVER_ERROR,
&format!("could not delete recording: {e}"),
),
}
}
// ---------------------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------------------
@@ -1231,6 +1424,15 @@ mod tests {
use std::net::{IpAddr, Ipv4Addr};
use tower::ServiceExt;
/// A throwaway stats recorder rooted in a unique temp dir (never touches the real config dir).
fn test_stats() -> Arc<crate::stats_recorder::StatsRecorder> {
crate::stats_recorder::StatsRecorder::new(std::env::temp_dir().join(format!(
"pf-mgmt-stats-{}-{:p}",
std::process::id(),
&0u8 as *const u8
)))
}
fn test_state() -> Arc<AppState> {
let host = Host {
hostname: "test-host".into(),
@@ -1240,18 +1442,20 @@ mod tests {
https_port: HTTPS_PORT,
};
let identity = ServerIdentity::ephemeral().expect("ephemeral identity");
Arc::new(AppState::new(host, identity))
Arc::new(AppState::new(host, identity, test_stats()))
}
// The mgmt API now always requires auth, so the router always has a token. A test that passes
// `None` gets the default "test-secret" (and `send` auto-attaches the matching bearer); a test
// that passes an explicit token exercises a mismatch (e.g. `bearer_token_is_enforced`).
fn test_app(state: Arc<AppState>, token: Option<&str>) -> Router {
let stats = state.stats.clone();
app(
state,
Some(token.unwrap_or("test-secret").to_string()),
DEFAULT_PORT,
None,
stats,
)
}
@@ -1261,11 +1465,13 @@ mod tests {
) -> Router {
// Auth required always; the paired-cert tests inject a fingerprint (cert branch wins), the
// rest authenticate via the `send`-attached default bearer.
let stats = state.stats.clone();
app(
state,
Some("test-secret".to_string()),
DEFAULT_PORT,
Some(np),
stats,
)
}
@@ -1474,6 +1680,7 @@ mod tests {
height: 1440,
fps: 120,
appid: 1,
peer_ip: None,
});
state.streaming.store(true, Ordering::SeqCst);
@@ -1580,7 +1787,9 @@ mod tests {
bind: "127.0.0.1:0".parse().unwrap(),
token: Some(" ".into()),
};
let err = run(test_state(), opts, None).await.unwrap_err();
let err = run(test_state(), opts, None, test_stats())
.await
.unwrap_err();
assert!(err.to_string().contains("no token"), "{err}");
}
@@ -1597,6 +1806,7 @@ mod tests {
height: 1080,
fps: 60,
appid: 1,
peer_ip: None,
});
let del = axum::http::Request::delete("/api/v1/session")
@@ -1663,14 +1873,14 @@ mod tests {
serde_json::json!([{}])
);
let checked_in = include_str!("../../../docs/api/openapi.json");
let checked_in = include_str!("../../../api/openapi.json");
// Compare content, not line-ending style: the generated `json` is LF (serde_json), but git
// may check the file out CRLF on Windows.
assert_eq!(
json.trim().replace('\r', ""),
checked_in.trim().replace('\r', ""),
"docs/api/openapi.json is stale — regenerate with: \
cargo run -p punktfunk-host -- openapi > docs/api/openapi.json"
"api/openapi.json is stale — regenerate with: \
cargo run -p punktfunk-host -- openapi > api/openapi.json"
);
}
+12 -17
View File
@@ -11,9 +11,6 @@
use anyhow::{Context, Result};
use rand::RngCore;
use std::fs;
use std::io::Write;
#[cfg(unix)]
use std::os::unix::fs::{OpenOptionsExt, PermissionsExt};
use std::path::Path;
const ENV_VAR: &str = "PUNKTFUNK_MGMT_TOKEN";
@@ -38,9 +35,10 @@ pub fn load_or_generate() -> Result<String> {
rand::thread_rng().fill_bytes(&mut buf);
let token = hex::encode(buf);
let dir = crate::gamestream::config_dir();
fs::create_dir_all(&dir).with_context(|| format!("create {}", dir.display()))?;
// Owner-private dir (0700 Unix / DACL-locked Windows) so the token can't leak via the config path.
crate::gamestream::create_private_dir(&dir).with_context(|| format!("create {}", dir.display()))?;
write_token(&path, &token)?;
tracing::info!(path = %path.display(), "generated and persisted management API token (0600)");
tracing::info!(path = %path.display(), "generated and persisted management API token (owner-only)");
Ok(token)
}
@@ -55,19 +53,15 @@ fn parse_token(contents: &str) -> Option<String> {
(!tok.is_empty()).then(|| tok.to_string())
}
/// Write `PUNKTFUNK_MGMT_TOKEN=<token>` to `path`, mode 0600 (never briefly world-readable).
/// Write `PUNKTFUNK_MGMT_TOKEN=<token>` to `path` as an owner-only secret — 0600 on Unix AND
/// DACL-locked to SYSTEM/Administrators on Windows. Routes through the shared `write_secret_file` so
/// the mgmt bearer token (full admin authority) gets the SAME Windows lockdown as the host key; the
/// bespoke `cfg(unix)`-only writer used to leave it readable by any local user (security-review
/// 2026-06-28 #2).
fn write_token(path: &Path, token: &str) -> Result<()> {
let mut opts = fs::OpenOptions::new();
opts.write(true).create(true).truncate(true);
#[cfg(unix)]
opts.mode(0o600);
let mut f = opts
.open(path)
.with_context(|| format!("write {}", path.display()))?;
writeln!(f, "PUNKTFUNK_MGMT_TOKEN={token}")?;
#[cfg(unix)]
let _ = fs::set_permissions(path, fs::Permissions::from_mode(0o600));
Ok(())
let line = format!("PUNKTFUNK_MGMT_TOKEN={token}\n");
crate::gamestream::write_secret_file(path, line.as_bytes())
.with_context(|| format!("write {}", path.display()))
}
#[cfg(test)]
@@ -95,6 +89,7 @@ mod tests {
assert_eq!(parse_token(&read).as_deref(), Some("cafef00d"));
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let mode = fs::metadata(&path).unwrap().permissions().mode() & 0o777;
assert_eq!(mode, 0o600);
}
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More