Compare commits

...

62 Commits

Author SHA1 Message Date
enricobuehler 78d018ae2f fix(apple/clipboard): announce + serve image/png from ANY pasteboard image type
apple / swift (push) Successful in 1m20s
ci / web (push) Successful in 49s
ci / docs-site (push) Successful in 1m4s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 5m43s
release / apple (push) Successful in 5m48s
docker / deploy-docs (push) Successful in 26s
apple / screenshots (push) Successful in 6m26s
android / android (push) Successful in 13m58s
arch / build-publish (push) Successful in 14m37s
ci / rust (push) Failing after 15m36s
deb / build-publish (push) Successful in 14m40s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m24s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m45s
macOS image copies rarely carry public.png — screenshots/Preview put TIFF on
the pasteboard, browsers add WebP/AVIF/GIF (observed live: TIFF+RTFD+WebP+AVIF+
8BPS+GIF, no PNG) — so the literal .png announce never fired and images
silently didn't sync. Announce image/png whenever a convertible image is
present (TIFF/HEIC alongside the native PNG check) and convert at serve time
via NSImage -> NSBitmapImageRep PNG (lazy, per design §3.5 — bytes still cross
only on a host paste).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 17:43:00 +02:00
enricobuehler 1b58130e68 merge(release): reconcile the 0.13.0 bump with today's landings
windows-drivers / probe-and-proto (push) Successful in 34s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m9s
decky / build-publish (push) Successful in 20s
apple / swift (push) Successful in 1m16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
windows-drivers / driver-build (push) Successful in 1m42s
ci / bench (push) Successful in 5m36s
release / apple (push) Successful in 5m36s
flatpak / build-publish (push) Successful in 7m1s
docker / deploy-docs (push) Successful in 24s
apple / screenshots (push) Successful in 5m41s
ci / rust (push) Failing after 14m16s
arch / build-publish (push) Successful in 14m37s
deb / build-publish (push) Successful in 16m50s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m6s
android / android (push) Successful in 17m4s
windows-host / package (push) Successful in 15m23s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m25s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m10s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m2s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m4s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m57s
Brings origin's release prep (widgets App Store profile mapping + workspace
version 0.13.0) into the local line that landed the Linux stream sink, the
Windows client-only audio rework, the July security-audit fixes, cross-client
render-scale, and the widgets scheme. release.yml was byte-identical on both
sides; no other file overlap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 17:26:59 +02:00
enricobuehler 12d4b025f7 ci(release): widgets-extension App Store profile mapping + shared Xcode scheme
Mirrors origin's 15233a68 (byte-identical release.yml, so it dedupes at the
next reconcile) and adds the shared PunktfunkWidgetsExtension scheme the
archive step builds with.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 17:20:15 +02:00
enricobuehler 871ebb31ce feat(clients): render-scale setting on every client — shared punktfunk_core::render_scale
Client-side supersampling/downscaling: the client asks the host to render
and encode at chosen-resolution × scale (the host does no scaling) and the
presenter rescales the decoded frame to the display. >1 supersamples for
sharpness; <1 lightens the host GPU and the link. Default 1.0 = Native, the
prior behavior.

The geometry lives once in punktfunk_core::render_scale (multiply, preserve
aspect ratio, floor to even, clamp to the codec's per-axis ceiling — 4096
for H.264, 8192 otherwise), the Rust twin of the Apple client's
RenderScale.swift, consumed by the native session client, the presenter's
match-window path, the Windows/Linux settings UIs, Decky, and Android
(settings + host connect + unit test).

Implemented and platform-verified by the Apple-client-features session
(Linux+Android+Apple green there); the punktfunk-core wiring
(pub mod render_scale) is restored here after being lost in a working-tree
reconciliation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 17:20:15 +02:00
enricobuehler 600693914f fix(security): land the 2026-07 audit fixes — SSRF guards, roster lane, SYSTEM path hygiene
The low/medium findings from the July host+Windows security review, as
implemented in the audit session's working tree:

- Webhooks and library-art fetches refuse loopback/link-local/metadata
  targets and no longer follow redirects (SSRF pivots from the privileged
  host process).
- The paired-client rosters (/clients, /native/clients) move off the
  streaming-client auth lane — one paired device could enumerate every other
  device's name + fingerprint; only the bearer/loopback console keeps them.
- Device-name sanitizing extends to bidi/format control characters
  (spoofable rendering) via the shared native_pairing::is_spoofy_char;
  stream-marker quoting uses the same set.
- The SYSTEM service resolves powershell by its full System32 path —
  CreateProcess checks the launching EXE's own directory first, so a planted
  powershell.exe beside the host binary would have run as SYSTEM.
- The pf-vdisplay driver's opt-in file log moves from world-writable
  C:\Users\Public to WUDFHost's own temp dir.
- GameStream pairing sessions are single-use (removed whatever the outcome).
- Uninstall also removes the pf_mouse driver-store entry (rider from the
  virtual-HID-mouse work).
- openapi.json regenerated (hardened-config-dir doc wording).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 17:20:15 +02:00
enricobuehler 4d89dcd3d7 fix(audio/windows): explicit-endpoint capture, client-only playback, self-healing watchdog
Root cause of the field report (Android client, Windows host: no audio until
the user manually cycled Sound-output devices, then audio on BOTH PC and
phone): the WASAPI loopback captured whatever the default render endpoint
was at open time — not the wiring plan's chosen endpoint — the plan's
IPolicyConfig default-set is warn-only and racy, and nothing reacted to
mid-stream default-device changes.

- Explicit-endpoint capture: the capture thread opens the plan's
  loopback_render by id, never "the default" (KEEP_DEFAULT preserves the old
  default-capturing behavior with the echo guard).
- Client-only playback default: wiring_plan::plan(..., host_audio) prefers a
  silent sink (Steam Streaming Microphone render side — loopback-validated,
  silent on host) over real hardware, so stream audio plays on the client
  only; PUNKTFUNK_HOST_AUDIO=1 restores real-hw-first (audible on the host).
  The capture side auto-installs the Steam pair once per process when no
  silent sink exists; open() handshake timeout 3s -> 30s to cover it, and a
  handshake timeout now stops the detached thread (it used to run for the
  process lifetime with the default still parked).
- Self-healing capture thread (wasapi_cap): outer capture_once loop
  (Assert|Follow) with a ~1s watchdog on the default render id. A user
  switch to a capturable endpoint is followed (their choice wins, audio on
  both); a switch to a dud (cable/SSS/mic target) re-asserts the plan;
  IPolicyConfig-denied converges to Follow instead of churning. Device
  errors reopen with 2s backoff; only the FIRST open failure is fatal.
  Zero-packets breadcrumb after 30s distinguishes broken-loopback from
  quiet-desktop.
- Park/restore of the default playback device (audio_control):
  wire_now(set_playback) parks the default on the loopback sink only for the
  capture's lifetime (the mic pump passes false — it runs while the host is
  idle); crash marker audio-default.prev + recover_orphaned_default() at
  first wire; restore is skipped if the operator changed the default
  themselves. A mic-default hygiene pass keeps VB-Cable installs audible and
  never records the mic target as the restore target.
- Session-end park_audio_capture(): Windows DROPS the capturer (thread join
  restores the default) instead of caching it; Linux keeps the parked
  PipeWire thread. Composes with the stream-sink idle() hook at all three
  park sites (idle is a no-op on Windows).

Verified: Linux (.21) clippy -D warnings + 176 punktfunk-host tests green
(incl. the new wiring-plan preference tests); Windows (.173) clippy with
nvenc,amf-qsv --all-targets -D warnings green at this exact tree.
On-glass winbox/Android validation still owed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 17:05:02 +02:00
enricobuehler 97c5778a36 merge(audio): land feat/linux-stream-sink — Linux host-owned stream sink
Merges 44b71e74 (see its message for the full design: the "Punktfunk Stream
Speaker" Audio/Sink claimed as the session's default output — the crackle
root fix for hardware-sink churn — plus the core-error liveness fix and true
5.1/7.1 capture) across the W1–W8 refactor that landed since the branch
forked:

- punktfunk1.rs was split on main: the audio_thread idle()/drain() hunks are
  hand-ported into native/audio.rs (same three sites: reuse-drain comment,
  encoder-fail park, end-of-thread park).
- pwinit moved to pf-capture (W6.2): the branch's crate::pwinit calls in
  pw_thread and stream_sink.rs become pf_capture::pwinit.
- Kept main's log-tier demotions (stream-state + setup lines at debug)
  inside the restructured pw_thread.

Resolution verified on Linux (home-worker-5): clippy -D warnings clean +
173 punktfunk-host tests green at the 691c064a-based equivalent ba5973a2;
re-verified at this base before landing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 16:35:02 +02:00
enricobuehler 22b352c1da chore(release): bump workspace version to 0.13.0
apple / swift (push) Successful in 1m19s
audit / bun-audit (push) Successful in 13s
audit / cargo-audit (push) Successful in 2m8s
ci / web (push) Successful in 53s
release / apple (push) Successful in 5m45s
ci / docs-site (push) Successful in 1m3s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
ci / bench (push) Successful in 5m26s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
apple / screenshots (push) Successful in 6m27s
windows-host / package (push) Successful in 15m6s
arch / build-publish (push) Successful in 10m50s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m1s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m58s
android / android (push) Successful in 16m0s
docker / deploy-docs (push) Successful in 11s
flatpak / build-publish (push) Failing after 8m9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m11s
deb / build-publish (push) Successful in 13m4s
ci / rust (push) Successful in 19m37s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m6s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m10s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m26s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m51s
The eight W6 leaf crates hardcoded 0.12.0 instead of inheriting the
workspace version — switched to version.workspace = true so the next bump
is one line again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 16:32:35 +02:00
enricobuehler 15233a68cf ci(release): map the widgets extension to its own App Store provisioning profile
The .ipa now embeds PunktfunkWidgetsExtension (io.unom.punktfunk.widgets), a
second distribution artifact, so the manual-signing exportArchive must map its
App ID to its own App Store profile — without it the export fails with "no
profile for io.unom.punktfunk.widgets". Requires the "Punktfunk iOS Widgets
App Store Distribution" profile on the runner.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 16:32:35 +02:00
enricobuehler 2a637eaf3f Merge origin/main: shared clipboard Phase 1 + first-frame-latency v4 into the local SDK line
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
Brings the day's landed work — clipboard (pf-clipboard + macOS client, ABI v8),
the perf/first-frame-latency merge (driver proto v4, Welcome-time display prep,
in-place resize), and the W7/W8 reconciliation — into the local main that had
diverged with the SDK-publish commits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:29:10 +02:00
enricobuehler 6c976e9dc5 fix(apple): land the RenderScale/DefaultsKeys definitions the client refactor references
apple / swift (push) Successful in 1m14s
ci / web (push) Successful in 1m42s
ci / docs-site (push) Successful in 2m0s
release / apple (push) Successful in 5m32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
apple / screenshots (push) Successful in 6m33s
ci / bench (push) Successful in 8m15s
decky / build-publish (push) Successful in 29s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 54s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
android / android (push) Successful in 14m14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
arch / build-publish (push) Successful in 18m11s
deb / build-publish (push) Successful in 18m29s
docker / deploy-docs (push) Successful in 28s
ci / rust (push) Successful in 28m37s
The W7/W8 client reconciliation committed the CONSUMERS of the render-scale
setting and the toggle-fullscreen notification (ContentView/FullscreenController
et al.) while their definitions were still uncommitted working-tree state from
the Apple-features effort — apple.yml red on main (run 10657). Lands the two
definition files (PunktfunkShared/RenderScale.swift + the DefaultsKeys additions)
so main builds; the rest of that effort's WIP stays in its working tree.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:20:39 +02:00
enricobuehler c8ee4b9902 fix(pf-vdisplay,pf-capture,pf-win-display): pre-split paths in the auto-merged v4 code
windows-drivers / probe-and-proto (push) Successful in 31s
ci / web (push) Successful in 43s
apple / swift (push) Failing after 1m1s
ci / docs-site (push) Successful in 1m0s
apple / screenshots (push) Has been skipped
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
windows-drivers / driver-build (push) Successful in 1m38s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m1s
ci / bench (push) Successful in 5m34s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5m53s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
windows-host / package (push) Successful in 15m6s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m48s
docker / deploy-docs (push) Successful in 23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m28s
The rename-followed perf hunks still said crate::win_display:: (the pre-W6
layout) — point them at pf_win_display::win_display:: and widen the four
helpers they call cross-crate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:13:15 +02:00
enricobuehler 1197415216 fix(pf-vdisplay,pf-win-display): v4 trait surface on the extracted driver.rs + cross-crate visibility
The W-refactor extracted VdisplayDriver into manager/driver.rs (the merge
resolution assumed it deleted) — carry the v4 changes there: open() returns the
driver's protocol version, update_modes() default-errs to the re-arrival
fallback. wait_target_departed goes pub for the manager's cross-crate call.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:11:34 +02:00
enricobuehler 18a5d93ae3 fix(host): allow too_many_arguments on the two fns the v4 merge grew
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:09:57 +02:00
enricobuehler 09849906e9 Merge perf/first-frame-latency: driver proto v4 + first-frame/resize latency (P0-P2)
Brings the first-frame-latency branch (P0.1 transition tracing, P1.1/P1.2
Welcome-time display prep, P2 in-place resize; pf-driver-proto v3 -> v4 with
IOCTL_UPDATE_MODES) onto current main. The branch predates the W6.2/W7 splits,
so git's rename detection carried most of it into the moved crates
(pf-capture idd_push, pf-vdisplay manager/pf_vdisplay, pf-win-display,
pf-driver-proto, the driver workspace) and the punktfunk1.rs remainder was
re-homed by hand:

- native/handshake.rs: welcome/start trace marks + the Welcome-time display
  prep spawn (the prep thread BECOMES the stream thread; hand-off via a
  SyncSender<SessionContext>). negotiate() gains bringup/quit/stop and returns
  the PrepHandle.
- native.rs: bringup/resize_ms creation + the stop/quit flags hoisted BEFORE
  the handshake (the close watcher splits: flags pre-handshake, lifecycle
  events post-handshake where `hello` exists); punch_done stamp; the data
  plane adopts the prep thread's result or builds inline.
- native/stream.rs: SessionContext/SendStats carry the trace; send_loop
  finishes it on the first video packet; the resize path gains the in-place
  fast path (try_inplace_resize) with the full rebuild as fallback, restructured
  so both share the post-rebuild bookkeeping; prepare_display/PreparedDisplay/
  PrepHandle; build_pipeline(+retry) thread the stage marks.
- session_status/mgmt: ttff_ms + last_resize_ms per session (union with the
  lifecycle-events fields main added to the same spots).
- pf-capture: Capturer gains capture_target_id() + resize_output() defaults.
- pf-vdisplay manager: perf's faster activation poll (60x50ms) + the settle
  floor before the PnP sweep, on main's knobs/no-trait shape.

Also: packaging/windows/build-gamepad-drivers.ps1 is ASCII again (an em-dash
from the pf-mouse work tripped windows-host.yml's locale-safety gate on main).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:08:16 +02:00
enricobuehler 691c064a37 build(sdk): publish @punktfunk/host to the Gitea npm registry
- package.json: drop private; point main/types/exports/bin at a tsc-built dist/;
  add publishConfig (unom/npm registry), files, repo metadata, and the
  MIT OR Apache-2.0 license; effect becomes a peerDependency (shared instance).
- tsconfig.build.json: emit dist/ JS + .d.ts (bun shebang preserved on the bin).
- .npmrc: map the @punktfunk scope to the registry (no token committed).
- sdk-publish.yml: publish on sdk-v* tags or manual dispatch, reusing the
  REGISTRY_TOKEN secret; typecheck/test/build/tag-matches-version gate.
- README: Install section for consumers.

Verified: build green, frozen lockfile stable, bun publish --dry-run packs
@punktfunk/host@0.1.0 (dist + README only) to the unom registry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:54:36 +02:00
enricobuehler 34519566ba feat(sdk): VirtualHere DualSense passthrough example + automation recipe
Add sdk/examples/virtualhere-dualsense.ts — bind a real USB DualSense (shared
from the couch via VirtualHere USB-over-IP) to the host for the length of each
connection and release it after, for full gyro/touchpad/adaptive-trigger/USB-
rumble passthrough instead of the emulated pad. Brackets on client.connected/
disconnected and releases the pad on SIGTERM for a clean runner stop.

Document it in the Events & hooks page with a zero-code hooks.json variant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:54:36 +02:00
enricobuehler 86d9f49473 style(pf-vdisplay): rustfmt under the pinned 1.96.0 toolchain
audit / bun-audit (push) Successful in 12s
windows-drivers / probe-and-proto (push) Successful in 31s
ci / web (push) Successful in 48s
apple / swift (push) Failing after 1m1s
ci / docs-site (push) Successful in 1m2s
decky / build-publish (push) Successful in 21s
windows-drivers / driver-build (push) Successful in 1m35s
windows-host / package (push) Failing after 16s
audit / cargo-audit (push) Successful in 2m46s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
release / apple (push) Successful in 3m56s
apple / screenshots (push) Has been skipped
ci / bench (push) Successful in 6m26s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m16s
flatpak / build-publish (push) Failing after 8m9s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 13m9s
android / android (push) Successful in 13m11s
deb / build-publish (push) Successful in 14m12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m8s
ci / rust (push) Successful in 19m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m10s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m1s
The W6.2 pf-vdisplay extraction (27a5d8da) committed six spots that the pinned
rustfmt (rust-toolchain.toml = 1.96.0) reformats — a short closure collapse and
chain-call wrapping. No semantic change; makes cargo fmt --all --check green so
the combined push does not land CI red on pre-existing format drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:38:12 +02:00
enricobuehler 2064c0780c merge(core): reconcile the W7/W8 client refactor with origin's shared-clipboard feature
origin/main landed the shared clipboard (design/clipboard-and-file-transfer.md) while
this branch split quic/msgs.rs -> quic/{caps,control,...} and client.rs ->
client/{mod,control,worker,pump,planes,...} (W7) and deleted the two monoliths. The
feature had modified both deleted files, so its delta is re-applied onto the split
instead of resurrecting the monoliths:

  - HOST_CAP_CLIPBOARD                         -> quic/caps.rs
  - MSG_CLIP_* / CLIP_* consts, the six Clip*
    structs, and their encode/decode impls     -> quic/control.rs (beside the clock codecs)
  - CtrlRequest::{ClipControl,ClipOffer} +
    Negotiated.host_caps                        -> client/control.rs
  - WorkerArgs.{clip_event_tx,clip_cmd_rx}      -> client/worker.rs
  - CLIP_EVENT_QUEUE                            -> client/planes.rs
  - NativeClient clip fields, the 7 clip_* /
    host_caps / next_clip methods, connect()
    channel wiring                              -> client/mod.rs
  - the control-task encode/decode arms and
    the clipboard-task spawn                     -> client/pump.rs

Cargo.lock reconciled (adds pf-clipboard), punktfunk-host/Cargo.toml unions the W6
pf-* subsystem deps with pf-clipboard, and include/punktfunk_core.h is the cbindgen
union (clipboard + rumble C-ABI). punktfunk-core builds --all-features and its 174
lib tests pass, including quic::tests::clip_loopback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:38:11 +02:00
enricobuehler f439b69451 refactor(android/W8): split decode.rs into decode/ directory module
Break the 1747-line clients/android/native/src/decode.rs into a decode/ directory
module (mod.rs + 5 concern submodules):
  - decode/setup.rs      : codec creation + low-latency config + thread/frame-rate
                           tuning + HDR static-info encode
  - decode/display.rs    : DisplayTracker + render-callback registration + HDR dataspace
  - decode/latency.rs    : realtime clock + decoded-pts / user-flags stat recording
  - decode/sync_loop.rs  : the synchronous poll decode loop (+ feed/drain) — moved WHOLE
  - decode/async_loop.rs : the event-driven async decode loop (+ helpers) — moved WHOLE
decode/mod.rs keeps the consts, DecodeOptions, and the `run` entry point + the
`codec_mime`/`codec_label` re-export, so every crate::decode::X path stays byte-stable.
The module has no decoder struct (free functions + small types), so both decode loops
move byte-for-byte and their separately-inlined received-stat recording is NOT unified.
16 helper fns/types became pub(super) for sibling access; zero field bumps. lib.rs
unchanged (`#[cfg(target_os="android")] mod decode;` resolves to decode/mod.rs).

Verified: cargo-ndk check (aarch64-linux-android, clean) + the gradle cargoNdkDebug
build (arm64-v8a / armeabi-v7a / x86_64). On-device runtime re-verification still owed
per the plan (the two decode loops are a hot path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:06:24 +02:00
enricobuehler 1eef55016d refactor(apple/W8): extract FullscreenController + ApprovalRequest from ContentView
Move the macOS `FullscreenController` (NSViewRepresentable that drives native
fullscreen) into its own FullscreenController.swift, and `ApprovalRequest` (the
pending-trust-decision value type) into ApprovalRequest.swift, out of the
1041-line ContentView.swift. Both were file-`private`; dropped to internal so
ContentView (same module) still references them across files. StreamView*.swift
and the connection body are untouched. Pure move; no behavior change.

Verified: `swift build` (macOS) — Build complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:02:34 +02:00
enricobuehler 61118cbdd4 refactor(presenter/W8): split vk.rs into vk/ directory module
Break the 2513-line pf-presenter/src/vk.rs into a vk/ directory module (mod.rs +
6 concern submodules), keeping ALL type definitions in vk/mod.rs so every submodule
(a descendant of vk) sees the Presenter/OverlayPipe/etc. private fields with no
field bumps:
  - vk/setup.rs      : Presenter::new + device/format/present-mode selection
  - vk/present.rs    : the per-frame present path (present + CSC record + AVVkFrame
                       sync) — HOT PATH, moved whole
  - vk/reconfig.rs   : swapchain recreate/resize + HDR reconfiguration
  - vk/resources.rs  : video-image/staging (re)build + Retired-frame destruction
  - vk/overlay_pipe.rs: the presenter-side overlay composite pipeline
  - vk/gpu.rs        : memory allocation, image barriers, geometry helpers (+ tests)
vk/mod.rs keeps FrameInput/Presenter/OverlayPipe/VideoImage/Staging/Retired/HwCtx*
+ the public accessors + Drop. Methods/free-fns a sibling submodule calls became
pub(super) (~18); zero field bumps, zero re-exports (Presenter/FrameInput never
leave mod.rs). lib.rs unchanged (`pub mod vk;` resolves to vk/mod.rs). The moved
overlay shader include_bytes! gained one `../` for the deeper dir. Pure move; no
behavior change; the hot present path keeps only static pub(super) calls (inlinable).

Verified both platforms: Linux (home-worker-5) clippy -p pf-presenter
(--all-targets -D warnings) + test; Windows (winbox, ASCII CARGO_HOME) clippy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:39:36 +02:00
enricobuehler 22a61e0b48 refactor(console-ui/W8): split shell.rs into shell/ facade + render/overlays/tests
Break the 1212-line pf-console-ui/src/shell.rs into a facade + shell/ subdir
(shell.rs stays the parent; `mod render;` resolves to shell/render.rs):
  - shell/render.rs   : the per-frame screen compose/transition path (Shell::render
                        + the LayerEnv paint helper)
  - shell/overlays.rs : the modal overlays (Shell::draw_overlays + draw_takeover)
  - shell/tests.rs    : the inline #[cfg(test)] module, extracted verbatim
The Shell struct + its public API + draw_aurora() stay in shell.rs (both children
reach draw_aurora + the private fields as descendants — no bumps). Sole visibility
change: draw_overlays -> pub(in crate::shell) (its caller Shell::render is now a
sibling). Zero re-exports needed (Shell/ConsoleOptions never leave the root). Pure
move; no behavior change.

Verified both platforms: Linux (home-worker-5) clippy -p pf-console-ui
(--all-targets -D warnings) + test; Windows (winbox, ASCII CARGO_HOME) clippy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:29:07 +02:00
enricobuehler 13b1f36d4a feat(core,clients): one rumble policy engine for every platform (rumble root fix D)
punktfunk-core client/rumble.rs: a per-connection policy engine consumes seq-gated wire
updates and emits EFFECTIVE actuator commands — re-emits on renewals (duration APIs stay
re-armed), self-silences at the v2 lease, a UNIFORM 1 s legacy-host staleness replacing the
per-platform zoo (Apple 1.6 s / Android 60 s / SDL 1.5 s / Deck 1 s), quirk-declared
actuator keepalives (Deck 40 ms + LSB dedupe-defeat jitter), and one stop per buzzing pad
on connection close. Per-pad mailbox semantics: a stalled embedder wakes to ONE current
command, and a stop can structurally never be the update an overflowing queue drops.

New API/ABI: NativeClient::{next_rumble_command,set_rumble_quirks} +
punktfunk_connection_next_rumble_cmd/_set_rumble_quirks (next_rumble/next_rumble2 stay for
un-migrated embedders; both consumers are fed). Migrations DELETE the platform forks:
pf-client-core loses RumbleState + the Deck keepalive loop + LEGACY_RUMBLE_CEILING_MS and
physically silences a slot at close; Android loses the 60 s legacy one-shot (backstop
repack, cancel-on-zero); Apple loses envelopeDeadline + sessionStaleSeconds + both tick
watchdogs (CoreHaptics realization untouched; mac xcframework rebuilt locally).

design/rumble-root-fix.md par. D. Engine 10/10 unit tests; core tests 176 Linux / 175
Windows + clippy -D warnings; swift build + RumbleTuningTests; Kotlin + android-native
compile green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 14:07:32 +02:00
enricobuehler 9e6fc6e071 fix(host/inject,drivers): rumble root fixes A-C — lossless report ring + rumble-keyed idle watchdogs
B: PadFeedback.game_drove -> rumble_drove, keyed on vibration-asserting reports — an
LED/adaptive-trigger stream can no longer feed the abandoned-rumble force-off while a
coalesced stop never re-asserts (the confirmed unbounded stuck-ON path). C: Linux parity —
every UHID backend now arms the shared watchdog (Steam Input drives these pads over hidraw
with Windows abandonment semantics) and the uinput mixer force-stops abandoned
infinite-replay FF effects (FfState, unit-tested). Shared PUNKTFUNK_RUMBLE_IDLE_MS hatch
(0 = off; non-zero floored above SDL's ~2 s rumble resend).

A: PadShm v2.1 — a 1024 B tail extension carrying an 8-slot lossless output-report ring,
feature-negotiated via zeroed reserved fields (out_ring_ver; deliberately NO
GAMEPAD_PROTO_VERSION bump — mixed generations degrade to the legacy latest-report slot
instead of failing closed). The pf-dualsense driver dual-writes both planes
(publish_output); the host's shared OutputDrain drains oldest->newest with a torn-read
recheck and an overflow->resync path (PadFeedback.resync force-stops + re-arms dedups).
pf-umdf-util grows a min_data_size map fallback. Ds*Feedback.fresh removed (dead).

design/rumble-root-fix.md par. A-C. Verified: pf-inject tests+clippy Linux+Windows (53/53
on winbox incl. the stop-coalesce repro); drivers ws check+clippy on the CI runner.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 14:07:32 +02:00
enricobuehler 570ff504ad refactor(client-core/W8): split video.rs into flat decoder-backend siblings
Break the 1974-line pf-client-core/src/video.rs into flat sibling modules
(matching the crate's video_d3d11.rs / video_pyrowave.rs convention), leaving
video.rs as the contract + Decoder dispatch facade:
  - video_color.rs   : ColorDesc + csc_rows (the Y'CbCr->RGB matrix)
  - video_software.rs : the libavcodec/swscale SoftwareDecoder
  - video_vaapi.rs   : the Linux-only VAAPI/DRM-PRIME backend (mod is cfg(linux))
  - video_vulkan.rs  : the FFmpeg Vulkan Video backend
Every crate::video::X / video::X path stays byte-stable (ColorDesc + csc_rows
re-exported from video.rs; frame POD, VulkanDecodeDevice, QueueLock, Decoder,
decodable_codecs*, ffmpeg_codec_id, fourcc/drm_fourcc_for all stay in video.rs).
Code-driven placements: averr, AVERROR_EAGAIN, frame_is_keyframe stay in video.rs
(shared by all three decoders); DrmFrameGuard's field + drm_fourcc_for +
Software/Vaapi/VulkanDecoder ctors/decode became pub(crate) (sibling access);
the test module split three ways (software tests need private decoder internals).
Pure move; no behavior change.

Verified on Linux (home-worker-5): cargo clippy -p pf-client-core (default
[pyrowave] + --no-default-features, --all-targets -D warnings) + cargo test.
Windows verify BLOCKED environmentally: pf-client-core -> sdl3 build-from-source
-> CMake/CL.exe fails on winbox's non-ASCII home path (fails the baseline too,
independent of this split); the split's Windows surface (facade cfg(windows) bits
+ video_d3d11) is verbatim-preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:06:57 +02:00
enricobuehler e8b64ffe43 Merge branch 'feat/shared-clipboard-v2': shared clipboard Phase 1 (wire + pf-clipboard host backends + macOS client)
audit / bun-audit (push) Successful in 13s
apple / swift (push) Successful in 1m19s
ci / web (push) Successful in 1m28s
audit / cargo-audit (push) Successful in 2m18s
ci / docs-site (push) Successful in 1m43s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
decky / build-publish (push) Successful in 38s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 1m10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m15s
ci / bench (push) Successful in 5m25s
release / apple (push) Successful in 6m4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m46s
arch / build-publish (push) Successful in 11m22s
docker / deploy-docs (push) Successful in 31s
apple / screenshots (push) Successful in 4m47s
flatpak / build-publish (push) Failing after 8m1s
windows-host / package (push) Successful in 15m58s
android / android (push) Successful in 16m40s
deb / build-publish (push) Successful in 15m58s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m57s
ci / rust (push) Successful in 23m13s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m53s
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m26s
2026-07-17 11:43:52 +00:00
enricobuehler ffa63a74f2 refactor(core/W7): split client.rs into client/ facade + submodules
Turn the 2674-line client.rs into a client/ directory module (mod.rs facade +
8 submodules) behind glob/`use self::` re-exports, so crate::client::X paths
(NativeClient, ProbeOutcome, AudioPacket, display_hdr_env_override) stay
byte-stable. Leaf lifts: frame_channel.rs (the FIFO hand-off + jump-to-live
consts + DecodeLatAcc), recovery.rs (RfiRecovery loss-range detector),
probe.rs (ProbeState/ProbeOutcome), planes.rs (side-plane queues + AudioPacket),
control.rs (CtrlRequest/Negotiated), worker.rs (WorkerArgs + reject_from_close),
pairing.rs (NativeClient::pair). The per-frame pump moves WHOLE as a plain
`pub(super) async fn run_pump` (was worker_main) — the only edit is the
signature line: no trait object, no Box, no per-frame allocation or indirection.
NativeClient + its public impl + Drop + the cfg-gated thread-pin/hot-tid helpers
stay in the facade. Visibility bumps are pub(crate) (struct + each field for
WorkerArgs/Negotiated/ProbeState; FrameChannel + each method); reject_from_close
is pub(crate) (sibling access). No behavior change.

Verified: Linux clippy (quic + no-default, -D warnings) + full cargo test;
Windows clippy (both) + test --lib; macOS clippy (apple thread-pin variant) +
165 lib tests. On-glass jump-to-live + ABR smoke still owed (pump is a pure
relocation, so this is a formality) per the plan's pump gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:29:16 +02:00
enricobuehler 716875dd09 refactor(core/W7): split quic/msgs.rs into handshake/caps/control/pairing
Break the 1302-line quic/msgs.rs into four flat sibling modules behind the
quic facade's glob re-exports, so every crate::quic::X path stays byte-stable:
handshake.rs (Hello/Welcome/Start + codecs), caps.rs (video-cap bits, codec &
chroma negotiation, ColorInfo), control.rs (typed CTL_MAGIC messages + frame),
pairing.rs (SPAKE2 ceremony messages). msgs.rs is deleted; quic/mod.rs gains the
four `mod`/`pub use` lines and the `pub use crate::reject::*` hoist (moved up from
msgs.rs). Pure move; no wire-format or behavior change. Private helpers
(truncate_to, put_bytes, get_bytes) stay with their sole callers; no visibility
changes.

Verified both platforms from clean HEAD snapshots: Linux clippy (quic +
no-default, -D warnings) + full cargo test (157 lib + integration); Windows
clippy (both) + test --lib (156).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:08:45 +02:00
enricobuehler ef736cb9d7 refactor(core/W7): split transport/udp.rs into udp/ facade + per-OS backends
Turn transport/udp.rs into a udp/ directory module: the cross-platform core
(UdpTransport, is_transient_io, spawn_data_punch, the Transport trait impl) stays
in mod.rs; the platform batched-I/O backends move to udp/{linux,windows,apple}.rs.
The trait impl is kept whole -- its per-OS send_batch/send_gso/recv_batch methods
become cfg-gated one-line delegators to pub(super) free fns that take &UdpTransport
(byte-identical bodies, self -> t). transport/mod.rs is unchanged (re-exports still
resolve; udp/mod.rs re-exports windows::send_uso_all). No behavior change.

Module gates: linux = any(linux, android) (Android uses sendmmsg/recvmmsg via its
bionic binding); windows = windows (USO); apple = all(unix, not(any(linux,android)))
(recvmsg_x on Darwin, recv-loop on BSD). GSO stays linux-only.

Verified on all four target families from clean HEAD snapshots: Linux clippy
(quic + no-default, -D warnings) + full test; Windows clippy (both) + test --lib
(156); macOS check (apple recvmsg_x path); aarch64-linux-android check (android_mmsg).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:01:53 +02:00
enricobuehler 93c8dc4712 refactor(core/W7): split packet.rs into packet/ facade + submodules
Turn the 1446-line packet.rs into a packet/ directory module (mod.rs facade
+ header/packetize/reassemble/tests) behind glob re-exports, so every
crate::packet::X path stays byte-stable. Pure move: the header consts +
PacketHeader -> header.rs; Packetizer -> packetize.rs; the Reassembler cluster
(kept WHOLE -- disjoint-borrow hot path) + loss-window consts -> reassemble.rs;
the inline #[cfg(test)] block -> tests.rs. Sole visibility change:
LOSS_WINDOW_NS -> pub(super) (a test imports it). No behavior change.

Verified on both platforms from a clean HEAD snapshot: Linux clippy
(--features quic and --no-default-features, --all-targets -D warnings) + full
cargo test; Windows clippy (both feature sets) + cargo test --lib (156 pass).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 12:48:48 +02:00
enricobuehler f012ebbcba feat(sdk): Effect v4 + @effect/openapi-generator; typed pf.api & example ladder
Drop Orval for the first-party @effect/openapi-generator (OpenAPI 3.1 ->
Effect Schema + a typed HttpClient client) and bump effect 3.19 ->
4.0.0-beta.98. Port the hand-written surfaces to the v4 API (Result over
Either, Context.Service, Codec, Literals/Union arrays, Stream/Schedule/
Effect renames). Transport (CA-pinning fetch) and the reconnecting SSE
source are kept intact.

Make the SDK approachable for non-Effect users:
- Add pf.api.* on the Promise facade: the generated client surfaced as
  typed, Promise-native methods (await pf.api.listPairedClients()), so REST
  calls are autocompleted and checked instead of stringly-typed
  pf.request(method, path, body) + `as` casts. Zero-drift veneer over
  make(httpClient), backed by the same pinning fetch. pf.request stays as
  the untyped escape hatch.
- Re-tier examples into a 1-4 complexity ladder, rewritten onto pf.api.*
  (the typed payloads caught a wrong `launch` shape in provider-sync);
  the Effect example is labelled advanced. Add examples/ to tsconfig so
  they are typechecked (stops rot).

typecheck + 19 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 12:40:04 +02:00
enricobuehler 27a5d8daac refactor(host/W6.2): extract virtual-display orchestration into the pf-vdisplay crate
vdisplay.rs + vdisplay/* (the per-compositor Linux backends — KWin zkde-screencast,
wlroots swaymsg, Mutter RemoteDesktop, Hyprland — and the Windows IddCx/pf-vdisplay
driver backend, behind one VirtualDisplay trait; the mode-conflict admission
registry, the display policy/identity/custom-preset state, and the session-env /
gamescope routing) move into crates/pf-vdisplay (plan §W6). The DDC/CI panel-power
control (used only here) and the KWin zkde protocol XML move with it. This
completes the host-crate decomposition: capture, encode, inject, and vdisplay are
now four subsystem crates over the shared leaves, and punktfunk-host is the
orchestrator (serve/supervisor + native + gamestream + mgmt).

Coupling breaks (all down-only, cargo-tree acyclic):
- capture::dxgi identity -> pf_frame::dxgi; win_display/monitor_devnode/
  console_session_mismatch -> pf-win-display leaf; can_open_another_session ->
  pf-encode (the NVENC session-budget admission gate — acyclic peer edge).
- The registry's DisplayCreated/DisplayReleased emits into the host SSE event bus
  invert to a leaf hook: pf-vdisplay emits a neutral DisplayEvent to a
  host-registered DISPLAY_EVENT_SINK, so it never reaches the orchestrator's
  events module.
- The IddCx driver module is renamed pf_vdisplay -> driver (its old name collided
  with the crate name through the host's `mod vdisplay` shim glob).

The host keeps `mod vdisplay { pub use pf_vdisplay::* }` so every crate::vdisplay::*
path (serve/mgmt/native/the capture FrameChannelSender seam) is unchanged; the
heavy deps (wayland/ashpd/tokio + the zkde protocol) moved with the crate.
Co-authored: a fail-closed IOCTL-reply-length security fix (reject short/zeroed
pf-vdisplay driver replies before trusting protocol_version/target_id/wudf_pid/luid,
security-review 2026-07-17) rides this commit in the moved driver module.

Verified: Linux clippy -D warnings (pf-vdisplay + host nvenc,vulkan-encode,pyrowave
--all-targets) + pf-vdisplay 63/63 + host 167/167 tests; Windows clippy -D warnings
(pf-vdisplay --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 12:14:08 +02:00
enricobuehler f6c6e4e594 refactor(host/W6.2): extract the input-injection backends into the pf-inject crate
inject.rs + inject/* (the per-OS injectors — wlroots virtual-input, KWin
fake_input, libei/reis, gamescope-EI on Linux; SendInput on Windows — plus the
virtual-gamepad HID stack: DualSense/DualShock4/Switch Pro/Steam Controller/Deck
over uhid/usbip and the Windows UMDF drivers, the proto codecs, the injector
service, and the uhid manager) move into crates/pf-inject behind the
InputInjector trait (plan §W6). It consumes punktfunk_core::input (the neutral
GamepadEvent/InputEvent vocabulary, moved to core in W5) + the pf-driver-proto
wire contract, and reaches pf-capture only for the Windows gamepad-channel
WUDFHost check + the resident-mouse compose-kick hook.

The one inject->vdisplay coupling (the libei gamescope-EI backend needs the EIS
relay socket path) is broken via a leaf: gamescope_ei_socket_file moves to
pf-paths as the shared contract — the gamescope producer (host vdisplay) keeps
its session-env-lock wrapper around it, the libei consumer (pf-inject) reads it
directly post-retarget. The host keeps a `mod inject { pub use pf_inject::* }`
shim so every crate::inject::* path (the native/gamestream input planes + devtest)
is unchanged; the heavy input deps (wayland/reis/xkbcommon/usbip + the KWin
fake-input protocol XML) moved with the crate.

Verified: Linux clippy -D warnings (pf-inject + host nvenc,vulkan-encode,pyrowave
--all-targets) + pf-inject 69/69 + host 230/230 tests; Windows clippy -D warnings
(pf-inject --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 11:52:02 +02:00
enricobuehler 0992548de7 feat(host/windows): HID compose kick — wake and dirty the display through the virtual mouse (lid-closed first-frame fix)
The final piece of the lid-closed field report (fixes 1-3: 3d9b3290;
prerequisites: 85dd2bb0 pf-mouse, 845a9760 leaf primitives, 94ca4041
pf-capture hook): the IDD first-frame gate can only pass if DWM composes
at least one frame, and a lid-closed/locked/idle machine is exactly the
state Windows has decided not to compose in. The SendInput compose kick
is conditional on this process's context — wrong session → wrong input
queue; secure desktop → blocked; display powered off → no wake. A report
from the resident pf-mouse HID device is REAL input to win32k:
session-independent, secure-desktop-proof, wakes a powered-off display,
counts as user presence. Parsec-class mechanism, and semantically honest
— a remote user starting a stream IS a user arriving at this machine.

- hid_kick(rect, bounds): newest-wins kick slot + condvar, serviced by
  the keeper thread that owns the ONE process-wide VirtualMouse (a
  second open() would squat the bootstrap mailbox). Not-ready (opt-out,
  driver missing, not yet attached) returns false → the capture crate
  falls back to SendInput.
- perform_kick: park the pointer at the target display's center, dwell
  35 ms (Stage-W3: DWM samples cursor position at the next vsync tick;
  the gaps also keep reports from coalescing in the driver's 8 ms
  timer), wiggle ~2 px, restore the saved position. Desktop→HID
  coordinates normalize against pf_win_display::desktop_bounds() (CCD
  union — correct from any session, unlike per-session GDI metrics).
- ensure_resident registers the hook (pf_capture::HID_COMPOSE_KICK) —
  the one-way-edge inversion: pf-capture never reaches into inject.
- keeper loop: condvar wait (250 ms tick) so a kick executes
  immediately, not at the next tick; publishes MOUSE_READY from
  driver_proto.

Paired with pf-frame's DisplayWakeRequest (held by the capturer from
before the first-frame gate): the power request keeps the display from
going dark mid-session, the HID kick wakes it when it already is.

Verified on winbox: combined-tree cargo check + clippy for
punktfunk-host, pf-capture, pf-frame, pf-win-display all EXIT 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 11:30:32 +02:00
enricobuehler 94ca4041ca refactor(host/W6.2): extract the frame-capture backends into the pf-capture crate
capture/linux (PipeWire portal) + capture/windows (IDD direct-push: dxgi
mechanics, idd_push + submodules, synthetic_nv12) + pwinit move into
crates/pf-capture behind the Capturer trait + synthetic sources (plan §W6).
The crate speaks pf-frame (CapturedFrame/PixelFormat + the DXGI identity),
pf-zerocopy (CUDA import), and the pf-win-display leaves, and NEVER pf-encode —
the capture->encode edge is one-way. This completes the deliberate capture/encode
crate split (the invasive path the plan had merged into one pf-media): capture
and encode are now separate subsystem crates sharing only pf-frame.

Four seams keep the capturer off the orchestrator:
- VirtualOutput is EXPLODED into primitives (remote_fd/node_id/preferred_mode/
  keepalive) by the host facade, so pf-capture never depends on the vdisplay type;
- FrameChannelSender: the sealed-channel delivery is a Send+Sync closure the host
  facade builds from the pf-vdisplay control device + send_frame_channel IOCTL and
  hands in; ChannelBroker holds the closure instead of the control HANDLE (the
  whole-desktop handle-duplication security boundary is byte-for-byte unchanged);
- console_session_mismatch + desktop_bounds live in pf-win-display (leaf peers);
- pwinit moves here (audio caller -> pf_capture::pwinit).

The host keeps capture.rs as a thin BRIDGE: it re-exports the vocabulary + capturer
types (every crate::capture::* path is unchanged) and keeps open_portal_monitor /
capture_virtual_output, which resolve the ZeroCopyPolicy + FrameChannelSender and
call into pf-capture. verify_is_wudfhost + install_gpu_pref_hook are re-exported
(the gamepad-channel bootstrap + the main.rs subcommand consume them).

Co-developed: the resident-HID-mouse compose-kick hook (HID_COMPOSE_KICK + the
HID-first cursor kick + _display_wake) rides this commit into pf-capture; the host
mouse_windows registration side lands separately on top.

Verified: Linux clippy -D warnings (pf-capture + host nvenc,vulkan-encode,pyrowave
--all-targets) + host tests 299/299; Windows clippy -D warnings (pf-capture
--all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 11:28:56 +02:00
enricobuehler 845a97601d feat(pf-frame,pf-win-display): leaf primitives for the lid-closed first-frame fix
Three leaf-crate additions the IDD-push capturer (pf-capture, plan §W6 C6)
builds on — committed ahead so the capture-crate extraction and the HID
compose kick can land on top:

- pf-frame session_tuning::DisplayWakeRequest — RAII PowerCreateRequest/
  PowerSetRequest(PowerRequestDisplayRequired + SystemRequired), the
  service-grade 'someone is watching this screen' assertion (visible in
  powercfg /requests), held for a capture session so the console cannot
  drop into display-off mid-stream. Object-lifetime, unlike the
  thread-bound ES_* flags in on_hot_thread. Prevention only: no power
  request turns an already-off display back on — that wake is input's
  job (the virtual-mouse compose kick).

- pf-win-display win_display::desktop_bounds() — the virtual-desktop
  bounds as the union of every ACTIVE CCD path's source rect. From the
  CCD database (global), NOT GetSystemMetrics (a per-session view), so
  a non-console-session host still aims HID absolute coordinates at the
  console's real layout.

- pf-win-display console_session_mismatch() — the session guard from
  3d9b3290, copied into the leaf so pf-capture reads it as a peer
  instead of reaching into the orchestrator (relocation authored by the
  W6 extraction session).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 11:17:29 +02:00
enricobuehler 85dd2bb077 feat(host/windows): resident virtual HID mouse (pf-mouse UMDF minidriver)
Headless Windows hosts (no dongle) stream an INVISIBLE cursor: with no
pointing device present win32k reports SM_MOUSEPRESENT=0 and DWM never
composites a pointer into the pf-vdisplay frame, even though SendInput
moves it. Keep ONE virtual HID mouse devnode alive for the host's
lifetime — the Sunshine/Parsec-class fix, zero client changes.

- pf-mouse: UMDF2 HID minidriver, one fixed identity (PF:MO 5046:4D4F,
  obviously virtual, nothing fingerprints it), one 8-byte input report
  (5 buttons + absolute 15-bit X/Y + wheel + AC-pan). Transport is the
  sealed pad channel verbatim (Global\pfmouse-boot-0 mailbox + unnamed
  MouseShm DATA section) so pf-umdf-util's audited layer serves it
  unchanged; report delivery is event-driven (idle = no HID traffic).
- host: inject::mouse_windows — VirtualMouse (SwDeviceCreate'd devnode +
  channel), ensure_resident() keeper thread started by every
  InjectorService (process-wide, PUNKTFUNK_NO_VIRTUAL_MOUSE opts out),
  vmouse-spike on-glass validation (cursor sweep via HID reports).
- proto: mouse module (magic, boot-name, identity, report layout,
  unit-tested input_report packing).
- SwDeviceProfile grows container_tag so the mouse's ContainerId family
  (PFMO) never groups with a pad's (PFDS) in the Devices UI.
- packaging: pf-mouse rides the gamepad-driver build + install pipeline
  (build-gamepad-drivers.ps1, windows-drivers.yml, driver install
  --gamepad picks up every staged .inf).

On-glass validated on winbox: devnode + HID child bind, SM_MOUSEPRESENT=1
with no physical mouse, cursor sweeps via HID reports (vmouse-spike).

This work was implemented in a parallel session; committed here as the
build prerequisite for the HID compose kick that follows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 11:02:10 +02:00
enricobuehler 3d9b329084 fix(host): name the lid-closed/no-frames failure — display-write decode, console-session guard, driver-truth attach diagnostics
Field report (Windows laptop, lid closed, Tailscale): v0.12.0's activation
fix works — the pf-vdisplay target activates in ~200ms — but the session
still dies at the first-frame gate: 'driver_status=1 but no frame published
within 4s'. Triage showed three independent blind spots; this names all of
them at their source instead of guessing downstream:

- pf-win-display: decode ChangeDisplaySettingsExW failures (-1 FAILED — a
  display write rejected, the wrong/remote-session signature — vs -2 BADMODE,
  which the old 'mode not advertised?' text conflated), and WARN on every
  non-zero SetDisplayConfig rc in the CCD isolate even when verification
  passes vacuously (the lid-closed case: nothing else active, so the INFO
  swallowed rc=0x5 ERROR_ACCESS_DENIED while the load-bearing COMMIT_MODES →
  ASSIGN_SWAPCHAIN re-commit silently never applied). Access-denied rcs get
  the remedy appended (console session / installed service).

- host: console-session guard (interactive::console_session_mismatch) — a
  host outside the active console session (a hand-launched host after an RDP
  round-trip) fails every display write, reads the wrong session's GDI view,
  and its SendInput compose kicks go nowhere. Named ERROR at vdisplay
  acquire + appended to the first-frame timeout, instead of the misleading
  generic failure. (The idd_push diagnosis half of this landed in 9a36ea21;
  this commit adds the proto helpers + session guard it references, healing
  the windows-cfg build.)

- proto + driver: while OPENED, driver_status_detail now carries a live
  packed word (bit31 live-marker | offered 15-bit | mismatch-dropped 16-bit)
  maintained by the publisher, so the host's first-frame timeout can tell
  apart: never-attached (no swap-chain worker ran), attached-but-DWM-composed-
  zero-frames (undamaged/powered-off desktop, kicks blocked on the secure
  desktop), and composed-but-every-frame-mismatched (ring sized from a stale/
  foreign-session GDI mode). Zero layout change, old drivers read as 'no
  detail'; unit-tested pack/unpack in pf-driver-proto.

Verified on winbox: cargo check + clippy -p punktfunk-host -p pf-win-display
-p pf-driver-proto EXIT 0, drivers ws cargo check -p pf-vdisplay EXIT 0
(Version_Number=10.0.26100.0), cargo fmt --all --check clean; pf-driver-proto
tests 13/13 pass locally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 10:47:47 +02:00
enricobuehler 9a36ea2132 refactor(host/W6.2): extract the video encode backends into the pf-encode crate
encode.rs + encode/* (NVENC, VAAPI, native AMF, AMF/QSV ffmpeg, direct-SDK
NVENC/CUDA, raw Vulkan-Video, PyroWave, openh264) move into crates/pf-encode
behind one Encoder trait + open_video selector (plan §W6). The crate speaks the
shared frame vocabulary (pf-frame: CapturedFrame/PixelFormat + the DXGI identity
D3d11Frame/make_device) and pf-zerocopy (CUDA context/buffers), and NEVER
pf-capture — the capture→encode edge is one-way (ZeroCopyPolicy, prior commit).

Dep moves: the heavy encoder deps (ffmpeg-next, the NVENC SDK, openh264,
pyrowave-sys) move from the host to pf-encode; the host's
nvenc/amf-qsv/vulkan-encode/pyrowave features now FORWARD to pf-encode/*. The
host keeps a mod-encode shim (pub use pf_encode) so every crate::encode::* path
(negotiator + GameStream/native/mgmt planes) is unchanged.

resolve_render_adapter_luid moves from the host's windows/win_adapter.rs into
pf-gpu (both pf-encode and pf-capture need it as a peer of GPU selection); its 5
call sites (encode amf/nvenc, capture idd_push/synthetic_nv12, vdisplay manager)
rewire to pf_gpu::resolve_render_adapter_luid and win_adapter.rs is deleted.
pf-frame's make_device gains a # Safety section (public-unsafe-fn lint, latent
since the pf-frame carve — a full-workspace -D warnings clippy catches it).

Verified: Linux clippy -D warnings (pf-encode + host nvenc,vulkan-encode,pyrowave
--all-targets) + 13/13 pf-encode + 299/299 host tests; Windows clippy -D warnings
(pf-encode nvenc,amf-qsv --all-targets + host nvenc,amf-qsv --all-targets)
Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:42:51 +02:00
enricobuehler 1de83ba51d refactor(host/W6.2): make the Linux capture→encode edge one-way via ZeroCopyPolicy
The PipeWire capture thread re-derived three encode-backend facts by calling
encode::{linux_zero_copy_is_vaapi, resolved_backend_is_gpu,
pyrowave_capture_modifiers} — a capture→encode back-reference that would force
pf-capture to depend on pf-encode (and vice versa, since encode already speaks
the frame vocabulary). Resolve them ONCE in the host capture facade (which may
reach crate::encode) as a ZeroCopyPolicy { backend_is_vaapi, backend_is_gpu,
pyrowave_modifiers } and thread it into PortalCapturer::open /
from_virtual_output → spawn_pipewire → pipewire_thread.

capture/linux/mod.rs now makes ZERO crate::encode calls — the edge is one-way
(plan §2.4 / §W6), so pf-capture can be extracted depending only on pf-frame
(not pf-encode). pyrowave_modifiers is computed by the facade whenever the
encoder pref is pyrowave (which implies the VAAPI backend); the thread still
consumes them only inside its existing vaapi_passthrough guard, so behavior is
unchanged.

Verified: Linux clippy -D warnings (host nvenc,vulkan-encode,pyrowave
--all-targets); Windows clippy nvenc,amf-qsv --all-targets Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:20:17 +02:00
enricobuehler ccc4b08d45 refactor(host/W6.2): extract the Windows display-topology cluster into the pf-win-display leaf crate
windows/{win_display,monitor_devnode,display_events}.rs move into
crates/pf-win-display: the CCD/GDI path-activation + mode-set + HDR
advanced-colour + source-rect helpers, the PnP monitor-devnode enable/disable
lever, and the WM_DISPLAYCHANGE / device-arrival watch. The coming pf-capture
crate's IDD-push capturer consumes all three; the host's pf-vdisplay backend
consumes win_display + monitor_devnode. A leaf lets both depend on them as a
PEER instead of the capturer reaching back into the orchestrator (plan §W6).

win_display's one external tie (crate::vdisplay::Mode) becomes the underlying
punktfunk_core::Mode; the cluster is otherwise self-contained (pf-paths for the
state file, serde_json for it, windows). pub(crate) items bump to pub at the
boundary; win_display carries a module-level allow(missing_safety_doc) to keep
the pre-carve behavior (the FFI helpers were pub(crate) unsafe fn with prose
safety docs — the lint only fires once they're pub, and this is an internal
publish=false leaf). The host imports the three modules at its crate root, so
every crate::{win_display,monitor_devnode,display_events}::* path is unchanged.

Verified: Linux clippy -D warnings (leaf empty + host
nvenc,vulkan-encode,pyrowave --all-targets); Windows clippy -D warnings
(pf-win-display --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:14:17 +02:00
enricobuehler b168790e0a refactor(host/W6.2): extract the shared frame/format vocabulary into the pf-frame leaf crate
The captured-frame types both capture (producer) and encode (consumer) speak —
PixelFormat, OutputFormat, CursorOverlay, CapturedFrame, FramePayload,
DmabufFrame, drm_fourcc — move into crates/pf-frame, alongside the small pure
helpers that ride the same seam: hdr (HDR static metadata / in-band SEI),
metronome (the metronomic-stall detector), thread_qos (per-thread scheduling
QoS), session_tuning (Windows process tuning), and the Windows DXGI capture
IDENTITY (WinCaptureTarget, D3d11Frame, pack_luid, make_device + the GPU
scheduling-priority hardening it applies) (plan §W6).

This is the crate that breaks the capture<->encode cycle: FramePayload's GPU
variants own their backends from BELOW (Cuda -> pf_zerocopy::DeviceBuffer,
D3d11 -> dxgi::D3d11Frame), so encode can speak the vocabulary without a path to
capture, and vice versa. The Windows DXGI identity moving here lets capture,
encode, and pf-vdisplay share ONE WinCaptureTarget/device factory instead of the
old capture<->encode<->vdisplay reach-in.

The host keeps thin facades: capture.rs re-exports the vocabulary
(crate::capture::{PixelFormat,…} unchanged); capture/windows/dxgi.rs keeps the
win32u GPU-preference hook + HDR/video-engine converters + self-test and
re-exports the identity; native.rs re-exports boost_thread_priority from
pf_frame. crate::hdr/metronome/session_tuning callers rewired to pf_frame::*.
metronome's Metronome::new gained a Default impl (new_without_default fires once
the type is public across the crate boundary).

Verified: Linux clippy -D warnings (pf-frame --all-targets + host
nvenc,vulkan-encode,pyrowave --all-targets) + 9/9 pf-frame tests; Windows clippy
nvenc,amf-qsv --all-targets Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:03:56 +02:00
enricobuehler 6824c1cc0c feat(core): Automatic bitrate climbs only through validated territory — utilization gate + proven-throughput cap
The controller's target is only a promise: on calm content the encoder
emits a fraction of it, every window looks clean while proving nothing,
and the climb drifts the target into rates the decoder has never seen.
The first motion spike is then the first real test — it fails, and the
decoder is overloaded for the two-window backoff latency (the reported
settle-calm-then-spike stutter).

Three changes, all client-side (no wire/ABI impact, old hosts unaffected):

- Climb gate: a clean window authorizes a climb only when its ACTUAL
  delivered throughput reached 3/4 of the current target — the target was
  genuinely tested. Calm windows still bank clean credit; the first
  loaded window after a clean run climbs immediately.
- Proven-throughput cap: climbs step at most x1.5 past the session's
  high-water mark of delivered-and-digested (decode-latency-flat)
  throughput, so slow start becomes a bounded experiment instead of a
  blind doubling. High-water never decays: calm periods keep a validated
  target, so returning motion gets the full rate instantly; shrinking
  capacity (thermals) stays the reactive decode signal's job.
- Severe decode excursion: a >45 ms-over-baseline decode spike backs off
  after ONE window instead of two — the overload is already on screen.

The pump feeds the window's wire-byte throughput; the byte baseline is
rebased when the startup capacity probe completes so FLAG_PROBE filler
can't poison the proven mark with the link rate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 09:55:09 +02:00
enricobuehler 85bc5b9a3f refactor(host/W6.2): extract the Linux zero-copy GPU plumbing into the pf-zerocopy leaf crate
linux/zerocopy/* (CUDA context/buffers + EGL/Vulkan dmabuf import + the isolated
import worker) and linux/dmabuf_fence.rs move wholesale into crates/pf-zerocopy,
so the coming pf-frame vocabulary crate (FramePayload::Cuda owns a DeviceBuffer)
and the pf-encode/pf-capture subsystem crates can reach the GPU plumbing without
the host orchestrator in between (plan §W6). Content stays Linux-only; the crate
compiles to an empty lib elsewhere, so dependents carry a plain dependency.

drm_fourcc deliberately does NOT move: it consumes the frame vocabulary
(PixelFormat), which sits ABOVE pf-zerocopy — it lives with capture for now and
moves into pf-frame next. cuda's ffi re-export bumps pub(crate)->pub (the raw
CUdeviceptr vocabulary is consumed across the crate boundary by the encode
backends). A crate::zerocopy shim module keeps every existing path valid until
capture/encode themselves move out.

Verified: Linux clippy -D warnings (pf-zerocopy --all-targets + host
nvenc,vulkan-encode,pyrowave --all-targets) + 17/17 pf-zerocopy tests + 321/321
host tests; Windows clippy nvenc,amf-qsv --all-targets Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 09:41:15 +02:00
enricobuehler 6ea036766a refactor(host/W6.1): extract GPU vendor/adapter detection into the pf-gpu leaf crate
Fourth de-coupling for the host crate carve (plan §W6.1 leaf). gpu.rs (inventory,
selection preference, active-session accounting — deps only pf-host-config + pf-paths, no
subsystem refs) moves to a new pf-gpu leaf so pf-encode/pf-capture/pf-vdisplay can consult
the selected GPU without an orchestrator edge. ~50 crate::gpu:: sites repoint to pf_gpu::;
the ~30 pub(crate) items become pub (crate API). assign_ids gets a macOS-only
allow(dead_code) (used only by the Linux/Windows enumerate arms).

Verified: Linux (home-worker-5) clippy -p pf-gpu -p punktfunk-host --all-targets
-D warnings + pf-gpu tests (12 pass); Windows (192.168.1.158) clippy --features
nvenc,amf-qsv --all-targets green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 09:00:53 +02:00
enricobuehler 3495d189e1 refactor(host/W6.1): extract the config() global into the pf-host-config leaf crate
Third de-coupling for the host crate carve (plan §W6.1 leaf). HostConfig + the config()
OnceLock (config.rs, pure std, zero deps) move to a new pf-host-config leaf so every
subsystem crate (pf-encode/pf-capture/pf-vdisplay/pf-gpu) can read process config WITHOUT
depending on the orchestrator. 34 crate::config::config() call sites across 19 files
repoint to pf_host_config::config(). thread_qos stays in the host for now (it calls
session_tuning::on_hot_thread — its own leaf-ification rides the encode carve).

Granular-crate decision (supersedes the plan's single pf-media): split capture/encode/
vdisplay into separate crates rather than one broad crate — the capture↔encode cycle is
broken by a shared frame-types leaf, and vdisplay→encode (can_open_another_session) is a
legal one-way edge since encode never references vdisplay.

Verified: Linux (home-worker-5) clippy -p pf-host-config -p punktfunk-host --all-targets
-D warnings; Windows (192.168.1.158) clippy --features nvenc,amf-qsv --all-targets green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 08:54:47 +02:00
enricobuehler c42ce88921 refactor(host/W6.1): extract secret/config-dir helpers into the pf-paths leaf crate
Second de-coupling for the host crate carve (plan §W6.1 leaf). config_dir /
create_private_dir / write_secret_file (+ the Windows DACL helpers) were pub(crate) in the
gamestream junk drawer, yet consumed by vdisplay, stats, gpu, library, mgmt_token,
native_pairing and the Windows service — many of which become pf-media / pf-vdisplay, for
which crate::gamestream would be an illegal upward edge. New leaf crate pf-paths (pure std
+ tracing) owns them; ~40 call sites across 14 files repoint to pf_paths::. gamestream
keeps only its own concerns.

Verified: Linux (home-worker-5) clippy -p pf-paths -p punktfunk-host --all-targets
-D warnings + tests (347 pass, incl. secrets_are_written_owner_only); Windows
(192.168.1.158) clippy --features nvenc,amf-qsv --all-targets green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 01:07:26 +02:00
enricobuehler 2e3208f75e refactor(host/W6.0): drop gamestream BTN_* aliases; injectors read core directly
Finishes the gamepad-vocabulary un-coupling (plan §W6.0): the Linux uinput button map now
names BTN_* straight from punktfunk_core::input::gamepad instead of the crate::gamestream
re-export aliases, so pf-inject will carry no edge into the gamestream junk drawer for
gamepad constants. Removes the now-dead alias block; the wire-bit pinning test
(gamepad_wire_bits_are_pinned) pins core directly (equally strong — core is the single
source). gamestream keeps only the decode path, which imports the types from core.

Verified: Linux (home-worker-5) clippy --all-targets -D warnings + gamepad tests green;
Windows (192.168.1.158) clippy -p punktfunk-host --features nvenc,amf-qsv --all-targets green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:55:44 +02:00
enricobuehler 6a0a97b702 Merge main into perf/first-frame-latency (controller fixes + Apple M0)
Keeps the latency branch current with 1a7e3a6e/b45323c0/4cae1b8b so the
eventual landing on main is a clean fast-forward-style merge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 18:35:20 +02:00
enricobuehler 45c29a99d5 perf(host+driver): in-place resize = advertised-mode fast path + mode-history union
On-glass round 2 settled the mechanism: after UpdateModes2 the OS re-parses
our description AND re-queries target modes (driver log — both callbacks
served the fresh list) yet the SETTABLE set stays pruned to the modes known
at monitor ARRIVAL; the monitor source-mode set is pinned then, below
anything the driver can refresh. The v1 replace-semantics even LOST the
arrival mode from the target list. Consequences:

- driver: UPDATE_MODES now UNIONs (new mode first, previous list kept,
  deduped by resolution, cap 12), and a re-created same-id monitor inherits
  its departed predecessor's list (MODE_HISTORY) — every size an identity
  ever served is settable at the next arrival, so returning to a
  previously-used size (windowed<->fullscreen, drag back) is IN-PLACE.
- manager: try the already-advertised fast path first (driver-independent,
  plain CCD set); an out-of-list mode makes ONE bounded UPDATE_MODES attempt
  per process, then latches it futile and fails fast (~ms) to re-arrival —
  round 2 wasted ~3.1 s per arbitrary resize on the doomed wait. Fallback
  log demoted warn->info (expected-normal for first-seen sizes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 18:01:36 +02:00
enricobuehler a738de6cd8 fix(host): force a CCD mode re-enumeration after UPDATE_MODES (in-place resize)
First on-glass run: the driver accepted every UpdateModes2 (0x0 in the driver
log) but the OS never re-enumerated the target's settable modes on its own —
'OS did not advertise 800x1050 within 2s' → re-arrival fallback every time.
Re-commit the current config with SDC_FORCE_MODE_ENUMERATION (the same nudge
the isolate/layout paths already rely on) before the advertised-wait, re-kick
up to 3x, and log the actually-offered resolutions when it still misses.
Driver: dbglog the *2 mode-query/parse callbacks so the re-enumeration story
is visible in pfvd-driver.log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:48:44 +02:00
enricobuehler 55e59458a2 test(host): instrument the live resize spike (tracing + CCD-visibility probe)
On-glass finding: from an ssh/schtasks session-0 context QueryDisplayConfig
returns nothing at all — the activation ladder is blind there, so the live
tests can only run from an INTERACTIVE (desktop) admin prompt on the box;
the probe line makes that precondition self-diagnosing. Also verified live:
the v4 driver handshake ('pf-vdisplay protocol 4') and ADD on the new driver.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:36:16 +02:00
enricobuehler f910d23fb2 fix(proto): drop the constant assertion clippy rejects (CI parity)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:18:06 +02:00
enricobuehler c95e9125b9 test(host): live in-place resize spike (PUNKTFUNK_PF_VDISPLAY_LIVE)
Answers the P2 open questions on real glass with no streaming client: a
second same-slot acquire at a different (never-advertised) mode drives the
manager's resize branch; in-place success = same OS target id + the new
active resolution, with the elapsed ms printed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:17:34 +02:00
enricobuehler c2b9b32904 perf(host): in-place mid-stream resize — mode-set the live monitor, keep the capturer
Latency plan P2.2/P2.3: against a v4 driver the manager's resize branch now
runs UPDATE_MODES -> wait-mode-advertised (the OS re-enumerates async) ->
set_active_mode -> verified-state settle (P0.2) on the SAME monitor — no
REMOVE->ADD hotplug, no departure settle, no activation ladder, no re-isolate;
Windows keeps the per-monitor DPI (identity preserved). Any failure (v3
driver, mode never advertised, settle miss) falls back to the proven
re-arrival path unchanged.

On top of that the session's resize handler keeps the WHOLE capture pipeline:
the IDD-push capturer re-sizes its ring immediately (Capturer::resize_output —
no DescriptorPoller two-strike debounce, which stays for EXTERNAL changes),
the driver re-attaches and the mode-set full redraw provides the first frame;
only the encoder is swapped once the first new-size frame arrives
(open_video is ms-scale — P2.4 deliberately skipped). The capturer, send
thread and session transport all survive; every decline routes to the full
rebuild. Resize-trace stages (display_resized, ring_recreated,
first_new_frame, encoder_open) extend the P0.1 timeline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:12:14 +02:00
enricobuehler 0899e53903 feat(driver): pf-vdisplay IOCTL_UPDATE_MODES — live monitor mode-list refresh (proto v4)
Latency plan P2.1 (design/first-frame-and-resize-latency.md): a new additive
control-plane op lets the host refresh a LIVE monitor's advertised target-mode
list to lead with an arbitrary new mode (IddCxMonitorUpdateModes2 — the same
IddCx 1.10 *2 family this driver already requires, so no new OS floor). This
removes the 'mode list frozen at ADD' constraint that forced the mid-stream
resize through a REMOVE->ADD monitor hotplug: the monitor's OS identity, its
swap-chain worker and the retained FrameStash all survive an in-place mode set.

Protocol v4 is ADDITIVE over v3: the host's handshake floor stays at v3
(MIN_DRIVER_PROTOCOL_VERSION) and gates the in-place path on the reported
version, keeping re-arrival as the permanent fallback. The driver's stored
mode list is swapped before the DDI and reverted if it fails, so the OS and
the mode-DDI callbacks always agree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:12:13 +02:00
enricobuehler 32ffe7d634 chore(api): regenerate openapi.json (transition-latency fields + held drift)
Adds StreamInfo.time_to_first_frame_ms / last_resize_ms (latency plan P0.1)
and folds in the drift the spec already owed from the held working-tree
consolidation (version 0.12.0, pnp_disable_monitors description, the
conflicting-host 'conflicts' summary field) — the drift test was already
red before this branch; it is green at this commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:48:35 +02:00
enricobuehler 8374dfedf3 perf(host): session-transition trace + Welcome-time display prep (native path)
Latency plan P0.1 + P1.1/P1.2 (design/first-frame-and-resize-latency.md):

P0.1 — every native session runs a bringup::Trace (hello -> welcome -> start
-> punch_done -> display_acquired -> capture_attached -> first_frame ->
encoder_open -> first_au -> first_packet), one summary info! line when the
first video packet leaves; each accepted resize runs its own trace
(reconfigure -> pipeline_rebuilt). Totals surface per session as
time_to_first_frame_ms / last_resize_ms in session_status -> mgmt /status,
so every subsequent latency change is measured, not vibed. (The Windows
manager logs its own activation/settle deltas — correlate by wall clock.)

P1.1/P1.2 — on the Windows native path the display bring-up no longer
serializes behind the Start round-trip and the up-to-2.5 s hole-punch wait:
a prep thread kicks off at Welcome (mode is final there) and runs monitor
create -> activation -> verified settle -> capture attach -> first frame ->
encoder open while the network waits are in flight; the data plane hands it
the post-punch SessionContext and it becomes the stream thread on a warm
pipeline. Abort between Welcome and Start drops the hand-off channel and the
prep result releases into the keep-alive machinery (stop/quit + watcher are
created pre-handshake so a vanished client also aborts the build retries).
Same slot-scoped begin_idd_setup serialization as the inline path. Linux
keeps the inline bring-up (launch semantics bind before create); GameStream
untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:45:34 +02:00
enricobuehler e62cd5448e perf(host): IDD-push open — poll the HDR-enable settle, wait on the frame event
Latency plan P0.4/P0.6: the fixed 250 ms advanced-color settle becomes a
25 ms poll of the CCD state (ceiling 250 ms, ring still sized FP16 from the
successful enable either way), and wait_for_attach waits on the driver's
frame-ready event (20 ms cap for the status-code polls) instead of a blind
20 ms sleep, which also sharpens the P0.1 stage stamps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:45:16 +02:00
enricobuehler 4ed5b88407 perf(host): replace the Windows bring-up/resize fixed sleeps with verified-state waits
Latency plan P0.2/P0.3/P0.5 (design/first-frame-and-resize-latency.md):
- topology settle: the unconditional 1500 ms sleeps after create_monitor's
  group-topology apply and re_add's reisolate become a 25 ms poll for the
  committed state (active path + active mode == requested), ceiling 1500 ms —
  worst case identical, typical case saves ~1.2-1.4 s on every fresh create
  AND every mid-stream resize. The experimental pnp_disable_monitors sweep
  keeps the full settle as its floor (it reads OTHER displays' active flags,
  which the target-scoped wait doesn't verify).
- monitor departure: the fixed 400 ms REMOVE settles (re_add + both preempt
  paths) become a 25 ms poll until the target leaves the active CCD set
  (2 consecutive absent samples), ceiling 400 ms; the driver-side ghost-reap
  ADD retry stays the backstop.
- activation ladder: 200 ms -> 50 ms sampling, same 3 s per-stage ceilings
  and the same 3-stage structure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:44:57 +02:00
enricobuehler 44b71e7460 feat(audio/linux): host-owned stream sink decouples capture from hardware-sink churn
The Linux desktop-audio capture stream now registers itself as an Audio/Sink
node ("Punktfunk Stream Speaker", default-on, PUNKTFUNK_STREAM_SINK=0 =
legacy escape hatch) and claims the configured default sink for the duration
of a session (saved and restored around it, refcounted across concurrent
sessions, crash-stale claims degrade to automatic election). Host apps play
directly into the capture stream, so the capture link no longer depends on
any hardware sink.

Root cause this fixes (live-diagnosed on a bazzite/LG-TV host): gamescope
modesets drop the NVIDIA HDMI audio endpoint, WirePlumber ping-pongs the
default sink HDMI<->auto_null ~8x/s, and the old monitor-follower relinked
its capture on every flip - Paused/renegotiate/Streaming storms (~1300 log
lines/min) heard as crackle on the client.

Also fixes a latent liveness bug in both modes: the capture thread had no
core-error listener, so a PipeWire daemon restart mid-session left a zombie
thread returning quiet-sink empty chunks forever (same class as the historic
virtual-mic death bug). Now the thread exits and sessions reopen with backoff.

Bonus: the sink advertises the session's true channel count, so games can
produce real 5.1/7.1 even when local hardware is stereo. New AudioCapturer::
idle() hook releases the routing claim when a capturer is parked between
sessions; drain() re-claims on reuse.

Verified: cargo clippy -D warnings + 295 tests green on Linux (.21).
On-glass validation on the bazzite host pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 09:34:36 +02:00
304 changed files with 25725 additions and 18032 deletions
+8 -1
View File
@@ -316,6 +316,10 @@ jobs:
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk iOS App Store Distribution"
# The embedded PunktfunkWidgetsExtension (bundle io.unom.punktfunk.widgets) is a second
# distribution artifact in the .ipa, so manual signing must map its App ID to its own
# App Store profile too — else exportArchive fails ("no profile for io.unom.punktfunk.widgets").
WIDGET_PROFILE="Punktfunk iOS Widgets App Store Distribution"
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \
-project "$PROJECT" -scheme Punktfunk-iOS \
-destination 'generic/platform=iOS' \
@@ -335,7 +339,10 @@ jobs:
<key>signingStyle</key><string>manual</string>
<key>signingCertificate</key><string>Apple Distribution</string>
<key>provisioningProfiles</key>
<dict><key>io.unom.punktfunk</key><string>$PROFILE</string></dict>
<dict>
<key>io.unom.punktfunk</key><string>$PROFILE</string>
<key>io.unom.punktfunk.widgets</key><string>$WIDGET_PROFILE</string>
</dict>
</dict>
</plist>
EOF
+60
View File
@@ -0,0 +1,60 @@
# Publish the TypeScript SDK (@punktfunk/host) to the Gitea npm registry
# (https://git.unom.io/api/packages/unom/npm/).
#
# Trigger: push a tag `sdk-vX.Y.Z` (must equal sdk/package.json "version"), or run manually.
# The SDK versions independently of the app's `v*` tags, so bumping the host doesn't republish it.
#
# Auth: REGISTRY_TOKEN — the same repo Actions secret docker.yml uses (a Gitea PAT with
# write:package scope). No new secret needed.
name: sdk-publish
on:
push:
tags: ['sdk-v*']
workflow_dispatch:
jobs:
publish:
runs-on: ubuntu-24.04
container:
image: oven/bun:1
timeout-minutes: 15
defaults:
run:
working-directory: sdk
steps:
# oven/bun's slim base ships neither git, a CA bundle, nor node — actions/checkout's HTTPS
# fetch needs git + ca-certificates, and the version-guard step below uses node.
- name: Install git + node + CA certs
working-directory: /
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates git nodejs
- uses: actions/checkout@v4
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
- name: Typecheck
run: bun run typecheck
- name: Test
run: bun test
- name: Build (dist/ JS + .d.ts)
run: bun run build
- name: Tag matches package version
if: startsWith(github.ref, 'refs/tags/')
run: |
TAG="${GITHUB_REF_NAME#sdk-v}"
PKG="$(node -p "require('./package.json').version")"
test "$TAG" = "$PKG" || { echo "tag $GITHUB_REF_NAME does not match package version $PKG"; exit 1; }
- name: Publish to Gitea registry
env:
NODE_AUTH_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
test -n "$NODE_AUTH_TOKEN" || { echo "REGISTRY_TOKEN secret is empty"; exit 1; }
# .npmrc already maps the @punktfunk scope to the registry; append the auth line.
printf '//git.unom.io/api/packages/unom/npm/:_authToken=%s\n' "$NODE_AUTH_TOKEN" >> .npmrc
bun publish
+3 -3
View File
@@ -153,9 +153,9 @@ jobs:
# `// SAFETY:` proof. Both invariants are lint-gated (`unsafe_op_in_unsafe_fn` +
# `undocumented_unsafe_blocks`); this step keeps them from regressing. (wdk-probe is a
# toolchain-only probe crate and is excluded.)
run: cargo clippy -p pf-umdf-util -p pf-xusb -p pf-dualsense -p wdk-iddcx -p pf-vdisplay --all-targets -- -D warnings
- name: cargo fmt --check the safe-layer + gamepad drivers
run: cargo fmt -p pf-umdf-util -p pf-xusb -p pf-dualsense --check
run: cargo clippy -p pf-umdf-util -p pf-xusb -p pf-dualsense -p pf-mouse -p wdk-iddcx -p pf-vdisplay --all-targets -- -D warnings
- name: cargo fmt --check the safe-layer + gamepad/mouse drivers
run: cargo fmt -p pf-umdf-util -p pf-xusb -p pf-dualsense -p pf-mouse --check
- name: Inspect /INTEGRITYCHECK (before) — expect FORCE_INTEGRITY set by wdk-build
run: |
# explicit --target (.cargo/config.toml) -> output under the triple subdir.
Generated
+188 -19
View File
@@ -2145,7 +2145,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.12.0"
version = "0.13.0"
[[package]]
name = "lazy_static"
@@ -2277,7 +2277,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"punktfunk-core",
]
@@ -2754,9 +2754,29 @@ version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.13.0"
dependencies = [
"anyhow",
"ashpd",
"libc",
"pf-driver-proto",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-win-display",
"pf-zerocopy",
"pipewire",
"punktfunk-core",
"tokio",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-client-core"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"anyhow",
"ash",
@@ -2797,7 +2817,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"anyhow",
"ash",
@@ -2816,9 +2836,31 @@ dependencies = [
"bytemuck",
]
[[package]]
name = "pf-encode"
version = "0.13.0"
dependencies = [
"anyhow",
"ash",
"ffmpeg-next",
"libc",
"libloading",
"nvidia-video-codec-sdk",
"openh264",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-zerocopy",
"punktfunk-core",
"pyrowave-sys",
"tracing",
"tracing-subscriber",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-ffvk"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"ash",
"bindgen",
@@ -2826,8 +2868,73 @@ dependencies = [
]
[[package]]
name = "pf-presenter"
name = "pf-frame"
version = "0.13.0"
dependencies = [
"anyhow",
"libc",
"pf-zerocopy",
"punktfunk-core",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-gpu"
version = "0.13.0"
dependencies = [
"anyhow",
"pf-host-config",
"pf-paths",
"serde",
"serde_json",
"tempfile",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-host-config"
version = "0.13.0"
[[package]]
name = "pf-inject"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"futures-util",
"libc",
"parking_lot",
"pf-capture",
"pf-driver-proto",
"pf-host-config",
"pf-paths",
"punktfunk-core",
"reis",
"tokio",
"tracing",
"usbip-sim",
"wayland-backend",
"wayland-client",
"wayland-protocols",
"wayland-protocols-misc",
"wayland-protocols-wlr",
"wayland-scanner",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"xkbcommon",
]
[[package]]
name = "pf-paths"
version = "0.13.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.13.0"
dependencies = [
"anyhow",
"ash",
@@ -2840,6 +2947,62 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "pf-vdisplay"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"bytemuck",
"futures-util",
"hex",
"libc",
"pf-driver-proto",
"pf-encode",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-paths",
"pf-win-display",
"punktfunk-core",
"serde",
"serde_json",
"sha2",
"tokio",
"tracing",
"utoipa",
"wayland-backend",
"wayland-client",
"wayland-scanner",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-win-display"
version = "0.13.0"
dependencies = [
"anyhow",
"pf-paths",
"punktfunk-core",
"serde_json",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-zerocopy"
version = "0.13.0"
dependencies = [
"anyhow",
"ash",
"khronos-egl",
"libc",
"libloading",
"serde",
"serde_json",
"tracing",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
@@ -3011,7 +3174,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"android_logger",
"jni",
@@ -3027,7 +3190,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3043,7 +3206,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3058,7 +3221,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3077,7 +3240,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"aes-gcm",
"bytes",
@@ -3108,7 +3271,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"aes",
"aes-gcm",
@@ -3120,7 +3283,6 @@ dependencies = [
"base64",
"bytemuck",
"cbc",
"ffmpeg-next",
"futures-util",
"hex",
"hmac",
@@ -3134,15 +3296,22 @@ dependencies = [
"log",
"mac_address",
"mdns-sd",
"nvidia-video-codec-sdk",
"openh264",
"opus",
"parking_lot",
"pf-capture",
"pf-clipboard",
"pf-driver-proto",
"pf-encode",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-inject",
"pf-paths",
"pf-vdisplay",
"pf-win-display",
"pf-zerocopy",
"pipewire",
"punktfunk-core",
"pyrowave-sys",
"quinn",
"rand 0.8.6",
"rcgen",
@@ -3184,7 +3353,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3198,7 +3367,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"anyhow",
"ksni",
@@ -3215,7 +3384,7 @@ dependencies = [
[[package]]
name = "pyrowave-sys"
version = "0.12.0"
version = "0.13.0"
dependencies = [
"bindgen",
"cmake",
+11 -1
View File
@@ -11,6 +11,16 @@ members = [
"crates/pf-console-ui",
"crates/pf-ffvk",
"crates/pf-driver-proto",
"crates/pf-paths",
"crates/pf-host-config",
"crates/pf-gpu",
"crates/pf-zerocopy",
"crates/pf-frame",
"crates/pf-win-display",
"crates/pf-encode",
"crates/pf-capture",
"crates/pf-inject",
"crates/pf-vdisplay",
"crates/pyrowave-sys",
"clients/probe",
"clients/linux",
@@ -37,7 +47,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.12.0"
version = "0.13.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+20 -2
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.12.0"
"version": "0.13.0"
},
"paths": {
"/api/v1/clients": {
@@ -2744,7 +2744,7 @@
},
"CustomEntry": {
"type": "object",
"description": "A user-added title, persisted in `~/.config/punktfunk/library.json`. Same shape the API\nreturns and the web console edits.",
"description": "A user-added title, persisted in the hardened host config dir's `library.json` (see\n[`custom_path`]). Same shape the API returns and the web console edits.",
"required": [
"id",
"title"
@@ -4617,6 +4617,15 @@
"format": "int32",
"minimum": 0
},
"last_resize_ms": {
"type": [
"integer",
"null"
],
"format": "int32",
"description": "Most recent mid-stream resize total, reconfigure → pipeline rebuilt, in ms (native sessions;\n`null` when no resize happened / GameStream).",
"minimum": 0
},
"min_fec": {
"type": "integer",
"format": "int32",
@@ -4629,6 +4638,15 @@
"description": "Video payload size per packet (bytes).",
"minimum": 0
},
"time_to_first_frame_ms": {
"type": [
"integer",
"null"
],
"format": "int32",
"description": "Session bring-up total, hello → first video packet, in ms (native sessions; `null` on the\nGameStream plane or while the session is still bringing up).",
"minimum": 0
},
"width": {
"type": "integer",
"format": "int32",
@@ -30,7 +30,13 @@ suspend fun connectToHost(
): Long {
// Advertise HDR only when the user enabled it AND this device's display can present it (else the
// host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val (w, h, hz) = settings.effectiveMode(context)
val (baseW, baseH, hz) = settings.effectiveMode(context)
// Render scale: ask the host for `chosen mode × scale` (even + codec-clamped) — > 1 supersamples
// (the compositor downscales the larger decoded frame to the SurfaceView), < 1 renders under
// native. 1.0 leaves the resolved mode untouched.
val (w, h) = RenderScale.apply(
baseW, baseH, settings.renderScale, RenderScale.maxDimension(settings.codec)
)
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID.
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
@@ -16,6 +16,14 @@ data class Settings(
val height: Int = 0,
val hz: Int = 0,
val bitrateKbps: Int = 0,
/**
* Render-resolution multiplier: the client asks the host to render/encode at `chosen mode ×
* renderScale` and the compositor downscales the larger decoded frame to the SurfaceView
* (`> 1` supersamples for sharpness, at more bandwidth AND decode; `< 1` renders under native
* for a lighter host/link). `1.0` = Native. Applied at connect via [RenderScale.apply], clamped
* even + to the codec's max dimension. Mirrors the Apple/Linux clients' render scale.
*/
val renderScale: Double = 1.0,
/**
* Advertise HDR (10-bit BT.2020 PQ) to the host. Default on, but only *effective* on a panel that
* can actually present HDR10 (see [displaySupportsHdr]) — on an SDR display HDR is never
@@ -137,6 +145,7 @@ class SettingsStore(context: Context) {
height = prefs.getInt(K_H, 0),
hz = prefs.getInt(K_HZ, 0),
bitrateKbps = prefs.getInt(K_BITRATE, 0),
renderScale = prefs.getFloat(K_RENDER_SCALE, 1.0f).toDouble(),
hdrEnabled = prefs.getBoolean(K_HDR, true),
compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0),
@@ -171,6 +180,7 @@ class SettingsStore(context: Context) {
.putInt(K_H, s.height)
.putInt(K_HZ, s.hz)
.putInt(K_BITRATE, s.bitrateKbps)
.putFloat(K_RENDER_SCALE, s.renderScale.toFloat())
.putBoolean(K_HDR, s.hdrEnabled)
.putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad)
@@ -193,6 +203,7 @@ class SettingsStore(context: Context) {
const val K_H = "height"
const val K_HZ = "hz"
const val K_BITRATE = "bitrate_kbps"
const val K_RENDER_SCALE = "render_scale"
const val K_HDR = "hdr_enabled"
const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad"
@@ -281,6 +292,54 @@ fun Settings.effectiveMode(context: Context): Triple<Int, Int, Int> {
return Triple(w, h, hz)
}
/**
* Client-side render-scale geometry — the Kotlin twin of `punktfunk-core`'s `render_scale` module
* (and the Apple client's `RenderScale`). Multiply a base size, preserve aspect, even-floor (the
* host rejects odd sizes), and clamp uniformly to the codec's per-axis ceiling so a connect can't
* ask for a size the encoder rejects. `1.0` = Native. Pure + covered by [RenderScaleTest].
*/
object RenderScale {
val PRESETS = listOf(0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0)
/** H.264 tops out at 4096 px/axis; HEVC/AV1/auto at 8192 — the host's `codec.rs` walls. */
fun maxDimension(codec: String): Int = if (codec == "h264") 4096 else 8192
/** Clamp a raw multiplier into [0.5, 4.0]; a missing / non-positive / NaN value → 1.0. */
fun sanitize(raw: Double): Double = if (raw > 0.0) raw.coerceIn(0.5, 4.0) else 1.0
/** "Native (1×)" / "1.5×" / "2× · supersample" — the picker label. */
fun label(scale: Double): String = when {
scale == 1.0 -> "Native (1×)"
scale > 1.0 -> "${trim(scale)}× · supersample"
else -> "${trim(scale)}×"
}
private fun trim(s: Double): String =
if (s == s.toLong().toDouble()) s.toLong().toString() else s.toString()
/** Apply [scale] to a base size → a host-valid even, aspect-preserved, codec-clamped (w, h). */
fun apply(baseW: Int, baseH: Int, scale: Double, maxDim: Int): Pair<Int, Int> {
val s = sanitize(scale)
var w = maxOf(baseW, 1) * s
var h = maxOf(baseH, 1) * s
val cap = maxDim.toDouble()
val over = maxOf(w / cap, h / cap)
if (over > 1.0) {
w /= over
h /= over
}
return Pair(evenFloor(w, 320), evenFloor(h, 200))
}
private fun evenFloor(value: Double, minimum: Int): Int {
val v = maxOf(kotlin.math.floor(value).toInt(), minimum).coerceAtLeast(0)
return v / 2 * 2
}
}
/** (scale, label) for the render-scale picker. `1.0` = Native. */
val RENDER_SCALE_OPTIONS = RenderScale.PRESETS.map { it to RenderScale.label(it) }
// ---- UI option tables (value, label). The first entry is always the "auto/native" default. ----
/** (width, height, label). `(0,0)` = native display. */
@@ -333,6 +333,15 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
update(s.copy(bitrateKbps = kbps))
}
SettingDropdown(
label = "Render scale",
options = RENDER_SCALE_OPTIONS,
// Snap the stored value (a Float round-tripped to Double) to the nearest preset so the
// exact Double keys match. > 1 supersamples for sharpness (more bandwidth AND decode);
// < 1 renders under native for a lighter host — this device resamples to the display.
selected = RenderScale.PRESETS.minByOrNull { kotlin.math.abs(it - s.renderScale) } ?: 1.0,
) { scale -> update(s.copy(renderScale = scale)) }
// AV1 is only offered when the device has a real AV1 decoder (it's never advertised to the
// host otherwise, so preferring it would be a dead setting). A stored "av1" from a capable
// device stays visible so the selection is always representable.
@@ -0,0 +1,73 @@
package io.unom.punktfunk
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
/**
* Pure JVM test of the client-side render-scale geometry ([RenderScale]) — the Kotlin twin of
* `punktfunk-core`'s `render_scale` module. Run: `./gradlew :app:testDebugUnitTest`.
*/
class RenderScaleTest {
@Test
fun sanitizeClampsAndDefaults() {
assertEquals(1.0, RenderScale.sanitize(0.0), 0.0) // absent / zero → Native
assertEquals(1.0, RenderScale.sanitize(-2.0), 0.0)
assertEquals(1.0, RenderScale.sanitize(Double.NaN), 0.0)
assertEquals(0.5, RenderScale.sanitize(0.1), 0.0) // below the floor
assertEquals(4.0, RenderScale.sanitize(9.0), 0.0) // above the ceiling
assertEquals(1.5, RenderScale.sanitize(1.5), 0.0)
}
@Test
fun maxDimensionIsCodecAware() {
assertEquals(4096, RenderScale.maxDimension("h264"))
assertEquals(8192, RenderScale.maxDimension("hevc"))
assertEquals(8192, RenderScale.maxDimension("av1"))
assertEquals(8192, RenderScale.maxDimension("auto"))
}
@Test
fun nativeIsIdentity() {
assertEquals(1920 to 1080, RenderScale.apply(1920, 1080, 1.0, 8192))
}
@Test
fun supersampleDoubles() {
assertEquals(3840 to 2160, RenderScale.apply(1920, 1080, 2.0, 8192))
}
@Test
fun underRenderHalves() {
assertEquals(960 to 540, RenderScale.apply(1920, 1080, 0.5, 8192))
}
@Test
fun resultsAreEven() {
// 1366×768 × 1.5 = 2049×1152 → even-floored to 2048×1152.
val (w, h) = RenderScale.apply(1366, 768, 1.5, 8192)
assertEquals(0, w % 2)
assertEquals(0, h % 2)
assertEquals(2048 to 1152, w to h)
}
@Test
fun overCeilingClampsUniformly() {
// 4K × 4 = 15360×8640; both exceed 8192 → width lands on cap, 16:9 kept (8192×4608).
val (w, h) = RenderScale.apply(3840, 2160, 4.0, 8192)
assertTrue(w <= 8192 && h <= 8192)
assertEquals(8192 to 4608, w to h)
}
@Test
fun h264CeilingIsTighter() {
// 1080p × 4 = 7680×4320; under H.264's 4096 wall → 4096×2304.
assertEquals(4096 to 2304, RenderScale.apply(1920, 1080, 4.0, 4096))
}
@Test
fun minimumFloorHonoured() {
val (w, h) = RenderScale.apply(400, 300, 0.5, 8192)
assertTrue(w >= 320 && h >= 200)
}
}
@@ -52,9 +52,6 @@ class GamepadFeedback(
const val TAG_PLAYER_LEDS: Byte = 0x02
const val TAG_TRIGGER: Byte = 0x03
const val TAG_HID_RAW: Byte = 0x05
// Fallback one-shot duration against a legacy host (no v2 TTL lease): the prior fixed value.
// A new host renews far below this, so it never actually holds this long there.
const val LEGACY_RUMBLE_MS = 60_000L
}
/** One controller's rumble binding — VibratorManager (API 31+) OR the legacy single Vibrator (API 2830). */
@@ -95,19 +92,19 @@ class GamepadFeedback(
while (running) {
val ev = NativeBridge.nativeNextRumble(handle)
if (ev < 0L) continue // timeout / closed
// ev bits 49..52 = wire pad index; bit 48 = has a v2 lease; bits 32..47 = ttl_ms;
// 16..31 = low; 0..15 = high. The lease flag is out-of-band, so any ttl_ms (incl.
// 0xFFFF) is a real lease — no in-band sentinel. No lease (legacy host) → the prior
// long one-shot.
// ev bits 49..52 = wire pad index; bits 32..47 = backstop duration (ms);
// 16..31 = low; 0..15 = high. These are EFFECTIVE commands from the core's shared
// rumble policy engine — it owns every lease/staleness/close decision (uniform
// across all clients; the old 60 s legacy-host exposure is gone) and emits
// explicit zeros, so apply verbatim: (0, 0) = cancel, non-zero = one-shot for
// the backstop (the hardware net under a stalled poll thread).
val pad = ((ev ushr 49) and 0xFL).toInt()
val hasLease = ((ev ushr 48) and 0x1L) == 0x1L
val ttl = ((ev ushr 32) and 0xFFFF).toInt()
val durationMs = if (hasLease) ttl.toLong() else LEGACY_RUMBLE_MS
val backstopMs = ((ev ushr 32) and 0xFFFF)
renderRumble(
pad,
((ev ushr 16) and 0xFFFF).toInt(),
(ev and 0xFFFF).toInt(),
durationMs,
backstopMs,
)
}
}, "pf-rumble").apply { isDaemon = true; start() }
@@ -212,12 +209,13 @@ class GamepadFeedback(
/**
* low = heavy/left motor, high = light/right motor; both 0..0xFFFF (the host's u16 amplitudes),
* addressed to wire pad [pad]. `durationMs` is the host's v2 envelope TTL — the one-shot self-
* terminates after it unless the host renews, so a lost stop (or a dead host) silences at the
* lease instead of the old fixed 60 s. Against a legacy host it is [LEGACY_RUMBLE_MS].
* addressed to wire pad [pad]. `durationMs` is the engine command's backstop — the one-shot's
* self-termination net under a stalled poll thread; the engine emits explicit zero commands at
* every policy stop (lease expiry, legacy staleness, session close), so cancel-on-zero is the
* real stop mechanism.
*/
private fun renderRumble(pad: Int, low: Int, high: Int, durationMs: Long) {
Log.i(TAG, "rumble pad=$pad low=$low high=$high ttlMs=$durationMs") // verification line — BEFORE any no-op return
Log.i(TAG, "rumble pad=$pad low=$low high=$high backstopMs=$durationMs") // verification line — BEFORE any no-op return
// Opt-in phone mirror, BEFORE the controller-bind early-return: the exact pads this
// serves have no vibrator of their own, so their bind below is null. It follows
// controller 1 unconditionally rather than only motor-less pads — capability probing
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,626 @@
//! The event-driven async MediaCodec decode loop (default) + its feeder/dispatch/present helpers.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{AsyncNotifyCallback, MediaCodec, MediaCodecDirection};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::reanchor::{GateVerdict, ReanchorGate};
use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, AtomicI64, Ordering};
use std::sync::{mpsc, Arc, Mutex};
use std::time::{Duration, Instant};
use super::display::{
apply_hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags};
use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
configure_low_latency, create_codec, try_set_frame_rate,
};
use super::{DecodeOptions, FRAME_PARK_CAP, IN_FLIGHT_CAP, PENDING_SPLIT_CAP};
/// One decoded output buffer ready to release: its codec buffer index + the pts the codec echoed
/// (from the output callback's `BufferInfo`), used to pair the `decode` HUD stat, and the
/// wall-clock instant the output callback fired — the spec's `decoded` point ("decoder output
/// frame available"), stamped at the callback so the event-channel hop + coalescing wait in the
/// loop never inflates the decode stage.
struct OutputReady {
index: usize,
pts_us: u64,
decoded_ns: i128,
}
/// Events the async decode loop reacts to. The codec's async-notify callbacks (which run on its
/// internal looper thread) push the codec ones; the feeder thread pushes `Au`. Each carries only
/// owned/`Copy` data so the callback closures satisfy the `Send` bound and never touch the codec.
enum DecodeEvent {
/// A received access unit from the feeder, ready to queue into the decoder. The `bool` is the
/// feeder's [`NativeClient::note_frame_index`] verdict — `true` when this AU revealed a forward
/// frame-index gap, so the loop arms the freeze gate (the feeder already fired the RFI request).
Au(Frame, bool),
/// An input buffer slot freed (index) — we can queue an AU into it.
InputAvailable(usize),
/// A decoded frame is ready (buffer index + echoed pts + the callback-time `decoded` stamp).
OutputAvailable {
index: usize,
pts_us: u64,
decoded_ns: i128,
},
/// The output format changed — re-check the stream's colour signalling (HDR DataSpace).
FormatChanged,
/// The codec reported an error; `fatal` when neither recoverable nor transient.
Error { fatal: bool },
}
/// The event-driven async decode loop (default; see [`run`]/[`USE_ASYNC_DECODE`]). The codec drives
/// us: an async-notify callback fires the instant an input buffer frees or a frame finishes
/// decoding, so a decoded frame is presented immediately instead of waiting out a poll interval (the
/// latency the sync loop left on the table). The callbacks run on the codec's internal looper thread
/// and only *push events* — every `AMediaCodec` buffer op stays on this thread, which owns the codec,
/// sidestepping the self-reference that would arise from a callback calling back into the codec it's
/// stored in. A small `pf-decode-feed` thread blocks on the network so this loop never does.
pub(super) fn run_async(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
let mime = codec_mime(client.codec);
let mut codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (async, low-latency feature: {ll_feature})"
);
// The event channel: the callbacks + feeder push, this loop pulls. `Sender` is `Send`, so the
// callback closures (each capturing a clone) satisfy the async-notify `Send` bound.
let (ev_tx, ev_rx) = mpsc::channel::<DecodeEvent>();
// Install the callbacks BEFORE configure()/start() so we're in async mode from the first buffer.
// Each just forwards an index/flag — no codec access here (the codec owns these closures).
{
let out_tx = ev_tx.clone();
let in_tx = ev_tx.clone();
let fmt_tx = ev_tx.clone();
let err_tx = ev_tx.clone();
let cb = AsyncNotifyCallback {
on_input_available: Some(Box::new(move |idx| {
let _ = in_tx.send(DecodeEvent::InputAvailable(idx));
})),
on_output_available: Some(Box::new(move |idx, info| {
let _ = out_tx.send(DecodeEvent::OutputAvailable {
index: idx,
pts_us: info.presentation_time_us().max(0) as u64,
// The `decoded` HUD point: stamp HERE, on the codec's looper thread, so the
// decode stage ends when the frame actually became available — not after the
// channel hop + whatever work the loop coalesces in front of presenting it.
decoded_ns: now_realtime_ns(),
});
})),
on_format_changed: Some(Box::new(move |_fmt| {
let _ = fmt_tx.send(DecodeEvent::FormatChanged);
})),
on_error: Some(Box::new(move |e, code, _detail| {
let fatal = !code.is_recoverable() && !code.is_transient();
if fatal {
log::error!("decode: fatal codec error — stream will stop: {e:?}");
} else {
log::warn!("decode: codec error {e:?} (recoverable)");
}
let _ = err_tx.send(DecodeEvent::Error { fatal });
})),
};
if let Err(e) = codec.set_async_notify_callback(Some(cb)) {
log::error!("decode: set_async_notify_callback failed: {e}");
return;
}
}
// Build the low-latency format (identical keys to the sync path).
let mut format = MediaFormat::new();
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
format.set_i32(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
configure_low_latency(&mut format, &codec_name, low_latency_mode);
if client.color.is_hdr() {
match client.next_hdr_meta(Duration::from_millis(250)) {
Ok(meta) => {
format.set_buffer("hdr-static-info", &android_hdr_static_info(&meta));
log::info!("decode: HDR static metadata applied (KEY_HDR_STATIC_INFO)");
}
Err(_) => {
log::info!("decode: HDR session but no mastering metadata yet — DataSpace only")
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
log::error!("decode: configure failed: {e}");
return;
}
if let Err(e) = codec.start() {
log::error!("decode: start failed: {e}");
return;
}
log::info!(
"decode: decoder started (async) at {}x{}",
mode.width,
mode.height
);
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
);
}
// Skew-corrected latency stats (spec: design/stats-unification.md). Receipt stamps (keyed by the
// pts we queue) live in a shared map: the feeder writes them at receipt, this loop pairs decoded
// output back to them. Behind a `Mutex` since two threads touch it — only ever locked while the
// HUD is visible.
let clock_offset = client.clock_offset_shared();
// Whether the adaptive-bitrate controller wants the `decode` stage as its decoder-backlog
// signal (Automatic, non-PyroWave): then `in_flight` is fed regardless of the HUD.
let measure_decode = client.wants_decode_latency();
let in_flight = Arc::new(Mutex::new(VecDeque::<(u64, i128)>::new()));
// Display stage (spec `display` + the capture→displayed headline): the rendered frame is
// parked in the tracker at release; the OnFrameRendered callback pairs it with
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
let tracker = DisplayTracker::new(stats.clone(), clock_offset.clone());
let render_cb = install_render_callback(&codec, &tracker);
// Feeder thread: block on the network so this loop doesn't (an AU's arrival becomes an event that
// wakes us immediately, with no input-side poll latency). It also records the `received` HUD stat.
let feeder = {
let client = client.clone();
let stats = stats.clone();
let in_flight = in_flight.clone();
let clock_offset = clock_offset.clone();
let shutdown = shutdown.clone();
let ev_tx = ev_tx.clone();
std::thread::Builder::new()
.name("pf-decode-feed".into())
.spawn(move || {
feeder_loop(
client,
stats,
measure_decode,
in_flight,
clock_offset,
shutdown,
ev_tx,
);
})
.ok()
};
drop(ev_tx); // only the feeder + callbacks keep the channel alive now
// ADPF: same as the sync path — register this thread now, create the session lazily on the first
// presented frame (by when the pump + audio + feeder threads have registered their tids too).
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread();
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
let mut free_inputs: VecDeque<usize> = VecDeque::new();
let mut pending_aus: VecDeque<Frame> = VecDeque::new();
let mut ready: Vec<OutputReady> = Vec::new();
let mut applied_ds: Option<DataSpace> = None;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
// AUs larger than the codec input buffer, dropped whole (see `feed`/`feed_ready`).
let mut oversized_dropped: u64 = 0;
// Freeze-until-reanchor gate (see the sync loop for the rationale). Armed on a frame-index gap
// (the feeder's Au verdict), a parked-AU overflow drop, a dropped-count climb, or a recoverable
// codec error; `recovery_flags` carries each AU's user_flags from `dispatch_event` (feed) to
// `present_ready` (present), keyed by the codec-echoed pts.
let mut gate = ReanchorGate::new(client.frames_dropped());
let mut recovery_flags: VecDeque<(u64, u32)> = VecDeque::new();
let mut last_kf_req: Option<Instant> = None;
// Productive (dispatch+feed+present) time between displayed frames; reported to ADPF once one is
// presented. The blocking event wait is excluded (idle, not work) — same accounting as the sync loop.
let mut work_accum_ns: i64 = 0;
let mut fatal = false;
while !shutdown.load(Ordering::Relaxed) && !fatal {
// Block for the next event (idle wait — excluded from the work tally). The short timeout
// drives loss-recovery housekeeping when the pipeline is momentarily quiet.
let ev0 = match ev_rx.recv_timeout(Duration::from_millis(5)) {
Ok(ev) => Some(ev),
Err(mpsc::RecvTimeoutError::Timeout) => None,
Err(mpsc::RecvTimeoutError::Disconnected) => break,
};
let work_t0 = Instant::now();
let mut fmt_dirty = false;
let mut aus_dropped: u64 = 0;
if let Some(ev) = ev0 {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
&mut gate,
&mut recovery_flags,
));
}
// Coalesce every other event already queued into this one work pass — correct newest-only
// presentation across a decode burst, and batched feeding.
while let Ok(ev) = ev_rx.try_recv() {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
&mut gate,
&mut recovery_flags,
));
}
stats.note_skipped(aus_dropped); // parked-AU overflow drops are client-side skips too
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
feed_ready(
&codec,
&client,
&mut pending_aus,
&mut free_inputs,
&mut fed,
&mut oversized_dropped,
);
let had_output = !ready.is_empty();
present_ready(
&codec,
&client,
measure_decode,
&mut ready,
&stats,
&in_flight,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut rendered,
&mut discarded,
&mut gate,
&mut recovery_flags,
);
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if had_output {
if !hint_tried {
hint_tried = true;
let tids = client.hot_thread_ids();
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
if rendered > 0 && rendered % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
}
// Loss recovery + overdue backstop, folded through the gate. A parked-AU overflow drop is itself
// a loss, so it arms the freeze directly; the gate's `poll` then arms on a dropped-count climb
// and re-asks on an overdue freeze. All keyframe intents route through the shared 100 ms
// throttle so a multi-frame recovery gap can't flood the control stream.
let now = Instant::now();
if aus_dropped > 0 {
gate.arm(now);
}
if (gate.poll(client.frames_dropped(), now) || aus_dropped > 0)
&& last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = client.request_keyframe();
}
}
let _ = codec.stop();
shutdown.store(true, Ordering::SeqCst); // ensure the feeder wakes and exits, then join it
if let Some(j) = feeder {
let _ = j.join();
}
drop(codec); // AMediaCodec_delete — after this no render callback can fire
if let Some(ud) = render_cb {
// SAFETY: the codec was dropped above; this registration's single reclaim.
unsafe { release_render_callback(ud) };
}
log::info!("decode: stopped (async, fed={fed} rendered={rendered} discarded={discarded})");
}
/// The `pf-decode-feed` thread: block on the connector for the next access unit so the async loop
/// never has to. Records the `received` HUD stat (receipt point) — including the Phase-2 host/network
/// split from any matching 0xCF host timings — then hands the AU to the loop via the event channel.
/// Exits when `shutdown` is set, the session closes, or the loop's receiver is gone.
fn feeder_loop(
client: Arc<NativeClient>,
stats: Arc<crate::stats::VideoStats>,
measure_decode: bool,
in_flight: Arc<Mutex<VecDeque<(u64, i128)>>>,
clock_offset: Arc<AtomicI64>,
shutdown: Arc<AtomicBool>,
ev_tx: mpsc::Sender<DecodeEvent>,
) {
// Received AUs awaiting their 0xCF host timing (Phase-2 split), as (pts_ns, capture→received µs).
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
while !shutdown.load(Ordering::Relaxed) {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
// Loss recovery (RFI): a forward frame-index gap fires a throttled reference-frame-
// invalidation request so an RFI-capable host recovers with a cheap clean P-frame
// instead of a full IDR (the frames_dropped keyframe path is the backstop). The gap
// verdict rides the Au event so the decode loop arms its freeze gate on the same signal.
let gap = client.note_frame_index(frame.frame_index);
// Park the receipt stamp (keyed by the pts the codec echoes) whenever the `decode`
// stage is consumed: the HUD, or the ABR decode signal (`measure_decode`). The
// HUD-only `received` point + host/network split stay gated on the overlay.
if stats.enabled() || measure_decode {
let received_ns = now_realtime_ns();
{
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.push_back((frame.pts_ns / 1000, received_ns));
if g.len() > IN_FLIGHT_CAP {
g.pop_front(); // stale — codec never echoed it back
}
}
if stats.enabled() {
let clock_offset = clock_offset.load(Ordering::Relaxed) as i128;
let lat_ns = received_ns + clock_offset - frame.pts_ns as i128;
let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000)
.then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front();
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|&(p, _)| p == t.pts_ns)
{
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
}
if ev_tx.send(DecodeEvent::Au(frame, gap)).is_err() {
break; // the decode loop is gone
}
}
Err(PunktfunkError::NoFrame) => {} // timeout — re-check shutdown and poll again
Err(_) => break, // session closed
}
}
}
/// Route one [`DecodeEvent`] into the loop's working sets. Returns `true` only when a parked AU was
/// dropped on overflow (the caller then requests a keyframe).
#[allow(clippy::too_many_arguments)] // two call sites; the freeze gate + flag map are threaded in
fn dispatch_event(
ev: DecodeEvent,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
ready: &mut Vec<OutputReady>,
fmt_dirty: &mut bool,
fatal: &mut bool,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) -> bool {
match ev {
DecodeEvent::Au(f, gap) => {
// A forward frame-index gap arms the freeze; park this AU's flags for the present side to
// fold `on_decoded` (keyed by the pts the codec will echo).
if gap {
gate.arm(Instant::now());
}
recovery_flags.push_back((f.pts_ns / 1000, f.flags));
if recovery_flags.len() > IN_FLIGHT_CAP {
recovery_flags.pop_front();
}
pending_aus.push_back(f);
if pending_aus.len() > FRAME_PARK_CAP {
pending_aus.pop_front(); // sustained overflow — drop oldest, signal a keyframe request
return true;
}
}
DecodeEvent::InputAvailable(i) => free_inputs.push_back(i),
DecodeEvent::OutputAvailable {
index,
pts_us,
decoded_ns,
} => ready.push(OutputReady {
index,
pts_us,
decoded_ns,
}),
DecodeEvent::FormatChanged => *fmt_dirty = true,
DecodeEvent::Error { fatal: f } => {
if f {
*fatal = true;
} else {
// A recoverable/transient codec error is a decode hiccup on a broken reference chain —
// arm the freeze so the concealed output it recovers into is held off the screen.
gate.arm(Instant::now());
}
}
}
false
}
/// Queue as many parked AUs as there are free input buffer slots (async mode: the indices come from
/// `InputAvailable` callbacks, not a dequeue). Each AU is copied into its codec input buffer and
/// submitted; an AU larger than the buffer is DROPPED (+ a recovery keyframe requested) — a
/// truncated AU is corrupt input the decoder chews on silently, poisoning the reference chain.
fn feed_ready(
codec: &MediaCodec,
client: &NativeClient,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
fed: &mut u64,
oversized_dropped: &mut u64,
) {
while !pending_aus.is_empty() && !free_inputs.is_empty() {
let idx = free_inputs.pop_front().unwrap();
let frame = pending_aus.pop_front().unwrap();
let pts_us = frame.pts_ns / 1000;
let Some(dst) = codec.input_buffer(idx) else {
log::warn!("decode: input_buffer({idx}) returned None — dropping AU");
continue;
};
let au = &frame.data;
if au.len() > dst.len() {
// The slot was never queued, so it stays ours — recycle it for the next AU.
free_inputs.push_front(idx);
*oversized_dropped += 1;
log::warn!(
"decode: AU {} > input buffer {} — dropped ({} so far), requesting keyframe",
au.len(),
dst.len(),
*oversized_dropped
);
let _ = client.request_keyframe();
continue;
}
let n = au.len();
// SAFETY: `au` (wire AU) and `dst` (codec input buffer) are distinct allocations, both valid
// for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so this initializes dst[..n].
unsafe {
std::ptr::copy_nonoverlapping(au.as_ptr(), dst.as_mut_ptr().cast::<u8>(), n);
}
if let Err(e) = codec.queue_input_buffer_by_index(idx, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer_by_index: {e}");
} else {
*fed += 1;
}
}
}
/// Present only the NEWEST ready output (render = true) and release the rest without rendering — a
/// burst of stale frames on glass is worse than skipping to the freshest (the sync loop's newest-ready
/// policy, callback-driven). Every dequeued buffer, rendered or not, is the HUD's `decoded`
/// measurement point (it finished decoding either way); samples are recorded in pts order so the
/// receipt-map eviction stays monotonic. The presented frame's `(pts, decoded stamp)` is parked in
/// `tracker` for the OnFrameRendered callback — the `display` stage's other endpoint. `ready` is
/// drained.
#[allow(clippy::too_many_arguments)] // one call site; mirrors the sync loop's drain
fn present_ready(
codec: &MediaCodec,
client: &NativeClient,
measure_decode: bool,
ready: &mut Vec<OutputReady>,
stats: &crate::stats::VideoStats,
in_flight: &Mutex<VecDeque<(u64, i128)>>,
clock_offset: i64,
tracker: &DisplayTracker,
rendered: &mut u64,
discarded: &mut u64,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) {
if ready.is_empty() {
return;
}
// Pair each output's decode stage (feeds the ABR decode signal always; the HUD histogram only
// while visible) — both consume the receipt map, so enter for either.
if stats.enabled() || measure_decode {
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
note_decoded_pts(
client,
measure_decode,
stats,
&mut g,
clock_offset,
o.pts_us,
o.decoded_ns,
);
}
}
// Fold EVERY output through the gate in pts (== decode) order — even the ones newest-wins discards —
// so the two-mark re-anchor count stays correct; the newest's verdict decides whether it reaches
// glass (`false` = withheld concealment; the SurfaceView keeps the last rendered frame frozen on).
let now = Instant::now();
let last = ready.len() - 1;
let mut skipped: u64 = 0;
for (i, o) in ready.drain(..).enumerate() {
let flags = take_flags(recovery_flags, o.pts_us);
let present = gate.on_decoded(flags, false, now) == GateVerdict::Present;
let render = i == last && present;
match codec.release_output_buffer_by_index(o.index, render) {
Ok(()) if render => {
*rendered += 1;
if stats.enabled() {
tracker.note_rendered(o.pts_us, o.decoded_ns);
}
}
Ok(()) => {
*discarded += 1;
skipped += 1;
}
Err(e) => {
log::warn!(
"decode: release_output_buffer_by_index({}, {render}): {e}",
o.index
)
}
}
}
stats.note_skipped(skipped); // HUD `skipped` counter (newest-wins + held-off drops); no-op hidden
}
@@ -0,0 +1,224 @@
//! Display/frame-rendered tracking, render-callback registration, HDR dataspace mapping.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::MediaCodec;
use ndk::native_window::NativeWindow;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicI64, Ordering};
use std::sync::{Arc, Mutex};
use super::latency::now_realtime_ns;
use super::RENDERED_CAP;
/// `CLOCK_MONOTONIC` now in nanoseconds — the base of the `systemNano` render timestamp the
/// `OnFrameRendered` callback reports (Android's `System.nanoTime`), read only to re-base that
/// stamp onto `CLOCK_REALTIME` (see [`on_frame_rendered`]).
fn now_monotonic_ns() -> i128 {
let mut ts = libc::timespec {
tv_sec: 0,
tv_nsec: 0,
};
// SAFETY: `clock_gettime` with a valid out-pointer is an always-safe syscall.
unsafe { libc::clock_gettime(libc::CLOCK_MONOTONIC, &mut ts) };
ts.tv_sec as i128 * 1_000_000_000 + ts.tv_nsec as i128
}
/// State shared between the decode loop and the `AMediaCodec` `OnFrameRendered` callback (which
/// fires on a codec-internal thread): rendered frames awaiting their render timestamp, so the HUD
/// gets the spec's `display` stage (decoded→displayed) and the `capture→displayed` end-to-end
/// headline (`design/stats-unification.md` — this replaces Android's v1 `capture→decoded`
/// endpoint whenever the platform delivers render callbacks).
pub(super) struct DisplayTracker {
stats: Arc<crate::stats::VideoStats>,
/// Live host-minus-client clock offset (ns) for the skew-corrected end-to-end sample —
/// loaded per callback so mid-stream re-syncs apply. Holding the handle (not the client)
/// keeps the leaked render-callback refcount from pinning the whole session alive.
clock_offset: Arc<AtomicI64>,
/// `(pts_us, decoded_real_ns)` of frames released with `render = true`, in release order,
/// awaiting their callback. Pushes are HUD-gated by the caller, so this stays empty (and the
/// callback early-outs) while the overlay is hidden.
rendered: Mutex<VecDeque<(u64, i128)>>,
}
impl DisplayTracker {
pub(super) fn new(
stats: Arc<crate::stats::VideoStats>,
clock_offset: Arc<AtomicI64>,
) -> Arc<DisplayTracker> {
Arc::new(DisplayTracker {
stats,
clock_offset,
rendered: Mutex::new(VecDeque::new()),
})
}
/// Park one just-rendered frame's `(pts, decoded stamp)` for the render callback to pair.
/// Caller gates on the HUD being visible.
pub(super) fn note_rendered(&self, pts_us: u64, decoded_ns: i128) {
let mut g = self
.rendered
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.push_back((pts_us, decoded_ns));
if g.len() > RENDERED_CAP {
g.pop_front(); // render callbacks stopped coming (allowed under load) — evict
}
}
}
/// Register [`on_frame_rendered`] on the codec (`AMediaCodec_setOnFrameRenderedCallback`,
/// **API 33** — "Available since Android T" per the NDK header; only the *Java* listener dates
/// back further). That sits above the API-28 floor, so the entry point is dlsym-resolved at
/// runtime like [`try_set_frame_rate`] — hard-linking it (as 0.9.0 shipped) made
/// `System.loadLibrary` fail on every pre-Android-13 device, taking down all of `NativeBridge`.
/// The `ndk` wrapper has no binding and the call needs the raw codec pointer, which is what the
/// vendored crate's public `as_ptr` patch is for. Returns the userdata pointer holding a leaked
/// `Arc<DisplayTracker>` refcount; the caller MUST reclaim it with [`release_render_callback`]
/// AFTER dropping the codec (`AMediaCodec_delete` is what guarantees no further callback can
/// fire). `None` (nothing to reclaim) if the symbol is absent (API < 33) or the platform refused —
/// the HUD then simply has no `display` stage, exactly the pre-callback behaviour.
pub(super) fn install_render_callback(
codec: &MediaCodec,
tracker: &Arc<DisplayTracker>,
) -> Option<*const DisplayTracker> {
// media_status_t AMediaCodec_setOnFrameRenderedCallback(
// AMediaCodec*, AMediaCodecOnFrameRendered, void*) (API 33)
type SetOnFrameRenderedFn = unsafe extern "C" fn(
*mut ndk_sys::AMediaCodec,
ndk_sys::AMediaCodecOnFrameRendered,
*mut c_void,
) -> ndk_sys::media_status_t;
// SAFETY: `dlopen` of `libmediandk.so`, which the `ndk` media wrapper already links — always
// mapped, so this only bumps its refcount (never closed — process-lifetime handle). `dlsym`
// returns null when the symbol is absent (device below API 33), checked before transmuting the
// non-null pointer to its fn-pointer type.
let set_on_frame_rendered = unsafe {
let lib = libc::dlopen(c"libmediandk.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return None;
}
let sym = libc::dlsym(lib, c"AMediaCodec_setOnFrameRenderedCallback".as_ptr());
if sym.is_null() {
log::info!("decode: no render callback on this API level (<33) — no display stage");
return None;
}
std::mem::transmute::<*mut c_void, SetOnFrameRenderedFn>(sym)
};
let ud = Arc::into_raw(tracker.clone());
// SAFETY: `codec.as_ptr()` is the live codec this thread owns; `ud` outlives the registration
// (reclaimed only after the codec is deleted, per this function's contract).
let status = unsafe {
set_on_frame_rendered(codec.as_ptr(), Some(on_frame_rendered), ud as *mut c_void)
};
if status == ndk_sys::media_status_t::AMEDIA_OK {
Some(ud)
} else {
log::warn!("decode: setOnFrameRenderedCallback failed ({status:?}) — no display stage");
// SAFETY: registration failed, so the codec never took the reference — reclaim it now.
unsafe { drop(Arc::from_raw(ud)) };
None
}
}
/// Reclaim [`install_render_callback`]'s leaked `Arc` refcount.
///
/// # Safety
/// Call exactly once, and only after the codec the callback was registered on has been dropped —
/// deleting the codec stops its internal threads, so no callback can still be running (or run
/// later) against this pointer.
pub(super) unsafe fn release_render_callback(ud: *const DisplayTracker) {
drop(Arc::from_raw(ud));
}
/// The `AMediaCodecOnFrameRendered` trampoline: fires (possibly batched) on a codec-internal
/// thread once per output frame actually placed on the output surface, with SurfaceFlinger's
/// render timestamp. That timestamp (`system_nano`) is on `CLOCK_MONOTONIC`, so it is re-based
/// onto `CLOCK_REALTIME` here — against monotonic-now at callback time, which also cancels any lag
/// between the frame rendering and the (batchable) callback delivery — to subtract against the
/// receipt/decode stamps and the host capture pts. Records the HUD's `displayed` point:
/// `end-to-end` = capture→displayed (skew-corrected) and `display` = decoded→displayed
/// (single-clock local). Panic-free by construction (poison-proof lock, saturating math) — an
/// unwind out of an `extern "C"` fn would abort the process.
unsafe extern "C" fn on_frame_rendered(
_codec: *mut ndk_sys::AMediaCodec,
userdata: *mut c_void,
media_time_us: i64,
system_nano: i64,
) {
let t = &*(userdata as *const DisplayTracker);
if !t.stats.enabled() {
return; // HUD hidden — the ring is empty too (pushes are caller-gated)
}
let displayed_ns = now_realtime_ns() - (now_monotonic_ns() - system_nano as i128);
let pts_us = media_time_us.max(0) as u64;
// Pair the frame back to its release record, evicting older entries (their callbacks were
// dropped by the platform, or the entry predates a HUD toggle) — same monotonic-eviction
// discipline as `note_decoded_pts`.
let mut decoded_ns = None;
{
let mut g = t
.rendered
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
while let Some(&(p, d)) = g.front() {
if p > pts_us {
break; // future frame — leave it for its own callback
}
g.pop_front();
if p == pts_us {
decoded_ns = Some(d);
break;
}
}
}
let e2e_ns =
displayed_ns + t.clock_offset.load(Ordering::Relaxed) as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
let display_us = decoded_ns.map(|d| ((displayed_ns - d).max(0) / 1000) as u64);
t.stats.note_displayed(e2e_us, display_us);
}
/// React to an output-format change by signalling the stream's HDR dataspace on the Surface (SDR
/// streams leave the default alone). The AMediaCodec analogue of the sync loop's `OutputFormatChanged`
/// handling; safe to call repeatedly (`applied_ds` dedups).
pub(super) fn apply_hdr_dataspace(
codec: &MediaCodec,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
) {
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
Ok(()) => {
*applied_ds = Some(ds);
log::info!("decode: HDR stream → Surface dataspace {ds}");
}
Err(e) => {
log::warn!("decode: set_buffers_data_space({ds}) failed (non-fatal): {e}")
}
}
}
}
}
/// Map the decoder's reported output colour to a BT.2020 HDR dataspace, or `None` for SDR. The
/// integer values are the Android MediaFormat colour constants the NDK shares: COLOR_TRANSFER
/// ST2084 = 6 (PQ/HDR10), HLG = 7; COLOR_RANGE FULL = 1, LIMITED = 2 (the host encodes limited).
pub(super) fn hdr_dataspace(codec: &MediaCodec) -> Option<DataSpace> {
let fmt = codec.output_format();
let full_range = fmt.i32("color-range") == Some(1);
match fmt.i32("color-transfer") {
Some(6) => Some(if full_range {
DataSpace::Bt2020Pq
} else {
DataSpace::Bt2020ItuPq
}),
Some(7) => Some(if full_range {
DataSpace::Bt2020Hlg
} else {
DataSpace::Bt2020ItuHlg
}),
_ => None, // SDR (BT.709 / SDR_VIDEO) or unspecified
}
}
@@ -0,0 +1,83 @@
//! Decode-latency bookkeeping: realtime clock + decoded-pts / user-flags stat recording.
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
/// Wall-clock now in nanoseconds (CLOCK_REALTIME basis), to compare against the host-stamped
/// capture `pts_ns` after the skew offset is applied.
pub(super) fn now_realtime_ns() -> i128 {
use std::time::{SystemTime, UNIX_EPOCH};
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos() as i128)
.unwrap_or(0)
}
/// HUD `decoded` point for one dequeued output frame, keyed by the echoed `presentationTimeUs`:
/// build the end-to-end (capture→decoded, skew-corrected, clamped to (0, 10 s)) and `decode`
/// (received→decoded, single-clock local, ≥ 0) samples and hand them to
/// [`crate::stats::VideoStats::note_decoded`]. The pts keys the receipt stamp in `in_flight`;
/// entries older than it are evicted (decode order == input order here — low-latency, no
/// B-frames — so anything before it was dropped inside the codec or stamped before a flush).
/// `decoded_ns` is the availability instant: the dequeue (sync loop) or the output callback's
/// stamp (async loop).
pub(super) fn note_decoded_pts(
client: &NativeClient,
measure_decode: bool,
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
pts_us: u64,
decoded_ns: i128,
) {
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
while let Some(&(p, r)) = in_flight.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
in_flight.pop_front();
if p == pts_us {
received_ns = Some(r);
break;
}
}
let decode_us = received_ns.map(|r| ((decoded_ns - r).max(0) / 1000) as u64);
// Adaptive bitrate: the `decode` stage (received→decoded, single-clock local) IS the decoder-
// backlog signal — the only bottleneck the host-side network signals can't see (a fast LAN
// feeding a slower mobile decoder). Report it whenever the controller is armed, regardless of
// the HUD; `report_decode_us` is a cheap accumulate the pump windows.
if measure_decode {
if let Some(us) = decode_us {
client.report_decode_us(us.min(u32::MAX as u64) as u32);
}
}
// HUD histogram: only while the overlay is visible (a measure-only caller enters here for the
// ABR report alone). `end-to-end` = capture→decoded (skew-corrected) tiles the `decode` stage.
// pts_us is the truncated frame.pts_ns/1000 we queued, so ×1000 re-approximates capture time to
// < 1 µs — negligible against the ms-scale figures shown.
if stats.enabled() {
let e2e_ns = decoded_ns + clock_offset as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
stats.note_decoded(e2e_us, decode_us);
}
}
/// The AU `user_flags` for a decoded output, keyed by the echoed `presentationTimeUs`. Recovery
/// signalling (FLAG_SOF IDR marker / RECOVERY_ANCHOR / RECOVERY_POINT) rides the AU's flags, which are
/// only in scope at feed time — so the feed side parks `(pts_us, flags)` here and the present side
/// looks them up to fold [`ReanchorGate::on_decoded`]. Decode order == input order (low-latency, no
/// B-frames), so this evicts entries older than `pts_us` as it goes; a miss (probe filler, or an entry
/// aged past the cap) reads `0` — no recovery flags, decoded normally.
pub(super) fn take_flags(map: &mut VecDeque<(u64, u32)>, pts_us: u64) -> u32 {
while let Some(&(p, f)) = map.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
map.pop_front();
if p == pts_us {
return f;
}
}
0
}
+84
View File
@@ -0,0 +1,84 @@
//! Android video decode (android-only): pull HEVC access units from the connector and render them
//! to the SurfaceView via NDK `AMediaCodec` — hardware decode, zero per-frame JNI.
//!
//! One-in/one-out: the host opens every stream with an IDR carrying VPS/SPS/PPS **in-band**, so the
//! decoder needs no out-of-band codec-specific data — we configure with mime + the negotiated
//! WxH (from [`NativeClient::mode`]) and feed each access unit as it arrives. The decode thread owns
//! the codec + window for its whole life; [`crate::session`] signals it to stop via the shared flag.
mod async_loop;
mod display;
mod latency;
mod setup;
mod sync_loop;
use async_loop::run_async;
pub(crate) use setup::{codec_label, codec_mime};
use sync_loop::run_sync;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
/// Cap on AUs parked in the async loop awaiting a free codec input slot. Matches the connector's
/// own frame-channel depth; on sustained overflow the oldest is dropped and a keyframe requested
/// (same recovery as a reassembler drop). In steady state this stays near-empty.
const FRAME_PARK_CAP: usize = 16;
/// Cap on the pts→received-timestamp map below: MediaCodec holds only a handful of frames in
/// flight, so anything beyond this is stale (codec flushed / HUD toggled) and gets evicted.
const IN_FLIGHT_CAP: usize = 64;
/// Cap on received AUs awaiting their 0xCF host timing (Phase 2 host/network split): the timing
/// datagram trails its AU by at most the wire, so a match lands within a frame or two — anything
/// this deep is a lost datagram (or an old host that never sends any) and gets evicted.
const PENDING_SPLIT_CAP: usize = 256;
/// Cap on rendered frames parked in [`DisplayTracker`] awaiting their `OnFrameRendered` render
/// timestamp: the callback trails its release by at most a vsync or two, so anything this deep
/// means the platform stopped delivering render callbacks (allowed under load, per the docs) and
/// gets evicted.
const RENDERED_CAP: usize = 64;
/// Whether low-latency mode uses the event-driven async decode loop (default) or the synchronous
/// poll loop. Flip to `false` to A/B the two on the HUD (`design/…`); the async loop presents a
/// decoded frame the instant it's ready instead of waiting out a poll interval. Only consulted when
/// the user's "Low-latency mode" toggle is ON (now the default) — off, the sync loop always runs (the
/// original pipeline, kept as the per-device escape hatch).
const USE_ASYNC_DECODE: bool = true;
/// Per-session decode configuration, resolved by the JNI layer (`nativeStartVideo`) and passed to
/// the decode loop. Bundled so the loop entry points don't sprout a wide argument list.
pub(crate) struct DecodeOptions {
/// The decoder Kotlin ranked from `MediaCodecList` (`VideoDecoders.pickDecoder`). `None`/empty ⇒
/// let the platform resolve the default decoder for the MIME.
pub decoder_name: Option<String>,
/// Whether Kotlin found the chosen decoder advertises `FEATURE_LowLatency` (queryable only via
/// the Java `CodecCapabilities` API) — surfaced on the HUD next to the decoder name.
pub ll_feature: bool,
/// The user's "Low-latency mode" master toggle. On (default) ⇒ the full fast pipeline: async
/// decode loop, per-SoC vendor keys, pipeline thread boosts, ADPF max-performance, forced TV
/// mode switch. Off ⇒ the original synchronous pre-overhaul pipeline, kept as the per-device
/// escape hatch.
pub low_latency_mode: bool,
/// TV form factor (Kotlin's `UiModeManager`): actively drive the HDMI output into the stream's
/// refresh mode, vs. the softer seamless hint on a phone/tablet.
pub is_tv: bool,
}
/// The decode entry point on the `pf-decode` thread: dispatches to the async or synchronous loop.
/// Both run until `shutdown` is set or the session closes.
pub fn run(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
if opts.low_latency_mode && USE_ASYNC_DECODE {
run_async(client, window, shutdown, stats, opts);
} else {
run_sync(client, window, shutdown, stats, opts);
}
}
+254
View File
@@ -0,0 +1,254 @@
//! Codec creation, low-latency config, thread/frame-rate tuning, HDR static-info encode.
use ndk::media::media_codec::MediaCodec;
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use std::ffi::c_void;
/// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). Shared by the decode
/// thread and `nativeVideoMime` (which tells Kotlin what to rank decoders for). AV1 uses the
/// AOSP `video/av01` type; anything not H.264/AV1 is treated as HEVC (every pre-negotiation host
/// emitted HEVC).
pub(crate) fn codec_mime(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "video/avc",
punktfunk_core::quic::CODEC_AV1 => "video/av01",
_ => "video/hevc",
}
}
/// A short human label for the codec the host resolved, for the stats HUD's video-feed line
/// (`"H.264"` / `"HEVC"` / `"AV1"` / `"PyroWave"`). Mirrors [`codec_mime`]'s fallback: anything
/// not H.264/AV1/PyroWave is reported as HEVC (every pre-negotiation host emitted HEVC). Kept
/// beside [`codec_mime`] because the MIME collapses PyroWave onto `video/hevc` and so can't name it.
pub(crate) fn codec_label(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "H.264",
punktfunk_core::quic::CODEC_AV1 => "AV1",
punktfunk_core::quic::CODEC_PYROWAVE => "PyroWave",
_ => "HEVC",
}
}
/// Create the decoder: prefer the specific codec Kotlin ranked from `MediaCodecList`
/// (`from_codec_name`), falling back to the platform's default decoder for the MIME
/// (`from_decoder_type`) if that name can't be created (codec busy / renamed across an OS update).
pub(super) fn create_codec(mime: &str, preferred: Option<&str>) -> Option<MediaCodec> {
if let Some(name) = preferred.filter(|n| !n.is_empty()) {
if let Some(c) = MediaCodec::from_codec_name(name) {
return Some(c);
}
log::warn!(
"decode: from_codec_name({name}) failed — falling back to default {mime} decoder"
);
}
MediaCodec::from_decoder_type(mime)
}
/// Apply the low-latency MediaFormat keys for `codec_name`.
///
/// `aggressive` = the "Low-latency mode" master toggle. **Off** ⇒ the pre-overhaul key set,
/// byte-for-byte — the standard `low-latency` key, the blind Qualcomm vendor twin, `priority = 0` AND
/// `operating-rate = MAX` set together — kept as the per-device escape hatch (the profile every device
/// streamed with before the overhaul). **On** (default) ⇒ the Moonlight-parity
/// profile: MediaTek's `vdec-lowlatency` (unconditionally — ignored off MediaTek), the per-SoC
/// vendor extension keys (gated on the decoder-name prefix the way Moonlight-Android does, since a
/// key one vendor honours is meaningless on another), and one *mutually exclusive* clock hint.
///
/// Vendor keys mirror Moonlight's `MediaCodecHelper` (verified against current source): Qualcomm
/// picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon, MediaTek. NVIDIA
/// Tegra / Rockchip / Realtek expose no such key (nor does Moonlight) — they're covered by the
/// standard key + clock hint + being ranked first in `VideoDecoders`.
pub(super) fn configure_low_latency(format: &mut MediaFormat, codec_name: &str, aggressive: bool) {
// Standard key: request the no-reorder low-latency path where the platform decoder supports it.
format.set_i32("low-latency", 1);
if !aggressive {
// The original profile: the Qualcomm vendor twin set blind (unknown keys are ignored by
// other vendors' codecs), realtime priority, and the AOSP "unbounded" operating-rate
// sentinel — decode each frame at max clocks rather than pacing to the frame rate.
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
format.set_i32("priority", 0); // 0 = realtime
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
return;
}
// MediaTek's low-latency key — very common (mid/budget phones + many Google TV / Fire TV boxes).
// Set unconditionally like the standard key: MediaTek decoders honour it, others ignore it, so it
// covers MediaTek whatever the exact decoder name (omx.mtk / c2.mtk / an OEM rename). Moonlight
// does the same, and also relies on it for Amazon's Amlogic fork.
format.set_i32("vdec-lowlatency", 1);
let name = codec_name.to_ascii_lowercase();
let is = |prefix: &str| name.starts_with(prefix);
// Qualcomm Snapdragon (the most common phone SoC): picture-order forces decode-order output
// (kills the reorder buffer on decoders that predate the standard key); low-latency is the older
// vendor twin.
if is("omx.qcom") || is("c2.qti") {
format.set_i32("vendor.qti-ext-dec-picture-order.enable", 1);
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
}
// Samsung Exynos — also covers Google Tensor (Pixel 6+), whose hardware decoder is `c2.exynos.*`.
if is("omx.exynos") || is("c2.exynos") {
format.set_i32("vendor.rtc-ext-dec-low-latency.enable", 1);
}
// Amlogic — the Android TV boxes (onn 4K, Chromecast w/ Google TV, Homatics).
if is("omx.amlogic") || is("c2.amlogic") {
format.set_i32("vendor.low-latency.enable", 1);
}
// HiSilicon / Kirin (older Huawei; paired req/rdy keys).
if is("omx.hisi") || is("c2.hisi") {
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-req",
1,
);
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-rdy",
-1,
);
}
// NVIDIA Tegra (Shield TV) and Rockchip/Realtek (budget TV boxes / smart TVs) expose no
// low-latency vendor key (Moonlight has none either) — their decoders are already low-latency
// oriented, so the standard `low-latency` key + the clock hint below + being ranked first
// (see `VideoDecoders`) is their treatment.
//
// Clock hint, mutually exclusive (matching Moonlight): the AOSP "unbounded" operating-rate
// sentinel (Short.MAX) tells the decoder to run each frame at max clocks and finish ASAP rather
// than pace to the frame rate — shaving per-frame decode latency at a power/heat cost. Only
// Qualcomm is known to handle the sentinel; every other vendor mis-paces on it, so they get the
// plain realtime `priority` hint instead.
if decoder_supports_max_operating_rate(&name) {
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
} else {
format.set_i32("priority", 0); // 0 = realtime
}
}
/// Whether a decoder tolerates `operating-rate = Short.MAX` rather than regressing on it. Follows
/// Moonlight's allowlist: Qualcomm decoders honour the sentinel (the Adreno 620 generation is the
/// known exception Moonlight excludes by GPU model — undetectable from native code here, so it
/// rides the master toggle as its escape hatch). Other vendors fall back to the plain `priority`
/// hint above.
fn decoder_supports_max_operating_rate(name_lower: &str) -> bool {
name_lower.starts_with("omx.qcom") || name_lower.starts_with("c2.qti")
}
/// Raise the pipeline's OTHER hot threads — the core's data-plane pump (UDP receive + FEC
/// reassembly) and the audio decode thread — toward the display band, matching this decode thread's
/// own boost. `setpriority(PRIO_PROCESS, tid)` targets any task in the process, so we do it from
/// here once their tids are known (the same set ADPF hints), without a per-platform priority hook
/// in the shared core. Slightly below the decode thread's -10 so the display path still wins.
/// Best-effort; skips this thread (already boosted) and is non-fatal if the platform refuses.
pub(super) fn boost_hot_threads(tids: &[i32]) {
// SAFETY: `gettid` is an always-safe syscall on the calling thread.
let self_tid = unsafe { libc::gettid() };
for &tid in tids {
if tid == self_tid {
continue;
}
// SAFETY: `setpriority` with PRIO_PROCESS + a live tid in our own process is an always-safe
// syscall; a refusal is reported via the return value, not UB.
unsafe {
if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -8) != 0 {
log::debug!("decode: setpriority(-8) on hot tid {tid} failed (non-fatal)");
}
}
}
}
/// Best-effort: raise the decode thread toward Android's URGENT_DISPLAY band so background work
/// can't preempt it under load (which shows up as late/dropped frames). Non-fatal if the platform
/// refuses (foreground apps may set their own threads; the exact floor is policy-dependent).
pub(super) fn boost_thread_priority() {
// SAFETY: `gettid`/`setpriority` on the calling thread are always-safe syscalls. PRIO_PROCESS
// with a TID targets that one task on Linux — the same idiom `Process.setThreadPriority` uses.
unsafe {
let tid = libc::gettid();
if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -10) != 0 {
log::warn!(
"decode: setpriority(-10) failed (non-fatal): {}",
std::io::Error::last_os_error()
);
}
}
}
/// Set the surface's frame-rate hint to the stream's refresh so SurfaceFlinger picks a matching
/// display mode and aligns vsync (no 60-in-120 judder). Both NDK entry points sit above our API-28
/// floor, so both are dlsym-resolved at runtime (a hard import of a >floor symbol makes
/// `dlopen`/`System.load` fail on every API-28/29 device, even where this path is never hit —
/// mirrors [`crate::adpf`]):
/// - On a **TV** (`is_tv`): `ANativeWindow_setFrameRateWithChangeStrategy` (**API 31**) with
/// `changeFrameRateStrategy = ALWAYS`, which actively drives the HDMI output into the matching
/// mode (e.g. 60↔120) instead of leaving the panel at its default and judder-matching. The
/// forced switch may blank the panel briefly — acceptable once at stream start, not wanted on a
/// phone. Falls through to the 2-arg hint on API 30.
/// - Otherwise: `ANativeWindow_setFrameRate` (**API 30**) with `compatibility = DEFAULT` — the
/// softer, seamless-preferred hint for phones/tablets and the universal fallback.
///
/// Returns `true` when the platform accepted a hint; `false` on API < 30 (symbols absent) or a
/// decline.
pub(super) fn try_set_frame_rate(window: &NativeWindow, frame_rate: f32, is_tv: bool) -> bool {
// int32_t ANativeWindow_setFrameRate(ANativeWindow*, float frameRate, int8_t compatibility)
type SetFrameRateFn = unsafe extern "C" fn(*mut c_void, f32, i8) -> i32;
// int32_t ANativeWindow_setFrameRateWithChangeStrategy(
// ANativeWindow*, float frameRate, int8_t compatibility, int8_t changeFrameRateStrategy)
type SetFrameRateStrategyFn = unsafe extern "C" fn(*mut c_void, f32, i8, i8) -> i32;
// SAFETY: `dlopen` of the always-mapped `libandroid.so` (only bumps its refcount; never closed —
// process-lifetime handle). Each `dlsym` returns null when the symbol is absent (device below the
// symbol's API level), checked before transmuting the non-null pointer to its fn-pointer type.
// `window.ptr()` is the live `ANativeWindow` this `NativeWindow` owns for the call's duration.
unsafe {
let lib = libc::dlopen(c"libandroid.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return false;
}
// TV: prefer the API-31 change-strategy form to force the mode switch (strategy 1 = ALWAYS,
// compatibility 0 = DEFAULT). Absent on API 30 ⇒ fall through to the 2-arg hint below.
if is_tv {
let sym = libc::dlsym(
lib,
c"ANativeWindow_setFrameRateWithChangeStrategy".as_ptr(),
);
if !sym.is_null() {
let set = std::mem::transmute::<*mut c_void, SetFrameRateStrategyFn>(sym);
return set(window.ptr().as_ptr().cast(), frame_rate, 0, 1) == 0;
}
}
let sym = libc::dlsym(lib, c"ANativeWindow_setFrameRate".as_ptr());
if sym.is_null() {
return false; // device API < 30 — no per-surface frame-rate hint
}
let set_frame_rate = std::mem::transmute::<*mut c_void, SetFrameRateFn>(sym);
set_frame_rate(window.ptr().as_ptr().cast(), frame_rate, 0) == 0
}
}
/// Serialize [`HdrMeta`](punktfunk_core::quic::HdrMeta) into Android's `KEY_HDR_STATIC_INFO`
/// (`hdr-static-info`) layout: a 25-byte CTA-861.3 / `HDRStaticInfo.Type1` blob — descriptor id 0,
/// then primaries in **R, G, B** order, white point, max/min display luminance, MaxCLL, MaxFALL, all
/// **little-endian** `u16`. Two conversions vs our wire form: HdrMeta stores primaries in ST.2086
/// **G, B, R** order (reorder to R, G, B), and `max_display_mastering_luminance` is in 0.0001-cd/m²
/// units while Android wants **whole nits** (min stays 0.0001-nit). Chromaticities (1/50000) and
/// MaxCLL/MaxFALL (nits) match 1:1.
pub(super) fn android_hdr_static_info(m: &punktfunk_core::quic::HdrMeta) -> [u8; 25] {
let [g, b_, r] = m.display_primaries; // ST.2086 G, B, R
let max_nits = (m.max_display_mastering_luminance / 10_000).min(u16::MAX as u32) as u16;
let min_units = m.min_display_mastering_luminance.min(u16::MAX as u32) as u16;
let fields: [u16; 12] = [
r[0],
r[1],
g[0],
g[1],
b_[0],
b_[1], // R, G, B primaries
m.white_point[0],
m.white_point[1], // white point
max_nits,
min_units, // max (nits) / min (0.0001-nit) display luminance
m.max_cll,
m.max_fall, // MaxCLL / MaxFALL (nits)
];
let mut out = [0u8; 25]; // out[0] = 0 (Type 1 descriptor id), already zero
for (i, v) in fields.iter().enumerate() {
out[1 + i * 2..3 + i * 2].copy_from_slice(&v.to_le_bytes());
}
out
}
@@ -0,0 +1,547 @@
//! The synchronous MediaCodec decode loop (the original poll path) + its feed/drain helpers.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{
DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection,
OutputBuffer,
};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::reanchor::{GateVerdict, ReanchorGate};
use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
use super::display::{
hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags};
use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
configure_low_latency, create_codec, try_set_frame_rate,
};
use super::{DecodeOptions, IN_FLIGHT_CAP, PENDING_SPLIT_CAP};
/// The synchronous poll loop — the original decode path: the only one when low-latency mode is off,
/// and the [`USE_ASYNC_DECODE`] A/B fallback when it's on. Feeds and drains on this one thread; the
/// only blocking wait is a short output dequeue while input is backed up.
pub(super) fn run_sync(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). AMediaCodec needs no
// out-of-band extradata — the in-band VPS/SPS/PPS on every IDR configure it either way.
let mime = codec_mime(client.codec);
let codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
// The decoder's *actual* resolved name (Kotlin's pick, or the platform default when it fell
// back) drives both the HUD label and which vendor low-latency keys apply below.
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (low-latency feature: {ll_feature})"
);
let mut format = MediaFormat::new();
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
// Generous input buffer so a large keyframe AU is never truncated.
format.set_i32(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
// Standard + per-SoC vendor low-latency keys and the clock hints, gated on the resolved decoder
// name and the master toggle (see `configure_low_latency`).
configure_low_latency(&mut format, &codec_name, low_latency_mode);
// HDR static metadata (ST.2086 mastering + content light level): when an HDR session was
// negotiated, set KEY_HDR_STATIC_INFO so the display tone-maps from the source's real grade.
// MediaCodec wants it BEFORE configure(), and the host sends a 0xCE right after the handshake,
// so it's typically already queued; wait briefly otherwise. The Surface DataSpace (applied on
// OutputFormatChanged below) carries transfer/primaries regardless — this adds the luminance the
// tone-mapper needs. A non-HDR display still gets sensible SurfaceFlinger tone-mapping.
if client.color.is_hdr() {
match client.next_hdr_meta(Duration::from_millis(250)) {
Ok(meta) => {
format.set_buffer("hdr-static-info", &android_hdr_static_info(&meta));
log::info!("decode: HDR static metadata applied (KEY_HDR_STATIC_INFO)");
}
Err(_) => {
log::info!("decode: HDR session but no mastering metadata yet — DataSpace only")
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
log::error!("decode: configure failed: {e}");
return;
}
if let Err(e) = codec.start() {
log::error!("decode: start failed: {e}");
return;
}
log::info!(
"decode: {mime} decoder started at {}x{}",
mode.width,
mode.height
);
// Tell the display the stream's refresh so Android can pick a matching display mode and align
// vsync (no 60-in-120 judder on high-refresh panels). `ANativeWindow_setFrameRate` is NDK API 30,
// above our API-28 floor, so we resolve it at runtime (see `try_set_frame_rate`) rather than link
// it — a hard import would stop `libpunktfunk_android.so` loading at all on API 28/29. Absent
// there ⇒ we simply skip the hint (non-fatal; the stream renders fine without it).
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
);
}
// ADPF: hint the platform that the whole video pipeline — this pf-decode feed/drain/present
// loop, the core's data-plane pump (UDP receive + FEC reassembly), and the audio thread — runs a
// per-frame real-time workload, so the CPU governor keeps those threads on fast cores at high
// clocks instead of down-clocking between frames or parking them on a little core. Snapdragon's
// ADPF backend responds well to this. We register this thread now but create the session lazily
// on the first presented frame: by then the pump + audio threads have registered their ids too,
// and ADPF `createSession` rejects a set with any not-yet-live/dead tid. No-op below API 33.
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread(); // this decode thread → the pipeline's hot-thread set
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
// Accumulates the loop's productive (feed+drain) time between displayed frames; reported to ADPF
// once per rendered frame against the frame-period target.
let mut work_accum_ns: i64 = 0;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
// AUs larger than the codec input buffer, dropped whole (see `feed`/`feed_ready`).
let mut oversized_dropped: u64 = 0;
// The AU waiting for a free codec input buffer. `feed` is non-blocking; on transient input
// pressure the AU stays parked here instead of being dropped (a drop forces a keyframe
// round-trip) and we only pop the next one once it's queued.
let mut pending: Option<Frame> = None;
// Freeze-until-reanchor: the shared post-loss gate ([`punktfunk_core::reanchor::ReanchorGate`]).
// Armed on a frame-index gap or a dropped-count climb, it withholds the decoder's concealed output
// (released WITHOUT rendering — the SurfaceView keeps the last rendered frame on glass) until a
// proven clean re-anchor lifts it: an IDR (wire FLAG_SOF), an RFI anchor, or the 2nd recovery mark.
// `last_kf_req` throttles the keyframe intents it emits; `recovery_flags` carries each AU's
// user_flags from feed to present (keyed by the codec-echoed pts) so `on_decoded` reads the
// re-anchor signalling the platform decoder doesn't expose.
let mut gate = ReanchorGate::new(client.frames_dropped());
let mut recovery_flags: VecDeque<(u64, u32)> = VecDeque::new();
let mut last_kf_req: Option<Instant> = None;
// Skew-corrected latency stats (spec: design/stats-unification.md) use the negotiated
// host-minus-client clock offset (0 if the host didn't answer the skew handshake — then the
// HUD flags it "(same-host clock)").
let clock_offset = client.clock_offset_shared();
// Display stage (spec `display` + the capture→displayed headline): frames released with
// render = true are parked in the tracker; the OnFrameRendered callback pairs them with
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
let tracker = DisplayTracker::new(stats.clone(), clock_offset.clone());
let render_cb = install_render_callback(&codec, &tracker);
// Receipt timestamps keyed by the pts we queue into the codec, so the decoded point (output-
// buffer dequeue — MediaCodec round-trips presentationTimeUs) can be paired back to its receipt
// for the `decode` stage. Fed while the HUD is visible OR the adaptive-bitrate controller wants
// the decode signal (`measure_decode`) — the decoder-backlog bottleneck the network can't see.
let measure_decode = client.wants_decode_latency();
let mut in_flight: VecDeque<(u64, i128)> = VecDeque::new();
// Phase-2 host/network split (design/stats-unification.md): received AUs awaiting their 0xCF
// host timing, as (pts_ns, capture→received µs). The timings are drained non-blockingly right
// where receipts are recorded and matched by pts; `network = hostnet host` (saturating).
// Only fed while the HUD is visible; an old host never sends a 0xCF, so entries just age out.
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
// The dataspace we've signalled on the Surface so far (None = default/SDR). Set reactively once
// the decoder reports an HDR stream (see `drain`); avoids re-applying every format event.
let mut applied_ds: Option<DataSpace> = None;
// One thread feeds AND drains: the NDK AMediaCodec wrapper isn't documented thread-safe for
// cross-thread feed/drain, so instead of splitting threads the loop decouples the two — input
// dequeue is non-blocking (never stalls presentation of already-decoded frames) and the only
// blocking wait is a short output dequeue while input is backed up (decoder progress is exactly
// what frees the next input buffer).
while !shutdown.load(Ordering::Relaxed) {
if pending.is_none() {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
// Loss recovery (RFI): feed the frame index so a forward gap fires a throttled
// reference-frame-invalidation request — an RFI-capable host (AMD LTR / NVENC)
// recovers with a cheap clean P-frame instead of a full IDR. The same forward gap
// arms the freeze gate so the decoder's concealment is held off the screen until the
// recovery re-anchors. The frames_dropped keyframe path below stays the backstop.
if client.note_frame_index(frame.frame_index) {
gate.arm(Instant::now());
}
// Park this AU's re-anchor flags for the present side (keyed by the pts the codec
// echoes on the output buffer) — unconditional, unlike the HUD's `in_flight` map.
recovery_flags.push_back((frame.pts_ns / 1000, frame.flags));
if recovery_flags.len() > IN_FLIGHT_CAP {
recovery_flags.pop_front();
}
if fed == 0 {
let p = &frame.data;
log::info!(
"decode: first AU {} bytes, head {:02x?}",
p.len(),
&p[..p.len().min(6)]
);
}
// Receipt stamp for the `decode` stage pairing, parked in `in_flight` (keyed by
// the pts the codec echoes on its output buffer) whenever it's needed: the HUD
// being visible, or the ABR decode signal (`measure_decode`). The HUD-only
// samplers (`received` point, host/network split) stay gated on the overlay so
// the hidden steady state adds only a wall-clock read + the receipt push.
if stats.enabled() || measure_decode {
let received_ns = now_realtime_ns();
in_flight.push_back((frame.pts_ns / 1000, received_ns));
if in_flight.len() > IN_FLIGHT_CAP {
in_flight.pop_front(); // stale — codec never echoed it back
}
// HUD stat, `received` point: host+network = client_now + (hostclient)
// capture_pts.
if stats.enabled() {
let clock_offset = clock_offset.load(Ordering::Relaxed);
let lat_ns = received_ns + clock_offset as i128 - frame.pts_ns as i128;
let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000)
.then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
// Phase-2 split: park this AU's capture→received sample, then match any
// 0xCF host timings that have arrived — host = the host's own
// capture→sent, network = our capture→received minus it (per-frame
// tiling; saturating in case of clock jitter).
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front(); // 0xCF lost / old host — evict
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) =
pending_split.iter().position(|&(p, _)| p == t.pts_ns)
{
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
}
pending = Some(frame);
}
Err(PunktfunkError::NoFrame) => {} // timeout — still drain output below
Err(_) => break, // session closed
}
}
// Time the productive work (feed + drain) only — the `next_frame` poll wait above is idle
// and excluded, so ADPF sees this thread's real per-frame CPU cost, not the poll timeout.
let work_t0 = Instant::now();
if let Some(frame) = pending.take() {
if feed(
&codec,
&client,
&frame.data,
frame.pts_ns / 1000,
&mut oversized_dropped,
) {
fed += 1;
if fed % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
} else {
// No input buffer free — transient back-pressure. Keep the AU and let `drain` block
// briefly below; a released output buffer is what recycles an input slot.
pending = Some(frame);
}
}
// Drain every iteration. When input is blocked, wait ~2 ms on output so the loop rides
// decoder progress instead of busy-spinning against a full input queue.
let wait = if pending.is_some() {
Duration::from_millis(2)
} else {
Duration::ZERO
};
let (r, d) = drain(
&codec,
&client,
measure_decode,
&window,
&mut applied_ds,
wait,
&stats,
&mut in_flight,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut gate,
&mut recovery_flags,
);
rendered += r;
discarded += d;
// ADPF: attribute this iteration's feed+drain time to the frame being produced, and report
// the accumulated per-frame work once one is actually presented (r > 0). Under back-pressure
// the short output-dequeue wait is included in the tally — for a latency-first client,
// biasing the governor toward "boost" is the desired behaviour. Cheap when `hint` is None
// (one `Instant` diff, no report).
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if r > 0 {
if !hint_tried {
// First presented frame: the pump + audio threads have registered their ids by now.
// Build one ADPF session over the whole pipeline's thread set (empty below API 33,
// or where the platform declines → `None`, and the loop runs unhinted).
hint_tried = true;
let tids = client.hot_thread_ids();
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
}
// Loss recovery + overdue backstop, folded through the gate. Under infinite GOP the only
// recovery keyframe is one we request; the reassembler drops unrecoverable AUs (frames_dropped)
// and the decoder then conceals the reference-missing deltas and renders them without error, so
// a decode-error trigger rarely fires — the gate arms the freeze on the drop-count climb
// instead. An overdue freeze (held REANCHOR_FREEZE_MAX with no clean re-anchor) re-asks while it
// keeps holding: never resume to gray — a dead stream is the QUIC idle-timeout watchdog's job.
let now = Instant::now();
if gate.poll(client.frames_dropped(), now)
&& last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = client.request_keyframe();
log::debug!("decode: requested keyframe (loss recovery / overdue re-anchor)");
}
}
let _ = codec.stop();
drop(codec); // AMediaCodec_delete — after this no render callback can fire
if let Some(ud) = render_cb {
// SAFETY: the codec was dropped above; this registration's single reclaim.
unsafe { release_render_callback(ud) };
}
log::info!("decode: stopped (fed={fed} rendered={rendered} discarded={discarded})");
}
/// Try to copy one access unit into a codec input buffer and queue it, without blocking. Returns
/// `false` only on `TryAgainLater` (no input buffer free) — the caller keeps the AU pending and
/// retries; a hard dequeue/queue error counts as consumed (retrying can't salvage the AU, and
/// parking it forever would wedge the loop on a broken codec). An AU larger than the input
/// buffer is DROPPED (+ a recovery keyframe requested), never truncated — a truncated AU is
/// corrupt input the decoder chews on silently, poisoning the reference chain.
fn feed(
codec: &MediaCodec,
client: &NativeClient,
au: &[u8],
pts_us: u64,
oversized_dropped: &mut u64,
) -> bool {
match codec.dequeue_input_buffer(Duration::ZERO) {
Ok(DequeuedInputBufferResult::Buffer(mut buf)) => {
let n = {
let dst = buf.buffer_mut();
if au.len() > dst.len() {
*oversized_dropped += 1;
log::warn!(
"decode: AU {} > input buffer {} — dropped ({} so far), requesting keyframe",
au.len(),
dst.len(),
*oversized_dropped
);
let _ = client.request_keyframe();
0 // return the slot with zero valid bytes — a no-op input, not corrupt data
} else {
let n = au.len();
// SAFETY: `au` and `dst` are distinct allocations (wire AU vs. codec buffer),
// both valid for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so
// the cast write initializes exactly `dst[..n]`.
unsafe {
std::ptr::copy_nonoverlapping(
au.as_ptr(),
dst.as_mut_ptr().cast::<u8>(),
n,
);
}
n
}
};
if let Err(e) = codec.queue_input_buffer(buf, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer: {e}");
}
true
}
Ok(DequeuedInputBufferResult::TryAgainLater) => false, // caller keeps the AU pending
Err(e) => {
log::warn!("decode: dequeue_input_buffer: {e}");
true
}
}
}
/// Dequeue every ready output buffer and present only the NEWEST (render = true), discarding the
/// rest (render = false) — when decode falls behind, a back-to-back burst of stale frames on glass
/// is worse than skipping straight to the freshest one (the Apple client's 1-slot newest-ready
/// ring, ported). `first_wait` is the timeout for the first dequeue only: zero normally, ~2 ms when
/// the caller's input is blocked so the loop waits on decoder progress instead of busy-spinning.
/// Returns `(rendered, discarded)`. Also reacts to `OutputFormatChanged` (which can interleave
/// between buffers — handled without losing the held buffer) to signal HDR on the Surface.
///
/// Each dequeued buffer is also the HUD's `decoded` measurement point (rendered or not — the frame
/// finished decoding either way): end-to-end = decoded + clock_offset capture pts, and the
/// `decode` stage pairs the buffer's echoed presentationTimeUs back to the receipt stamp in
/// `in_flight` (single-clock local difference, no skew involved). The presented frame's
/// `(pts, decoded stamp)` is additionally parked in `tracker` for the OnFrameRendered callback —
/// the `display` stage's other endpoint.
#[allow(clippy::too_many_arguments)] // one call site; mirrors the async loop's present_ready
fn drain(
codec: &MediaCodec,
client: &NativeClient,
measure_decode: bool,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
first_wait: Duration,
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
tracker: &DisplayTracker,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) -> (u64, u64) {
// Newest ready buffer so far (presented after the loop) with its HUD metadata —
// `Some((pts_us, decoded_ns))` only while the HUD is visible. `held_present` is the freeze gate's
// verdict for that newest buffer (`false` = a post-loss concealment to withhold).
let mut held: Option<(OutputBuffer<'_>, Option<(u64, i128)>)> = None;
let mut held_present = true;
let mut discarded: u64 = 0;
let mut wait = first_wait;
loop {
match codec.dequeue_output_buffer(wait) {
Ok(DequeuedOutputBufferInfoResult::Buffer(buf)) => {
// Only the first dequeue may block; later ones poll (wait == ZERO).
wait = Duration::ZERO;
// Fold every dequeued frame through the gate in pts (== decode) order — even the ones
// the newest-wins policy discards — so the two-mark re-anchor count stays correct; the
// verdict of the newest (last folded) buffer decides whether it reaches glass.
let pts_us = buf.info().presentation_time_us().max(0) as u64;
let flags = take_flags(recovery_flags, pts_us);
held_present =
gate.on_decoded(flags, false, Instant::now()) == GateVerdict::Present;
let meta = if stats.enabled() || measure_decode {
// The dequeue IS the sync loop's decoded-availability instant.
let decoded_ns = now_realtime_ns();
note_decoded_pts(
client,
measure_decode,
stats,
in_flight,
clock_offset,
pts_us,
decoded_ns,
);
// The tracker's `display` stage is a HUD concern — park only when visible.
stats.enabled().then_some((pts_us, decoded_ns))
} else {
None
};
if let Some((stale, _)) = held.replace((buf, meta)) {
// A newer frame is ready — drop the held one without rendering.
if let Err(e) = codec.release_output_buffer(stale, false) {
log::warn!("decode: release_output_buffer(discard): {e}");
}
discarded += 1;
stats.note_skipped(1); // HUD `skipped` counter; no-op while hidden
}
}
Ok(DequeuedOutputBufferInfoResult::OutputFormatChanged) => {
// The decoder has parsed the SPS and now reports the stream's real colour signalling
// (the AMediaCodec analogue of VideoToolbox's format description on the Apple client).
// If it's HDR (BT.2020 PQ/HLG), tell the Surface so the compositor/display switch to
// HDR; SDR streams leave the default dataspace alone. The decoder itself picks a
// Main10 path from the SPS — no profile override needed. Keep looping (buffers
// follow, and any held buffer stays held across this event).
wait = Duration::ZERO;
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
Ok(()) => {
*applied_ds = Some(ds);
log::info!("decode: HDR stream → Surface dataspace {ds}");
}
Err(e) => log::warn!(
"decode: set_buffers_data_space({ds}) failed (non-fatal): {e}"
),
}
}
}
}
// TryAgainLater / OutputBuffersChanged — nothing more to dequeue now.
Ok(_) => break,
Err(e) => {
log::warn!("decode: dequeue_output_buffer: {e}");
break;
}
}
}
// Present the newest ready frame — UNLESS the gate is withholding it as a post-loss concealment,
// in which case release it without rendering (the SurfaceView keeps the last rendered frame frozen
// on glass) and count it as a discard rather than a display.
let mut rendered = 0;
if let Some((buf, meta)) = held {
match codec.release_output_buffer(buf, held_present) {
Ok(()) if held_present => {
rendered = 1;
if let Some((pts_us, decoded_ns)) = meta {
tracker.note_rendered(pts_us, decoded_ns);
}
}
Ok(()) => discarded += 1, // held off the screen — awaiting a clean re-anchor
Err(e) => log::warn!("decode: release_output_buffer: {e}"),
}
}
(rendered, discarded)
}
+23 -25
View File
@@ -24,14 +24,19 @@ const TAG_PLAYER_LEDS: u8 = 0x02;
const TAG_TRIGGER: u8 = 0x03;
const TAG_HID_RAW: u8 = 0x05;
/// `NativeBridge.nativeNextRumble(handle): Long` — block up to ~100 ms for the next rumble update.
/// Returns a packed positive long: bits 49..52 = wire `pad` index (0..15), bit 48 = "has a v2 lease",
/// bits 32..47 = `ttl_ms`, bits 16..31 = `low`, bits 0..15 = `high` (`low`/`high` 0..=0xFFFF, `0/0` =
/// stop). The lease flag is out-of-band so ANY 16-bit `ttl_ms` — including 0xFFFF — is unambiguous (no
/// in-band sentinel to collide with a real 65535 ms lease). No lease (legacy host) → bit 48 clear, and
/// Kotlin falls back to its long one-shot. `-1` on timeout / session closed (all packed values are
/// positive, so `-1` stays unambiguous). Kotlin routes the update back to the controller holding that
/// wire `pad` index (multi-pad rumble). Run from a Kotlin poll thread.
/// `NativeBridge.nativeNextRumble(handle): Long` — block up to ~100 ms for the next EFFECTIVE
/// rumble command from the core's shared policy engine (`design/rumble-root-fix.md` §D). The
/// engine owns ALL rumble policy — v2 lease expiry, legacy-host staleness (a uniform 1 s, ending
/// the old 60 s Android exposure), connection-close drain zeros — so Kotlin applies commands
/// verbatim: `(0, 0)` = cancel now, non-zero = one-shot at this level.
///
/// Returns a packed positive long: bits 49..52 = wire `pad` index (0..15), bits 32..47 = the
/// command's `backstop_ms` (≤ 5000 — the one-shot duration, i.e. the hardware net under a stalled
/// poll thread; the engine emits explicit zeros at every policy stop, so it is never the stop
/// mechanism), bits 16..31 = `low`, bits 0..15 = `high` (0..=0xFFFF). `-1` on timeout / session
/// closed (all packed values are positive, so `-1` stays unambiguous). Kotlin routes the command
/// back to the controller holding that wire `pad` index (multi-pad rumble). Run from a Kotlin
/// poll thread.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextRumble(
_env: JNIEnv,
@@ -43,24 +48,17 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextRumble(
if handle == 0 {
return -1;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; next_rumble_ttl is &self on
// the Sync connector — safe alongside the decode/audio/input threads. Kotlin stops these poll
// threads (and joins them — unbounded) before nativeClose frees the handle.
// SAFETY: live handle per the nativeConnect/nativeClose contract; next_rumble_command is
// &self on the Sync connector — safe alongside the decode/audio/input threads. Kotlin
// stops these poll threads (and joins them — unbounded) before nativeClose frees the
// handle.
let h = unsafe { &*(handle as *const SessionHandle) };
match h.client.next_rumble_ttl(PULL_TIMEOUT) {
Ok((pad, low, high, ttl)) => {
// The reorder gate already ran in the core, so this update is fresh. Encode the
// Option out-of-band: a real lease sets bit 48 and carries ttl_ms verbatim. The pad
// index rides above the lease flag (bits 49..52), keeping the whole word positive.
let (lease_flag, ttl_bits) = match ttl {
Some(ms) => (1i64 << 48, jlong::from(ms) << 32),
None => (0, 0),
};
(jlong::from(pad & 0xF) << 49)
| lease_flag
| ttl_bits
| (jlong::from(low) << 16)
| jlong::from(high)
match h.client.next_rumble_command(PULL_TIMEOUT) {
Ok(cmd) => {
(jlong::from(cmd.pad & 0xF) << 49)
| (jlong::from(cmd.backstop_ms.min(0xFFFF) as u16) << 32)
| (jlong::from(cmd.low) << 16)
| jlong::from(cmd.high)
}
Err(_) => -1, // NoFrame (timeout) or Closed — Kotlin loops on its running flag
}
@@ -0,0 +1,120 @@
<?xml version="1.0" encoding="UTF-8"?>
<Scheme
LastUpgradeVersion = "2700"
wasCreatedForAppExtension = "YES"
version = "2.0">
<BuildAction
parallelizeBuildables = "YES"
buildImplicitDependencies = "YES"
buildArchitectures = "Automatic">
<BuildActionEntries>
<BuildActionEntry
buildForTesting = "YES"
buildForRunning = "YES"
buildForProfiling = "YES"
buildForArchiving = "YES"
buildForAnalyzing = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "E2955696300948B9009F939C"
BuildableName = "PunktfunkWidgetsExtension.appex"
ReferencedContainer = "container:Punktfunk.xcodeproj">
</BuildableReference>
</BuildActionEntry>
<BuildActionEntry
buildForTesting = "YES"
buildForRunning = "YES"
buildForProfiling = "YES"
buildForArchiving = "YES"
buildForAnalyzing = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "BB0000000000000000000009"
BuildableName = "Punktfunk-iOS.app"
ReferencedContainer = "container:Punktfunk.xcodeproj">
</BuildableReference>
</BuildActionEntry>
</BuildActionEntries>
</BuildAction>
<TestAction
buildConfiguration = "Debug"
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
shouldUseLaunchSchemeArgsEnv = "YES"
shouldAutocreateTestPlan = "YES">
</TestAction>
<LaunchAction
buildConfiguration = "Debug"
selectedDebuggerIdentifier = ""
selectedLauncherIdentifier = "Xcode.IDEFoundation.Launcher.PosixSpawn"
launchStyle = "0"
askForAppToLaunch = "Yes"
useCustomWorkingDirectory = "NO"
ignoresPersistentStateOnLaunch = "NO"
debugDocumentVersioning = "YES"
debugServiceExtension = "internal"
allowLocationSimulation = "YES"
launchAutomaticallySubstyle = "2"
queueDebuggingEnabled = "No">
<RemoteRunnable
runnableDebuggingMode = "2"
BundleIdentifier = "com.apple.springboard">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "E2955696300948B9009F939C"
BuildableName = "PunktfunkWidgetsExtension.appex"
ReferencedContainer = "container:Punktfunk.xcodeproj">
</BuildableReference>
</RemoteRunnable>
<MacroExpansion>
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "BB0000000000000000000009"
BuildableName = "Punktfunk-iOS.app"
ReferencedContainer = "container:Punktfunk.xcodeproj">
</BuildableReference>
</MacroExpansion>
<EnvironmentVariables>
<EnvironmentVariable
key = "_XCWidgetKind"
value = ""
isEnabled = "YES">
</EnvironmentVariable>
<EnvironmentVariable
key = "_XCWidgetDefaultView"
value = "timeline"
isEnabled = "YES">
</EnvironmentVariable>
<EnvironmentVariable
key = "_XCWidgetFamily"
value = "systemMedium"
isEnabled = "YES">
</EnvironmentVariable>
</EnvironmentVariables>
</LaunchAction>
<ProfileAction
buildConfiguration = "Release"
shouldUseLaunchSchemeArgsEnv = "YES"
savedToolIdentifier = ""
useCustomWorkingDirectory = "NO"
debugDocumentVersioning = "YES"
askForAppToLaunch = "Yes"
launchAutomaticallySubstyle = "2">
<BuildableProductRunnable
runnableDebuggingMode = "0">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "BB0000000000000000000009"
BuildableName = "Punktfunk-iOS.app"
ReferencedContainer = "container:Punktfunk.xcodeproj">
</BuildableReference>
</BuildableProductRunnable>
</ProfileAction>
<AnalyzeAction
buildConfiguration = "Debug">
</AnalyzeAction>
<ArchiveAction
buildConfiguration = "Release"
revealArchiveInOrganizer = "YES">
</ArchiveAction>
</Scheme>
@@ -0,0 +1,10 @@
import Foundation
import PunktfunkKit
/// A fresh `pair=required`/unknown host pending a trust decision: drives both the "request access
/// vs. pair with PIN" choice and the subsequent approval wait. `advertisedFingerprint` is the
/// discovered host's advertised cert (nil for a manually-typed host trust-on-first-use).
struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
@@ -24,6 +24,7 @@ struct ContentView: View {
@AppStorage(DefaultsKey.streamWidth) private var width = 1920
@AppStorage(DefaultsKey.streamHeight) private var height = 1080
@AppStorage(DefaultsKey.streamHz) private var hz = 60
@AppStorage(DefaultsKey.renderScale) private var renderScale = 1.0
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@@ -338,18 +339,20 @@ struct ContentView: View {
+ "approve it — no need to reconnect.")
}
// Informational deep-link outcome (unknown host / already streaming). Not an error.
.alert(
"Can't open",
isPresented: Binding(
get: { deepLinkNotice != nil },
set: { if !$0 { deepLinkNotice = nil } })
) {
.alert("Can't open", isPresented: deepLinkNoticePresented) {
Button("OK", role: .cancel) {}
} message: {
Text(deepLinkNotice ?? "")
}
}
/// Presentation flag for the informational deep-link alert. Extracted from the `.alert` call so
/// the manual get/set Binding type-checks on its own instead of inflating the body chain's
/// budget (adding it inline tips SwiftUI's per-expression limit see the split sections idiom).
private var deepLinkNoticePresented: Binding<Bool> {
Binding(get: { deepLinkNotice != nil }, set: { if !$0 { deepLinkNotice = nil } })
}
#if os(iOS)
/// The Live Activity mode line, e.g. "2560×1440 @120 · HEVC · HDR", from the live connection.
private func currentModeLine() -> String {
@@ -736,6 +739,17 @@ struct ContentView: View {
/// host is back online. `prepareWake` still runs here to LEARN/refresh the MAC now that the host
/// is advertising (and is a harmless no-op otherwise). `onUnreachable` hands a plain connect
/// failure back to the caller (the wake-wait fallback) instead of the error alert.
/// The stream mode to request = the chosen resolution × the render scale, aspect-preserved,
/// even, and clamped to the codec's max dimension. > 1 supersamples for sharpness (the presenter
/// downscales the larger decoded frame to this display); < 1 renders under native and upscales.
/// The match-window path applies the SAME scale to the live window size in `MatchWindowFollower`.
private func scaledMode() -> (width: UInt32, height: UInt32) {
RenderScale.apply(
baseWidth: width, baseHeight: height,
scale: renderScale,
maxDimension: RenderScale.maxDimension(codec: codec))
}
private func startSessionDirect(
_ host: StoredHost, launchID: String? = nil,
allowTofu: Bool, requestAccess: Bool = false, approvalReq: ApprovalRequest? = nil,
@@ -747,7 +761,7 @@ struct ContentView: View {
if let approvalReq { awaitingApproval = approvalReq }
model.connect(
to: host,
width: UInt32(clamping: width), height: UInt32(clamping: height),
width: scaledMode().width, height: scaledMode().height,
hz: UInt32(clamping: hz),
compositor: PunktfunkConnection.Compositor(
rawValue: UInt32(clamping: compositor)) ?? .auto,
@@ -930,7 +944,7 @@ struct ContentView: View {
}
model.connect(
to: host,
width: UInt32(clamping: width), height: UInt32(clamping: height),
width: scaledMode().width, height: scaledMode().height,
hz: UInt32(clamping: hz),
compositor: pref,
gamepad: pad,
@@ -941,71 +955,3 @@ struct ContentView: View {
autoTrust: true)
}
}
#if os(macOS)
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
private struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
if want != isFull { window.toggleFullScreen(nil) }
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
}
}
#endif
/// A fresh `pair=required`/unknown host pending a trust decision: drives both the "request access
/// vs. pair with PIN" choice and the subsequent approval wait. `advertisedFingerprint` is the
/// discovered host's advertised cert (nil for a manually-typed host trust-on-first-use).
private struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
@@ -0,0 +1,83 @@
import PunktfunkKit
import SwiftUI
#if os(macOS)
import AppKit
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
/// The last `active` value we DROVE the window to. We toggle only when `active` itself
/// changes (stream start/end) never to correct a mismatch so a deliberate mid-session
/// toggle (F / the green button) isn't snapped back on the next SwiftUI update.
var lastActive: Bool?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
// Drive the window only on an `active` EDGE (stream start/end), not to close a mismatch
// so a user's F / green-button toggle stays put. First pass (lastActive == nil) just
// records the state without toggling, so mounting never yanks a window into fullscreen.
if coordinator.lastActive != want {
coordinator.lastActive = want
if want != isFull { window.toggleFullScreen(nil) }
}
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
// The Stream menu's "Toggle Fullscreen" (F) and InputCapture's captured-state interception
// both post this; flip the KEY window only (posted app-wide, object nil). The transition
// observers above then mirror the real state back into the binding.
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: .punktfunkToggleFullscreen, object: nil, queue: .main
) { [weak window] _ in
guard let window, window.isKeyWindow else { return }
window.toggleFullScreen(nil)
})
}
}
#endif
@@ -202,9 +202,17 @@ public final class ClipboardSync: NSObject {
let types = pb.types ?? []
if types.contains(Self.concealed) || types.contains(Self.transient) { return }
offerSeq &+= 1
let kinds = Self.wireToPasteboard
var kinds = Self.wireToPasteboard
.filter { types.contains($0.type) }
.map { PunktfunkConnection.ClipKind(mime: $0.wire) }
// Images: macOS image copies usually carry TIFF (browsers add WebP/AVIF/GIF, screenshots
// TIFF) and only sometimes PNG announce the portable `image/png` whenever ANY
// convertible image type is present; `serveFetch` converts at fetch time (lazy, §3.5).
if !kinds.contains(where: { $0.mime == "image/png" }),
types.contains(.tiff) || types.contains(NSPasteboard.PasteboardType("public.heic"))
{
kinds.append(PunktfunkConnection.ClipKind(mime: "image/png"))
}
// Empty = the pasteboard holds nothing we sync (or was cleared) clears the host side.
connection.clipOffer(seq: offerSeq, kinds: kinds)
}
@@ -305,8 +313,7 @@ public final class ClipboardSync: NSObject {
private func serveFetch(reqId: UInt32, seq: UInt32, mime: String) {
let pb = NSPasteboard.general
guard seq == offerSeq, pb.changeCount == lastSeenChangeCount,
let type = Self.wireToPasteboard.first(where: { $0.wire == mime })?.type,
let data = pb.data(forType: type)
let data = Self.readWireData(pb, mime)
else {
connection.clipCancel(id: reqId)
return
@@ -322,6 +329,30 @@ public final class ClipboardSync: NSObject {
connection.clipServe(reqId: reqId, data: Data(), last: true)
}
}
/// Read one wire format from the pasteboard, converting where macOS stores a different
/// native type: `image/png` is served from a real `.png` entry when present, else converted
/// from whatever image representation the pasteboard holds (TIFF from screenshots/Preview,
/// WebP/AVIF/GIF from browsers `NSImage` decodes them all) into PNG at fetch time.
private static func readWireData(_ pb: NSPasteboard, _ mime: String) -> Data? {
guard mime == "image/png" else {
guard let type = wireToPasteboard.first(where: { $0.wire == mime })?.type else {
return nil
}
return pb.data(forType: type)
}
if let png = pb.data(forType: .png) {
return png
}
// No native PNG: decode whatever image the pasteboard carries and re-encode.
guard let img = NSImage(pasteboard: pb),
let tiff = img.tiffRepresentation,
let rep = NSBitmapImageRep(data: tiff)
else {
return nil
}
return rep.representation(using: .png, properties: [:])
}
}
/// The lazy paste hook: AppKit calls `provideDataForType` only when a Mac app actually pastes;
@@ -794,6 +794,34 @@ public final class PunktfunkConnection {
}
}
/// Pull the next EFFECTIVE rumble command from the core's shared rumble policy engine the
/// uniform replacement for per-platform rumble policy. The engine owns every decision
/// (v2 lease expiry, legacy-host staleness at a uniform 1 s, connection-close drain zeros),
/// so apply commands verbatim: `(0, 0)` = stop now, non-zero = run at this level.
/// `backstopMs` is a safety-net duration for duration-parameterized platform APIs the
/// CoreHaptics renderer ignores it (its finite segment ceiling is the equivalent net).
/// Drain from the (single) feedback thread, alongside `nextHidOutput`.
public func nextRumbleCommand(timeoutMs: UInt32 = 0) throws
-> (pad: UInt16, low: UInt16, high: UInt16, backstopMs: UInt32)?
{
feedbackLock.lock()
defer { feedbackLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var pad: UInt16 = 0, low: UInt16 = 0, high: UInt16 = 0, backstop: UInt32 = 0
let rc = punktfunk_connection_next_rumble_cmd(h, &pad, &low, &high, &backstop, timeoutMs)
switch rc {
case statusOK:
return (pad, low, high, backstop)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// One DualSense feedback event a game wrote to the host's virtual pad replay it on
/// the real controller (GCDeviceLight, GCControllerPlayerIndex,
/// GCDualSenseAdaptiveTrigger). Only a `.dualSense` session emits these.
@@ -155,21 +155,18 @@ public final class GamepadFeedback {
// meta, was unaffected). Pacing with a short sleep OUTSIDE the lock (below) keeps
// rumble/HID latency low while leaving the lock free between polls.
//
// Rumble is idempotent state, so drain the plane DRY and apply only the newest
// level PER PAD. The old one-datagram-per-cycle shape let a burst outpace the
// ~125 Hz drain: levels rendered up to ~130 ms late through the core's 16-deep
// queue, and its drop-newest overflow could shed a stop while stale nonzero
// states queued ahead of it buzzing until the host's next 500 ms refresh.
var newestByPad: [UInt8: (low: UInt16, high: UInt16, ttl: UInt32)] = [:]
// Rumble arrives as EFFECTIVE commands from the core's shared policy engine
// (design/rumble-root-fix.md §D): the engine owns leases, legacy staleness,
// and close-drain zeros, and its per-pad mailbox already coalesces a
// stalled drain wakes to ONE current-level command per pad, and a stop can
// never be shed by a queue. Apply verbatim, in order.
var rumbleBurst = 0
while rumbleBurst < 64, !flag.isStopped,
let r = try connection.nextRumble2(timeoutMs: 0) {
newestByPad[UInt8(truncatingIfNeeded: r.pad)] = (r.low, r.high, r.ttlMs)
let c = try connection.nextRumbleCommand(timeoutMs: 0) {
self?.routeRumble(
pad: UInt8(truncatingIfNeeded: c.pad), low: c.low, high: c.high)
rumbleBurst += 1
}
for (pad, n) in newestByPad {
self?.routeRumble(pad: pad, low: n.low, high: n.high, ttlMs: n.ttl)
}
// Drain a BOUNDED burst of hidout events so sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) can't spin here or block stop() past one cycle.
var burst = 0
@@ -218,15 +215,15 @@ public final class GamepadFeedback {
}
}
/// Route one rumble envelope to its pad's renderer (drain thread). An update for a pad with no
/// Route one engine command to its pad's renderer (drain thread). A command for a pad with no
/// live renderer one that just left the forwarded set is dropped.
private func routeRumble(pad: UInt8, low: UInt16, high: UInt16, ttlMs: UInt32) {
private func routeRumble(pad: UInt8, low: UInt16, high: UInt16) {
let renderer = withRouting { rumbleByPad[pad] }
renderer?.apply(low: low, high: high, ttlMs: ttlMs)
renderer?.apply(low: low, high: high)
// The opt-in device mirror follows controller 1 unconditionally the pads it exists for
// have no motors (their renderer above no-ops), and mirroring deliberately isn't gated on
// that: capability probing can't see a motor-less MFi pad, and the user opted in.
if pad == 0 { deviceRumble?.apply(low: low, high: high, ttlMs: ttlMs) }
if pad == 0 { deviceRumble?.apply(low: low, high: high) }
}
private func withRouting<R>(_ body: () -> R) -> R {
@@ -23,23 +23,6 @@ enum RumbleTuning {
/// the churn that lost stops inside CoreHaptics. Newest level wins when the window opens;
/// zero is never throttled.
static let minRebakeSeconds: TimeInterval = 0.025
/// Session watchdog: silence the motors when no wire command arrived for this long. This is
/// the **legacy-host fallback only** an old host sends no self-termination lease, so its
/// periodic re-send (every 500 ms) is the sole liveness signal and 3 vanished refreshes means
/// the channel or host died while audible. A v2 host instead supplies a per-command TTL (see
/// [`leaseSeconds`]); that deadline supersedes this watchdog.
static let sessionStaleSeconds: TimeInterval = 1.6
/// The legacy no-lease sentinel a v2 `ttl_ms` carries for an old host (mirrors the C ABI's
/// `PUNKTFUNK_RUMBLE_NO_TTL`). `UInt32.max` by construction.
static let noTTL: UInt32 = .max
/// Interpret a wire TTL (ms) from a rumble update: `nil` for the legacy no-lease sentinel
/// ([`noTTL`]) the renderer falls back to [`sessionStaleSeconds`] else the self-termination
/// lease in seconds (render the level for at most this long unless the host renews it).
static func leaseSeconds(ttlMs: UInt32) -> TimeInterval? {
ttlMs == noTTL ? nil : TimeInterval(ttlMs) / 1000
}
/// Levels closer than this (0.4 % of full scale) are the same level an identical host
/// refresh must never rebuild a player.
static let levelEpsilon: Float = 1.0 / 256.0
@@ -110,13 +93,15 @@ enum RumbleTuning {
/// `@unchecked Sendable` is sound because every property is read and written only inside
/// `queue` closures the serial queue is the synchronization.
final class RumbleRenderer: @unchecked Sendable {
/// What an un-refreshed nonzero target means. A live session ties motor life to wire
/// liveness (the host refreshes state every 500 ms); the controller test panel holds a
/// slider level indefinitely.
/// Who ends an un-refreshed nonzero target. Session mode applies the core policy engine's
/// commands verbatim the engine (punktfunk-core `client/rumble.rs`) owns every lease,
/// staleness, and close decision and emits explicit zeros, so the renderer keeps NO
/// staleness policy of its own anymore. The controller test panel (`manual`) holds a slider
/// level indefinitely; both are identical renderer-side today, the distinction is kept for
/// the call sites' intent.
struct Policy {
let staleAfter: TimeInterval?
static let session = Policy(staleAfter: RumbleTuning.sessionStaleSeconds)
static let manual = Policy(staleAfter: nil)
static let session = Policy()
static let manual = Policy()
}
/// Which physical actuator this renderer drives: the forwarded controller's haptics engine
@@ -160,13 +145,9 @@ final class RumbleRenderer: @unchecked Sendable {
private var controller: GCController?
private var low: Motor?
private var high: Motor?
/// Wire-truth target (raw wire units) and when it was last confirmed by any command.
/// Wire-truth target (raw wire units) the engine command's level, applied verbatim; the
/// core policy engine owns when it ends (explicit zero commands), so no deadline lives here.
private var target: (low: UInt16, high: UInt16) = (0, 0)
private var lastCommand = DispatchTime(uptimeNanoseconds: 0)
/// The v2 envelope lease: the active level is authorized until here unless the host renews it
/// (`tick` silences at the deadline). `nil` against a legacy host (no lease the
/// `sessionStaleSeconds` watchdog is the backstop) and while silent.
private var envelopeDeadline: DispatchTime?
/// Runs while anything is (or should be) audible: staleness watchdog, segment re-arm,
/// throttled-level catch-up, engine rebuild after a reset, HID keepalive. Nil while silent,
/// so an idle controller costs no timer wakeups and no radio traffic.
@@ -247,17 +228,9 @@ final class RumbleRenderer: @unchecked Sendable {
/// against a legacy host (no lease the staleness watchdog is the backstop). Renewals at an
/// unchanged level extend the deadline before the idempotence guard, so a held rumble never
/// lapses mid-effect.
func apply(low lowAmp: UInt16, high highAmp: UInt16, ttlMs: UInt32 = RumbleTuning.noTTL) {
func apply(low lowAmp: UInt16, high highAmp: UInt16) {
queue.async {
self.lastCommand = .now()
let active = lowAmp != 0 || highAmp != 0
// v2 lease: a nonzero level gets an explicit deadline; a stop or a legacy update clears
// it. Set BEFORE the idempotence guard so an identical renewal still extends the lease.
if let lease = RumbleTuning.leaseSeconds(ttlMs: ttlMs), active {
self.envelopeDeadline = .now() + lease
} else {
self.envelopeDeadline = nil
}
if active != self.wasActive {
self.wasActive = active
log.debug(
@@ -275,7 +248,6 @@ final class RumbleRenderer: @unchecked Sendable {
self.ticker?.cancel()
self.ticker = nil
self.target = (0, 0)
self.envelopeDeadline = nil
self.wasActive = false
self.teardown()
self.closeHID()
@@ -331,25 +303,11 @@ final class RumbleRenderer: @unchecked Sendable {
healthSink?(problem)
}
/// Watchdog + housekeeping heartbeat while audible.
/// Housekeeping heartbeat while audible: segment re-arm, HID keepalive, backoff retries.
/// Every liveness decision (lease expiry, legacy-host staleness, session close) lives in the
/// core policy engine now it emits explicit zero commands, so the renderer never guesses
/// when a level should end.
private func tick() {
if let deadline = envelopeDeadline {
// v2 host lease: silence the moment it lapses unrenewed. This firing in the wild is the
// observable signature of a host that stopped renewing (a dropped stop, or a dead host)
// the whole point of the envelope model: the motor can't outlive the host's intent.
if target != (0, 0), DispatchTime.now() >= deadline {
log.warning("rumble: envelope expired unrenewed — silencing")
target = (0, 0)
envelopeDeadline = nil
}
} else if let after = policy.staleAfter, target != (0, 0), seconds(since: lastCommand) > after {
// Legacy host (no lease): it re-sends state every 500 ms, so this much silence means the
// channel (or host) died while a motor was on. A direct-connected pad would have been
// stopped by its game long ago force the same outcome.
log.warning(
"rumble: no wire refresh for \(after, format: .fixed(precision: 1), privacy: .public)s — auto-silencing")
target = (0, 0)
}
render()
}
@@ -21,6 +21,14 @@ public enum DefaultsKey {
/// is native either way, so this degenerates to Auto-native there). Read per session by the
/// stream views' `MatchWindowFollower`.
public static let matchWindow = "punktfunk.matchWindow"
/// Render-resolution multiplier (a `RenderScale` value, default 1.0): the client asks the host
/// to render/encode at `chosen resolution × scale`, then the presenter downscales the larger
/// decoded frame to this display in one Catmull-Rom pass. > 1 supersamples (sharper, at the cost
/// of more bandwidth AND client decode both grow scale²); < 1 renders below native for a
/// weak host GPU / constrained link (the presenter upscales). Purely client-side the host just
/// sees a normal (larger/smaller) `Mode`, and Automatic bitrate scales with it. Clamped even +
/// to the codec's max dimension at connect. Applies to the fixed mode and the match-window path.
public static let renderScale = "punktfunk.renderScale"
public static let compositor = "punktfunk.compositor"
public static let gamepadType = "punktfunk.gamepadType"
public static let gamepadID = "punktfunk.gamepadID"
@@ -69,6 +77,21 @@ public enum DefaultsKey {
public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never".
public static let cursorMode = "punktfunk.cursorMode"
/// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by
/// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is
/// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across
/// the macOS wheel, the iOS trackpad pan, and a GCMouse wheel. For users whose host expects the
/// opposite convention from their local OS preference.
public static let invertScroll = "punktfunk.invertScroll"
/// Location-based modifier mapping (a `ModifierLayout` value, default `.mac`): which Windows VK
/// each PHYSICAL modifier position forwards to the host. `.mac` keeps Option Alt and
/// Command Super/Win (the Apple positions). `.windows` swaps the Alt/Super ROLE between the
/// Option and Command keys preserving side (L/R) so the key nearest the space bar acts as
/// Alt and the next one as the Windows key, matching a Windows keyboard's `Ctrl / / Alt` row.
/// Only what's FORWARDED changes; client-local shortcuts ( &co.) stay on the physical key.
/// Read live at the wire boundary by `InputCapture`. Control/Shift never move (same position on
/// both keyboards).
public static let modifierLayout = "punktfunk.modifierLayout"
/// iPad: capture the mouse/trackpad pointer (pointer lock relative movement) for games,
/// rather than forwarding an absolute cursor position. On by default. Only meaningful on iPad
/// with a hardware mouse/trackpad; the system grants the lock only to a full-screen, frontmost
@@ -132,6 +155,12 @@ extension Notification.Name {
/// discoverable menu-bar surface.
public static let punktfunkReleaseCapture = Notification.Name("io.unom.punktfunk.release-capture")
/// Posted by the app's Stream menu ("Toggle Fullscreen", F) and by InputCapture's monitor
/// when the same combo fires while input is captured (the menu key-equivalent never reaches a
/// captured stream view). The key window's `FullscreenController` flips the window's fullscreen
/// state. macOS only.
public static let punktfunkToggleFullscreen = Notification.Name("io.unom.punktfunk.toggle-fullscreen")
/// Posted by the Live Activity's / Shortcuts' End-stream intent (`EndStreamIntent.perform`,
/// which runs in the app's process): the app tears the active session down deliberately
/// (quit-close the host). Same cross-process-signal pattern as `punktfunkReleaseCapture`
@@ -0,0 +1,72 @@
// Render-resolution scaling the pure geometry behind `DefaultsKey.renderScale`. The client asks
// the host to render/encode at `chosen resolution × scale` and lets the presenter downscale the
// larger decoded frame to the display (a Catmull-Rom minification, > 1 = supersampling for
// sharpness) or upscale a smaller one (< 1 = a performance mode for a weak host GPU / thin link).
//
// This is where the multiplier is turned into a host-valid `Mode` dimension: multiply, preserve the
// aspect ratio, floor to even (the host's `validate_dimensions` rejects odd sizes), and clamp to the
// codec's per-axis ceiling so the connect can't ask for something the encoder will reject. Kept
// dependency-free + side-effect-free so it's unit-tested (`RenderScaleTests`) and reused by both the
// fixed-mode connect and the match-window follower.
import Foundation
public enum RenderScale {
/// The supported multiplier range. Below 1 renders under native (upscaled on present); above 1
/// supersamples. The UI clamps its slider to this and the connect clamps the raw stored value.
public static let range: ClosedRange<Double> = 0.5...4.0
/// The multipliers the picker offers. 1.0 (Native) is the default; the rest are the round stops
/// users reason about.
public static let presets: [Double] = [0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0]
/// The encoder/host per-axis ceiling for a codec preference string (`DefaultsKey.codec`). H.264
/// tops out at 4096 px/axis; HEVC / AV1 / PyroWave (and "auto", which negotiates one of those in
/// practice) at 8192. The host enforces the same walls in `codec.rs::validate_dimensions`.
public static func maxDimension(codec: String) -> Int {
codec == "h264" ? 4096 : 8192
}
/// A compact user-facing label for a multiplier: "Native (1×)", "1.5×", "2× · supersample".
/// Shared by every platform's picker so the wording stays identical.
public static func label(_ scale: Double) -> String {
if scale == 1.0 { return "Native (1×)" }
let magnitude = String(format: "%g×", scale)
return scale > 1 ? "\(magnitude) · supersample" : magnitude
}
/// Clamp a raw stored multiplier into `range`, treating a missing/zero value as 1.0 (Native).
public static func sanitize(_ raw: Double) -> Double {
guard raw > 0 else { return 1.0 }
return min(max(raw, range.lowerBound), range.upperBound)
}
/// Apply `scale` to a base pixel size, preserving aspect, even-flooring each axis, and clamping
/// uniformly so neither axis exceeds `maxDimension` (the larger axis lands on the cap, the ratio
/// is kept). Also floors each axis at `minWidth`/`minHeight` (the host never accepts < 320×200).
/// The result is a directly host-valid `Mode` width/height.
public static func apply(
baseWidth: Int,
baseHeight: Int,
scale rawScale: Double,
maxDimension: Int,
minWidth: Int = 320,
minHeight: Int = 200
) -> (width: UInt32, height: UInt32) {
let scale = sanitize(rawScale)
var w = Double(max(baseWidth, 1)) * scale
var h = Double(max(baseHeight, 1)) * scale
// Uniform down-clamp if either axis blew past the ceiling keep the aspect ratio intact.
let cap = Double(maxDimension)
let over = max(w / cap, h / cap)
if over > 1 {
w /= over
h /= over
}
let evenFloor: (Double, Int) -> UInt32 = { value, minimum in
let clamped = max(Int(value.rounded(.down)), minimum)
return UInt32(clamped / 2 * 2)
}
return (evenFloor(w, minWidth), evenFloor(h, minHeight))
}
}
@@ -52,13 +52,6 @@ final class RumbleTuningTests: XCTestCase {
XCTAssertEqual(RumbleTuning.handoffStart(endsAt: 100, now: 100.5), 100.5)
}
func testPolicies() {
// The session policy ties motor life to wire liveness; the manual (test-panel) policy
// holds a level indefinitely.
XCTAssertNotNil(RumbleRenderer.Policy.session.staleAfter)
XCTAssertNil(RumbleRenderer.Policy.manual.staleAfter)
}
/// Exercise the renderer's queue/ticker machinery without a physical pad: a wire-rate call
/// storm, an audible target left to the ticker (watchdog path), then `stop()` which runs
/// `queue.sync` against the same serial queue the ticker fires on and must not deadlock.
@@ -75,45 +68,22 @@ final class RumbleTuningTests: XCTestCase {
renderer.stop()
}
func testLeaseSecondsInterpretsWireTTL() {
// The legacy no-lease sentinel nil (fall back to the staleness watchdog).
XCTAssertNil(RumbleTuning.leaseSeconds(ttlMs: RumbleTuning.noTTL))
XCTAssertEqual(RumbleTuning.noTTL, UInt32.max)
// A real lease its duration in seconds (non-nil for any ttl != noTTL).
XCTAssertEqual(RumbleTuning.leaseSeconds(ttlMs: 400) ?? .nan, 0.4, accuracy: 1e-9)
XCTAssertEqual(RumbleTuning.leaseSeconds(ttlMs: 0) ?? .nan, 0, accuracy: 1e-9)
XCTAssertEqual(RumbleTuning.leaseSeconds(ttlMs: 150) ?? .nan, 0.15, accuracy: 1e-9)
}
func testEnvelopeLeaseBoundsMotorLifeTighterThanTheLegacyWatchdog() {
// The whole point of v2: a host-supplied lease silences the motor faster than the
// legacy staleness watchdog ever could (which needs sessionStaleSeconds of silence). The
// default 400 ms TTL is well under that, on every platform.
let defaultTTL = RumbleTuning.leaseSeconds(ttlMs: 400)
XCTAssertNotNil(defaultTTL)
XCTAssertLessThan(defaultTTL!, RumbleTuning.sessionStaleSeconds)
// The ticker must be able to observe an expired lease promptly (well within one TTL).
XCTAssertLessThan(RumbleTuning.tickSeconds, defaultTTL!)
}
/// A v2 envelope with a short TTL, left unrenewed, must self-silence the renderer's core
/// promise. Drive the real queue/ticker (no physical pad) and confirm it doesn't wedge.
func testEnvelopeExpiresWhenUnrenewed() {
/// A zero command must silence promptly the engine (punktfunk-core) emits explicit zeros at
/// every policy stop (lease expiry, legacy staleness, session close), and the renderer's only
/// job is to apply them. Drive the real queue/ticker (no physical pad) and confirm no wedge.
func testZeroCommandSilencesAndTeardownDoesNotDeadlock() {
let renderer = RumbleRenderer(policy: .session)
renderer.retarget(nil)
// A 100 ms lease, then no renewal the ticker (50 ms) must silence it on its own.
renderer.apply(low: 0x8000, high: 0x8000, ttlMs: 100)
Thread.sleep(forTimeInterval: 0.3)
// No assertion on private state; this exercises the expiry path + serial-queue teardown
renderer.apply(low: 0x8000, high: 0x8000)
Thread.sleep(forTimeInterval: 0.1)
renderer.apply(low: 0, high: 0)
Thread.sleep(forTimeInterval: 0.1)
// No assertion on private state; this exercises the stop path + serial-queue teardown
// without deadlock (the ticker fires on the same queue stop() sync-hops onto).
renderer.stop()
}
func testTuningRelationsTheDesignDependsOn() {
// The watchdog must tolerate a couple of lost 500 ms host refreshes (heals, not gaps)
// but trip well before a stuck rumble reads as "still going".
XCTAssertGreaterThan(RumbleTuning.sessionStaleSeconds, 2 * 0.5)
XCTAssertLessThanOrEqual(RumbleTuning.sessionStaleSeconds, 2.5)
// Re-arm headroom must clear several ticker periods, or a steady rumble could miss the
// segment boundary and gap.
XCTAssertGreaterThanOrEqual(
@@ -123,9 +93,8 @@ final class RumbleTuningTests: XCTestCase {
// The rebake throttle must be far under the host refresh period, or refreshed level
// changes would queue behind it; and under a frame at 30 fps so ramps stay smooth.
XCTAssertLessThan(RumbleTuning.minRebakeSeconds, 1.0 / 30)
// The ticker (which lands throttled levels) must outpace the HID keepalive and the
// watchdog, or those deadlines could be overshot by a full period.
// The ticker (which lands throttled levels) must outpace the HID keepalive, or its
// deadline could be overshot by a full period.
XCTAssertLessThan(RumbleTuning.tickSeconds, RumbleTuning.hidKeepaliveSeconds)
XCTAssertLessThan(RumbleTuning.tickSeconds, RumbleTuning.sessionStaleSeconds)
}
}
+2 -2
View File
@@ -896,8 +896,8 @@ class Plugin:
except (OSError, json.JSONDecodeError):
# The client's own defaults (native display, host-default bitrate, auto pad).
return {
"width": 0, "height": 0, "refresh_hz": 0, "bitrate_kbps": 0,
"codec": "auto", "gamepad": "auto", "compositor": "auto",
"width": 0, "height": 0, "refresh_hz": 0, "render_scale": 1.0,
"bitrate_kbps": 0, "codec": "auto", "gamepad": "auto", "compositor": "auto",
"inhibit_shortcuts": True, "mic_enabled": False,
}
+1
View File
@@ -99,6 +99,7 @@ export interface StreamSettings {
width: number; // 0 = native
height: number; // 0 = native
refresh_hz: number; // 0 = native
render_scale?: number; // render-resolution multiplier; 1.0 = native (absent in pre-scale files)
bitrate_kbps: number; // 0 = host default
codec?: string; // "auto" | "hevc" | "h264" | "av1" — soft preference (absent in pre-codec files)
gamepad: string; // "auto" | "xbox360" | "xboxone" | "dualsense" | "dualshock4" | "steamdeck"
+22
View File
@@ -25,6 +25,10 @@ const RESOLUTIONS: [number, number, string][] = [
[2560, 1440, "2560 × 1440"],
];
const REFRESH = [0, 30, 60, 90, 120];
// Render-resolution multipliers (mirrors punktfunk_core::render_scale::PRESETS). 1.0 = native.
const RENDER_SCALES = [0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0];
const renderScaleLabel = (x: number): string =>
x === 1 ? "Native (1×)" : x > 1 ? `${x}× · supersample` : `${x}×`;
const GAMEPADS = ["auto", "xbox360", "xboxone", "dualsense", "dualshock4", "steamdeck"];
const GAMEPAD_LABELS: Record<string, string> = {
auto: "Automatic",
@@ -106,6 +110,24 @@ export const SettingsSection: FC = () => {
</div>
</RowActions>
</Field>
<Field
label="Render scale"
description="Supersample for sharpness (> 1×, more bandwidth) or render below native (< 1×) — the Deck resamples to its screen"
childrenContainerWidth="max"
>
<RowActions>
<div style={selectShell}>
<Dropdown
rgOptions={RENDER_SCALES.map((x) => ({ data: x, label: renderScaleLabel(x) }))}
// Snap the stored value to the nearest preset so the dropdown always shows a match.
selectedOption={RENDER_SCALES.reduce((best, x) =>
Math.abs(x - (s.render_scale ?? 1)) < Math.abs(best - (s.render_scale ?? 1)) ? x : best,
)}
onChange={(o) => patch({ render_scale: o.data as number })}
/>
</div>
</RowActions>
</Field>
<SliderField
label="Bitrate"
description="Mbit/s · 0 = host default"
+34
View File
@@ -17,6 +17,20 @@ const RESOLUTIONS: &[(u32, u32)] = &[
];
/// `0` = the monitor's native refresh, resolved at connect.
const REFRESH: &[u32] = &[0, 30, 60, 90, 120, 144, 165, 240];
/// Render-scale multipliers (persisted as f64; mirrors [`punktfunk_core::render_scale::PRESETS`]).
/// `1.0` = Native. Applied at connect and each match-window resize.
const RENDER_SCALES: &[f64] = &[0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0];
/// A compact label for a render-scale multiplier: "Native" / "1.5×" / "2× (supersample)".
fn render_scale_label(scale: f64) -> String {
if scale == 1.0 {
"Native".to_string()
} else if scale > 1.0 {
format!("{scale}× (supersample)")
} else {
format!("{scale}×")
}
}
const GAMEPADS: &[&str] = &[
"auto",
"xbox360",
@@ -304,6 +318,18 @@ pub fn show(
"",
&hz_names.iter().map(String::as_str).collect::<Vec<_>>(),
);
let scale_names: Vec<String> = RENDER_SCALES
.iter()
.map(|&s| render_scale_label(s))
.collect();
let scale_row = ChoiceRow::new(
&dialog,
inline,
"Render scale",
"Supersample for sharpness (> 1×, more bandwidth and decode) or render below native \
(< 1×) for a lighter host this device resamples to the window",
&scale_names.iter().map(String::as_str).collect::<Vec<_>>(),
);
let bitrate_row = adw::SpinRow::with_range(0.0, 3000.0, 5.0);
bitrate_row.set_title("Bitrate");
bitrate_row.set_subtitle("Mbit/s · 0 = host default · run a speed test before going high");
@@ -346,6 +372,7 @@ pub fn show(
.build();
stream.add(res_row.widget());
stream.add(hz_row.widget());
stream.add(scale_row.widget());
stream.add(&bitrate_row);
stream.add(compositor_row.widget());
stream.add(decoder_row.widget());
@@ -500,6 +527,11 @@ pub fn show(
res_row.set_selected(res_i as u32);
let hz_i = REFRESH.iter().position(|&r| r == s.refresh_hz).unwrap_or(0);
hz_row.set_selected(hz_i as u32);
let scale_i = RENDER_SCALES
.iter()
.position(|&x| (x - s.render_scale).abs() < 1e-6)
.unwrap_or_else(|| RENDER_SCALES.iter().position(|&x| x == 1.0).unwrap());
scale_row.set_selected(scale_i as u32);
bitrate_row.set_value(f64::from(s.bitrate_kbps) / 1000.0);
let pad_i = GAMEPADS.iter().position(|&g| g == s.gamepad).unwrap_or(0);
pad_row.set_selected(pad_i as u32);
@@ -545,6 +577,8 @@ pub fn show(
RESOLUTIONS[res_i - 1]
};
s.refresh_hz = REFRESH[(hz_row.selected() as usize).min(REFRESH.len() - 1)];
s.render_scale =
RENDER_SCALES[(scale_row.selected() as usize).min(RENDER_SCALES.len() - 1)];
s.bitrate_kbps = (bitrate_row.value() * 1000.0) as u32;
s.gamepad = GAMEPADS[(pad_row.selected() as usize).min(GAMEPADS.len() - 1)].to_string();
s.touch_mode =
+2
View File
@@ -175,6 +175,8 @@ pub fn run(target: Option<&str>) -> u8 {
// Latched at console start (like the stats tier above): toggling Match window in
// the console's settings screen applies from the next console launch.
match_window: crate::session_main::match_window(&settings_at_start),
render_scale: settings_at_start.render_scale,
render_scale_max_dim: punktfunk_core::render_scale::max_dimension(&settings_at_start.codec),
};
let result =
+16
View File
@@ -127,6 +127,20 @@ mod session_main {
settings.refresh_hz
},
};
// Render scale: multiply the resolved mode (even + codec-clamped) so the host renders
// larger/smaller and the presenter resamples to the window. 1.0 = Native. Applied after the
// Native/explicit resolution so it composes uniformly with both.
let (sw, sh) = punktfunk_core::render_scale::apply(
mode.width,
mode.height,
settings.render_scale,
punktfunk_core::render_scale::max_dimension(&settings.codec),
);
let mode = Mode {
width: sw,
height: sh,
..mode
};
SessionParams {
host: addr,
port,
@@ -372,6 +386,8 @@ mod session_main {
overlay: None,
window_size: window_size(&settings),
match_window: match_window(&settings),
render_scale: settings.render_scale,
render_scale_max_dim: punktfunk_core::render_scale::max_dimension(&settings.codec),
};
let outcome =
+33
View File
@@ -19,6 +19,20 @@ const RESOLUTIONS: &[(u32, u32)] = &[
];
/// `0` = the display's native refresh, resolved at connect.
const REFRESH: &[u32] = &[0, 30, 60, 90, 120, 144, 165, 240];
/// Render-scale multipliers (persisted as f64; mirrors [`punktfunk_core::render_scale::PRESETS`]).
/// `1.0` = Native. Applied at connect and each match-window resize.
const RENDER_SCALES: &[f64] = &[0.5, 0.67, 0.75, 1.0, 1.25, 1.5, 2.0, 3.0, 4.0];
/// A compact label for a render-scale multiplier: "Native" / "1.5×" / "2× (supersample)".
fn render_scale_label(scale: f64) -> String {
if scale == 1.0 {
"Native".to_string()
} else if scale > 1.0 {
format!("{scale}\u{00D7} (supersample)")
} else {
format!("{scale}\u{00D7}")
}
}
/// Decode backend presets: `(stored value, display label)`.
// A stored legacy "hardware" (the D3D11VA era) matches no preset, so the combo shows
// Automatic — which is exactly how the session's decoder chain reads that value.
@@ -193,6 +207,24 @@ pub(crate) fn settings_page(
s.refresh_hz = REFRESH[i];
})
.tooltip("\u{201C}Native\u{201D} resolves to this display's refresh rate at connect.");
let (scale_names, scale_i) = {
let names: Vec<String> = RENDER_SCALES
.iter()
.map(|&x| render_scale_label(x))
.collect();
let i = RENDER_SCALES
.iter()
.position(|&x| (x - s.render_scale).abs() < 1e-6)
.unwrap_or_else(|| RENDER_SCALES.iter().position(|&x| x == 1.0).unwrap());
(names, i)
};
let scale_combo = setting_combo(ctx, "Render scale", scale_names, scale_i, |s, i| {
s.render_scale = RENDER_SCALES[i];
})
.tooltip(
"Supersample for sharpness (above 1\u{00D7}, more bandwidth and decode) or render below \
native (below 1\u{00D7}) for a lighter host \u{2014} this device resamples to the window.",
);
let (comp_names, comp_i) = presets(COMPOSITORS, |v| *v == s.compositor);
let comp_combo = setting_combo(ctx, "Host compositor", comp_names, comp_i, |s, i| {
s.compositor = COMPOSITORS[i].0.to_string();
@@ -441,6 +473,7 @@ pub(crate) fn settings_page(
settings_card(vec![
res_combo.into(),
hz_combo.into(),
scale_combo.into(),
fullscreen_toggle.into(),
comp_combo.into(),
]),
+52
View File
@@ -0,0 +1,52 @@
# Frame capture (plan §7 / §W6): the Linux xdg-ScreenCast/PipeWire portal capturer and the Windows
# IDD direct-push capturer, plus the synthetic sources + the Capturer trait, extracted into a
# subsystem crate. Depends on the shared frame vocabulary (pf-frame), the zero-copy plumbing
# (pf-zerocopy), and the display leaves (pf-win-display) — never on pf-encode: the encode-backend
# facts arrive pre-resolved (ZeroCopyPolicy) and the sealed-channel delivery as a closure
# (FrameChannelSender), so the capture→encode edge is one-way (plan §2.4).
[package]
name = "pf-capture"
version.workspace = true
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
description = "punktfunk host frame capture: Linux PipeWire portal + Windows IDD direct-push capturers behind one Capturer trait."
publish = false
[dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
pf-frame = { path = "../pf-frame" }
pf-zerocopy = { path = "../pf-zerocopy" }
pf-win-display = { path = "../pf-win-display" }
pf-gpu = { path = "../pf-gpu" }
pf-host-config = { path = "../pf-host-config" }
anyhow = "1"
tracing = "0.1"
[target.'cfg(target_os = "linux")'.dependencies]
# The xdg ScreenCast + RemoteDesktop portals, and the PipeWire consumer for the capture frames.
ashpd = { version = "0.13", features = ["screencast", "remote_desktop"] }
pipewire = "0.9"
libc = "0.2"
# ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane).
tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
[target.'cfg(target_os = "windows")'.dependencies]
# The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header).
pf-driver-proto = { path = "../pf-driver-proto" }
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_Security",
"Win32_Security_Authorization",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_Graphics_Direct3D_Fxc",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
"Win32_System_LibraryLoader",
"Win32_System_Memory",
"Win32_System_Threading",
"Win32_UI_HiDpi",
"Win32_UI_Input_KeyboardAndMouse",
"Win32_UI_WindowsAndMessaging",
] }
+357
View File
@@ -0,0 +1,357 @@
//! Frame capture (plan §7 / §W6): the capturers themselves — the Linux xdg-ScreenCast/PipeWire
//! portal capturer and the Windows IDD direct-push capturer — plus the synthetic test sources and
//! the `Capturer` trait, extracted into a subsystem crate. Speaks the shared frame vocabulary
//! (`pf-frame`) + the zero-copy plumbing (`pf-zerocopy`) and the display leaves (`pf-win-display`),
//! and NEVER `pf-encode` — the capture→encode edge is one-way (the encode-backend facts arrive
//! pre-resolved in a [`ZeroCopyPolicy`], and the Windows sealed-channel delivery arrives as a
//! [`FrameChannelSender`] closure, so this crate reaches neither the encoder nor the host
//! orchestrator).
// Scaffold: trait defaults + synthetic sources are defined ahead of the backends that use them.
#![allow(dead_code)]
// Every unsafe block in this crate carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::Result;
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
// The Linux capturer reaches `DmabufFrame` through `super::`; `CursorOverlay` it names directly as
// `pf_frame::CursorOverlay`, so only `DmabufFrame` needs to sit in this crate root's scope.
#[cfg(target_os = "linux")]
use pf_frame::DmabufFrame;
/// Produces frames from a captured output. Lives on its own thread, feeding the encoder
/// over a bounded drop-oldest channel (never block the compositor).
pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>;
/// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal
/// overrides it to drain its channel without blocking.
fn try_latest(&mut self) -> Result<Option<CapturedFrame>> {
self.next_frame().map(Some)
}
/// Gate expensive per-frame work so the capturer can be kept alive (reused) between
/// streams without burning CPU. The portal capturer skips the de-pad copy while inactive;
/// the default is a no-op (synthetic sources are produced on demand). Set `true` for the
/// duration of a stream, `false` when it ends.
fn set_active(&self, _active: bool) {}
/// The source's static HDR mastering metadata (SMPTE ST.2086 + content light level), when the
/// capturer can read it from the output (Windows `IDXGIOutput6::GetDesc1`). `None` = unknown /
/// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet).
/// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram),
/// so the two stay a single source of truth. May change mid-session if the source is regraded.
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None
}
/// How many frames the encode loop may keep in flight (submitted but not yet polled) before it
/// blocks. `1` (the default) is the synchronous loop: capture → submit → poll-blocks, so the
/// per-frame wall time is `capture+convert + encode`. A capturer that hands a fresh output texture
/// per frame (so the encode of N reads a different texture than the convert of N+1 writes) can return
/// `>1` to PIPELINE: the loop submits N+1 before polling N, overlapping the convert/copy on the 3D
/// engine with the NVENC-ASIC encode of the prior frame, dropping per-frame wall toward `max(...)`.
fn pipeline_depth(&self) -> usize {
1
}
/// The OS display-target id this capturer is bound to (Windows IDD-push), so the resize path
/// can verify the display it just reconfigured is STILL the one this capturer serves (an
/// in-place resize keeps the target; a re-arrival fallback mints a new one, which needs a
/// fresh capturer). `None` = the backend has no such identity (every non-IDD backend).
fn capture_target_id(&self) -> Option<u32> {
None
}
/// HOST-INITIATED output resize (latency plan P2.3): the session's resize handler has ALREADY
/// committed the display's new mode (the manager's in-place mode set), so a capable capturer
/// re-sizes its capture surface NOW — no descriptor-poll debounce (that machinery stays, for
/// EXTERNAL changes only) and no teardown: the capture pipeline and its send thread survive;
/// only the encoder is swapped by the caller once the first new-size frame arrives. Returns
/// `true` when handled; `false` (the default) routes the caller to the full-rebuild path.
fn resize_output(&mut self, _width: u32, _height: u32) -> bool {
false
}
}
/// A deterministic moving test pattern (BGRx). Lets the spike exercise the encode → file →
/// `punktfunk_core` path with no live capture session, and produces obviously non-static
/// content (a sweeping bar + animated gradient) so the encoded output is verifiable.
pub struct SyntheticCapturer {
width: u32,
height: u32,
fps: u32,
frame_idx: u64,
buf: Vec<u8>,
}
impl SyntheticCapturer {
const BPP: usize = 4; // emits BGRx
pub fn new(width: u32, height: u32, fps: u32) -> Self {
assert!(width > 0 && height > 0 && fps > 0);
let buf = vec![0u8; width as usize * height as usize * Self::BPP];
SyntheticCapturer {
width,
height,
fps,
frame_idx: 0,
buf,
}
}
}
impl Capturer for SyntheticCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
let w = self.width as usize;
let h = self.height as usize;
let bpp = Self::BPP;
let t = self.frame_idx;
// A vertical bar sweeps left→right once every ~2s; the background is a gradient
// whose phase advances each frame, so every pixel changes frame-to-frame.
let bar_x = ((t * w as u64) / (self.fps as u64 * 2)) % w as u64;
let phase = (t % 256) as usize;
for y in 0..h {
let row = y * w * bpp;
for x in 0..w {
let i = row + x * bpp;
let on_bar = (x as u64).abs_diff(bar_x) < 8;
// BGRx byte order: [B, G, R, x]
self.buf[i] = if on_bar {
255
} else {
((x + phase) & 0xff) as u8
};
self.buf[i + 1] = if on_bar {
255
} else {
((y + phase) & 0xff) as u8
};
self.buf[i + 2] = if on_bar { 255 } else { ((x + y) & 0xff) as u8 };
self.buf[i + 3] = 0;
}
}
let pts_ns = self.frame_idx * 1_000_000_000 / self.fps as u64;
self.frame_idx += 1;
Ok(CapturedFrame {
width: self.width,
height: self.height,
pts_ns,
format: PixelFormat::Bgrx,
payload: FramePayload::Cpu(self.buf.clone()),
cursor: None,
})
}
}
/// A cheap moving test pattern (BGRx) for the streaming path: a pulsing field + a white band
/// sweeping down, generated with whole-buffer `fill`s so it stays real-time even at 5K.
pub struct FastSyntheticCapturer {
width: u32,
height: u32,
frame_idx: u64,
buf: Vec<u8>,
/// PUNKTFUNK_SYNTH_NOISE: every frame is fresh high-entropy noise NVENC can't compress or
/// predict, so the encoder hits its (CBR) bitrate target — a throughput test of the real
/// encode→FEC→send→recv path. The default flat/band content compresses to ~nothing, so it
/// can't generate real Mbps (the encoder is content-driven). xorshift over u64 chunks.
noise: bool,
rng: u64,
}
impl FastSyntheticCapturer {
pub fn new(width: u32, height: u32) -> Self {
assert!(width > 0 && height > 0);
FastSyntheticCapturer {
width,
height,
frame_idx: 0,
buf: vec![0u8; width as usize * height as usize * 4],
noise: std::env::var_os("PUNKTFUNK_SYNTH_NOISE").is_some(),
rng: 0x9e3779b97f4a7c15,
}
}
}
impl Capturer for FastSyntheticCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
if self.noise {
// Fresh, every-frame-decorrelated noise: reseed from the frame index so consecutive
// frames share no structure (forces large P-frames too, not just the keyframe).
let mut s = self
.rng
.wrapping_add(self.frame_idx.wrapping_mul(0x2545F491_4F6CDD1D))
| 1;
for c in self.buf.chunks_exact_mut(8) {
s ^= s << 13;
s ^= s >> 7;
s ^= s << 17;
c.copy_from_slice(&s.to_le_bytes());
}
self.rng = s;
} else {
let (w, h) = (self.width as usize, self.height as usize);
let row = w * 4;
let shade = (self.frame_idx % 256) as u8;
self.buf.fill(shade);
let band_h = (h / 20).max(1);
let band_y = (self.frame_idx as usize * 6) % h;
for y in band_y..(band_y + band_h).min(h) {
self.buf[y * row..(y + 1) * row].fill(0xff);
}
}
self.frame_idx += 1;
Ok(CapturedFrame {
width: self.width,
height: self.height,
pts_ns: 0,
format: PixelFormat::Bgrx,
payload: FramePayload::Cpu(self.buf.clone()),
cursor: None,
})
}
}
/// The encode-backend facts the Linux zero-copy negotiation needs, resolved **once** here (the host
/// facade, which may reach the host `encode`) and passed **into** the capturer — so the capturer never
/// calls back into `encode`, keeping the capture→encode dependency one-way (plan §2.4 / §W6). The
/// three facts were formerly re-derived inside the PipeWire thread via
/// `encode::{linux_zero_copy_is_vaapi, resolved_backend_is_gpu, pyrowave_capture_modifiers}`.
#[cfg(target_os = "linux")]
#[derive(Clone, Default)]
pub struct ZeroCopyPolicy {
/// The GPU encode backend resolves to VAAPI (AMD/Intel) — the capturer hands raw dmabufs
/// straight through instead of the EGL→CUDA import (the host `encode::linux_zero_copy_is_vaapi`).
pub backend_is_vaapi: bool,
/// The resolved backend produces GPU-resident frames (everything but the software encoder) —
/// used only to phrase the CPU-fallback warning (the host `encode::resolved_backend_is_gpu`).
pub backend_is_gpu: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the encoder pref is `pyrowave` (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
pub pyrowave_modifiers: Vec<u64>,
}
#[cfg(target_os = "linux")]
pub fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
true
}
#[cfg(target_os = "windows")]
pub fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
// but only a backend that ingests RGB and CSCs it to 4:4:4 itself can use it — today just
// direct-NVENC (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). An HDR
// display can't be known here (the virtual display's mode settles after the Welcome); that
// combination downgrades at capture time — the capturer emits P010 and the encoder's caps
// cross-check reports the 4:2:0 truth (the in-band SPS keeps the client correct either way).
encoder_ingests_rgb_444
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
false
}
/// Host-registered HID compose-kick hook: `(target_rect, desktop_bounds) -> accepted`, both
/// `(x, y, w, h)` in desktop coordinates (from CCD). The host facade registers it once at startup
/// when the resident virtual HID mouse exists (`inject::mouse_windows::hid_kick`); the IDD-push
/// capturer's compose kick then prefers it over `SendInput`, because device-level input is
/// delivered regardless of this process's session or the active desktop and wakes a powered-off
/// display — the lid-closed first-frame fix. Same one-way-edge philosophy as
/// [`FrameChannelSender`]: this crate never reaches back into the host's inject module. `false`
/// from the hook = mouse not available right now → the caller falls back to `SendInput`.
#[cfg(target_os = "windows")]
pub static HID_COMPOSE_KICK: std::sync::OnceLock<HidKickFn> = std::sync::OnceLock::new();
/// The [`HID_COMPOSE_KICK`] hook's shape: `(target_rect, desktop_bounds) -> accepted`, both
/// `(x, y, w, h)` in desktop coordinates.
#[cfg(target_os = "windows")]
pub type HidKickFn = fn((i32, i32, i32, i32), (i32, i32, i32, i32)) -> bool;
/// Delivers a monitor's sealed frame channel to the pf-vdisplay driver (`IOCTL_SET_FRAME_CHANNEL`) —
/// the ONE reach the IDD-push capturer would otherwise make into the host's `vdisplay` module. The
/// host facade builds this closure (capturing the pf-vdisplay control device handle + the
/// `send_frame_channel` IOCTL wrapper) and hands it in, so this crate delivers the channel without a
/// path back to the orchestrator. Called once per ring generation (at attach), never per-frame —
/// guardrail-compliant. The handle values in `req` were just duplicated into the driver's WUDFHost
/// by the capturer's [`windows::idd_push`] broker; on IOCTL success the DRIVER owns them.
#[cfg(target_os = "windows")]
pub type FrameChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync,
>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")]
pub mod pwinit;
// The Windows backend lives under `windows/`, the Linux one under `linux/`. Windows capture is IDD
// direct-push only (DXGI Desktop Duplication + the WGC relay were removed).
#[cfg(target_os = "windows")]
#[path = "windows/dxgi.rs"]
pub mod dxgi;
#[cfg(target_os = "windows")]
#[path = "windows/idd_push.rs"]
mod idd_push;
// The WUDFHost-identity check the IDD-push broker uses is reused by the host's gamepad-channel
// bootstrap (`inject::windows::gamepad_raii`); re-export it so that reach stays a leaf dependency.
#[cfg(target_os = "windows")]
pub use idd_push::verify_is_wudfhost;
#[cfg(target_os = "linux")]
#[path = "linux/mod.rs"]
mod linux;
#[cfg(target_os = "windows")]
#[path = "windows/synthetic_nv12.rs"]
pub mod synthetic_nv12;
/// Open the Linux xdg-ScreenCast portal capturer for a client-sized monitor. `anchored` drives
/// ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits that grant headlessly. The
/// [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
#[cfg(target_os = "linux")]
pub fn open_portal_monitor(anchored: bool, policy: ZeroCopyPolicy) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::open(anchored, policy).map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// Open the Linux portal capturer bound to an already-created virtual output's PipeWire node. The
/// caller (host facade) explodes its `VirtualOutput` into these primitives + owns nothing after —
/// the capturer takes `keepalive`, so dropping it releases the output. `allow_zerocopy` mirrors
/// `OutputFormat::gpu`; `want_444` selects the planar-YUV444 GPU convert.
#[cfg(target_os = "linux")]
#[allow(clippy::too_many_arguments)]
pub fn open_virtual_output(
remote_fd: Option<std::os::fd::OwnedFd>,
node_id: u32,
preferred_mode: Option<(u32, u32, u32)>,
keepalive: Box<dyn Send>,
allow_zerocopy: bool,
want_444: bool,
policy: ZeroCopyPolicy,
) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::from_virtual_output(
remote_fd,
node_id,
preferred_mode,
keepalive,
allow_zerocopy,
want_444,
policy,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// Open the Windows IDD direct-push capturer on a pf-vdisplay target. `sender` delivers the sealed
/// frame channel to the driver (the host facade builds it from the vdisplay control device). On
/// failure the `keepalive` is handed back so the caller can retire the display.
#[cfg(target_os = "windows")]
#[allow(clippy::too_many_arguments)]
pub fn open_idd_push(
target: pf_frame::dxgi::WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
client_10bit: bool,
want_444: bool,
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(target, preferred, client_10bit, want_444, keepalive, sender)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
@@ -20,7 +20,7 @@
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat};
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy};
use anyhow::{anyhow, Context, Result};
use std::os::fd::OwnedFd;
use std::sync::atomic::{AtomicBool, Ordering};
@@ -55,7 +55,7 @@ pub struct PortalCapturer {
stall_since: Option<std::time::Instant>,
/// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If
/// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches
/// the process-wide downgrade ([`crate::zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
/// rebuild retries on the CPU offer instead of failing identically forever.
vaapi_dmabuf: bool,
/// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis.
@@ -77,7 +77,7 @@ impl PortalCapturer {
/// `anchored` drives ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits the
/// RemoteDesktop grant and never raises a separate ScreenCast dialog; `false` uses a plain
/// ScreenCast session (wlroots, which has no RemoteDesktop portal).
pub fn open(anchored: bool) -> Result<PortalCapturer> {
pub fn open(anchored: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> {
// Portal handshake (async) on its own thread; hands back the PW fd + node id.
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
thread::Builder::new()
@@ -101,37 +101,45 @@ impl PortalCapturer {
"ScreenCast portal session started; connecting PipeWire"
);
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok(spawn_pipewire(Some(fd), node_id, None, true, false)?.into_capturer(node_id, None))
Ok(
spawn_pipewire(Some(fd), node_id, None, true, false, policy)?
.into_capturer(node_id, None),
)
}
/// Build a capturer from an already-created virtual output ([`crate::vdisplay::VirtualOutput`]):
/// connect PipeWire to its node (`remote_fd` selects portal-remote vs. default-daemon) and
/// take ownership of its keepalive so the output lives exactly as long as this capturer. This
/// is how the client's requested resolution becomes the captured resolution without scaling.
/// `allow_zerocopy` mirrors [`OutputFormat::gpu`](crate::capture::OutputFormat): `false` forces
/// the CPU mmap path, `true` keeps the GPU zero-copy path subject to `PUNKTFUNK_ZEROCOPY`.
/// `want_444` (a 4:4:4 session) makes the zero-copy worker convert tiled dmabufs to planar
/// YUV444 on the GPU instead of NV12/RGB.
/// Build a capturer from an already-created virtual output's PipeWire node. The host facade
/// explodes its `vdisplay::VirtualOutput` into these primitives so this crate never depends on
/// the vdisplay type: `remote_fd` selects portal-remote vs. default-daemon, `node_id` is the
/// output's screencast node, `preferred_mode` seeds format negotiation, and `keepalive` owns the
/// output (dropping the capturer releases it). `allow_zerocopy` mirrors
/// [`OutputFormat::gpu`](pf_frame::OutputFormat): `false` forces the CPU mmap path, `true` keeps
/// the GPU zero-copy path subject to `PUNKTFUNK_ZEROCOPY`. `want_444` (a 4:4:4 session) makes the
/// zero-copy worker convert tiled dmabufs to planar YUV444 on the GPU instead of NV12/RGB.
#[allow(clippy::too_many_arguments)]
pub fn from_virtual_output(
vout: crate::vdisplay::VirtualOutput,
remote_fd: Option<OwnedFd>,
node_id: u32,
preferred_mode: Option<(u32, u32, u32)>,
keepalive: Box<dyn Send>,
allow_zerocopy: bool,
want_444: bool,
policy: ZeroCopyPolicy,
) -> Result<PortalCapturer> {
tracing::info!(
node_id = vout.node_id,
node_id,
allow_zerocopy,
want_444,
"connecting PipeWire to virtual output"
);
let node_id = vout.node_id;
Ok(spawn_pipewire(
vout.remote_fd,
remote_fd,
node_id,
vout.preferred_mode,
preferred_mode,
allow_zerocopy,
want_444,
policy,
)?
.into_capturer(node_id, Some(vout.keepalive)))
.into_capturer(node_id, Some(keepalive)))
}
}
@@ -186,6 +194,9 @@ fn spawn_pipewire(
// 4:4:4 session: tiled dmabufs convert to planar YUV444 on the GPU (`ImportKind::Tiled444`)
// instead of NV12/RGB, so the session stays zero-copy at full chroma.
want_444: bool,
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6).
policy: ZeroCopyPolicy,
) -> Result<PwHandles> {
// Frames flow from the pipewire thread over a small bounded channel.
let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8);
@@ -201,13 +212,13 @@ fn spawn_pipewire(
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope.
let (quit_tx, quit_rx) = ::pipewire::channel::channel::<()>();
let zerocopy = allow_zerocopy && crate::zerocopy::enabled();
let zerocopy = allow_zerocopy && pf_zerocopy::enabled();
// Mirror of the thread's `vaapi_passthrough` decision (deterministic from here: on a VAAPI
// backend the EGL→CUDA importer is never built) — kept on the capturer so `next_frame`'s
// negotiation-timeout branch knows a failed negotiation was the LINEAR-dmabuf offer.
let vaapi_dmabuf = zerocopy
&& std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() != Ok("1")
&& crate::encode::linux_zero_copy_is_vaapi();
&& policy.backend_is_vaapi;
let join = thread::Builder::new()
.name("punktfunk-pipewire".into())
.spawn(move || {
@@ -223,6 +234,7 @@ fn spawn_pipewire(
want_444,
preferred,
quit_rx,
policy,
) {
tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed");
}
@@ -330,11 +342,11 @@ impl PortalCapturer {
or capture never started)",
self.node_id
))
} else if self.vaapi_dmabuf && !crate::zerocopy::vaapi_dmabuf_forced() {
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
// The LINEAR-dmabuf-only offer (VAAPI passthrough default) was never accepted.
// Latch the process-wide downgrade so the encode loop's pipeline rebuild
// retries on the CPU offer instead of failing this same negotiation forever.
crate::zerocopy::note_vaapi_dmabuf_failed();
pf_zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this host to the \
@@ -604,7 +616,7 @@ fn portal_thread_remote_desktop(setup_tx: std::sync::mpsc::Sender<Result<(OwnedF
mod pipewire {
//! The PipeWire consumer, confined to its own thread (the PW types are `!Send`).
use super::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat};
use super::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy};
use anyhow::{Context, Result};
use pipewire as pw;
use pw::{properties::properties, spa};
@@ -655,11 +667,11 @@ mod pipewire {
impl CursorState {
/// A shareable overlay for the GPU encode paths (blended at encode time), or `None` when
/// there is nothing to draw. Cheap: clones an `Arc` + a few scalars.
fn overlay(&self) -> Option<crate::capture::CursorOverlay> {
fn overlay(&self) -> Option<pf_frame::CursorOverlay> {
if !self.visible || self.rgba.is_empty() {
return None;
}
Some(crate::capture::CursorOverlay {
Some(pf_frame::CursorOverlay {
x: self.x,
y: self.y,
w: self.bw,
@@ -693,8 +705,8 @@ mod pipewire {
/// Consecutive tiled-import failures (reset on success); see [`IMPORT_FAIL_POISON`].
import_fail_streak: u32,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer,
/// normally via the isolated worker process (`crate::zerocopy::Importer::Remote`).
importer: Option<crate::zerocopy::Importer>,
/// normally via the isolated worker process (`pf_zerocopy::Importer::Remote`).
importer: Option<pf_zerocopy::Importer>,
/// VAAPI zero-copy: hand the raw dmabuf to the encoder (which imports + GPU-CSCs it) instead
/// of a CUDA import. Set when zero-copy is on, the EGL→CUDA importer is unavailable, and the
/// encoder backend is VAAPI (AMD/Intel).
@@ -1205,7 +1217,7 @@ mod pipewire {
// closing the stale/old-frame race on NVIDIA. No-op for shm buffers or drivers that
// attach no fence. Covers both the GPU import and the CPU mmap read below.
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
match crate::dmabuf_fence::wait_read_ready(datas[0].fd(), 100) {
match pf_zerocopy::dmabuf_fence::wait_read_ready(datas[0].fd(), 100) {
Ok(waited) => {
static F1: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(true);
@@ -1237,7 +1249,7 @@ mod pipewire {
if ud.vaapi_passthrough {
if let Some(fmt) = ud.format {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
if let Some(fourcc) = crate::zerocopy::drm_fourcc(fmt) {
if let Some(fourcc) = pf_frame::drm_fourcc(fmt) {
let chunk = datas[0].chunk();
let offset = chunk.offset();
let stride = chunk.stride().max(0) as u32;
@@ -1300,7 +1312,7 @@ mod pipewire {
let mut gpu_import_broken = false;
if let (Some(importer), Some(fmt)) = (ud.importer.as_mut(), ud.format) {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
let plane = crate::zerocopy::DmabufPlane {
let plane = pf_zerocopy::DmabufPlane {
fd: datas[0].fd(),
offset: datas[0].chunk().offset(),
stride: datas[0].chunk().stride().max(0) as u32,
@@ -1309,7 +1321,7 @@ mod pipewire {
// gamescope) → direct CUDA external-memory import (NVIDIA EGL can't
// sample LINEAR).
let modifier = (ud.modifier != 0).then_some(ud.modifier);
if let Some(fourcc) = crate::zerocopy::drm_fourcc(fmt) {
if let Some(fourcc) = pf_frame::drm_fourcc(fmt) {
// GPU converts only on the tiled EGL/GL path (`modifier.is_some()`): a 4:4:4
// session gets the planar-YUV444 convert (full chroma, takes precedence over
// NV12 — 4:4:4 must never subsample), otherwise `PUNKTFUNK_NV12` gets NV12 —
@@ -1333,7 +1345,7 @@ mod pipewire {
match imported {
Ok(devbuf) => {
ud.import_fail_streak = 0;
crate::zerocopy::note_gpu_import_ok();
pf_zerocopy::note_gpu_import_ok();
static ONCE: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
@@ -1371,7 +1383,7 @@ mod pipewire {
Err(e) => {
let dead = importer.dead();
if dead {
crate::zerocopy::note_gpu_import_death();
pf_zerocopy::note_gpu_import_death();
}
if modifier.is_some() {
// Tiled buffer: the CPU fallback below would mmap TILED bytes
@@ -1549,6 +1561,9 @@ mod pipewire {
want_444: bool,
preferred: Option<(u32, u32, u32)>,
quit_rx: pw::channel::Receiver<()>,
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6).
policy: ZeroCopyPolicy,
) -> Result<()> {
crate::pwinit::ensure_init();
@@ -1581,15 +1596,15 @@ mod pipewire {
// would waste a CUDA probe — or worse, on an NVIDIA box forced to PUNKTFUNK_ENCODER=vaapi,
// succeed and produce CUDA payloads the VAAPI encoder must reject. Also skipped once
// repeated worker deaths latched the import off (a wedged GPU stack must not crash-loop).
let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi();
let backend_is_vaapi = policy.backend_is_vaapi;
let mut importer = if zerocopy && !backend_is_vaapi {
if crate::zerocopy::gpu_import_disabled() {
if pf_zerocopy::gpu_import_disabled() {
tracing::warn!(
"zero-copy GPU import disabled after repeated import-worker deaths — using CPU path"
);
None
} else {
match crate::zerocopy::Importer::new_for_capture() {
match pf_zerocopy::Importer::new_for_capture() {
Ok(i) => Some(i),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "zero-copy import unavailable — using CPU path");
@@ -1617,19 +1632,19 @@ mod pipewire {
// radeonsi/iHD import it and any compositor can allocate it.
let mut modifiers = importer
.as_mut()
.map(|i| i.supported_modifiers(crate::zerocopy::drm_fourcc(PixelFormat::Bgrx).unwrap()))
.map(|i| i.supported_modifiers(pf_frame::drm_fourcc(PixelFormat::Bgrx).unwrap()))
.unwrap_or_default();
if (importer.is_some() || vaapi_passthrough) && !modifiers.contains(&0) {
modifiers.push(0); // DRM_FORMAT_MOD_LINEAR
}
// PyroWave passthrough: the encoder imports through Vulkan, not libva — extend the
// advertisement with every modifier its device samples from, so compositors that
// never allocate LINEAR (Mutter+NVIDIA) still negotiate zero-copy dmabufs.
#[cfg(feature = "pyrowave")]
if vaapi_passthrough && crate::config::config().encoder_pref.as_str() == "pyrowave" {
for m in crate::encode::pyrowave_capture_modifiers(
crate::zerocopy::drm_fourcc(PixelFormat::Bgrx).unwrap(),
) {
// never allocate LINEAR (Mutter+NVIDIA) still negotiate zero-copy dmabufs. The modifiers
// were resolved by the facade (`ZeroCopyPolicy::pyrowave_modifiers`) — non-empty only when
// the host's `pyrowave` feature is on AND the encoder pref is `pyrowave` — so capture never
// calls back into `encode` and needs no feature gate of its own (the emptiness check gates it).
if vaapi_passthrough && !policy.pyrowave_modifiers.is_empty() {
for &m in &policy.pyrowave_modifiers {
if !modifiers.contains(&m) {
modifiers.push(m);
}
@@ -1657,7 +1672,7 @@ mod pipewire {
sample = ?&modifiers[..modifiers.len().min(6)],
"zero-copy: advertising EGL-importable dmabuf modifiers"
);
} else if backend_is_vaapi && crate::encode::resolved_backend_is_gpu() {
} else if backend_is_vaapi && policy.backend_is_gpu {
// A VAAPI session on the CPU path pays three full-frame CPU touches (mmap de-pad +
// swscale RGB→NV12 + surface upload) — make the silent fallback visible.
tracing::warn!(
@@ -1676,7 +1691,7 @@ mod pipewire {
"4:4:4 zero-copy: tiled dmabufs convert to planar YUV444 (BT.709) on the GPU — \
NVENC fed native full-chroma YUV, no CPU pixel path"
);
} else if want_dmabuf && !vaapi_passthrough && crate::zerocopy::nv12_enabled() {
} else if want_dmabuf && !vaapi_passthrough && pf_zerocopy::nv12_enabled() {
tracing::info!(
"PUNKTFUNK_NV12: tiled dmabufs convert to NV12 (BT.709 limited) on the GPU — NVENC \
fed native YUV (no internal RGBYUV CSC)"
@@ -1695,7 +1710,7 @@ mod pipewire {
import_fail_streak: 0,
importer,
vaapi_passthrough,
nv12: crate::zerocopy::nv12_enabled(),
nv12: pf_zerocopy::nv12_enabled(),
yuv444: want_444,
dbg_log_n: 0,
cursor: CursorState::default(),
@@ -1,20 +1,27 @@
//! Shared Windows GPU primitives — D3D11 device creation, GPU scheduling priority hooks,
//! HLSL shader compilation, HDR FP16→P010 conversion ([`HdrP010Converter`]), video-engine
//! colour conversion ([`VideoConverter`]), and the IDD-push capture identity
//! ([`WinCaptureTarget`], [`pack_luid`]). Consumed by [`super::idd_push`].
//! DXGI Desktop Duplication has been removed; this module contains no capturer.
//! Windows capture GPU mechanics — the win32u GPU-preference hook, HLSL shader compilation, HDR
//! FP16→P010 conversion ([`HdrP010Converter`]), video-engine colour conversion ([`VideoConverter`]),
//! and the P010 self-test. Consumed by [`super::idd_push`].
//!
//! The shared IDD-push capture IDENTITY — [`WinCaptureTarget`], [`D3d11Frame`], [`pack_luid`], and
//! [`make_device`] (the D3D11 device factory + GPU scheduling-priority hardening) — moved into the
//! `pf-frame` leaf crate so capture, encode, and pf-vdisplay share one identity type without a
//! capture↔encode↔vdisplay cycle (plan §W6); this module re-exports it so every existing
//! `crate::dxgi::*` path keeps resolving. DXGI Desktop Duplication has been removed; this
//! module contains no capturer.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
pub use pf_frame::dxgi::{make_device, pack_luid, D3d11Frame, WinCaptureTarget};
use anyhow::{bail, Context, Result};
use std::ffi::c_void;
use std::sync::atomic::{AtomicU64, Ordering};
use windows::core::{s, Interface, PCSTR};
use windows::Win32::Foundation::{HMODULE, LUID};
use windows::Win32::Foundation::HMODULE;
use windows::Win32::Graphics::Direct3D::Fxc::D3DCompile;
use windows::Win32::Graphics::Direct3D::{
ID3DBlob, D3D_DRIVER_TYPE_UNKNOWN, D3D_FEATURE_LEVEL_11_0, D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST,
ID3DBlob, D3D_FEATURE_LEVEL_11_0, D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST,
};
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, ID3D11Buffer, ID3D11Device, ID3D11DeviceContext, ID3D11PixelShader,
@@ -32,205 +39,6 @@ use windows::Win32::Graphics::Dxgi::Common::{
DXGI_FORMAT, DXGI_FORMAT_P010, DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM,
DXGI_FORMAT_R16_UNORM, DXGI_SAMPLE_DESC,
};
use windows::Win32::Graphics::Dxgi::{IDXGIAdapter1, IDXGIDevice, IDXGIDevice1};
#[derive(Clone)]
pub struct WinCaptureTarget {
/// Packed DXGI adapter LUID (`(HighPart << 32) | (LowPart & 0xffff_ffff)`).
pub adapter_luid: i64,
/// The output's GDI device name, e.g. `\\.\DISPLAY3`. Can CHANGE across a secure-desktop switch.
pub gdi_name: String,
/// Stable virtual-display (IddCx) target id — re-resolved to the current GDI name on every recovery.
pub target_id: u32,
/// The pf-vdisplay driver's WUDFHost pid (from the ADD reply) — the process the IDD-push capturer
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32,
}
/// A GPU-resident captured texture (future NVENC-D3D11 zero-copy path).
pub struct D3d11Frame {
pub texture: ID3D11Texture2D,
pub device: ID3D11Device,
}
// SAFETY: `D3d11Frame` owns an `ID3D11Texture2D` + `ID3D11Device`, which are COM interface pointers.
// D3D11 devices/resources use thread-safe (interlocked) COM reference counting, and the device is
// created free-threaded (`make_device` passes no `D3D11_CREATE_DEVICE_SINGLETHREADED`), so handing
// ownership of the frame to another thread — the capture→encode handoff — and releasing it there is
// sound. The value is moved, never aliased (no `Sync`), so there is no concurrent use of the
// single-threaded immediate context.
unsafe impl Send for D3d11Frame {}
pub fn pack_luid(luid: LUID) -> i64 {
((luid.HighPart as i64) << 32) | (luid.LowPart as i64 & 0xffff_ffff)
}
/// Create a fresh D3D11 device + context on a specific adapter (driver_type UNKNOWN with an explicit
/// adapter). Used at open and on every ACCESS_LOST: a device created on one desktop cannot sustain a
/// duplication on a *different* desktop (perpetual ACCESS_LOST), so the secure-desktop switch needs a
/// device made while the thread is attached to that desktop.
pub(crate) unsafe fn make_device(
adapter: &IDXGIAdapter1,
) -> Result<(ID3D11Device, ID3D11DeviceContext)> {
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice")?;
let device = device.context("null D3D11 device")?;
let context = context.context("null D3D11 context")?;
// GPU scheduling hardening — the same approach Sunshine/Apollo use, reimplemented here via the
// documented D3DKMT/DXGI APIs (no GPL source copied). Our capture+encode
// shares the GPU with the streamed game; when the game saturates the GPU our process is starved of
// GPU time slices, so NVENC sits near-idle yet `lock_bitstream` waits ~20 ms for our context to be
// scheduled — capping the stream (~47 fps measured at 5K@240) and stuttering. Per-frame copy/convert
// is NOT the cause (zero-copy + thread-priority alone didn't move it); the PROCESS-level GPU
// scheduling priority class is the decisive cross-process lever. Secondary: the absolute per-device
// GPU thread priority and a 1-frame latency cap.
elevate_process_gpu_priority();
if let Ok(dxgi_dev) = device.cast::<IDXGIDevice>() {
// The absolute max GPU thread priority (0x4000001E; the same value Sunshine/Apollo use); fall back to relative +7.
if dxgi_dev.SetGPUThreadPriority(0x4000_001E).is_err()
&& dxgi_dev.SetGPUThreadPriority(7).is_err()
{
tracing::warn!("SetGPUThreadPriority failed (run as admin/SYSTEM for GPU priority)");
}
}
if let Ok(dxgi1) = device.cast::<IDXGIDevice1>() {
let _ = dxgi1.SetMaximumFrameLatency(1);
}
Ok((device, context))
}
/// Resolve the configured GPU scheduling-priority class from `PUNKTFUNK_GPU_PRIORITY_CLASS`
/// (`off|normal|high|realtime`, default high). `None` = leave it at the OS default (the `off` opt-out).
/// D3DKMT_SCHEDULINGPRIORITYCLASS: IDLE 0, BELOW_NORMAL 1, NORMAL 2, ABOVE_NORMAL 3, HIGH 4, REALTIME 5.
fn configured_gpu_priority_class() -> Option<i32> {
match std::env::var("PUNKTFUNK_GPU_PRIORITY_CLASS")
.ok()
.as_deref()
{
Some("off") => None,
Some("normal") => Some(2),
Some("realtime") => Some(5),
_ => Some(4), // HIGH — safe on NVIDIA+HAGS (realtime can freeze NVENC)
}
}
/// Enable SE_INC_BASE_PRIORITY on the CURRENT process token (best-effort) — the kernel gates the
/// HIGH/REALTIME GPU scheduling-priority bump on it. Held by SYSTEM/Administrators; a UAC-FILTERED
/// token does NOT have it, which is why `elevate_process_gpu_priority` may silently no-op in a
/// restricted service context.
unsafe fn enable_inc_base_priority() {
use windows::core::PCWSTR;
use windows::Win32::Foundation::{CloseHandle, HANDLE, LUID};
use windows::Win32::Security::{
AdjustTokenPrivileges, LookupPrivilegeValueW, LUID_AND_ATTRIBUTES,
SE_INC_BASE_PRIORITY_NAME, SE_PRIVILEGE_ENABLED, TOKEN_ADJUST_PRIVILEGES, TOKEN_PRIVILEGES,
TOKEN_QUERY,
};
use windows::Win32::System::Threading::{GetCurrentProcess, OpenProcessToken};
let mut token = HANDLE::default();
if OpenProcessToken(
GetCurrentProcess(),
TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY,
&mut token,
)
.is_ok()
{
let mut luid = LUID::default();
if LookupPrivilegeValueW(PCWSTR::null(), SE_INC_BASE_PRIORITY_NAME, &mut luid).is_ok() {
let tp = TOKEN_PRIVILEGES {
PrivilegeCount: 1,
Privileges: [LUID_AND_ATTRIBUTES {
Luid: luid,
Attributes: SE_PRIVILEGE_ENABLED,
}],
};
if AdjustTokenPrivileges(
token,
false,
Some(&tp as *const TOKEN_PRIVILEGES),
0,
None,
None,
)
.is_err()
{
tracing::warn!("could not enable SE_INC_BASE_PRIORITY for GPU priority");
}
}
let _ = CloseHandle(token);
}
}
/// Call `gdi32!D3DKMTSetProcessSchedulingPriorityClass(process, prio)` (no stable windows-rs binding —
/// loaded by name). Returns the NTSTATUS (0 = success) or `None` if the export can't be resolved. The
/// CALLING process must hold SE_INC_BASE_PRIORITY ([`enable_inc_base_priority`]) for HIGH/REALTIME; the
/// kernel checks the caller's privilege whether the target is self or a child we created.
unsafe fn d3dkmt_set_scheduling_priority_class(
process: windows::Win32::Foundation::HANDLE,
prio: i32,
) -> Option<i32> {
use windows::core::s;
use windows::Win32::Foundation::HANDLE;
use windows::Win32::System::LibraryLoader::{GetProcAddress, LoadLibraryA};
let gdi32 = LoadLibraryA(s!("gdi32.dll")).ok()?;
let p = GetProcAddress(gdi32, s!("D3DKMTSetProcessSchedulingPriorityClass"))?;
type SetPrio = unsafe extern "system" fn(HANDLE, i32) -> i32;
let f: SetPrio = std::mem::transmute(p);
Some(f(process, prio))
}
/// GPU scheduling-priority hardening — the same approach as Sunshine/Apollo, independently
/// implemented via the documented D3DKMT APIs (no GPL source copied). On a
/// GPU-saturated game our capture+encode process is starved of GPU time slices — NVENC sits ~idle but
/// `lock_bitstream` waits ~20 ms for our context to be scheduled. Elevating the PROCESS GPU scheduling
/// priority class (the strong cross-process lever — far more effective than `SetGPUThreadPriority`
/// alone, which we measured as no help) lets our brief encode preempt the game. Uses HIGH, NOT
/// realtime: realtime on NVIDIA + HAGS can freeze/crash NVENC (Apollo downgrades it for exactly this).
/// Runs once per process; best-effort. `PUNKTFUNK_GPU_PRIORITY_CLASS = off|normal|high|realtime`
/// (default high). Best-effort: silently no-ops under a UAC-filtered token (the process will not
/// hold SE_INC_BASE_PRIORITY, so the D3DKMT call is a no-op).
fn elevate_process_gpu_priority() {
use std::sync::Once;
static ONCE: Once = Once::new();
// SAFETY: the closure calls two of this module's `unsafe fn`s — `enable_inc_base_priority`
// (adjusts the current-process token; it has no caller precondition and builds all its FFI args
// locally) and `d3dkmt_set_scheduling_priority_class` (loads gdi32 by name and calls the export).
// The latter requires `process` to be a valid process handle; `GetCurrentProcess()` returns the
// current-process pseudo-handle, which is always valid and needs no close. Runs once via
// `Once::call_once`; no raw pointers are dereferenced here.
ONCE.call_once(|| unsafe {
use windows::Win32::System::Threading::GetCurrentProcess;
let Some(prio) = configured_gpu_priority_class() else {
tracing::info!("GPU process scheduling priority class left at default (off)");
return;
};
enable_inc_base_priority();
match d3dkmt_set_scheduling_priority_class(GetCurrentProcess(), prio) {
Some(0) => tracing::info!(
priority_class = prio,
"GPU process scheduling priority class set (2=normal 4=high 5=realtime)"
),
Some(st) => tracing::warn!(
status = format!("0x{st:08X}"),
"D3DKMTSetProcessSchedulingPriorityClass failed (run as admin/SYSTEM for GPU priority)"
),
None => tracing::warn!("D3DKMTSetProcessSchedulingPriorityClass export not found"),
}
});
}
/// How many times DXGI has actually called our hooked `NtGdiDdDDIGetCachedHybridQueryValue`. If this
/// stays 0 while DDA churns with ACCESS_LOST, the hook is NOT on DXGI's GPU-preference path on this
@@ -269,7 +77,7 @@ unsafe extern "system" fn hybrid_query_hook(gpu_preference: *mut u32) -> i32 {
/// a cached preference of UNSPECIFIED makes DXGI skip the resolution, so the output is NOT reparented
/// and DDA stays stable on one adapter (this is what makes Apollo's DDA work on this hardware).
/// Installed once, before the first DXGI factory/enumeration; lasts the process lifetime (like Apollo).
pub(crate) fn install_gpu_pref_hook() {
pub fn install_gpu_pref_hook() {
use std::sync::Once;
static HOOK: Once = Once::new();
// SAFETY: this one-time hook install only touches a region it has just validated.
@@ -65,8 +65,8 @@ use windows::Win32::UI::WindowsAndMessaging::{GetCursorPos, SetCursorPos};
// `DRV_STATUS_*` codes and the channel-delivery struct — lives in `pf_driver_proto`; both sides
// `use` it, so a layout/code drift is a compile error (the proto has `const` size asserts).
use frame::{
SharedHeader, DRV_STATUS_BIND_FAIL, DRV_STATUS_NO_DEVICE1, DRV_STATUS_OPENED,
DRV_STATUS_TEX_FAIL, MAGIC, RING_LEN, VERSION,
unpack_opened_detail, SharedHeader, DRV_STATUS_BIND_FAIL, DRV_STATUS_NONE,
DRV_STATUS_NO_DEVICE1, DRV_STATUS_OPENED, DRV_STATUS_TEX_FAIL, MAGIC, RING_LEN, VERSION,
};
/// `DXGI_SHARED_RESOURCE_READ | _WRITE` for `CreateSharedHandle`/`OpenSharedResourceByName`. Local (not
@@ -219,6 +219,14 @@ impl Drop for KeyedMutexGuard<'_> {
/// the cursor layer of the display it lands on, so the target composes at least one frame; the
/// round trip is sub-millisecond and throttled). Best-effort — injection can be unavailable on the
/// secure desktop, where a fresh compose just happened anyway.
///
/// **HID-first**: when the host has registered [`HID_COMPOSE_KICK`] (the resident pf-mouse virtual
/// HID pointer), the kick goes through it INSTEAD of the `SendInput` paths below. A report from a
/// HID device is real input to win32k — delivered regardless of this process's session or the
/// active desktop, it wakes a powered-off display subsystem (lid-closed laptop / display idle-off /
/// modern standby) and counts as user presence — every condition under which `SendInput` is
/// silently impotent (wrong session → wrong input queue; secure desktop → blocked; display off →
/// nothing composes at all). That set is exactly the lid-closed field-report state.
fn kick_dwm_compose(target_id: u32) {
// Process-GLOBAL throttle (Stage W3): with N parallel capturers each nudging on its own
// schedule, DWM needs only one dirty per composition window — and the nudge is synthetic INPUT
@@ -240,7 +248,21 @@ fn kick_dwm_compose(target_id: u32) {
let have_pos = unsafe { GetCursorPos(&mut pos) }.is_ok();
// SAFETY: `source_desktop_rect` only runs the CCD QueryDisplayConfig FFI over owned local
// buffers; the `Copy` target id crosses by value.
let rect = unsafe { crate::win_display::source_desktop_rect(target_id) };
let rect = unsafe { pf_win_display::win_display::source_desktop_rect(target_id) };
// HID-first (see the doc comment): the registered virtual-mouse kick works from any
// session/desktop and wakes an off display. Both geometries come from CCD (global database),
// NOT per-session GDI metrics, so the aim is right even from a non-console session. Fall
// through to SendInput only when the hook isn't registered / the mouse isn't up.
if let (Some(kick), Some(rect)) = (crate::HID_COMPOSE_KICK.get(), rect) {
// SAFETY: `desktop_bounds` only runs the CCD QueryDisplayConfig FFI over owned local
// buffers.
let bounds = unsafe { pf_win_display::win_display::desktop_bounds() };
if let Some(bounds) = bounds {
if kick(rect, bounds) {
return;
}
}
}
if let (true, Some((x, y, w, h))) = (have_pos, rect) {
let inside = pos.x >= x && pos.x < x + w.max(1) && pos.y >= y && pos.y < y + h.max(1);
if !inside {
@@ -295,7 +317,7 @@ fn kick_dwm_compose(target_id: u32) {
///
/// # Safety
/// `process` must be a live process handle carrying `PROCESS_QUERY_LIMITED_INFORMATION`.
pub(crate) unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &str) -> Result<()> {
pub unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &str) -> Result<()> {
let mut buf = [0u16; 512];
let mut len = buf.len() as u32;
// SAFETY: `process` carries QUERY_LIMITED per the contract; `buf`/`len` are a valid out-buffer and
@@ -395,7 +417,7 @@ pub struct IddPushCapturer {
/// periodic-stutter diagnostic.
stall_watch: StallWatch,
/// Stall↔OS-event correlation counters for the metronomic warn: how many stalls this session,
/// and how many had a coinciding [`crate::display_events`] event in their gap window — the
/// and how many had a coinciding a `pf_win_display::display_events` event in their gap window — the
/// discriminator between "Windows re-enumerates a monitor each cycle" (devnode churn the
/// `pnp_disable_monitors` axis suppresses) and "the disturbance is below the OS" (GPU driver
/// servicing a standby sink / display-poller software).
@@ -420,6 +442,13 @@ pub struct IddPushCapturer {
last_seq: u64,
last_present: Option<(ID3D11Texture2D, PixelFormat)>,
status_logged: bool,
/// Session-lifetime `PowerRequestDisplayRequired` (RAII, `powercfg /requests`-visible): keeps
/// the console out of display-off while this capturer lives — DWM composes nothing (for ANY
/// display) once the console's displays power down, so without this a lid-closed/idle box can
/// go dark mid-stream and the ring runs dry. Prevention only; waking an ALREADY-off display is
/// the HID compose kick's job ([`crate::HID_COMPOSE_KICK`]). `None` when the kernel refused
/// (best-effort, the pre-existing behavior).
_display_wake: Option<pf_frame::session_tuning::DisplayWakeRequest>,
_keepalive: Box<dyn Send>,
}
// SAFETY: `IddPushCapturer` is `!Send` only because of its `*mut SharedHeader` raw pointer (and the
@@ -533,11 +562,12 @@ impl IddPushCapturer {
client_10bit: bool,
want_444: bool,
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
crate::display_events::spawn_once();
match Self::open_inner(target, preferred, client_10bit, want_444) {
pf_win_display::display_events::spawn_once();
match Self::open_inner(target, preferred, client_10bit, want_444, sender) {
Ok(mut me) => {
me._keepalive = keepalive;
Ok(me)
@@ -551,6 +581,7 @@ impl IddPushCapturer {
preferred: Option<(u32, u32, u32)>,
client_10bit: bool,
want_444: bool,
sender: crate::FrameChannelSender,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -561,11 +592,18 @@ impl IddPushCapturer {
// the driver HAVE drifted — identical twin GPUs whose max-VRAM tie moved between ADD and
// this open, or a stale kept monitor across an adapter re-init — the driver reports
// TEX_FAIL plus the adapter it actually renders on, and the rebind below reopens on that.
let luid = crate::win_adapter::resolve_render_adapter_luid().unwrap_or(LUID {
let luid = pf_gpu::resolve_render_adapter_luid().unwrap_or(LUID {
LowPart: (target.adapter_luid & 0xffff_ffff) as u32,
HighPart: (target.adapter_luid >> 32) as i32,
});
match Self::open_on(target.clone(), preferred, client_10bit, want_444, luid) {
match Self::open_on(
target.clone(),
preferred,
client_10bit,
want_444,
luid,
sender.clone(),
) {
Ok(me) => Ok(me),
Err(e) => {
// Self-heal a render-adapter mismatch ONCE: on TEX_FAIL the driver has reported the
@@ -576,7 +614,7 @@ impl IddPushCapturer {
let driver_luid = e
.downcast_ref::<AttachTexFail>()
.map(|tf| tf.driver_luid)
.filter(|d| *d != 0 && *d != crate::capture::dxgi::pack_luid(luid));
.filter(|d| *d != 0 && *d != crate::dxgi::pack_luid(luid));
let Some(packed) = driver_luid else {
return Err(e);
};
@@ -590,7 +628,7 @@ impl IddPushCapturer {
"IDD push: ring/driver render-adapter mismatch — rebinding the ring to the \
driver's reported adapter"
);
Self::open_on(target, preferred, client_10bit, want_444, drv)
Self::open_on(target, preferred, client_10bit, want_444, drv, sender)
.context("IDD-push rebind to the driver's reported render adapter")
}
}
@@ -602,6 +640,7 @@ impl IddPushCapturer {
client_10bit: bool,
want_444: bool,
luid: LUID,
sender: crate::FrameChannelSender,
) -> Result<Self> {
let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -612,8 +651,8 @@ impl IddPushCapturer {
// SAFETY: `active_resolution` is an `unsafe fn` (Win32 CCD `QueryDisplayConfig`) that takes only a
// copy of the plain `u32` CCD target id and returns owned `(w, h)` values; it forms no borrows from
// us and validates the id internally, returning `None` on any failure (handled by `unwrap_or`).
let (w, h) =
unsafe { crate::win_display::active_resolution(target.target_id) }.unwrap_or((pw, ph));
let (w, h) = unsafe { pf_win_display::win_display::active_resolution(target.target_id) }
.unwrap_or((pw, ph));
if (w, h) != (pw, ph) {
tracing::info!(
target_id = target.target_id,
@@ -656,16 +695,35 @@ impl IddPushCapturer {
// size the ring FP16 directly — don't race the advanced_color_enabled poll, which may not have
// settled within 250 ms and would size the ring SDR while the driver composes FP16 → a format
// mismatch → an immediate ring recreate + dropped first frames (audit §5.4).
let enabled_hdr =
client_10bit && crate::win_display::set_advanced_color(target.target_id, true);
let enabled_hdr = client_10bit
&& pf_win_display::win_display::set_advanced_color(target.target_id, true);
if enabled_hdr {
// Let the colorspace change settle before the driver composes + we size the ring.
std::thread::sleep(Duration::from_millis(250));
// Let the colorspace change settle before the driver composes + we size the ring:
// poll the CCD advanced-color state instead of a fixed sleep (latency plan P0.4),
// ceiling = the old 250 ms. A read that never flips within the ceiling proceeds
// exactly like the fixed sleep did — the ring is sized FP16 from `enabled_hdr`
// either way (the set succeeded; only the driver's compose flip may lag, which the
// stash/format-guard machinery absorbs).
let hdr_settle = Instant::now();
while hdr_settle.elapsed() < Duration::from_millis(250) {
if pf_win_display::win_display::advanced_color_enabled(target.target_id)
== Some(true)
{
break;
}
std::thread::sleep(Duration::from_millis(25));
}
tracing::debug!(
target_id = target.target_id,
settle_ms = hdr_settle.elapsed().as_millis() as u64,
"IDD push: advanced-color (HDR) enable settle"
);
}
// A failed open-time read defaults to SDR (unless the 10-bit path enabled HDR above) —
// there is no "last known" yet; the descriptor poller corrects a wrong guess mid-session.
let display_hdr = enabled_hdr
|| crate::win_display::advanced_color_enabled(target.target_id).unwrap_or(false);
|| pf_win_display::win_display::advanced_color_enabled(target.target_id)
.unwrap_or(false);
// Downgrade point D (design/hdr-10bit-default-and-av1.md item 2d): the session was
// NEGOTIATED 10-bit (the client was told HDR in the Welcome), but the virtual display
// could not enable advanced color — the ring sizes SDR and the encoder will emit 8-bit
@@ -757,7 +815,7 @@ impl IddPushCapturer {
// driver's WUDFHost and hand it the values over the control device. All-or-nothing (the
// broker reaps its remote duplicates on failure), and a failure fails the open — without
// the delivery the driver can never attach.
let broker = ChannelBroker::open(target.wudf_pid)?;
let broker = ChannelBroker::open(target.wudf_pid, sender)?;
broker
.send(
target.target_id,
@@ -819,6 +877,9 @@ impl IddPushCapturer {
last_seq: 0,
last_present: None,
status_logged: false,
// Held from BEFORE the first-frame gate (the display must not idle off while we
// wait for the first compose) until the capturer drops with the session.
_display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(),
// Placeholder; `open()` attaches the real keepalive on success, so a FAILED open can hand
// it back to the caller for the DDA fallback (audit §5.1).
_keepalive: Box::new(()),
@@ -935,12 +996,67 @@ impl IddPushCapturer {
}
if Instant::now() > deadline {
bail!(
"IDD-push: driver_status={st} but no frame published within 4s (despite compose \
kicks) the virtual display is likely in a format/size the ring can't match \
(fullscreen game?); falling back"
"IDD-push: no frame published within 4s (despite compose kicks) — {}; \
falling back",
self.no_first_frame_diagnosis(st)
);
}
std::thread::sleep(Duration::from_millis(20));
// Event-driven wait (latency plan P0.6): the driver signals the frame-ready event on
// every publish, so wake on it instead of a blind sleep — the 20 ms timeout keeps the
// driver_status polls above live (status writes don't signal the event). Consuming a
// signal here is fine: `next_frame` re-checks the atomic `latest` token, never the
// event, for truth.
// SAFETY: `self.event` is this capturer's owned, live auto-reset event handle;
// `WaitForSingleObject` only reads the handle and the 20 ms timeout bounds the wait.
let _ = unsafe { WaitForSingleObject(HANDLE(self.event.as_raw_handle()), 20) };
}
}
/// Name a first-frame timeout from the driver's own evidence — `driver_status` plus the live
/// OPENED detail word (proto `pack_opened_detail`) — instead of guessing. The three no-frames
/// states look identical from the host side but have disjoint causes and fixes; the lid-closed
/// field report burned days for lack of exactly this line. Appends a console-session hint when
/// the host itself is in the wrong session (display writes + input kicks can't work from there).
fn no_first_frame_diagnosis(&self, st: u32) -> String {
let what = match st {
// The delivery was never consumed: no swap-chain worker ran for this monitor at all.
DRV_STATUS_NONE => "the driver never attached — the channel delivery was never \
consumed, so the OS ran no swap-chain worker for this monitor (display not \
composed at all: console display-off / modern standby, or the mode commit \
never reached the adapter)"
.to_string(),
DRV_STATUS_OPENED => {
// SAFETY: in-bounds, aligned u32 read of the live, owned shared-header mapping
// (same best-effort diagnostic access as the `driver_status` read in the caller);
// no reference into the shared region is formed.
let detail = unsafe { (*self.header).driver_status_detail };
match unpack_opened_detail(detail) {
Some((0, _)) => "driver attached with a live swap-chain, but DWM composed \
ZERO frames an undamaged or powered-off desktop, and the compose \
kicks didn't bite (synthetic input is blocked on the secure desktop)"
.to_string(),
Some((offered, mismatched)) => format!(
"driver attached and DWM composed {offered} frame(s), but none matched \
the ring {mismatched} dropped for a size/format mismatch (the \
display's actual mode differs from what the host sized the ring to: \
a mid-open mode-set, a fullscreen game, or a stale GDI view)"
),
// A pre-detail driver never stamps the live bit — say so rather than guess.
None => "driver attached but published nothing; this pf-vdisplay build \
predates attach diagnostics, so the cause can't be named update the \
driver for a precise line here"
.to_string(),
}
}
other => format!("driver_status={other} (unexpected at this point)"),
};
match pf_win_display::console_session_mismatch() {
Some((own, console)) => format!(
"{what} [host is in session {own} but the console is session {console} — display \
writes and input kicks cannot work from a non-console session; reconnect the \
console or run via the installed service]"
),
None => what,
}
}
@@ -1340,7 +1456,7 @@ impl IddPushCapturer {
let window = stall.gap + Duration::from_millis(300);
let events = now
.checked_sub(window)
.map(|from| crate::display_events::events_between(from, now))
.map(|from| pf_win_display::display_events::events_between(from, now))
.unwrap_or_default();
self.stalls_seen = self.stalls_seen.saturating_add(1);
if !events.is_empty() {
@@ -1351,12 +1467,12 @@ impl IddPushCapturer {
// at debug level, and the web-console debug ring captures these.
tracing::debug!(
gap_ms = stall.gap.as_millis() as u64,
os_display_events = %crate::display_events::summarize(&events),
os_display_events = %pf_win_display::display_events::summarize(&events),
"IDD-push capture stall — the desktop was composing at speed, then DWM \
delivered no frame for the gap; the present path stalled below capture"
);
if let Some(period) = stall.metronomic {
let suspects = crate::display_events::connected_inactive_externals();
let suspects = pf_win_display::display_events::connected_inactive_externals();
let suspects = if suspects.is_empty() {
"none".to_string()
} else {
@@ -1514,7 +1630,7 @@ impl Capturer for IddPushCapturer {
// PQ VUI; pair that with a mastering-display SEI so any decoder tone-maps from a real grade. The
// driver doesn't (yet) forward the OS's IDDCX_HDR10_METADATA, so use the generic HDR10 baseline
// (the same metadata the native HDR path sends on the 0xCE datagram).
self.display_hdr.then(crate::hdr::generic_hdr10)
self.display_hdr.then(pf_frame::hdr::generic_hdr10)
}
fn pipeline_depth(&self) -> usize {
@@ -1522,7 +1638,40 @@ impl Capturer for IddPushCapturer {
// NVENC encodes N on the ASIC. We hand a rotating `OUT_RING` of output textures, so this is safe.
// `PUNKTFUNK_IDD_DEPTH` overrides (1 disables pipelining; clamp to ≤ OUT_RING so a frame in flight
// always has its own texture).
crate::config::config().idd_depth.clamp(1, OUT_RING)
pf_host_config::config().idd_depth.clamp(1, OUT_RING)
}
fn capture_target_id(&self) -> Option<u32> {
Some(self.target_id)
}
fn resize_output(&mut self, width: u32, height: u32) -> bool {
// Host-initiated resize (latency plan P2.3): the session's resize handler has already
// committed the display's new mode (the manager's in-place mode set), so recreate the ring
// at the new size NOW — no DescriptorPoller two-strike debounce (that stays, unchanged,
// for EXTERNAL changes: HDR flips, game mode-sets). The driver re-attaches to the fresh
// ring and republishes; on an in-place mode set the OS's mode-set full redraw gives the
// stash/first frame within the recover window. Same recover-or-drop arming as the
// poller-driven recreate, so a ring that can't re-attach still fails the session cleanly
// instead of freezing.
if (width, height) == (self.width, self.height) {
return true; // already at the requested size (refresh-only change) — nothing to do
}
tracing::info!(
target_id = self.target_id,
from = format!("{}x{}", self.width, self.height),
to = format!("{width}x{height}"),
"IDD push: host-initiated resize — recreating the ring at the new mode"
);
self.recovering_since.get_or_insert_with(Instant::now);
if let Err(e) = self.recreate_ring(self.display_hdr, width, height) {
tracing::warn!(
error = %format!("{e:#}"),
"IDD push: host-initiated ring recreate failed — falling back to a full rebuild"
);
return false;
}
true
}
}
@@ -20,9 +20,11 @@ pub(super) struct ChannelBroker {
process: OwnedHandle,
/// The WUDFHost pid `process` refers to (diagnostics for the driver-death bail).
pub(super) wudf_pid: u32,
/// The pf-vdisplay control device — owned by the `VirtualDisplayManager`, never closed for the
/// process lifetime (a dead one is retired, kept alive), so holding the bare `HANDLE` is sound.
control: HANDLE,
/// Delivers a filled `SetFrameChannelRequest` to the pf-vdisplay driver
/// (`IOCTL_SET_FRAME_CHANNEL`). The host facade builds this from the vdisplay control device +
/// `send_frame_channel` IOCTL wrapper, so this crate delivers the channel without reaching into
/// the orchestrator's `vdisplay` module (plan §W6). Called once per generation, never per-frame.
sender: crate::FrameChannelSender,
}
impl ChannelBroker {
@@ -35,13 +37,10 @@ impl ChannelBroker {
/// spoofed devnode (same interface GUID) could name an arbitrary process and receive the frames; a
/// fully-compromised REAL pf_vdisplay driver is already a frame endpoint, so this specifically closes
/// the reachable-without-owning-the-driver case (`design/idd-push-security.md` §hardening).
pub(super) fn open(wudf_pid: u32) -> Result<Self> {
pub(super) fn open(wudf_pid: u32, sender: crate::FrameChannelSender) -> Result<Self> {
if wudf_pid == 0 {
bail!("driver reported no WUDFHost pid for the frame channel");
}
let control = crate::vdisplay::manager::control_device_handle().context(
"pf-vdisplay control device not open (monitor not created via the manager?)",
)?;
// SAFETY: plain FFI; `wudf_pid` is a copy. The handle (checked by `?`) is owned solely here and
// moved into the `OwnedHandle` (single owner, closes on drop); `verify_is_wudfhost` borrows it
// for the duration of the synchronous check and forms no lasting alias.
@@ -59,7 +58,7 @@ impl ChannelBroker {
Ok(Self {
process,
wudf_pid,
control,
sender,
})
}
@@ -182,8 +181,9 @@ impl ChannelBroker {
slots: &[HostSlot],
) -> Result<()> {
// SAFETY: forwarded from the caller's contract — `header`/`event`/each `slot.shared` are live
// handles of this process, and `self.control` is the manager's control handle, never closed for
// the process lifetime (`send_frame_channel`'s precondition).
// handles of this process. The `sender` closure encapsulates the manager's control handle +
// the `send_frame_channel` IOCTL (its precondition — a live control handle — is upheld by the
// host facade that built it).
unsafe {
// Least privilege per handle: the header maps read/write, the event is only signalled, and
// the textures keep their already-scoped `CreateSharedHandle` access (see `dup_into`).
@@ -192,7 +192,7 @@ impl ChannelBroker {
for (k, s) in slots.iter().enumerate() {
req.texture_handles[k] = self.dup_into(HANDLE(s.shared.as_raw_handle()), None)?;
}
crate::vdisplay::pf_vdisplay::send_frame_channel(self.control, req)
(self.sender)(req)
}
}
}
@@ -63,8 +63,8 @@ impl DescriptorPoller {
// target id (see their own SAFETY docs); nothing is borrowed across the calls.
let (hdr, res) = unsafe {
(
crate::win_display::advanced_color_enabled(target_id),
crate::win_display::active_resolution(target_id),
pf_win_display::win_display::advanced_color_enabled(target_id),
pf_win_display::win_display::active_resolution(target_id),
)
};
let took = t.elapsed();
@@ -12,7 +12,7 @@ pub(super) struct Stall {
/// How long the hole lasted (last fresh frame → the frame that ended it).
pub(super) gap: Duration,
/// `Some(mean period)` when this stall completes a metronomic cycle (see
/// [`crate::metronome::Metronome`]).
/// [`pf_frame::metronome::Metronome`]).
pub(super) metronomic: Option<Duration>,
}
@@ -23,14 +23,14 @@ pub(super) struct Stall {
/// On a damage-driven capture an idle desktop legitimately goes quiet (no damage → no frames), so a
/// gap only counts as a stall when the [`Self::RECENT`] frames before it all arrived within
/// [`Self::ACTIVE_SPAN`] — sustained ≥ ~20 fps flow (a game or video), not a blinking caret or a
/// mouse twitch. Each stall feeds a [`crate::metronome::Metronome`], so periodic stalls self-diagnose
/// mouse twitch. Each stall feeds a [`pf_frame::metronome::Metronome`], so periodic stalls self-diagnose
/// in the log WITHOUT needing any client keyframe request — discriminating "DWM stopped composing"
/// from encode/network causes that the recovery-cadence detector covers. Pure logic — unit-tested
/// below; the caller does the logging.
pub(super) struct StallWatch {
/// The last [`Self::RECENT`] fresh-frame instants (pre-gap history for the activity gate).
recent: std::collections::VecDeque<Instant>,
cadence: crate::metronome::Metronome,
cadence: pf_frame::metronome::Metronome,
}
impl StallWatch {
@@ -47,7 +47,7 @@ impl StallWatch {
pub(super) fn new() -> Self {
Self {
recent: std::collections::VecDeque::with_capacity(Self::RECENT + 1),
cadence: crate::metronome::Metronome::new(),
cadence: pf_frame::metronome::Metronome::new(),
}
}
@@ -2,15 +2,15 @@
//! without a real capture session.
//!
//! The native AMF path (and the D3D11 zero-copy NVENC/QSV paths) require an NV12 texture that lives
//! on the GPU — the CPU-Bgrx [`SyntheticCapturer`](crate::capture::SyntheticCapturer) can't provide
//! on the GPU — the CPU-Bgrx [`SyntheticCapturer`](crate::SyntheticCapturer) can't provide
//! one, and DXGI Desktop Duplication can't create one under an ssh session-0 (E_ACCESSDENIED). This
//! source builds an NV12 texture on the selected render adapter and fills it with a **moving** luma
//! ramp each frame, so the encoder sees genuine motion (P-frame residuals + the intra-refresh wave
//! under content change) — exactly what an intra-refresh recovery validation needs. Driven by
//! `spike --source synthetic-nv12`.
use crate::capture::dxgi::{make_device, D3d11Frame};
use crate::capture::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use crate::dxgi::{make_device, D3d11Frame};
use crate::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{Context, Result};
use windows::Win32::Graphics::Direct3D11::{
ID3D11Device, ID3D11DeviceContext, ID3D11Texture2D, D3D11_BIND_SHADER_RESOURCE,
@@ -140,7 +140,7 @@ impl Capturer for SyntheticNv12Capturer {
/// Calls DXGI factory/adapter enumeration; returns owned COM objects or an error.
unsafe fn resolve_render_adapter() -> Result<IDXGIAdapter1> {
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("CreateDXGIFactory1")?;
if let Some(luid) = crate::win_adapter::resolve_render_adapter_luid() {
if let Some(luid) = pf_gpu::resolve_render_adapter_luid() {
if let Ok(a) = factory.EnumAdapterByLuid::<IDXGIAdapter1>(luid) {
return Ok(a);
}
+50 -130
View File
@@ -31,7 +31,7 @@
//!
//! This thread is also the single consumer of the rumble and HID-output pull planes.
use punktfunk_core::client::NativeClient;
use punktfunk_core::client::{ActuatorQuirks, NativeClient};
use punktfunk_core::config::GamepadPref;
use punktfunk_core::input::{gamepad as wire, InputEvent, InputKind};
use punktfunk_core::quic::{HidOutput, RichInput};
@@ -61,24 +61,14 @@ const ESCAPE_CHORD: [u32; 4] = [wire::BTN_LB, wire::BTN_RB, wire::BTN_START, wir
/// Hold the [`ESCAPE_CHORD`] at least this long to disconnect (escalates the leave-fullscreen press).
const DISCONNECT_HOLD: Duration = Duration::from_millis(1500);
/// Steam Deck built-in haptic keep-alive interval. The Deck's actuator decays inside SDL's
/// ~2 s internal rumble resend (`SDL_RUMBLE_RESEND_MS`), and SDL short-circuits a repeated
/// identical `set_rumble` value to a no-op device write — so a STEADY host value (which the
/// host delivers only as unchanging 500 ms refreshes) never re-kicks the motor and is felt as
/// a periodic pulse. We re-issue below the decay so the bursts fuse into a continuous buzz;
/// 40 ms mirrors SDL's sibling Steam-Controller driver keep-alive. Deck-only (see
/// [`Worker::issue_rumble`]); every other pad sustains rumble at the hardware level and is
/// left untouched.
const DECK_RUMBLE_KEEPALIVE_MS: u64 = 40;
/// Ceiling on a *legacy* (no-TTL) host's Steam Deck rumble: silence the actuator once a real host
/// update has been absent this long. A legacy host re-sends the held level as a flat 500 ms refresh,
/// so a genuinely-held rumble refreshes the per-slot update clock (`RumbleState::updated_at`) every
/// 500 ms and never approaches this — only a lost *stop* datagram (the host went quiet entirely)
/// lets the 40 ms keep-alive drone on. 2× the 500 ms refresh bounds that lost stop to ~1 s,
/// mirroring the Windows host's `RUMBLE_IDLE_TIMEOUT` residual cutoff. The v2 path is bounded by its
/// lease `deadline` instead and never trips this (see [`Worker::render_feedback`]).
const LEGACY_RUMBLE_CEILING_MS: u64 = 1_000;
/// Steam Deck actuator-decay keepalive cadence, declared to the core's rumble policy engine as an
/// [`ActuatorQuirks`] at slot open. The Deck's built-in actuator decays inside SDL's ~2 s internal
/// rumble resend (`SDL_RUMBLE_RESEND_MS`) and SDL short-circuits an identical `set_rumble` value
/// to a no-op device write — so a steady level is felt as a periodic pulse without sub-decay
/// re-kicks; 40 ms mirrors SDL's sibling Steam-Controller driver keep-alive. The engine owns the
/// re-kick timing, the 1-LSB dedupe-defeat jitter, and every staleness/lease bound — this worker
/// only applies the commands it emits (`design/rumble-root-fix.md` §D).
const DECK_RUMBLE_KEEPALIVE_MS: u16 = 40;
/// Stick deflection below this is ignored for menu navigation (0.5 of full scale — Apple
/// `GamepadMenuInput` parity; menus want deliberate flicks, not drift).
@@ -626,32 +616,6 @@ impl Ds5Feedback {
}
}
/// Per-controller rumble render state (the Steam Deck keep-alive + the host's v2 lease). Held
/// per [`Slot`] so a rumble the host addressed to pad N drives only pad N's actuator.
#[derive(Default)]
struct RumbleState {
/// Last rumble value handed to this pad (the logical host value, pre-jitter) and when —
/// drives the Steam Deck haptic keep-alive in [`Worker::render_feedback`].
last: (u16, u16),
last_at: Option<Instant>,
/// When the last *real* host rumble datagram landed on this slot — set only in the feedback
/// drain, never bumped by the Deck keep-alive re-kick (unlike `last_at`, which the keep-alive
/// refreshes every ~40 ms). A legacy host carries no lease, so this per-slot clock is what
/// bounds a lost stop-frame: once it is stale past `LEGACY_RUMBLE_CEILING_MS` the keep-alive
/// stops and issues one (0, 0). See [`Worker::render_feedback`].
updated_at: Option<Instant>,
/// Toggles the 1-LSB low-motor nudge that forces SDL past its identical-value dedupe on a
/// Deck keep-alive re-issue (see [`Worker::issue_rumble`]).
jitter: bool,
/// The host lease from a v2 rumble envelope: last non-zero level expires at this instant
/// unless the host renews it. `None` outside a live rumble or against a legacy host (which
/// sends no lease — the pad then relies on SDL's own duration expiry as before).
deadline: Option<Instant>,
/// The host-supplied TTL (ms) of the current envelope, handed to SDL as the `set_rumble`
/// duration; `0` = legacy host (fall back to the proven 1.5 s duration).
ttl_ms: u16,
}
/// One forwarded controller during an attached session: the open SDL handle, its stable wire
/// pad index (0..[`MAX_PADS`](punktfunk_core::input::MAX_PADS)), and the per-pad wire/feedback
/// state that used to be single-scalar on the Worker. Opening the device is what grabs the
@@ -683,7 +647,6 @@ struct Slot {
/// close lift a click held across detach/unplug.
held_clicks: [bool; 2],
last_accel: [i16; 3],
rumble: RumbleState,
}
impl Slot {
@@ -699,7 +662,6 @@ impl Slot {
surface_last: [(0, 0, false); 2],
held_clicks: [false; 2],
last_accel: [0; 3],
rumble: RumbleState::default(),
}
}
@@ -928,6 +890,20 @@ impl Worker {
// uses the session-default kind.
if let Some(c) = &self.attached {
send(c, InputKind::GamepadArrival, pref.to_u8() as u32, 0, index);
// Declare the actuator's quirks to the shared rumble policy engine. ALWAYS
// set (defaults for a well-behaved pad): wire indices are reused within a
// connection, so a Deck slot that closes must not leave its keepalive quirk
// behind for the next pad on the same index.
let quirks = if pref == GamepadPref::SteamDeck {
ActuatorQuirks {
keepalive_ms: DECK_RUMBLE_KEEPALIVE_MS,
min_pulse_ms: 0,
dedup_jitter: true,
}
} else {
ActuatorQuirks::default()
};
c.set_rumble_quirks(index as u16, quirks);
}
tracing::info!(id, index, pref = ?pref, "gamepad forwarding (slot opened)");
self.slots.push(slot);
@@ -940,6 +916,10 @@ impl Worker {
/// the SDL handle. The flush only emits wire events, so it is safe even when the device is
/// already gone (unplug).
fn close_slot_at(&mut self, i: usize) {
// Best-effort physical silence before the handle drops: a slot closed mid-buzz (detach /
// unplug) must not depend on what SDL does to a rumbling device at close. Errors are
// expected for an already-unplugged pad.
let _ = self.slots[i].pad.set_rumble(0, 0, 100);
if let Some(c) = self.attached.clone() {
Self::flush_slot(&c, &mut self.slots[i]);
// Signal the host to tear down this pad's virtual device (native hot-unplug). Sent
@@ -1464,107 +1444,47 @@ impl Worker {
}
}
/// Hand a rumble value to SDL on one slot's pad, remembering it for the Deck keep-alive.
/// SDL short-circuits an identical `(low, high)` with NO device write (it only re-arms its
/// expiration), so on a Deck keep-alive re-issue of the same non-zero value we flip a single
/// low-motor LSB — an imperceptible amplitude nudge — to force the write through and keep the
/// actuator physically fed. The SDL duration is the host's envelope TTL (a lease continuously
/// refreshed by renewals, so a sustained rumble never dies mid-effect and an abandoned one
/// self-silences at the TTL); against a legacy host (`ttl_ms == 0`) it stays the proven 1.5 s.
fn issue_rumble(slot: &mut Slot, low: u16, high: u16, deck: bool) {
let dur_ms: u32 = if slot.rumble.ttl_ms == 0 {
1_500 // legacy host: no lease — keep the proven duration
/// Hand one policy-engine command to SDL on a slot's pad, verbatim. The core engine owns all
/// rumble policy — leases, legacy-host staleness, the Deck keepalive + its dedupe-defeat
/// jitter (declared as quirks at slot open) — so this worker keeps no rumble state at all.
/// `backstop_ms` becomes the SDL duration: the hardware-level net under a stalled worker
/// thread (the engine emits explicit zeros at every policy stop, so it is never the stop
/// mechanism).
fn issue_rumble(slot: &mut Slot, low: u16, high: u16, backstop_ms: u32) {
let dur_ms: u32 = if (low, high) == (0, 0) {
100 // a stop takes effect immediately; the duration is irrelevant
} else {
// Floor the lease so a jittered renewal (or the ~40 ms Deck re-kick) can never gap the
// actuator between SDL writes.
(slot.rumble.ttl_ms as u32).max(DECK_RUMBLE_KEEPALIVE_MS as u32 * 4)
backstop_ms.max(160) // floor: a jittered renewal can never gap the actuator
};
let (out_low, out_high) =
if deck && (low, high) == slot.rumble.last && (low, high) != (0, 0) {
slot.rumble.jitter = !slot.rumble.jitter;
(low ^ slot.rumble.jitter as u16, high)
} else {
(low, high)
};
// Surface a failed SDL rumble write: a swallowed error here (DualSense not in the right
// HIDAPI mode, etc.) reads exactly like "rumble doesn't work". The host logs the send side
// on 0xCA with the pad index, so the two together pinpoint host-game vs client-render.
match slot.pad.set_rumble(out_low, out_high, dur_ms) {
match slot.pad.set_rumble(low, high, dur_ms) {
Err(e) => {
tracing::warn!(pad = slot.index, low, high, error = %e, "rumble: SDL set_rumble failed")
}
Ok(()) => tracing::trace!(pad = slot.index, low, high, "rumble: rendered"),
}
slot.rumble.last = (low, high);
slot.rumble.last_at = Some(Instant::now());
}
/// Drain and render the feedback planes — rumble plus HID output (lightbar / player LEDs /
/// adaptive triggers) — routing each update to the forwarded slot on its wire pad index; this
/// thread is their single consumer. Rumble arrives as self-terminating v2 envelopes: each
/// carries a TTL the host renews while the level holds and lets expire when it stops, so the
/// actuator's divergence from the host's intent is bounded by the wire, not by a client guess.
/// A legacy host (`ttl == None`) has no lease — the pad falls back to SDL's own 1.5 s duration
/// expiry as before.
/// thread is their single consumer. Rumble arrives as EFFECTIVE commands from the core's
/// shared policy engine, which already applied every policy — v2 lease expiry, legacy-host
/// staleness, the Deck actuator keepalive + jitter (via the quirks declared at slot open),
/// and connection-close drain zeros — so this worker applies commands verbatim and keeps no
/// rumble state of its own (`design/rumble-root-fix.md` §D).
fn render_feedback(&mut self) {
let Some(connector) = self.attached.clone() else {
return;
};
// Rumble envelopes (0xCA) → the slot holding that wire pad index. An update for an index
// with no live slot (a pad that just unplugged) is dropped.
while let Ok((pad, low, high, ttl)) = connector.next_rumble_ttl(Duration::ZERO) {
if let Some(slot) = self.slots.iter_mut().find(|s| s.index as u16 == pad) {
let deck = slot.pref == GamepadPref::SteamDeck;
slot.rumble.ttl_ms = ttl.unwrap_or(0);
// A v2 lease sets an explicit client-side deadline; a legacy update clears it and
// leans on SDL's own duration expiry (unchanged behaviour).
slot.rumble.deadline = match ttl {
Some(ms) if (low, high) != (0, 0) => {
Some(Instant::now() + Duration::from_millis(ms as u64))
}
_ => None,
};
// Mark this as a real host update. Unlike `last_at` (which the Deck keep-alive
// re-kick refreshes every ~40 ms), this clock advances only here, so a legacy
// lost-stop can be bounded by `LEGACY_RUMBLE_CEILING_MS` in the keep-alive below.
slot.rumble.updated_at = Some(Instant::now());
Self::issue_rumble(slot, low, high, deck);
}
}
// Steam Deck keep-alive, per slot: the built-in actuator decays inside SDL's ~2 s internal
// rumble resend, and SDL dedupes an unchanged `set_rumble` to a no-op device write — so a
// steady host value is felt as a periodic pulse. Re-kick a Deck slot below the decay
// (`DECK_RUMBLE_KEEPALIVE_MS`) so its discrete bursts fuse into a continuous buzz, but
// silence it once the host's lease expires (the host stopped renewing — a lost stop, or the
// host died). The per-slot timing guards make this idempotent with a fresh datagram this
// tick (a just-set `last_at`/`deadline` fails both checks). Non-Deck slots sustain/expire at
// the SDL/hardware level and never enter here.
for slot in self.slots.iter_mut() {
if slot.pref != GamepadPref::SteamDeck || slot.rumble.last == (0, 0) {
continue;
}
if slot.rumble.deadline.is_some_and(|d| Instant::now() >= d) {
slot.rumble.deadline = None;
slot.rumble.ttl_ms = 0;
Self::issue_rumble(slot, 0, 0, true);
} else if slot.rumble.ttl_ms == 0
&& slot
.rumble
.updated_at
.is_some_and(|t| t.elapsed() >= Duration::from_millis(LEGACY_RUMBLE_CEILING_MS))
{
// Legacy host (no v2 lease): a held rumble refreshes `updated_at` every ~500 ms, so
// this only trips on a lost stop-frame the host never followed up — silence the
// actuator once instead of letting the 40 ms keep-alive drone forever. `issue_rumble`
// sets `last` to (0, 0), so the top-of-loop guard skips this slot on later ticks.
Self::issue_rumble(slot, 0, 0, true);
} else if slot
.rumble
.last_at
.is_none_or(|t| t.elapsed() >= Duration::from_millis(DECK_RUMBLE_KEEPALIVE_MS))
{
let (low, high) = slot.rumble.last;
Self::issue_rumble(slot, low, high, true);
// Engine commands → the slot holding that wire pad index. A command for an index with no
// live slot (a pad that just unplugged) is dropped. The loop ends on NoFrame (drained
// dry this tick) or Closed (session over — the engine delivered its close-drain zeros
// first; the physical silence backstop is in `close_slot_at`).
while let Ok(cmd) = connector.next_rumble_command(Duration::ZERO) {
if let Some(slot) = self.slots.iter_mut().find(|s| s.index as u16 == cmd.pad) {
Self::issue_rumble(slot, cmd.low, cmd.high, cmd.backstop_ms);
}
}
// HID output (lightbar / player LEDs / adaptive triggers) → the slot on that wire index.
+8
View File
@@ -33,6 +33,14 @@ pub mod session;
pub mod trust;
#[cfg(any(target_os = "linux", windows))]
pub mod video;
#[cfg(any(target_os = "linux", windows))]
mod video_color;
#[cfg(any(target_os = "linux", windows))]
mod video_software;
#[cfg(target_os = "linux")]
mod video_vaapi;
#[cfg(any(target_os = "linux", windows))]
mod video_vulkan;
// PyroWave decode — Linux + `pyrowave` feature only (plan §4.5; the Windows client's
// present-path decision and the Apple Metal port are their own phases).
#[cfg(windows)]
+9
View File
@@ -461,6 +461,14 @@ pub struct Settings {
pub refresh_hz: u32,
/// Requested encoder bitrate (kbps); 0 = host default.
pub bitrate_kbps: u32,
/// Render-resolution multiplier: the client asks the host to render/encode at
/// `resolved mode × render_scale` and the presenter downscales the larger decoded frame to the
/// window (`> 1` supersamples for sharpness, at more bandwidth AND decode; `< 1` renders under
/// native for a lighter host/link). `1.0` = Native (the prior behaviour). Applied at connect
/// (and each match-window resize) via [`punktfunk_core::render_scale`], clamped even + to the
/// codec's max dimension. Missing in a pre-existing store → the `Default` (1.0) via the
/// container `#[serde(default)]`.
pub render_scale: f64,
pub gamepad: String,
/// Stable identity (`vid:pid:name`, see `PadInfo::key`) of the physical controller
/// forwarded as pad 0; empty = automatic (most recently connected). Applied to the
@@ -590,6 +598,7 @@ impl Default for Settings {
height: 0,
refresh_hz: 0,
bitrate_kbps: 0,
render_scale: 1.0,
gamepad: "auto".into(),
forward_pad: String::new(),
compositor: "auto".into(),
File diff suppressed because it is too large Load Diff
+210
View File
@@ -0,0 +1,210 @@
//! The stream's per-frame colour signalling (`ColorDesc`) + the YCbCr→RGB CSC matrix (`csc_rows`).
#![allow(clippy::unnecessary_cast)]
use ffmpeg_next as ffmpeg;
/// The stream's colour signaling, read PER-FRAME from the decoder (HEVC VUI → the
/// `AVFrame` CICP fields). The Windows host switches an HDR desktop to Main10 BT.2020 PQ
/// **in-band** (the Welcome still says SDR — clients are expected to follow the VUI, as
/// the Windows/Apple/Android clients do), so rendering must follow the frames, not the
/// handshake — else PQ content drawn as BT.709 comes out washed out and desaturated.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct ColorDesc {
/// H.273 code points as signaled (2 = unspecified → the renderer picks the SDR default).
pub primaries: u8,
pub transfer: u8,
pub matrix: u8,
pub full_range: bool,
}
impl ColorDesc {
/// Read the CICP fields off a raw decoded frame. Public: the Windows client's raw-FFI
/// D3D11VA/software decoders build their per-frame `ColorDesc` with it too (same
/// `ffmpeg-next` major, so the `AVFrame` type unifies across the workspace).
///
/// # Safety
/// `frame` must point to a valid `AVFrame` (alive for the duration of the call).
pub unsafe fn from_raw(frame: *const ffmpeg::ffi::AVFrame) -> ColorDesc {
// SAFETY: caller guarantees a live AVFrame; these are plain enum field reads.
unsafe {
ColorDesc {
primaries: (*frame).color_primaries as u32 as u8,
transfer: (*frame).color_trc as u32 as u8,
matrix: (*frame).colorspace as u32 as u8,
full_range: (*frame).color_range == ffmpeg::ffi::AVColorRange::AVCOL_RANGE_JPEG,
}
}
}
/// PQ (SMPTE ST.2084) transfer — the HDR10 signal.
pub fn is_pq(&self) -> bool {
self.transfer == 16
}
}
/// The YCbCr→RGB conversion as three vec4 rows for a shader constant buffer / push-constant
/// block: `rgb[i] = dot(r[i].xyz, yuv) + r[i].w` — bit-depth exact. The ONE coefficient
/// implementation every presenter derives its CSC from (Vulkan push constants, the Windows
/// client's D3D11 constant buffer), so a stream's signaled matrix/range is honored identically
/// everywhere; the Apple client ports this function (and its tests) to Swift.
///
/// `depth` picks the limited-range code points (8-bit: 16/235/240 over 255; 10-bit:
/// 64/940/960 over 1023 — NOT the same normalized values, the difference is ~half a
/// code). `msb_packed` folds in the P010/X6 packing factor: 10 significant bits live in
/// the MSBs of 16, so a UNORM16 sample reads `code·64/65535` — multiplying by
/// `65535/65472` recovers exact `code/1023`.
pub fn csc_rows(desc: ColorDesc, depth: u8, msb_packed: bool) -> [[f32; 4]; 3] {
// BT.601 (5/6), BT.2020 (9/10); everything else — incl. unspecified — is the host's
// BT.709 SDR default (mirrors the software path's swscale coefficient choice).
let (kr, kb) = match desc.matrix {
5 | 6 => (0.299, 0.114),
9 | 10 => (0.2627, 0.0593),
_ => (0.2126, 0.0722),
};
let kg = 1.0 - kr - kb;
let max = f64::from((1u32 << depth) - 1); // 255 / 1023
let step = f64::from(1u32 << (depth - 8)); // code points per 8-bit step: 1 / 4
let pack = if msb_packed { 65535.0 / 65472.0 } else { 1.0 };
let (sy, oy, sc) = if desc.full_range {
(pack, 0.0f64, pack)
} else {
(
pack * max / (219.0 * step),
-(16.0 * step) / max,
pack * max / (224.0 * step),
)
};
// rgb = M * (yuv + off) = M*yuv + M*off — rows of M with the offset dot folded into
// w. `yuv` is the SAMPLED (packed) value, so the offsets divide by the packing
// factor to land on the same scale.
let off = [oy / pack, -0.5 / pack, -0.5 / pack];
let m = [
[sy, 0.0, 2.0 * (1.0 - kr) * sc],
[
sy,
-2.0 * (1.0 - kb) * kb / kg * sc,
-2.0 * (1.0 - kr) * kr / kg * sc,
],
[sy, 2.0 * (1.0 - kb) * sc, 0.0],
];
core::array::from_fn(|r| {
let w: f64 = (0..3).map(|c| m[r][c] * off[c]).sum();
[m[r][0] as f32, m[r][1] as f32, m[r][2] as f32, w as f32]
})
}
#[cfg(test)]
mod tests {
use super::*;
fn desc(matrix: u8, full_range: bool) -> ColorDesc {
ColorDesc {
primaries: 1,
transfer: 1,
matrix,
full_range,
}
}
fn apply(rows: &[[f32; 4]; 3], yuv: [f32; 3]) -> [f32; 3] {
core::array::from_fn(|r| {
rows[r][0] * yuv[0] + rows[r][1] * yuv[1] + rows[r][2] * yuv[2] + rows[r][3]
})
}
/// 10-bit limited MSB-packed (P010/X6): reference white Y=940, black Y=64, neutral
/// chroma 512 — sampled as UNORM16 of `code << 6`.
#[test]
fn bt2020_10bit_limited_white_black() {
let rows = csc_rows(desc(9, false), 10, true);
let s = |code: u32| ((code << 6) as f32) / 65535.0;
let white = apply(&rows, [s(940), s(512), s(512)]);
let black = apply(&rows, [s(64), s(512), s(512)]);
for (w, b) in white.iter().zip(black) {
assert!((w - 1.0).abs() < 0.002, "white {white:?}");
assert!(b.abs() < 0.002, "black {black:?}");
}
}
/// Reference white (Y=235, U=V=128 limited) → RGB 1.0; reference black (Y=16) → 0.0
/// — the GL presenter's test, in row form.
#[test]
fn bt709_limited_white_black() {
let rows = csc_rows(desc(1, false), 8, false);
let white = apply(&rows, [235.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0]);
let black = apply(&rows, [16.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0]);
for (w, b) in white.iter().zip(black) {
assert!((w - 1.0).abs() < 0.005, "white {white:?}");
assert!(b.abs() < 0.005, "black {black:?}");
}
}
/// Full-range identity points + the 601-vs-709 red excursion (guards the
/// matrix-code dispatch), same as the GL presenter's test.
#[test]
fn full_range_and_red_excursion() {
let rows = csc_rows(desc(5, true), 8, false);
let white = apply(&rows, [1.0, 0.5, 0.5]);
assert!(white.iter().all(|v| (v - 1.0).abs() < 1e-5), "{white:?}");
let red = apply(&rows, [0.0, 0.5, 1.0]);
assert!((red[0] - 2.0 * (1.0 - 0.299) * 0.5).abs() < 1e-4, "{red:?}");
let rows709 = csc_rows(desc(1, true), 8, false);
let red709 = apply(&rows709, [0.0, 0.5, 1.0]);
assert!(
(red709[0] - 2.0 * (1.0 - 0.2126) * 0.5).abs() < 1e-4,
"{red709:?}"
);
assert!((red[0] - red709[0]).abs() > 0.05);
}
/// The row form must agree with the GL presenter's column-major `yuv_to_rgb` on a
/// grid of inputs — same math, different packing.
#[test]
fn rows_match_the_gl_matrix_form() {
for (matrix, full) in [(1u8, false), (1, true), (5, false), (9, false), (9, true)] {
let d = desc(matrix, full);
let rows = csc_rows(d, 8, false);
// Reimplementation of video_gl::yuv_to_rgb's application for comparison.
let (kr, kb) = match matrix {
5 | 6 => (0.299f32, 0.114f32),
9 | 10 => (0.2627, 0.0593),
_ => (0.2126, 0.0722),
};
let kg = 1.0 - kr - kb;
let (sy, oy, sc) = if full {
(1.0f32, 0.0f32, 1.0f32)
} else {
(255.0 / 219.0, -16.0 / 255.0, 255.0 / 224.0)
};
let mat = [
sy,
sy,
sy,
0.0,
-2.0 * (1.0 - kb) * kb / kg * sc,
2.0 * (1.0 - kb) * sc,
2.0 * (1.0 - kr) * sc,
-2.0 * (1.0 - kr) * kr / kg * sc,
0.0,
];
let off = [oy, -0.5, -0.5];
for yuv in [
[0.1f32, 0.3, 0.7],
[0.9, 0.5, 0.5],
[0.5, 0.2, 0.8],
[16.0 / 255.0, 0.5, 0.5],
] {
let v = [yuv[0] + off[0], yuv[1] + off[1], yuv[2] + off[2]];
let gl: [f32; 3] =
core::array::from_fn(|r| (0..3).map(|c| mat[c * 3 + r] * v[c]).sum());
let ours = apply(&rows, yuv);
for (a, b) in gl.iter().zip(ours) {
assert!(
(a - b).abs() < 1e-5,
"{matrix}/{full}: gl {gl:?} rows {ours:?}"
);
}
}
}
}
}
+264
View File
@@ -0,0 +1,264 @@
//! CPU/libavcodec software decode backend (swscale → RGBA).
use crate::video::{averr, CpuFrame};
use crate::video_color::ColorDesc;
use anyhow::{anyhow, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
use std::ptr;
// --- software backend ---------------------------------------------------------------
pub(crate) struct SoftwareDecoder {
decoder: ffmpeg::decoder::Video,
/// Rebuilt whenever the decoded format/size — or the colour signaling (a mid-stream
/// SDR↔HDR flip) — changes.
sws: Option<(scaling::Context, Pixel, u32, u32, ColorDesc)>,
}
impl SoftwareDecoder {
pub(crate) fn new(codec_id: ffmpeg::codec::Id) -> Result<SoftwareDecoder> {
let codec = ffmpeg::decoder::find(codec_id)
.ok_or_else(|| anyhow!("no {codec_id:?} decoder in libavcodec"))?;
let mut ctx = ffmpeg::codec::Context::new_with_codec(codec);
unsafe {
let raw = ctx.as_mut_ptr();
(*raw).flags |= ffmpeg::ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
// Slice threading adds no frame delay (frame threading adds thread_count-1).
(*raw).thread_type = ffmpeg::ffi::FF_THREAD_SLICE;
(*raw).thread_count = 0; // auto
}
let decoder = ctx.decoder().video().context("open video decoder")?;
Ok(SoftwareDecoder { decoder, sws: None })
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<CpuFrame>> {
let packet = ffmpeg::Packet::copy(au);
self.decoder
.send_packet(&packet)
.map_err(|e| anyhow!("send_packet: {e}"))?;
let mut frame = AvFrame::empty();
let mut out = None;
while self.decoder.receive_frame(&mut frame).is_ok() {
out = Some(self.convert_rgba(&frame)?);
}
Ok(out)
}
fn convert_rgba(&mut self, frame: &AvFrame) -> Result<CpuFrame> {
let (fmt, w, h) = (frame.format(), frame.width(), frame.height());
// SAFETY: `frame.as_ptr()` is the decoder-owned live AVFrame for this call.
let color = unsafe { ColorDesc::from_raw(frame.as_ptr()) };
let rebuild = !matches!(&self.sws,
Some((_, f, sw, sh, c)) if *f == fmt && *sw == w && *sh == h && *c == color);
if rebuild {
let mut ctx =
scaling::Context::get(fmt, w, h, Pixel::RGBA, w, h, scaling::Flags::POINT)
.context("swscale context")?;
// swscale defaults to BT.601 coefficients — set them from the FRAME's signaling
// (unspecified → BT.709 limited, the host's SDR default; a Windows HDR desktop
// streams BT.2020 in-band). Without this, YUV→RGB decodes with the wrong matrix
// and colours shift. Destination = full-range RGB; the transfer function stays
// baked in (the presenter tags PQ textures so GTK applies the EOTF).
const SWS_CS_ITU709: i32 = 1;
const SWS_CS_ITU601: i32 = 5;
const SWS_CS_BT2020: i32 = 9;
let cs = match color.matrix {
9 | 10 => SWS_CS_BT2020,
5 | 6 => SWS_CS_ITU601,
_ => SWS_CS_ITU709,
};
unsafe {
let coeffs = ffmpeg::ffi::sws_getCoefficients(cs);
ffmpeg::ffi::sws_setColorspaceDetails(
ctx.as_mut_ptr(),
coeffs, // inv_table: source (YUV) coefficients per the VUI
color.full_range as i32, // srcRange: 0 = limited/studio (MPEG)
coeffs, // table: destination coefficients (ignored for RGB output)
1, // dstRange: 1 = full-range RGB
0,
1 << 16,
1 << 16, // brightness, contrast, saturation (defaults)
);
}
self.sws = Some((ctx, fmt, w, h, color));
}
let (sws, ..) = self.sws.as_mut().unwrap();
// Single-pass conversion: swscale writes straight into the Vec the texture will
// wrap. (The old path scaled into a scratch AVFrame and then copied `data(0)` out
// — a second full-frame pass per frame.) 64-byte row alignment keeps swscale on
// aligned SIMD stores; `GdkMemoryTexture` takes the resulting stride explicitly.
const ALIGN: i32 = 64;
use ffmpeg::ffi;
let dst_fmt = ffi::AVPixelFormat::AV_PIX_FMT_RGBA;
// SAFETY: pure size computation from format/dimensions; no pointers involved.
let size = unsafe { ffi::av_image_get_buffer_size(dst_fmt, w as i32, h as i32, ALIGN) };
if size < 0 {
return Err(averr("av_image_get_buffer_size", size));
}
let rgba = vec![0u8; size as usize];
let mut dst_data: [*mut u8; 4] = [ptr::null_mut(); 4];
let mut dst_linesize: [i32; 4] = [0; 4];
// SAFETY: fill_arrays only derives plane pointers/strides into `rgba` (sized by
// av_image_get_buffer_size above, same format/align) — no allocation, no
// ownership transfer; `rgba` outlives the scale below.
let r = unsafe {
ffi::av_image_fill_arrays(
dst_data.as_mut_ptr(),
dst_linesize.as_mut_ptr(),
rgba.as_ptr(),
dst_fmt,
w as i32,
h as i32,
ALIGN,
)
};
if r < 0 {
return Err(averr("av_image_fill_arrays", r));
}
// SAFETY: src pointers/strides belong to the decoder-owned `frame` (alive for the
// call); dst pointers were just filled over `rgba`, and sws_scale writes rows
// [0, h) only — exactly the buffer fill_arrays sized.
let r = unsafe {
ffi::sws_scale(
sws.as_mut_ptr(),
(*frame.as_ptr()).data.as_ptr() as *const *const u8,
(*frame.as_ptr()).linesize.as_ptr(),
0,
h as i32,
dst_data.as_ptr(),
dst_linesize.as_ptr(),
)
};
if r < 0 {
return Err(averr("sws_scale", r));
}
Ok(CpuFrame {
width: w,
height: h,
stride: dst_linesize[0] as usize,
rgba,
color,
// `is_key()` reads the same intra flag `frame_is_keyframe` derives from pict_type
// for the hardware paths; ffmpeg-next handles the FFmpeg-version binding split.
keyframe: frame.is_key(),
})
}
}
#[cfg(test)]
mod tests {
use super::*;
/// The wire → `ColorDesc` plumbing: an HDR10 stream's VUI (BT.2020 primaries, PQ
/// transfer, BT.2020-NCL matrix, limited range) must arrive on the decoded frame —
/// this is what the Windows host emits in-band for an HDR desktop, and mis-rendering
/// it as BT.709 is the washed-out-colors bug. Fixture: one 64×64 Main10 IDR
/// (`tests/pq-frame.h265`, x265 with explicit VUI).
#[test]
fn software_decode_carries_pq_signaling() {
let au = include_bytes!("../tests/pq-frame.h265");
let mut dec = SoftwareDecoder::new(ffmpeg::codec::Id::HEVC).expect("hevc decoder");
let mut got = dec.decode(au).expect("decode");
if got.is_none() {
// Low-delay decoders may still hold the frame until a flush — send EOF.
dec.decoder.send_eof().ok();
let mut frame = AvFrame::empty();
if dec.decoder.receive_frame(&mut frame).is_ok() {
got = Some(dec.convert_rgba(&frame).expect("convert"));
}
}
let f = got.expect("no frame decoded from the PQ fixture");
assert_eq!(
f.color,
ColorDesc {
primaries: 9,
transfer: 16,
matrix: 9,
full_range: false
}
);
assert!(f.color.is_pq());
assert_eq!((f.width, f.height), (64, 64));
}
/// Golden colour fixtures: one 256×64 LOSSLESS x265 IDR of 8 fully-saturated colour bars per
/// signaling variant (generated offline with ffmpeg/libx265; the RGB→YUV conversion matched
/// to the VUI each fixture declares, so the original RGB is recoverable ±1 code). Decoding
/// through the real CPU path (`SoftwareDecoder` → per-frame `ColorDesc` → swscale with the
/// signaled matrix/range) must reproduce the bars — the end-to-end guard for the
/// signaling-driven CSC across BT.601/709 × limited/full. A hardcoded-709 regression fails
/// the 601 fixture by tens of code points; a range mix-up fails the full-range one.
#[test]
fn software_decode_reproduces_golden_bars() {
const BARS: [(u8, u8, u8); 8] = [
(255, 255, 255),
(255, 255, 0),
(0, 255, 255),
(0, 255, 0),
(255, 0, 255),
(255, 0, 0),
(0, 0, 255),
(0, 0, 0),
];
let fixtures: [(&str, &[u8], ColorDesc); 3] = [
(
"601-limited",
include_bytes!("../tests/bars-601-limited.h265"),
ColorDesc {
primaries: 1,
transfer: 1,
matrix: 5, // BT.470BG — what a Linux host's RGB-input NVENC signals
full_range: false,
},
),
(
"709-limited",
include_bytes!("../tests/bars-709-limited.h265"),
ColorDesc {
primaries: 1,
transfer: 1,
matrix: 1,
full_range: false,
},
),
(
"709-full",
include_bytes!("../tests/bars-709-full.h265"),
ColorDesc {
primaries: 1,
transfer: 1,
matrix: 1,
full_range: true, // the PUNKTFUNK_444_FULLRANGE experiment's signaling
},
),
];
for (name, au, want_color) in fixtures {
let mut dec = SoftwareDecoder::new(ffmpeg::codec::Id::HEVC).expect("hevc decoder");
let mut got = dec.decode(au).expect("decode");
if got.is_none() {
dec.decoder.send_eof().ok();
let mut frame = AvFrame::empty();
if dec.decoder.receive_frame(&mut frame).is_ok() {
got = Some(dec.convert_rgba(&frame).expect("convert"));
}
}
let f = got.unwrap_or_else(|| panic!("{name}: no frame decoded"));
assert_eq!(f.color, want_color, "{name}: signaling");
assert_eq!((f.width, f.height), (256, 64), "{name}: dims");
for (i, (r, g, b)) in BARS.iter().enumerate() {
let (cx, cy) = (i * 32 + 16, 32usize);
let o = cy * f.stride + cx * 4;
let px = &f.rgba[o..o + 3];
for (got, want) in px.iter().zip([r, g, b]) {
assert!(
got.abs_diff(*want) <= 3,
"{name} bar {i}: got {px:?}, want ({r},{g},{b})"
);
}
}
}
}
}
+243
View File
@@ -0,0 +1,243 @@
//! VAAPI (libavcodec hwaccel) decode backend → DRM-PRIME dmabuf for the presenter. Linux-only.
use crate::video::{
averr, drm_fourcc_for, frame_is_keyframe, DmabufFrame, DmabufPlane, DrmFrameGuard,
AVERROR_EAGAIN,
};
use crate::video_color::ColorDesc;
use anyhow::{anyhow, bail, Result};
use ffmpeg_next as ffmpeg;
use std::ptr;
/// libavcodec offers the formats it can decode into; pick the VAAPI hw surface. Falling
/// back to the first (software) entry would silently decode on the CPU *and* break our
/// dmabuf mapping — return NONE instead so the error surfaces and the session demotes
/// to the software backend explicitly.
#[cfg(target_os = "linux")]
unsafe extern "C" fn pick_vaapi(
_ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
) -> ffmpeg::ffi::AVPixelFormat {
unsafe {
while *list != ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE {
if *list == ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_VAAPI {
return ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_VAAPI;
}
list = list.add(1);
}
}
ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE
}
#[cfg(target_os = "linux")]
pub(crate) struct VaapiDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
}
// Single-owner pointers, only touched from the session pump thread.
#[cfg(target_os = "linux")]
unsafe impl Send for VaapiDecoder {}
#[cfg(target_os = "linux")]
impl VaapiDecoder {
pub(crate) fn new(codec_id: ffmpeg::codec::Id) -> Result<VaapiDecoder> {
use ffmpeg::ffi;
unsafe {
let mut hw_device: *mut ffi::AVBufferRef = ptr::null_mut();
let r = ffi::av_hwdevice_ctx_create(
&mut hw_device,
ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VAAPI,
ptr::null(),
ptr::null_mut(),
0,
);
if r < 0 {
bail!("no VAAPI device ({})", ffmpeg::Error::from(r));
}
// The negotiated codec's decoder id (av_codec_id maps 1:1 from ffmpeg::codec::Id).
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
ffi::av_buffer_unref(&mut hw_device);
bail!("no {codec_id:?} decoder");
}
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device);
(*ctx).get_format = Some(pick_vaapi);
(*ctx).flags |= ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
(*ctx).thread_count = 1; // hwaccel: threads only add latency
// The presenter holds mapped surfaces PAST receive_frame (the paintable's
// current texture + the newest frame in flight each pin one until GDK's
// release func) — surfaces libavcodec doesn't know are missing from its
// fixed-size VAAPI pool. Without headroom the decoder can recycle a surface
// the renderer is still sampling (intermittent block corruption) or fail
// allocation under scheduling jitter.
(*ctx).extra_hw_frames = 4;
let r = ffi::avcodec_open2(ctx, codec, ptr::null_mut());
if r < 0 {
let mut ctx = ctx;
ffi::avcodec_free_context(&mut ctx);
let mut hw_device = hw_device;
ffi::av_buffer_unref(&mut hw_device);
bail!("avcodec_open2: {}", ffmpeg::Error::from(r));
}
Ok(VaapiDecoder {
ctx,
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
})
}
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<DmabufFrame>> {
use ffmpeg::ffi;
unsafe {
let r = ffi::av_new_packet(self.packet, au.len() as i32);
if r < 0 {
return Err(averr("av_new_packet", r));
}
ptr::copy_nonoverlapping(au.as_ptr(), (*self.packet).data, au.len());
let r = ffi::avcodec_send_packet(self.ctx, self.packet);
ffi::av_packet_unref(self.packet);
if r < 0 {
return Err(averr("send_packet", r));
}
let mut out = None;
loop {
let r = ffi::avcodec_receive_frame(self.ctx, self.frame);
if r == AVERROR_EAGAIN {
break;
}
if r < 0 {
return Err(averr("receive_frame", r));
}
out = Some(self.map_dmabuf()?); // newest wins; older guards drop here
ffi::av_frame_unref(self.frame);
}
Ok(out)
}
}
/// Map the VAAPI surface to DRM PRIME (zero copy) and lift the descriptor into a
/// `DmabufFrame`. The mapped frame keeps the surface alive via its buffer refs.
///
/// FFmpeg's VAAPI export uses `VA_EXPORT_SURFACE_SEPARATE_LAYERS`, so an NV12 surface
/// comes back as TWO layers (`R8` luma + `GR88` chroma), each one plane — NOT a single
/// `NV12` layer. The previous code took `layers[0]` only: GTK then saw an `R8`
/// single-plane texture with the chroma dropped, painting the screen green. The fix:
/// derive the COMBINED fourcc from the decoder's software pixel format (NV12 →
/// `DRM_FORMAT_NV12`) and flatten every plane across every layer in order (Y then UV).
unsafe fn map_dmabuf(&mut self) -> Result<DmabufFrame> {
use ffmpeg::ffi;
unsafe {
if (*self.frame).format != ffi::AVPixelFormat::AV_PIX_FMT_VAAPI as i32 {
bail!("decoder returned a software frame (no VAAPI surface)");
}
// The real pixel layout lives on the hardware frames context, not the
// DRM-PRIME layer formats (those are the per-plane R8/GR88 component formats).
let sw_format = {
let hwfc = (*self.frame).hw_frames_ctx;
if hwfc.is_null() {
bail!("VAAPI frame without a hardware frames context");
}
(*((*hwfc).data as *const ffi::AVHWFramesContext)).sw_format
};
let fourcc = drm_fourcc_for(sw_format)
.ok_or_else(|| anyhow!("unsupported VAAPI output format {sw_format:?}"))?;
let drm = ffi::av_frame_alloc();
(*drm).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME as i32;
let r = ffi::av_hwframe_map(drm, self.frame, ffi::AV_HWFRAME_MAP_READ as i32);
if r < 0 {
let mut drm = drm;
ffi::av_frame_free(&mut drm);
return Err(averr("av_hwframe_map", r));
}
let desc = (*drm).data[0] as *const ffi::AVDRMFrameDescriptor;
let guard = DrmFrameGuard(drm);
let d = &*desc;
if d.nb_layers < 1 || d.nb_objects < 1 {
bail!("DRM descriptor without layers/objects");
}
// Flatten planes across ALL layers, in declared order — the combined fourcc's
// plane order (Y, then UV for NV12) matches the layer order FFmpeg emits.
let mut planes = Vec::new();
for layer in &d.layers[..d.nb_layers as usize] {
for p in &layer.planes[..layer.nb_planes as usize] {
let obj = &d.objects[p.object_index as usize];
planes.push(DmabufPlane {
fd: obj.fd,
offset: p.offset as u32,
stride: p.pitch as u32,
});
}
}
// The whole surface shares one tiling modifier (one BO on radeonsi); GTK takes
// a single modifier for the texture.
let modifier = d.objects[0].format_modifier;
log_descriptor_once(d, sw_format, fourcc, modifier);
Ok(DmabufFrame {
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
fourcc,
modifier,
planes,
// SAFETY: `self.frame` is the live decoded AVFrame (unref'd only after
// this returns); plain CICP field reads.
color: ColorDesc::from_raw(self.frame),
keyframe: frame_is_keyframe(self.frame),
guard,
})
}
}
}
/// One-time dump of the DRM descriptor layout (objects, layers, planes, modifier) — so a
/// new client/driver combination's real layout is visible in the logs without a debugger.
#[cfg(target_os = "linux")]
fn log_descriptor_once(
d: &ffmpeg_next::ffi::AVDRMFrameDescriptor,
sw: ffmpeg_next::ffi::AVPixelFormat,
fourcc: u32,
modifier: u64,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if !ONCE.swap(false, Ordering::Relaxed) {
return;
}
let layers: Vec<(u32, i32)> = d.layers[..d.nb_layers.max(0) as usize]
.iter()
.map(|l| (l.format, l.nb_planes))
.collect();
tracing::info!(
sw_format = ?sw,
chosen_fourcc = format_args!("{:#010x}", fourcc),
nb_objects = d.nb_objects,
nb_layers = d.nb_layers,
?layers,
modifier = format_args!("{:#018x}", modifier),
"VAAPI dmabuf descriptor layout (first frame)"
);
}
#[cfg(target_os = "linux")]
impl Drop for VaapiDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
unsafe {
ffi::av_packet_free(&mut self.packet);
ffi::av_frame_free(&mut self.frame);
ffi::avcodec_free_context(&mut self.ctx);
ffi::av_buffer_unref(&mut self.hw_device);
}
}
}
+419
View File
@@ -0,0 +1,419 @@
//! FFmpeg Vulkan Video decode over the presenter's own VkDevice (zero-copy VkImage).
#![allow(clippy::unnecessary_cast)]
use crate::video::{
averr, frame_is_keyframe, DrmFrameGuard, QueueLock, VkVideoFrame, VulkanDecodeDevice,
AVERROR_EAGAIN,
};
use crate::video_color::ColorDesc;
use anyhow::{bail, Result};
use ffmpeg_next as ffmpeg;
use std::ptr;
// --- Vulkan Video backend -------------------------------------------------------------
/// FFmpeg's Vulkan Video decoder over the PRESENTER's device: the hwdevice context is
/// built from [`VulkanDecodeDevice`]'s handles (not `av_hwdevice_ctx_create`, which
/// would make FFmpeg create its own device the presenter can't sample from). Output
/// frames are `AVVkFrame`s whose VkImage the presenter feeds straight to its CSC pass.
pub(crate) struct VulkanDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
/// `vkWaitSemaphores` on the shared device — the decode-complete measurement
/// (resolved through the same get_proc_addr chain FFmpeg uses).
wait_semaphores: pf_ffvk::PFN_vkWaitSemaphores,
vk_device: pf_ffvk::VkDevice,
/// Storage `AVVulkanDeviceContext` points into (extension string arrays + the
/// feature chain) — FFmpeg reads the extension lists past init (frames-context
/// setup keys code paths off them), so this lives exactly as long as `hw_device`.
_ctx_storage: Box<VkCtxStorage>,
}
// Single-owner pointers, only touched from the session pump thread.
unsafe impl Send for VulkanDecoder {}
struct VkCtxStorage {
_inst: Vec<std::ffi::CString>,
inst_ptrs: Vec<*const std::os::raw::c_char>,
_dev: Vec<std::ffi::CString>,
dev_ptrs: Vec<*const std::os::raw::c_char>,
f11: pf_ffvk::VkPhysicalDeviceVulkan11Features,
f12: pf_ffvk::VkPhysicalDeviceVulkan12Features,
f13: pf_ffvk::VkPhysicalDeviceVulkan13Features,
/// Keeps the shared queue lock alive for `AVHWDeviceContext.user_opaque` — the
/// `lock_queue`/`unlock_queue` trampolines below dereference it for as long as the
/// hw device context can fire them.
_queue_lock: std::sync::Arc<QueueLock>,
}
/// FFmpeg `AVVulkanDeviceContext.lock_queue` trampoline: take the device's shared
/// [`QueueLock`] (stashed in `AVHWDeviceContext.user_opaque`; owned by
/// [`VkCtxStorage`], which outlives the context). Replaces FFmpeg's internal default,
/// which only serializes FFmpeg against itself — the presenter submits to the same
/// graphics queue from another thread and holds this same lock around its calls.
unsafe extern "C" fn ffvk_lock_queue(
ctx: *mut pf_ffvk::AVHWDeviceContext,
_queue_family: u32,
_index: u32,
) {
let dev = ctx as *mut ffmpeg::ffi::AVHWDeviceContext;
let lock = (*dev).user_opaque as *const QueueLock;
(*lock).lock();
}
/// The matching `unlock_queue` trampoline — see [`ffvk_lock_queue`].
unsafe extern "C" fn ffvk_unlock_queue(
ctx: *mut pf_ffvk::AVHWDeviceContext,
_queue_family: u32,
_index: u32,
) {
let dev = ctx as *mut ffmpeg::ffi::AVHWDeviceContext;
let lock = (*dev).user_opaque as *const QueueLock;
(*lock).unlock();
}
impl VulkanDecoder {
pub(crate) fn new(
codec_id: ffmpeg::codec::Id,
vk: &VulkanDecodeDevice,
) -> Result<VulkanDecoder> {
use ffmpeg::ffi;
unsafe {
let mut hw_device =
ffi::av_hwdevice_ctx_alloc(ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VULKAN);
if hw_device.is_null() {
bail!("av_hwdevice_ctx_alloc(VULKAN) failed (FFmpeg built without Vulkan?)");
}
let devctx = (*hw_device).data as *mut ffi::AVHWDeviceContext;
let hwctx = (*devctx).hwctx as *mut pf_ffvk::AVVulkanDeviceContext;
// Pinned storage for everything the context points into.
let mut store = Box::new(VkCtxStorage {
_inst: vk.instance_extensions.clone(),
inst_ptrs: Vec::new(),
_dev: vk.device_extensions.clone(),
dev_ptrs: Vec::new(),
f11: std::mem::zeroed(),
f12: std::mem::zeroed(),
f13: std::mem::zeroed(),
_queue_lock: vk.queue_lock.clone(),
});
store.inst_ptrs = store._inst.iter().map(|c| c.as_ptr()).collect();
store.dev_ptrs = store._dev.iter().map(|c| c.as_ptr()).collect();
// The features enabled at device creation, as the 1.1/1.2/1.3 chain FFmpeg
// walks to learn what it may use (sType values are vulkan.h constants).
store.f11.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_1_FEATURES;
store.f11.samplerYcbcrConversion = vk.f_sampler_ycbcr as u32;
store.f12.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_2_FEATURES;
store.f12.timelineSemaphore = vk.f_timeline_semaphore as u32;
store.f13.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_FEATURES;
store.f13.synchronization2 = vk.f_synchronization2 as u32;
store.f11.pNext = &mut store.f12 as *mut _ as *mut std::ffi::c_void;
store.f12.pNext = &mut store.f13 as *mut _ as *mut std::ffi::c_void;
(*hwctx).get_proc_addr = std::mem::transmute::<usize, pf_ffvk::PFN_vkGetInstanceProcAddr>(
vk.get_instance_proc_addr,
);
(*hwctx).inst = vk.instance as pf_ffvk::VkInstance;
(*hwctx).phys_dev = vk.physical_device as pf_ffvk::VkPhysicalDevice;
(*hwctx).act_dev = vk.device as pf_ffvk::VkDevice;
(*hwctx).device_features.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2;
(*hwctx).device_features.pNext = &mut store.f11 as *mut _ as *mut std::ffi::c_void;
(*hwctx).enabled_inst_extensions = store.inst_ptrs.as_ptr();
(*hwctx).nb_enabled_inst_extensions = store.inst_ptrs.len() as i32;
(*hwctx).enabled_dev_extensions = store.dev_ptrs.as_ptr();
(*hwctx).nb_enabled_dev_extensions = store.dev_ptrs.len() as i32;
// Queue map: the deprecated per-role indices (tx/comp are "Required") plus
// the qf[] list, which per the header must also carry every family named
// above. One merged entry when decode shares the graphics family.
let g = vk.graphics_qf as i32;
let d = vk.decode_qf as i32;
(*hwctx).queue_family_index = g;
(*hwctx).nb_graphics_queues = 1;
(*hwctx).queue_family_tx_index = g;
(*hwctx).nb_tx_queues = 1;
(*hwctx).queue_family_comp_index = g;
(*hwctx).nb_comp_queues = 1;
(*hwctx).queue_family_encode_index = -1;
(*hwctx).nb_encode_queues = 0;
(*hwctx).queue_family_decode_index = d;
(*hwctx).nb_decode_queues = 1;
const VIDEO_DECODE_BIT: u32 = 0x20; // VK_QUEUE_VIDEO_DECODE_BIT_KHR
// `flags`/`video_caps` are bindgen enum types: i32 under MSVC, u32 under
// Linux clang — the `as _` casts absorb the difference.
if g == d {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: (vk.graphics_queue_flags | VIDEO_DECODE_BIT) as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 1;
} else {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags as _,
video_caps: 0,
};
(*hwctx).qf[1] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: d,
num: 1,
flags: VIDEO_DECODE_BIT as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 2;
}
// Shared-queue external sync (see [`QueueLock`]): FFmpeg must take the
// same lock the presenter holds around its own submits/presents — set
// BEFORE init so FFmpeg never installs its internal defaults (which only
// serialize FFmpeg against itself; the cross-thread race with the
// presenter's queue was an intermittent VK_ERROR_DEVICE_LOST).
(*devctx).user_opaque =
std::sync::Arc::as_ptr(&store._queue_lock) as *mut std::ffi::c_void;
(*hwctx).lock_queue = Some(ffvk_lock_queue);
(*hwctx).unlock_queue = Some(ffvk_unlock_queue);
let r = ffi::av_hwdevice_ctx_init(hw_device);
if r < 0 {
ffi::av_buffer_unref(&mut hw_device);
return Err(averr("av_hwdevice_ctx_init(VULKAN)", r));
}
// vkWaitSemaphores for the pump's decode-complete stat: loader →
// vkGetDeviceProcAddr → device fn (core 1.2, guaranteed by our gate).
let gipa = (*hwctx)
.get_proc_addr
.expect("get_proc_addr was just set above");
let gdpa: pf_ffvk::PFN_vkGetDeviceProcAddr =
std::mem::transmute(gipa((*hwctx).inst, c"vkGetDeviceProcAddr".as_ptr()));
let wait_semaphores: pf_ffvk::PFN_vkWaitSemaphores = std::mem::transmute(gdpa
.expect("vkGetDeviceProcAddr resolvable")(
(*hwctx).act_dev,
c"vkWaitSemaphores".as_ptr(),
));
if wait_semaphores.is_none() {
ffi::av_buffer_unref(&mut hw_device);
bail!("vkWaitSemaphores unresolvable on this device");
}
let vk_device = (*hwctx).act_dev;
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
ffi::av_buffer_unref(&mut hw_device);
bail!("no {codec_id:?} decoder");
}
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device);
(*ctx).get_format = Some(pick_vulkan);
(*ctx).flags |= ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
(*ctx).thread_count = 1; // hwaccel: threads only add latency
// Same pool headroom rationale as VAAPI: the presenter pins the on-screen
// frame + the newest in flight past receive_frame.
(*ctx).extra_hw_frames = 4;
let r = ffi::avcodec_open2(ctx, codec, ptr::null_mut());
if r < 0 {
let mut ctx = ctx;
ffi::avcodec_free_context(&mut ctx);
ffi::av_buffer_unref(&mut hw_device);
return Err(averr("avcodec_open2 (vulkan)", r));
}
Ok(VulkanDecoder {
ctx,
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
wait_semaphores,
vk_device,
_ctx_storage: store,
})
}
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<VkVideoFrame>> {
use ffmpeg::ffi;
unsafe {
let r = ffi::av_new_packet(self.packet, au.len() as i32);
if r < 0 {
return Err(averr("av_new_packet", r));
}
ptr::copy_nonoverlapping(au.as_ptr(), (*self.packet).data, au.len());
let r = ffi::avcodec_send_packet(self.ctx, self.packet);
ffi::av_packet_unref(self.packet);
if r < 0 {
return Err(averr("send_packet", r));
}
let mut out = None;
loop {
let r = ffi::avcodec_receive_frame(self.ctx, self.frame);
if r == AVERROR_EAGAIN {
break;
}
if r < 0 {
return Err(averr("receive_frame", r));
}
out = Some(self.extract()?); // newest wins; older guards drop here
ffi::av_frame_unref(self.frame);
}
Ok(out)
}
}
/// Block until the timeline semaphore reaches `value` (GPU decode complete) or the
/// timeout passes. Pure measurement — the presenter's own GPU wait is what gates
/// sampling, so a timeout here only degrades the stat, never the picture.
pub(crate) fn wait_timeline(&self, sem: u64, value: u64, timeout_ns: u64) -> bool {
let sems = [sem as pf_ffvk::VkSemaphore];
let values = [value];
let info = pf_ffvk::VkSemaphoreWaitInfo {
sType: pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_SEMAPHORE_WAIT_INFO,
pNext: std::ptr::null(),
flags: 0,
semaphoreCount: 1,
pSemaphores: sems.as_ptr(),
pValues: values.as_ptr(),
};
// SAFETY: resolved from this device at init; handles outlive the decoder.
let r = unsafe {
self.wait_semaphores.expect("checked at init")(self.vk_device, &info, timeout_ns)
};
r == 0 // VK_SUCCESS (VK_TIMEOUT = 2)
}
/// Lift the decoded `AVVkFrame` into a [`VkVideoFrame`]: clone the AVFrame (the
/// guard — keeps the image + frames context alive through present) and ship the
/// POINTERS; the presenter reads the live sync state under the frames-context lock
/// at its own submit time.
unsafe fn extract(&mut self) -> Result<VkVideoFrame> {
use ffmpeg::ffi;
unsafe {
if (*self.frame).format != ffi::AVPixelFormat::AV_PIX_FMT_VULKAN as i32 {
bail!("decoder returned a non-Vulkan frame");
}
let hwfc_ref = (*self.frame).hw_frames_ctx;
if hwfc_ref.is_null() {
bail!("Vulkan frame without a hardware frames context");
}
let fc = (*hwfc_ref).data as *mut ffi::AVHWFramesContext;
let sw = (*fc).sw_format;
if sw != ffi::AVPixelFormat::AV_PIX_FMT_NV12
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P010LE
{
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010 only)");
}
let vkfc = (*fc).hwctx as *const pf_ffvk::AVVulkanFramesContext;
let vk_format = (*vkfc).format[0] as i32;
let lock_frame = (*vkfc).lock_frame.map_or(0, |f| f as usize);
let unlock_frame = (*vkfc).unlock_frame.map_or(0, |f| f as usize);
if lock_frame == 0 || unlock_frame == 0 {
bail!("Vulkan frames context without lock functions");
}
let clone = ffi::av_frame_clone(self.frame);
if clone.is_null() {
bail!("av_frame_clone failed");
}
let vkf = (*clone).data[0] as *mut pf_ffvk::AVVkFrame;
// v1 handles the (default) single multiplanar image; a disjoint/multi-image
// pool would need per-plane images — bail so the session demotes cleanly.
if !(*vkf).img[1].is_null() {
let mut clone = clone;
ffi::av_frame_free(&mut clone);
bail!("multi-image Vulkan frames unsupported (disjoint pool)");
}
// Safe without the frames lock: the handle is creation-constant and
// sem_value was last written by the decode submission on THIS thread.
let timeline_sem = (*vkf).sem[0] as u64;
let decode_done_value = (*vkf).sem_value[0];
Ok(VkVideoFrame {
vkframe: vkf as usize,
frames_ctx: fc as usize,
lock_frame,
unlock_frame,
vk_format,
timeline_sem,
decode_done_value,
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
color: ColorDesc::from_raw(self.frame),
keyframe: frame_is_keyframe(self.frame),
guard: DrmFrameGuard(clone),
})
}
}
}
impl Drop for VulkanDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
unsafe {
ffi::av_packet_free(&mut self.packet);
ffi::av_frame_free(&mut self.frame);
ffi::avcodec_free_context(&mut self.ctx);
ffi::av_buffer_unref(&mut self.hw_device);
}
}
}
/// libavcodec offers the formats it can decode into; pick the Vulkan hw surface and
/// hand the decoder OUR frames context — the default one lacks
/// `VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT`, without which the presenter can't create the
/// per-plane views its CSC pass samples. Returning NONE (over the software entry) keeps
/// failures loud: the session demotes explicitly instead of silently CPU-decoding.
unsafe extern "C" fn pick_vulkan(
ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
) -> ffmpeg::ffi::AVPixelFormat {
use ffmpeg::ffi;
unsafe {
let mut offered = false;
while *list != ffi::AVPixelFormat::AV_PIX_FMT_NONE {
if *list == ffi::AVPixelFormat::AV_PIX_FMT_VULKAN {
offered = true;
break;
}
list = list.add(1);
}
if !offered {
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
let mut fr: *mut ffi::AVBufferRef = ptr::null_mut();
let r = ffi::avcodec_get_hw_frames_parameters(
ctx,
(*ctx).hw_device_ctx,
ffi::AVPixelFormat::AV_PIX_FMT_VULKAN,
&mut fr,
);
if r < 0 || fr.is_null() {
tracing::warn!(code = r, "avcodec_get_hw_frames_parameters(VULKAN) failed");
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
let fc = (*fr).data as *mut ffi::AVHWFramesContext;
let vkfc = (*fc).hwctx as *mut pf_ffvk::AVVulkanFramesContext;
// MUTABLE_FORMAT: per-plane views (spec requirement); ALIAS is FFmpeg's default.
// (`as _`: the FlagBits constants are i32 under MSVC, the img_flags field u32.)
(*vkfc).img_flags = (pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT)
as _;
let r = ffi::av_hwframe_ctx_init(fr);
if r < 0 {
tracing::warn!(code = r, "av_hwframe_ctx_init(VULKAN) failed");
let mut fr = fr;
ffi::av_buffer_unref(&mut fr);
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
if !(*ctx).hw_frames_ctx.is_null() {
ffi::av_buffer_unref(&mut (*ctx).hw_frames_ctx);
}
(*ctx).hw_frames_ctx = fr; // the codec owns our ref now
ffi::AVPixelFormat::AV_PIX_FMT_VULKAN
}
}
+7 -774
View File
@@ -9,22 +9,22 @@
//! `save_layer_alpha` so a screen fades as a unit, never element by element. The
//! backdrop crossfades in parallel when the screens disagree (aurora ↔ form).
use crate::anim::{approach, ease_out_cubic, Progress};
use crate::glyphs::{hint_bar, GlyphStyle, Hint, HintKey};
use crate::anim::Progress;
use crate::glyphs::GlyphStyle;
use crate::library::{mesh_sksl, LibraryShared};
use crate::model::{ConsoleBus, ConsoleCmd, ConsoleShared, HostRow, PairPhase, WakeStatus};
use crate::screens::{Bg, ConnectIntent, Ctx, Nav, Outbox, Screen};
use crate::theme::{white, Fonts, PanelStroke, DIM, W, WHITE};
use anyhow::{anyhow, Result};
use pf_client_core::gamepad::{MenuDir, MenuEvent, MenuPulse, PadInfo};
use pf_client_core::trust;
use pf_presenter::overlay::OverlayAction;
use skia_safe::{
gradient_shader, Canvas, Color4f, Data, Paint, Point, Rect, RuntimeEffect, TileMode,
};
use skia_safe::{Canvas, Color4f, Data, Paint, Rect, RuntimeEffect};
use std::collections::VecDeque;
use std::time::Instant;
mod overlays;
mod render;
const TRANSITION_S: f64 = 0.26;
/// Chrome bands (design units): the pinned title above, hints below.
const TOP_BAND: f64 = 64.0;
@@ -425,176 +425,6 @@ impl Shell {
}
}
// --- Rendering -------------------------------------------------------------------------
#[allow(clippy::too_many_arguments)]
pub(crate) fn render(
&mut self,
canvas: &Canvas,
width: u32,
height: u32,
fonts: &Fonts,
pad: Option<&str>,
pad_pref: Option<punktfunk_core::config::GamepadPref>,
pads: &[PadInfo],
) {
let now = Instant::now();
let dt = self
.last_frame
.replace(now)
.map_or(1.0 / 60.0, |t| (now - t).as_secs_f64().clamp(0.0, 0.05));
self.sync();
self.pads = pads.to_vec();
self.glyphs = GlyphStyle::from_pref(pad_pref);
self.chip = Some(pad.map_or_else(
|| "No controller — keyboard works too".to_string(),
str::to_owned,
));
let (w, h) = (f64::from(width), f64::from(height));
let k = (h / 800.0).clamp(0.75, 3.0);
let t = self.t();
// Advance the transition; a finished pop finally drops its leaving screen.
let motion_p = match &mut self.motion {
Motion::None => None,
Motion::Push(p) => {
p.advance(dt);
let v = p.value();
if p.done() {
self.motion = Motion::None;
None
} else {
Some(v)
}
}
Motion::Pop { t, .. } => {
t.advance(dt);
let v = t.value();
if t.done() {
self.motion = Motion::None;
None
} else {
Some(v)
}
}
};
// Backdrop crossfade follows the top screen.
let bg_target = match self.stack.last().expect("non-empty").background() {
Bg::Aurora => 0.0,
Bg::Form => 1.0,
};
self.bg_mix = approach(self.bg_mix, bg_target, dt, 0.12);
if (self.bg_mix - bg_target).abs() < 0.005 {
self.bg_mix = bg_target;
}
if self.bg_mix < 1.0 {
self.draw_aurora(canvas, w, h, t);
} else {
canvas.clear(Color4f::new(0.0, 0.0, 0.0, 1.0));
}
if self.bg_mix > 0.0 {
canvas.save_layer_alpha_f(None, self.bg_mix as f32);
crate::theme::draw_form_background(canvas, w, h);
canvas.restore();
}
// The screens, through the transition choreography.
let content = Rect::from_ltrb(
0.0,
(TOP_BAND * k) as f32,
w as f32,
(h - BOTTOM_BAND * k) as f32,
);
// One paint recipe per layer: (alpha, slide, scale). Everything below borrows
// disjoint fields of `self` per call, so the borrow checker stays happy.
let mut env = LayerEnv {
canvas,
w,
h,
content,
k,
dt,
fonts,
hosts: &self.hosts,
library: &self.library,
settings: &mut self.settings,
pads: &self.pads,
deck: self.deck,
device_name: &self.device_name,
t,
glyphs: self.glyphs,
// A modal card owns B/A while it's up — the screen's legend would lie.
show_hints: self.connecting.is_none() && self.wake.is_none(),
};
match (&mut self.motion, motion_p) {
(Motion::Push(_), Some(raw)) => {
let p = ease_out_cubic(raw);
let n = self.stack.len();
// Outgoing recedes underneath…
if n >= 2 {
let (below, top) = self.stack.split_at_mut(n - 1);
env.paint(&mut below[n - 2], 1.0 - p, 0.0, 1.0 - 0.04 * p);
// …while the incoming slides up out of a fade.
env.paint(&mut top[0], p, 36.0 * k * (1.0 - p), 0.985 + 0.015 * p);
} else {
env.paint(
&mut self.stack[0],
p,
36.0 * k * (1.0 - p),
0.985 + 0.015 * p,
);
}
}
(Motion::Pop { leaving, .. }, Some(raw)) => {
let p = ease_out_cubic(raw);
// The revealed screen grows back in…
let n = self.stack.len();
env.paint(&mut self.stack[n - 1], 0.4 + 0.6 * p, 0.0, 0.96 + 0.04 * p);
// …while the leaving one slides down into a fade.
env.paint(leaving.as_mut(), 1.0 - p, 36.0 * k * p, 1.0);
}
_ => {
let n = self.stack.len();
env.paint(&mut self.stack[n - 1], 1.0, 0.0, 1.0);
}
}
// Persistent chrome: the controller chip (top-right, above every layer).
if let Some(chip) = &self.chip {
let size = 12.0 * k;
let tw = f64::from(fonts.measure(chip, W::Medium, size));
let (bh, pad_x) = (24.0 * k, 12.0 * k);
let bx = w - 24.0 * k - tw - 2.0 * pad_x;
let rect = Rect::from_xywh(
bx as f32,
(18.0 * k) as f32,
(tw + 2.0 * pad_x) as f32,
bh as f32,
);
crate::theme::panel(
canvas,
rect,
(bh / 2.0 / k) as f32,
None,
PanelStroke::Plain(0.12),
k as f32,
);
fonts.draw(
canvas,
chip,
bx + pad_x,
18.0 * k + 16.0 * k,
W::Medium,
size,
white(0.7),
);
}
self.draw_overlays(canvas, w, h, k, dt, t, fonts);
}
fn draw_aurora(&self, canvas: &Canvas, w: f64, h: f64, t: f64) {
let uniforms: [f32; 3] = [w as f32, h as f32, t as f32];
let bytes = unsafe { std::slice::from_raw_parts(uniforms.as_ptr().cast::<u8>(), 12) };
@@ -609,604 +439,7 @@ impl Shell {
}
}
}
// --- Overlays (connecting / waking / toast) --------------------------------------------
#[allow(clippy::too_many_arguments)]
fn draw_overlays(
&mut self,
canvas: &Canvas,
w: f64,
h: f64,
k: f64,
dt: f64,
t: f64,
fonts: &Fonts,
) {
// Resolve the connect/wake takeover — the two phases of reaching a host — into one
// full-screen shape (spinner, title, one detail line, its own hints). Connecting flows
// straight out of a wake (see `sync`) so they share the same backdrop and never blink
// between them. Mirrors the Android client's unified `ConnectOverlay`.
let takeover: Option<(f64, bool, String, String, Vec<Hint>)> =
if let Some(c) = &mut self.connecting {
c.appear = approach(c.appear, 1.0, dt, 0.07);
if c.canceling {
Some((
c.appear,
true,
"Canceling…".to_string(),
String::new(),
vec![],
))
} else if c.request_access {
Some((
c.appear,
true,
"Waiting for approval…".to_string(),
format!(
"Approve this device in {}'s console or web UI — no PIN needed.",
c.title
),
vec![Hint::new(HintKey::Back, "Cancel")],
))
} else {
Some((
c.appear,
true,
format!("Connecting to {}", c.title),
"Starting the stream in this window.".to_string(),
vec![Hint::new(HintKey::Back, "Cancel")],
))
}
} else if let Some(wk) = &self.wake {
// Service-driven, so it appears settled (no fade-in).
if wk.timed_out {
Some((
1.0,
false,
format!("{} didn't wake", wk.name),
"Check its power settings, or wake it manually and try again.".to_string(),
vec![
Hint::new(HintKey::Confirm, "Try Again"),
Hint::new(HintKey::Back, "Cancel"),
],
))
} else {
Some((
1.0,
true,
format!("Waking {}", wk.name),
format!("Waiting for it to come online · {} s", wk.seconds),
// A wake-only wait (no dial after) offers "Stop Waiting"; a wake-&-connect
// is a plain "Cancel".
vec![Hint::new(
HintKey::Back,
if wk.then_connect {
"Cancel"
} else {
"Stop Waiting"
},
)],
))
}
} else {
None
};
if let Some((appear, spinner, title, body, hints)) = takeover {
self.draw_takeover(
canvas, w, h, k, appear, t, fonts, spinner, &title, &body, &hints,
);
}
// The toast: a transient pill above the hint bar; slides in, fades out.
if self.toast.as_ref().is_some_and(|toast| t - toast.at > 4.0) {
self.toast = None;
}
if let Some(toast) = &self.toast {
let age = t - toast.at;
{
let slide = ease_out_cubic((age / 0.25).min(1.0));
let fade = if age > 3.4 {
(1.0 - (age - 3.4) / 0.6).max(0.0)
} else {
1.0
};
let alpha = (slide * fade) as f32;
let size = 13.0 * k;
let tw = f64::from(fonts.measure(&toast.text, W::Medium, size));
let (pad_x, bh) = (16.0 * k, 34.0 * k);
let bw = tw + 2.0 * pad_x;
let bx = (w - bw) / 2.0;
let by = h - BOTTOM_BAND * k - bh - 8.0 * k + (1.0 - slide) * 12.0 * k;
canvas.save_layer_alpha_f(None, alpha);
let rect = Rect::from_xywh(bx as f32, by as f32, bw as f32, bh as f32);
canvas.draw_rrect(
skia_safe::RRect::new_rect_xy(rect, (bh / 2.0) as f32, (bh / 2.0) as f32),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.6), None),
);
crate::theme::panel(
canvas,
rect,
(bh / 2.0 / k) as f32,
None,
PanelStroke::Plain(0.14),
k as f32,
);
fonts.draw(
canvas,
&toast.text,
bx + pad_x,
by + bh / 2.0 + size * 0.36,
W::Medium,
size,
white(0.92),
);
canvas.restore();
}
}
}
}
/// Everything one screen layer needs to paint — bundled so the transition arms stay
/// readable and each `paint` call borrows `Shell` fields disjointly.
struct LayerEnv<'a> {
canvas: &'a Canvas,
w: f64,
h: f64,
content: Rect,
k: f64,
dt: f64,
fonts: &'a Fonts,
hosts: &'a [HostRow],
library: &'a LibraryShared,
settings: &'a mut trust::Settings,
pads: &'a [PadInfo],
deck: bool,
device_name: &'a str,
t: f64,
glyphs: GlyphStyle,
show_hints: bool,
}
impl LayerEnv<'_> {
/// One screen composited as a unit: `alpha` fade, `dy` vertical slide, `scale`
/// about the screen center — its pinned title and hint bar ride inside the layer,
/// so chrome travels with content through a transition.
fn paint(&mut self, screen: &mut Screen, alpha: f64, dy: f64, scale: f64) {
let canvas = self.canvas;
canvas.save_layer_alpha_f(None, alpha.clamp(0.0, 1.0) as f32);
canvas.translate((0.0, dy as f32));
let (cx, cy) = ((self.w / 2.0) as f32, (self.h / 2.0) as f32);
canvas.translate((cx, cy));
canvas.scale((scale as f32, scale as f32));
canvas.translate((-cx, -cy));
let mut ctx = Ctx {
hosts: self.hosts,
library: self.library,
settings: self.settings,
pads: self.pads,
deck: self.deck,
device_name: self.device_name,
t: self.t,
};
self.fonts.centered(
canvas,
&screen.title(&ctx),
W::Bold,
30.0 * self.k,
WHITE,
self.w / 2.0,
18.0 * self.k,
self.w * 0.7,
);
screen.render(canvas, self.content, self.k, self.dt, self.fonts, &mut ctx);
if self.show_hints {
let hints = screen.hints(&ctx);
hint_bar(
canvas,
self.fonts,
&hints,
self.glyphs,
18.0 * self.k,
self.h - 18.0 * self.k,
self.k,
);
}
canvas.restore();
}
}
impl Shell {
/// A full-screen connect/wake takeover: a fresh aurora over everything (so the carousel and
/// chrome fall away), a centered spinner (or none, when a wake has timed out), a title, one
/// detail line, and its own bottom hint row. `appear` fades the whole thing in over the home;
/// a wake that hands off to a connect passes 1.0 so the two never blink between them. The
/// console counterpart of the Android/Apple `ConnectOverlay` — one full-screen shape, not a
/// centered modal card.
#[allow(clippy::too_many_arguments)]
fn draw_takeover(
&self,
canvas: &Canvas,
w: f64,
h: f64,
k: f64,
appear: f64,
t: f64,
fonts: &Fonts,
spinner: bool,
title: &str,
body: &str,
hints: &[Hint],
) {
let cx = w / 2.0;
canvas.save_layer_alpha_f(None, appear as f32);
// Opaque aurora — the same living backdrop the home wears, so the takeover reads as the
// console taking over rather than a card popping up.
self.draw_aurora(canvas, w, h, t);
// A soft pool of shade under the centre seats the white text against a bright aurora.
let mut vignette = Paint::default();
vignette.set_shader(gradient_shader::radial(
Point::new(cx as f32, (h / 2.0) as f32),
(w.max(h) * 0.42) as f32,
gradient_shader::GradientShaderColors::Colors(&[
Color4f::new(0.0, 0.0, 0.0, 0.5).to_color(),
Color4f::new(0.0, 0.0, 0.0, 0.0).to_color(),
]),
None,
TileMode::Clamp,
None,
None,
));
canvas.draw_rect(Rect::from_wh(w as f32, h as f32), &vignette);
// Centre the spinner + title + detail as a group around the middle of the screen.
let title_y = h / 2.0 + if spinner { 14.0 * k } else { 0.0 };
if spinner {
crate::theme::spinner(canvas, cx, title_y - 52.0 * k, 22.0 * k, t);
}
fonts.centered(
canvas,
title,
W::SemiBold,
23.0 * k,
WHITE,
cx,
title_y,
w * 0.82,
);
if !body.is_empty() {
fonts.centered(
canvas,
body,
W::Regular,
14.0 * k,
DIM,
cx,
title_y + 32.0 * k,
w * 0.66,
);
}
if !hints.is_empty() {
// Centered near the bottom, where every console screen's legend sits.
let probe = hint_bar(canvas, fonts, hints, self.glyphs, -10_000.0, -10_000.0, k);
hint_bar(
canvas,
fonts,
hints,
self.glyphs,
cx - probe.0 / 2.0,
h - 34.0 * k,
k,
);
}
canvas.restore();
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::model::WakeStatus;
use crate::screens::home::HomeScreen;
use crate::screens::library::LibraryScreen;
use punktfunk_core::config::GamepadPref;
/// Point the settings/known-hosts stores at a throwaway HOME — the settings screen
/// SAVES on adjust, and a test must never write the developer's real config.
fn fake_home() {
use std::sync::OnceLock;
static HOME: OnceLock<std::path::PathBuf> = OnceLock::new();
let dir = HOME.get_or_init(|| {
let dir = std::env::temp_dir().join(format!("pf-console-test-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
std::env::set_var("HOME", &dir);
dir.clone()
});
std::env::set_var("HOME", dir);
}
fn hosts() -> Vec<HostRow> {
let base = HostRow {
key: String::new(),
name: String::new(),
addr: "10.0.0.20".into(),
port: 9777,
fp_hex: String::new(),
paired: false,
saved: true,
online: false,
mgmt_port: 47990,
can_wake: false,
last_used: None,
};
vec![
HostRow {
key: "aa11".into(),
name: "Living Room PC".into(),
fp_hex: "aa11".into(),
paired: true,
online: true,
last_used: Some(1),
..base.clone()
},
HostRow {
key: "bb22".into(),
name: "Office Tower".into(),
addr: "10.0.0.21".into(),
fp_hex: "bb22".into(),
paired: true,
can_wake: true,
..base.clone()
},
HostRow {
key: "10.0.0.30:9777".into(),
name: "steambox".into(),
addr: "10.0.0.30".into(),
saved: false,
online: true,
..base
},
]
}
fn shell(stack: Vec<Screen>) -> (Shell, ConsoleShared, LibraryShared) {
fake_home();
let console = ConsoleShared::default();
console.set_hosts(hosts());
let library = LibraryShared::default();
let bus = ConsoleBus::default();
let shell = Shell::new(
console.clone(),
library.clone(),
bus,
ConsoleOptions {
device_name: "deck".into(),
deck: false,
},
stack,
)
.unwrap();
(shell, console, library)
}
/// The shell survives a full navigation lap (a smoke test over every screen's
/// input handling — no rendering, no GPU).
#[test]
fn navigation_lap() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
// Home → Settings (X), adjust something, back out.
s.handle_menu(MenuEvent::Tertiary);
assert_eq!(s.stack.len(), 2);
finish_motion(&mut s);
s.handle_menu(MenuEvent::Move(MenuDir::Down));
s.handle_menu(MenuEvent::Move(MenuDir::Right));
s.handle_menu(MenuEvent::Back);
finish_motion(&mut s);
assert_eq!(s.stack.len(), 1);
// Home → Library on the paired host (Y), then back.
s.handle_menu(MenuEvent::Secondary);
assert_eq!(s.stack.len(), 2);
finish_motion(&mut s);
s.handle_menu(MenuEvent::Back);
finish_motion(&mut s);
assert_eq!(s.stack.len(), 1);
// B at the root quits.
s.handle_menu(MenuEvent::Back);
assert!(matches!(s.take_action(), Some(OverlayAction::Quit)));
}
#[test]
fn connect_flow_raises_launch_and_cancel() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
s.handle_menu(MenuEvent::Confirm); // paired+online host focused first
assert!(matches!(
s.take_action(),
Some(OverlayAction::Launch { launch: None, .. })
));
assert!(s.connecting.is_some());
// While connecting: B cancels exactly once.
s.handle_menu(MenuEvent::Back);
assert!(matches!(
s.take_action(),
Some(OverlayAction::CancelConnect)
));
s.handle_menu(MenuEvent::Back);
assert!(s.take_action().is_none(), "cancel is idempotent");
// The canceled dial ends silently.
s.session_ended(None);
assert!(s.connecting.is_none());
}
fn finish_motion(s: &mut Shell) {
// Transitions block input; tests fast-forward them.
s.motion = Motion::None;
}
#[test]
fn wake_gates_input_in_the_same_press() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
// Focus "Office Tower" (offline + wakeable), then A: the wake starts.
s.handle_menu(MenuEvent::Move(MenuDir::Right));
s.handle_menu(MenuEvent::Confirm);
let w = s
.wake
.as_ref()
.expect("Waking card raised in the SAME call as the A press");
assert_eq!(w.name, "Office Tower");
assert!(!w.online);
// The very next input is modal-gated — the cursor can't drift onto Add Host —
// and sync (which runs first in handle_menu) must not clear the placeholder
// before the service thread reports its first real status.
assert!(s.handle_menu(MenuEvent::Move(MenuDir::Right)).is_none());
assert!(
s.wake.is_some(),
"optimistic card survived a sync with no service status"
);
// B cancels: the gate releases and navigation works again.
s.handle_menu(MenuEvent::Back);
assert!(s.wake.is_none());
assert!(s.handle_menu(MenuEvent::Move(MenuDir::Left)).is_some());
}
/// Render every console scene to PNGs for the eyeball pass (ignored; run with
/// `PF_CONSOLE_DUMP=<dir> cargo test -p pf-console-ui --release -- --ignored dump`).
/// CPU raster — the SkSL aurora, layers and text all run without a GPU.
#[test]
#[ignore]
fn dump_console_screens() {
let dir = std::env::var("PF_CONSOLE_DUMP").expect("set PF_CONSOLE_DUMP to an output dir");
let fonts = crate::theme::build_fonts().unwrap();
let (w, h) = (1280, 800);
let pads: Vec<PadInfo> = Vec::new();
let dump = |shell: &mut Shell, frames: usize, sleep_ms: u64, name: &str, pad: bool| {
let mut surface = skia_safe::surfaces::raster_n32_premul((w, h)).unwrap();
for _ in 0..frames {
shell.render(
surface.canvas(),
w as u32,
h as u32,
&fonts,
pad.then_some("Xbox Wireless Controller"),
pad.then_some(GamepadPref::Xbox360),
&pads,
);
std::thread::sleep(std::time::Duration::from_millis(sleep_ms));
}
let png = surface
.image_snapshot()
.encode(None, skia_safe::EncodedImageFormat::PNG, 100)
.unwrap();
std::fs::write(format!("{dir}/{name}.png"), png.as_bytes()).unwrap();
};
// Home, settled, with a pad (Letters glyphs).
let (mut s, console, library) = shell(vec![Screen::Home(HomeScreen::new())]);
dump(&mut s, 40, 8, "01-home", true);
// Mid-push into Settings (the transition still): a couple of fast frames land
// the capture around p ≈ 0.4 — both layers visible.
s.handle_menu(MenuEvent::Tertiary);
dump(&mut s, 3, 25, "02-transition", true);
dump(&mut s, 40, 8, "03-settings", true);
// Add Host with the keyboard tray up (keyboard glyph style: no pad).
s.handle_menu(MenuEvent::Back);
dump(&mut s, 40, 8, "_back", true);
for _ in 0..3 {
s.handle_menu(MenuEvent::Move(MenuDir::Right));
}
s.handle_menu(MenuEvent::Confirm); // Add Host screen
dump(&mut s, 40, 8, "04-addhost", false);
s.handle_menu(MenuEvent::Confirm); // open the Name keyboard
for ev in [
MenuEvent::Move(MenuDir::Down),
MenuEvent::Confirm,
MenuEvent::Confirm,
] {
s.handle_menu(ev);
}
dump(&mut s, 40, 8, "05-addhost-keyboard", false);
// Pair (focused on the unpaired discovered host).
s.handle_menu(MenuEvent::Back); // close keyboard
s.handle_menu(MenuEvent::Back); // leave add-host
dump(&mut s, 40, 8, "_back2", true);
s.handle_menu(MenuEvent::Move(MenuDir::Left)); // onto "steambox"
s.handle_menu(MenuEvent::Confirm);
dump(&mut s, 40, 8, "06-pair", true);
// Library with placeholder posters.
library.set_games(
[
"Hades II",
"Elden Ring",
"Hollow Knight",
"Baldur's Gate 3",
"Celeste",
"Deep Rock Galactic",
"Portal 2",
]
.iter()
.enumerate()
.map(|(i, t)| crate::library::LibraryGame {
id: format!("steam:{i}"),
title: (*t).to_string(),
store: "steam".into(),
})
.collect(),
);
let (mut s2, _c2, _l2) = {
let console2 = ConsoleShared::default();
console2.set_hosts(hosts());
let bus = ConsoleBus::default();
let sh = Shell::new(
console2.clone(),
library.clone(),
bus,
ConsoleOptions {
device_name: "deck".into(),
deck: false,
},
vec![
Screen::Home(HomeScreen::new()),
Screen::Library(LibraryScreen::new(&hosts()[0])),
],
)
.unwrap();
(sh, console2, library.clone())
};
s2.handle_menu(MenuEvent::Move(MenuDir::Right));
s2.handle_menu(MenuEvent::Move(MenuDir::Right));
dump(&mut s2, 40, 8, "07-library", true);
// The wake and connecting overlays + a toast.
console.set_wake(Some(WakeStatus {
key: "bb22".into(),
name: "Office Tower".into(),
seconds: 12,
timed_out: false,
online: false,
then_connect: true,
}));
dump(&mut s, 10, 8, "08-waking", true);
console.set_wake(Some(WakeStatus {
key: "bb22".into(),
name: "Office Tower".into(),
seconds: 90,
timed_out: true,
online: false,
then_connect: true,
}));
dump(&mut s, 10, 8, "08b-wake-timed-out", true);
console.set_wake(None);
s.set_connecting(Some("Elden Ring".into()));
dump(&mut s, 10, 8, "09-connecting", true);
s.set_connecting(None);
s.session_failed("Connection timed out");
dump(&mut s, 10, 8, "10-toast", true);
}
}
mod tests;
+228
View File
@@ -0,0 +1,228 @@
//! The console shell's modal overlays (connecting / waking / toast / full-screen takeover).
use crate::anim::{approach, ease_out_cubic};
use crate::glyphs::{hint_bar, Hint, HintKey};
use crate::theme::{white, Fonts, PanelStroke, DIM, W, WHITE};
use skia_safe::{gradient_shader, Canvas, Color4f, Paint, Point, Rect, TileMode};
use super::{Shell, BOTTOM_BAND};
impl Shell {
#[allow(clippy::too_many_arguments)]
pub(in crate::shell) fn draw_overlays(
&mut self,
canvas: &Canvas,
w: f64,
h: f64,
k: f64,
dt: f64,
t: f64,
fonts: &Fonts,
) {
// Resolve the connect/wake takeover — the two phases of reaching a host — into one
// full-screen shape (spinner, title, one detail line, its own hints). Connecting flows
// straight out of a wake (see `sync`) so they share the same backdrop and never blink
// between them. Mirrors the Android client's unified `ConnectOverlay`.
let takeover: Option<(f64, bool, String, String, Vec<Hint>)> =
if let Some(c) = &mut self.connecting {
c.appear = approach(c.appear, 1.0, dt, 0.07);
if c.canceling {
Some((
c.appear,
true,
"Canceling…".to_string(),
String::new(),
vec![],
))
} else if c.request_access {
Some((
c.appear,
true,
"Waiting for approval…".to_string(),
format!(
"Approve this device in {}'s console or web UI — no PIN needed.",
c.title
),
vec![Hint::new(HintKey::Back, "Cancel")],
))
} else {
Some((
c.appear,
true,
format!("Connecting to {}", c.title),
"Starting the stream in this window.".to_string(),
vec![Hint::new(HintKey::Back, "Cancel")],
))
}
} else if let Some(wk) = &self.wake {
// Service-driven, so it appears settled (no fade-in).
if wk.timed_out {
Some((
1.0,
false,
format!("{} didn't wake", wk.name),
"Check its power settings, or wake it manually and try again.".to_string(),
vec![
Hint::new(HintKey::Confirm, "Try Again"),
Hint::new(HintKey::Back, "Cancel"),
],
))
} else {
Some((
1.0,
true,
format!("Waking {}", wk.name),
format!("Waiting for it to come online · {} s", wk.seconds),
// A wake-only wait (no dial after) offers "Stop Waiting"; a wake-&-connect
// is a plain "Cancel".
vec![Hint::new(
HintKey::Back,
if wk.then_connect {
"Cancel"
} else {
"Stop Waiting"
},
)],
))
}
} else {
None
};
if let Some((appear, spinner, title, body, hints)) = takeover {
self.draw_takeover(
canvas, w, h, k, appear, t, fonts, spinner, &title, &body, &hints,
);
}
// The toast: a transient pill above the hint bar; slides in, fades out.
if self.toast.as_ref().is_some_and(|toast| t - toast.at > 4.0) {
self.toast = None;
}
if let Some(toast) = &self.toast {
let age = t - toast.at;
{
let slide = ease_out_cubic((age / 0.25).min(1.0));
let fade = if age > 3.4 {
(1.0 - (age - 3.4) / 0.6).max(0.0)
} else {
1.0
};
let alpha = (slide * fade) as f32;
let size = 13.0 * k;
let tw = f64::from(fonts.measure(&toast.text, W::Medium, size));
let (pad_x, bh) = (16.0 * k, 34.0 * k);
let bw = tw + 2.0 * pad_x;
let bx = (w - bw) / 2.0;
let by = h - BOTTOM_BAND * k - bh - 8.0 * k + (1.0 - slide) * 12.0 * k;
canvas.save_layer_alpha_f(None, alpha);
let rect = Rect::from_xywh(bx as f32, by as f32, bw as f32, bh as f32);
canvas.draw_rrect(
skia_safe::RRect::new_rect_xy(rect, (bh / 2.0) as f32, (bh / 2.0) as f32),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.6), None),
);
crate::theme::panel(
canvas,
rect,
(bh / 2.0 / k) as f32,
None,
PanelStroke::Plain(0.14),
k as f32,
);
fonts.draw(
canvas,
&toast.text,
bx + pad_x,
by + bh / 2.0 + size * 0.36,
W::Medium,
size,
white(0.92),
);
canvas.restore();
}
}
}
/// A full-screen connect/wake takeover: a fresh aurora over everything (so the carousel and
/// chrome fall away), a centered spinner (or none, when a wake has timed out), a title, one
/// detail line, and its own bottom hint row. `appear` fades the whole thing in over the home;
/// a wake that hands off to a connect passes 1.0 so the two never blink between them. The
/// console counterpart of the Android/Apple `ConnectOverlay` — one full-screen shape, not a
/// centered modal card.
#[allow(clippy::too_many_arguments)]
fn draw_takeover(
&self,
canvas: &Canvas,
w: f64,
h: f64,
k: f64,
appear: f64,
t: f64,
fonts: &Fonts,
spinner: bool,
title: &str,
body: &str,
hints: &[Hint],
) {
let cx = w / 2.0;
canvas.save_layer_alpha_f(None, appear as f32);
// Opaque aurora — the same living backdrop the home wears, so the takeover reads as the
// console taking over rather than a card popping up.
self.draw_aurora(canvas, w, h, t);
// A soft pool of shade under the centre seats the white text against a bright aurora.
let mut vignette = Paint::default();
vignette.set_shader(gradient_shader::radial(
Point::new(cx as f32, (h / 2.0) as f32),
(w.max(h) * 0.42) as f32,
gradient_shader::GradientShaderColors::Colors(&[
Color4f::new(0.0, 0.0, 0.0, 0.5).to_color(),
Color4f::new(0.0, 0.0, 0.0, 0.0).to_color(),
]),
None,
TileMode::Clamp,
None,
None,
));
canvas.draw_rect(Rect::from_wh(w as f32, h as f32), &vignette);
// Centre the spinner + title + detail as a group around the middle of the screen.
let title_y = h / 2.0 + if spinner { 14.0 * k } else { 0.0 };
if spinner {
crate::theme::spinner(canvas, cx, title_y - 52.0 * k, 22.0 * k, t);
}
fonts.centered(
canvas,
title,
W::SemiBold,
23.0 * k,
WHITE,
cx,
title_y,
w * 0.82,
);
if !body.is_empty() {
fonts.centered(
canvas,
body,
W::Regular,
14.0 * k,
DIM,
cx,
title_y + 32.0 * k,
w * 0.66,
);
}
if !hints.is_empty() {
// Centered near the bottom, where every console screen's legend sits.
let probe = hint_bar(canvas, fonts, hints, self.glyphs, -10_000.0, -10_000.0, k);
hint_bar(
canvas,
fonts,
hints,
self.glyphs,
cx - probe.0 / 2.0,
h - 34.0 * k,
k,
);
}
canvas.restore();
}
}
+254
View File
@@ -0,0 +1,254 @@
//! The console shell's per-frame screen compose/transition render path.
use crate::anim::{approach, ease_out_cubic};
use crate::glyphs::{hint_bar, GlyphStyle};
use crate::library::LibraryShared;
use crate::model::HostRow;
use crate::screens::{Bg, Ctx, Screen};
use crate::theme::{white, Fonts, PanelStroke, W, WHITE};
use pf_client_core::gamepad::PadInfo;
use pf_client_core::trust;
use skia_safe::{Canvas, Color4f, Rect};
use std::time::Instant;
use super::{Motion, Shell, BOTTOM_BAND, TOP_BAND};
impl Shell {
#[allow(clippy::too_many_arguments)]
pub(crate) fn render(
&mut self,
canvas: &Canvas,
width: u32,
height: u32,
fonts: &Fonts,
pad: Option<&str>,
pad_pref: Option<punktfunk_core::config::GamepadPref>,
pads: &[PadInfo],
) {
let now = Instant::now();
let dt = self
.last_frame
.replace(now)
.map_or(1.0 / 60.0, |t| (now - t).as_secs_f64().clamp(0.0, 0.05));
self.sync();
self.pads = pads.to_vec();
self.glyphs = GlyphStyle::from_pref(pad_pref);
self.chip = Some(pad.map_or_else(
|| "No controller — keyboard works too".to_string(),
str::to_owned,
));
let (w, h) = (f64::from(width), f64::from(height));
let k = (h / 800.0).clamp(0.75, 3.0);
let t = self.t();
// Advance the transition; a finished pop finally drops its leaving screen.
let motion_p = match &mut self.motion {
Motion::None => None,
Motion::Push(p) => {
p.advance(dt);
let v = p.value();
if p.done() {
self.motion = Motion::None;
None
} else {
Some(v)
}
}
Motion::Pop { t, .. } => {
t.advance(dt);
let v = t.value();
if t.done() {
self.motion = Motion::None;
None
} else {
Some(v)
}
}
};
// Backdrop crossfade follows the top screen.
let bg_target = match self.stack.last().expect("non-empty").background() {
Bg::Aurora => 0.0,
Bg::Form => 1.0,
};
self.bg_mix = approach(self.bg_mix, bg_target, dt, 0.12);
if (self.bg_mix - bg_target).abs() < 0.005 {
self.bg_mix = bg_target;
}
if self.bg_mix < 1.0 {
self.draw_aurora(canvas, w, h, t);
} else {
canvas.clear(Color4f::new(0.0, 0.0, 0.0, 1.0));
}
if self.bg_mix > 0.0 {
canvas.save_layer_alpha_f(None, self.bg_mix as f32);
crate::theme::draw_form_background(canvas, w, h);
canvas.restore();
}
// The screens, through the transition choreography.
let content = Rect::from_ltrb(
0.0,
(TOP_BAND * k) as f32,
w as f32,
(h - BOTTOM_BAND * k) as f32,
);
// One paint recipe per layer: (alpha, slide, scale). Everything below borrows
// disjoint fields of `self` per call, so the borrow checker stays happy.
let mut env = LayerEnv {
canvas,
w,
h,
content,
k,
dt,
fonts,
hosts: &self.hosts,
library: &self.library,
settings: &mut self.settings,
pads: &self.pads,
deck: self.deck,
device_name: &self.device_name,
t,
glyphs: self.glyphs,
// A modal card owns B/A while it's up — the screen's legend would lie.
show_hints: self.connecting.is_none() && self.wake.is_none(),
};
match (&mut self.motion, motion_p) {
(Motion::Push(_), Some(raw)) => {
let p = ease_out_cubic(raw);
let n = self.stack.len();
// Outgoing recedes underneath…
if n >= 2 {
let (below, top) = self.stack.split_at_mut(n - 1);
env.paint(&mut below[n - 2], 1.0 - p, 0.0, 1.0 - 0.04 * p);
// …while the incoming slides up out of a fade.
env.paint(&mut top[0], p, 36.0 * k * (1.0 - p), 0.985 + 0.015 * p);
} else {
env.paint(
&mut self.stack[0],
p,
36.0 * k * (1.0 - p),
0.985 + 0.015 * p,
);
}
}
(Motion::Pop { leaving, .. }, Some(raw)) => {
let p = ease_out_cubic(raw);
// The revealed screen grows back in…
let n = self.stack.len();
env.paint(&mut self.stack[n - 1], 0.4 + 0.6 * p, 0.0, 0.96 + 0.04 * p);
// …while the leaving one slides down into a fade.
env.paint(leaving.as_mut(), 1.0 - p, 36.0 * k * p, 1.0);
}
_ => {
let n = self.stack.len();
env.paint(&mut self.stack[n - 1], 1.0, 0.0, 1.0);
}
}
// Persistent chrome: the controller chip (top-right, above every layer).
if let Some(chip) = &self.chip {
let size = 12.0 * k;
let tw = f64::from(fonts.measure(chip, W::Medium, size));
let (bh, pad_x) = (24.0 * k, 12.0 * k);
let bx = w - 24.0 * k - tw - 2.0 * pad_x;
let rect = Rect::from_xywh(
bx as f32,
(18.0 * k) as f32,
(tw + 2.0 * pad_x) as f32,
bh as f32,
);
crate::theme::panel(
canvas,
rect,
(bh / 2.0 / k) as f32,
None,
PanelStroke::Plain(0.12),
k as f32,
);
fonts.draw(
canvas,
chip,
bx + pad_x,
18.0 * k + 16.0 * k,
W::Medium,
size,
white(0.7),
);
}
self.draw_overlays(canvas, w, h, k, dt, t, fonts);
}
}
/// Everything one screen layer needs to paint — bundled so the transition arms stay
/// readable and each `paint` call borrows `Shell` fields disjointly.
struct LayerEnv<'a> {
canvas: &'a Canvas,
w: f64,
h: f64,
content: Rect,
k: f64,
dt: f64,
fonts: &'a Fonts,
hosts: &'a [HostRow],
library: &'a LibraryShared,
settings: &'a mut trust::Settings,
pads: &'a [PadInfo],
deck: bool,
device_name: &'a str,
t: f64,
glyphs: GlyphStyle,
show_hints: bool,
}
impl LayerEnv<'_> {
/// One screen composited as a unit: `alpha` fade, `dy` vertical slide, `scale`
/// about the screen center — its pinned title and hint bar ride inside the layer,
/// so chrome travels with content through a transition.
fn paint(&mut self, screen: &mut Screen, alpha: f64, dy: f64, scale: f64) {
let canvas = self.canvas;
canvas.save_layer_alpha_f(None, alpha.clamp(0.0, 1.0) as f32);
canvas.translate((0.0, dy as f32));
let (cx, cy) = ((self.w / 2.0) as f32, (self.h / 2.0) as f32);
canvas.translate((cx, cy));
canvas.scale((scale as f32, scale as f32));
canvas.translate((-cx, -cy));
let mut ctx = Ctx {
hosts: self.hosts,
library: self.library,
settings: self.settings,
pads: self.pads,
deck: self.deck,
device_name: self.device_name,
t: self.t,
};
self.fonts.centered(
canvas,
&screen.title(&ctx),
W::Bold,
30.0 * self.k,
WHITE,
self.w / 2.0,
18.0 * self.k,
self.w * 0.7,
);
screen.render(canvas, self.content, self.k, self.dt, self.fonts, &mut ctx);
if self.show_hints {
let hints = screen.hints(&ctx);
hint_bar(
canvas,
self.fonts,
&hints,
self.glyphs,
18.0 * self.k,
self.h - 18.0 * self.k,
self.k,
);
}
canvas.restore();
}
}
+303
View File
@@ -0,0 +1,303 @@
use super::*;
use crate::model::WakeStatus;
use crate::screens::home::HomeScreen;
use crate::screens::library::LibraryScreen;
use punktfunk_core::config::GamepadPref;
/// Point the settings/known-hosts stores at a throwaway HOME — the settings screen
/// SAVES on adjust, and a test must never write the developer's real config.
fn fake_home() {
use std::sync::OnceLock;
static HOME: OnceLock<std::path::PathBuf> = OnceLock::new();
let dir = HOME.get_or_init(|| {
let dir = std::env::temp_dir().join(format!("pf-console-test-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
std::env::set_var("HOME", &dir);
dir.clone()
});
std::env::set_var("HOME", dir);
}
fn hosts() -> Vec<HostRow> {
let base = HostRow {
key: String::new(),
name: String::new(),
addr: "10.0.0.20".into(),
port: 9777,
fp_hex: String::new(),
paired: false,
saved: true,
online: false,
mgmt_port: 47990,
can_wake: false,
last_used: None,
};
vec![
HostRow {
key: "aa11".into(),
name: "Living Room PC".into(),
fp_hex: "aa11".into(),
paired: true,
online: true,
last_used: Some(1),
..base.clone()
},
HostRow {
key: "bb22".into(),
name: "Office Tower".into(),
addr: "10.0.0.21".into(),
fp_hex: "bb22".into(),
paired: true,
can_wake: true,
..base.clone()
},
HostRow {
key: "10.0.0.30:9777".into(),
name: "steambox".into(),
addr: "10.0.0.30".into(),
saved: false,
online: true,
..base
},
]
}
fn shell(stack: Vec<Screen>) -> (Shell, ConsoleShared, LibraryShared) {
fake_home();
let console = ConsoleShared::default();
console.set_hosts(hosts());
let library = LibraryShared::default();
let bus = ConsoleBus::default();
let shell = Shell::new(
console.clone(),
library.clone(),
bus,
ConsoleOptions {
device_name: "deck".into(),
deck: false,
},
stack,
)
.unwrap();
(shell, console, library)
}
/// The shell survives a full navigation lap (a smoke test over every screen's
/// input handling — no rendering, no GPU).
#[test]
fn navigation_lap() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
// Home → Settings (X), adjust something, back out.
s.handle_menu(MenuEvent::Tertiary);
assert_eq!(s.stack.len(), 2);
finish_motion(&mut s);
s.handle_menu(MenuEvent::Move(MenuDir::Down));
s.handle_menu(MenuEvent::Move(MenuDir::Right));
s.handle_menu(MenuEvent::Back);
finish_motion(&mut s);
assert_eq!(s.stack.len(), 1);
// Home → Library on the paired host (Y), then back.
s.handle_menu(MenuEvent::Secondary);
assert_eq!(s.stack.len(), 2);
finish_motion(&mut s);
s.handle_menu(MenuEvent::Back);
finish_motion(&mut s);
assert_eq!(s.stack.len(), 1);
// B at the root quits.
s.handle_menu(MenuEvent::Back);
assert!(matches!(s.take_action(), Some(OverlayAction::Quit)));
}
#[test]
fn connect_flow_raises_launch_and_cancel() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
s.handle_menu(MenuEvent::Confirm); // paired+online host focused first
assert!(matches!(
s.take_action(),
Some(OverlayAction::Launch { launch: None, .. })
));
assert!(s.connecting.is_some());
// While connecting: B cancels exactly once.
s.handle_menu(MenuEvent::Back);
assert!(matches!(
s.take_action(),
Some(OverlayAction::CancelConnect)
));
s.handle_menu(MenuEvent::Back);
assert!(s.take_action().is_none(), "cancel is idempotent");
// The canceled dial ends silently.
s.session_ended(None);
assert!(s.connecting.is_none());
}
fn finish_motion(s: &mut Shell) {
// Transitions block input; tests fast-forward them.
s.motion = Motion::None;
}
#[test]
fn wake_gates_input_in_the_same_press() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
// Focus "Office Tower" (offline + wakeable), then A: the wake starts.
s.handle_menu(MenuEvent::Move(MenuDir::Right));
s.handle_menu(MenuEvent::Confirm);
let w = s
.wake
.as_ref()
.expect("Waking card raised in the SAME call as the A press");
assert_eq!(w.name, "Office Tower");
assert!(!w.online);
// The very next input is modal-gated — the cursor can't drift onto Add Host —
// and sync (which runs first in handle_menu) must not clear the placeholder
// before the service thread reports its first real status.
assert!(s.handle_menu(MenuEvent::Move(MenuDir::Right)).is_none());
assert!(
s.wake.is_some(),
"optimistic card survived a sync with no service status"
);
// B cancels: the gate releases and navigation works again.
s.handle_menu(MenuEvent::Back);
assert!(s.wake.is_none());
assert!(s.handle_menu(MenuEvent::Move(MenuDir::Left)).is_some());
}
/// Render every console scene to PNGs for the eyeball pass (ignored; run with
/// `PF_CONSOLE_DUMP=<dir> cargo test -p pf-console-ui --release -- --ignored dump`).
/// CPU raster — the SkSL aurora, layers and text all run without a GPU.
#[test]
#[ignore]
fn dump_console_screens() {
let dir = std::env::var("PF_CONSOLE_DUMP").expect("set PF_CONSOLE_DUMP to an output dir");
let fonts = crate::theme::build_fonts().unwrap();
let (w, h) = (1280, 800);
let pads: Vec<PadInfo> = Vec::new();
let dump = |shell: &mut Shell, frames: usize, sleep_ms: u64, name: &str, pad: bool| {
let mut surface = skia_safe::surfaces::raster_n32_premul((w, h)).unwrap();
for _ in 0..frames {
shell.render(
surface.canvas(),
w as u32,
h as u32,
&fonts,
pad.then_some("Xbox Wireless Controller"),
pad.then_some(GamepadPref::Xbox360),
&pads,
);
std::thread::sleep(std::time::Duration::from_millis(sleep_ms));
}
let png = surface
.image_snapshot()
.encode(None, skia_safe::EncodedImageFormat::PNG, 100)
.unwrap();
std::fs::write(format!("{dir}/{name}.png"), png.as_bytes()).unwrap();
};
// Home, settled, with a pad (Letters glyphs).
let (mut s, console, library) = shell(vec![Screen::Home(HomeScreen::new())]);
dump(&mut s, 40, 8, "01-home", true);
// Mid-push into Settings (the transition still): a couple of fast frames land
// the capture around p ≈ 0.4 — both layers visible.
s.handle_menu(MenuEvent::Tertiary);
dump(&mut s, 3, 25, "02-transition", true);
dump(&mut s, 40, 8, "03-settings", true);
// Add Host with the keyboard tray up (keyboard glyph style: no pad).
s.handle_menu(MenuEvent::Back);
dump(&mut s, 40, 8, "_back", true);
for _ in 0..3 {
s.handle_menu(MenuEvent::Move(MenuDir::Right));
}
s.handle_menu(MenuEvent::Confirm); // Add Host screen
dump(&mut s, 40, 8, "04-addhost", false);
s.handle_menu(MenuEvent::Confirm); // open the Name keyboard
for ev in [
MenuEvent::Move(MenuDir::Down),
MenuEvent::Confirm,
MenuEvent::Confirm,
] {
s.handle_menu(ev);
}
dump(&mut s, 40, 8, "05-addhost-keyboard", false);
// Pair (focused on the unpaired discovered host).
s.handle_menu(MenuEvent::Back); // close keyboard
s.handle_menu(MenuEvent::Back); // leave add-host
dump(&mut s, 40, 8, "_back2", true);
s.handle_menu(MenuEvent::Move(MenuDir::Left)); // onto "steambox"
s.handle_menu(MenuEvent::Confirm);
dump(&mut s, 40, 8, "06-pair", true);
// Library with placeholder posters.
library.set_games(
[
"Hades II",
"Elden Ring",
"Hollow Knight",
"Baldur's Gate 3",
"Celeste",
"Deep Rock Galactic",
"Portal 2",
]
.iter()
.enumerate()
.map(|(i, t)| crate::library::LibraryGame {
id: format!("steam:{i}"),
title: (*t).to_string(),
store: "steam".into(),
})
.collect(),
);
let (mut s2, _c2, _l2) = {
let console2 = ConsoleShared::default();
console2.set_hosts(hosts());
let bus = ConsoleBus::default();
let sh = Shell::new(
console2.clone(),
library.clone(),
bus,
ConsoleOptions {
device_name: "deck".into(),
deck: false,
},
vec![
Screen::Home(HomeScreen::new()),
Screen::Library(LibraryScreen::new(&hosts()[0])),
],
)
.unwrap();
(sh, console2, library.clone())
};
s2.handle_menu(MenuEvent::Move(MenuDir::Right));
s2.handle_menu(MenuEvent::Move(MenuDir::Right));
dump(&mut s2, 40, 8, "07-library", true);
// The wake and connecting overlays + a toast.
console.set_wake(Some(WakeStatus {
key: "bb22".into(),
name: "Office Tower".into(),
seconds: 12,
timed_out: false,
online: false,
then_connect: true,
}));
dump(&mut s, 10, 8, "08-waking", true);
console.set_wake(Some(WakeStatus {
key: "bb22".into(),
name: "Office Tower".into(),
seconds: 90,
timed_out: true,
online: false,
then_connect: true,
}));
dump(&mut s, 10, 8, "08b-wake-timed-out", true);
console.set_wake(None);
s.set_connecting(Some("Elden Ring".into()));
dump(&mut s, 10, 8, "09-connecting", true);
s.set_connecting(None);
s.session_failed("Connection timed out");
dump(&mut s, 10, 8, "10-toast", true);
}
+304 -7
View File
@@ -59,7 +59,19 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// attach a ring naming a different monitor ([`frame::DRV_STATUS_BIND_FAIL`], the gamepad channel's
/// `pad_index` validation applied to frames). A v2 host never stamps the field, so a v3 driver
/// against a v2 host would refuse every attach — lockstep by the handshake, as ever.
pub const PROTOCOL_VERSION: u32 = 3;
/// v4: ADDITIVE — [`control::IOCTL_UPDATE_MODES`] (the in-place mid-stream resize,
/// `design/first-frame-and-resize-latency.md` P2): the driver refreshes a LIVE monitor's advertised
/// target-mode list (`IddCxMonitorUpdateModes2`) so the OS can mode-set to an arbitrary new mode
/// without a REMOVE→ADD monitor hotplug. Nothing existing changed, so the host accepts a v3 driver
/// too ([`MIN_DRIVER_PROTOCOL_VERSION`]) and simply falls back to the re-arrival resize against it;
/// a v4 driver serving an older (v3-asserting) host fails that host's strict handshake — ship
/// driver+host together, as ever.
pub const PROTOCOL_VERSION: u32 = 4;
/// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on
/// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the
/// handshake-reported version and covers with the re-arrival fallback.
pub const MIN_DRIVER_PROTOCOL_VERSION: u32 = 3;
/// `CTL_CODE(FILE_DEVICE_UNKNOWN = 0x22, func, METHOD_BUFFERED = 0, FILE_ANY_ACCESS = 0)`.
pub const fn ctl_code(func: u32) -> u32 {
@@ -91,6 +103,13 @@ pub mod control {
/// host duplicated into the driver's WUDFHost process. Input [`SetFrameChannelRequest`]. Sent once
/// after the ring is created and again on every mid-session ring recreate (HDR-mode flip).
pub const IOCTL_SET_FRAME_CHANNEL: u32 = ctl_code(0x906);
/// Refresh a LIVE monitor's advertised target-mode list to a new preferred mode (+ the built-in
/// fallbacks) via `IddCxMonitorUpdateModes2` — the in-place mid-stream resize (v4,
/// `design/first-frame-and-resize-latency.md` P2). Input [`UpdateModesRequest`]. The host then
/// CCD-forces the new mode active on the SAME monitor: no REMOVE→ADD hotplug, the monitor's OS
/// identity (saved per-monitor DPI) and the driver's swap-chain/stash machinery survive. A v3
/// driver fails this unknown IOCTL → the host falls back to the re-arrival resize.
pub const IOCTL_UPDATE_MODES: u32 = ctl_code(0x907);
/// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns
/// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this
@@ -164,6 +183,22 @@ pub mod control {
pub session_id: u64,
}
/// `IOCTL_UPDATE_MODES` input (v4): the live monitor (by its ADD `session_id`) and the new
/// preferred mode its target-mode list should lead with. The driver replaces the stored list
/// (new mode first, then its built-in fallbacks — the same shape ADD produces) and pushes it to
/// the OS via `IddCxMonitorUpdateModes2`; success means the OS accepted the new list, after
/// which the host force-sets the mode via CCD/GDI as usual.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct UpdateModesRequest {
pub session_id: u64,
pub width: u32,
pub height: u32,
pub refresh_hz: u32,
/// Pads the `u64`-aligned struct to a multiple of 8 (Pod forbids implicit tail padding).
pub _reserved: u32,
}
/// `IOCTL_SET_RENDER_ADAPTER` input (the GPU the IddCx swap-chain should render on).
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
@@ -253,6 +288,12 @@ pub mod control {
assert!(size_of::<RemoveRequest>() == 8);
assert!(offset_of!(RemoveRequest, session_id) == 0);
assert!(size_of::<UpdateModesRequest>() == 24);
assert!(offset_of!(UpdateModesRequest, session_id) == 0);
assert!(offset_of!(UpdateModesRequest, width) == 8);
assert!(offset_of!(UpdateModesRequest, height) == 12);
assert!(offset_of!(UpdateModesRequest, refresh_hz) == 16);
assert!(size_of::<SetRenderAdapterRequest>() == 8);
assert!(offset_of!(SetRenderAdapterRequest, luid_low) == 0);
assert!(offset_of!(SetRenderAdapterRequest, luid_high) == 4);
@@ -379,6 +420,46 @@ pub mod frame {
/// `design/idd-push-security.md`); `driver_status_detail` carries the target id the ring claims.
pub const DRV_STATUS_BIND_FAIL: u32 = 4;
/// While `driver_status` is [`DRV_STATUS_OPENED`], `driver_status_detail` carries a LIVE
/// diagnostic word maintained by the driver's publisher (best-effort plain writes — the same
/// visibility contract as `driver_status` itself). Layout:
///
/// - bit 31 (this constant): stamped at attach by a detail-capable driver, so the host can
/// tell "pre-detail driver, no information" (field = 0) from "zero frames offered".
/// - bits 30..16: surfaces the swap-chain worker OFFERED to the ring (15-bit, saturating) —
/// every DWM compose that reached `publish()`, whatever its outcome.
/// - bits 15..0: publishes DROPPED for a descriptor mismatch (16-bit, saturating) — the
/// surface's size/format didn't match the ring's.
///
/// The host's wait-for-attach reads this on its first-frame timeout to NAME the failure
/// instead of guessing (the lid-closed field report was undiagnosable without it):
/// `offered == 0` → DWM never composed the display (powered-off / undamaged desktop, compose
/// kicks blocked); `offered > 0` with `seq` still 0 → every compose was dropped mismatched
/// (the host sized the ring from a stale or foreign-session GDI mode).
pub const OPENED_DETAIL_LIVE: u32 = 0x8000_0000;
/// Pack the live OPENED diagnostic word (see [`OPENED_DETAIL_LIVE`]); both counters saturate.
#[must_use]
pub const fn pack_opened_detail(offered: u32, mismatched: u32) -> u32 {
let o = if offered > 0x7FFF { 0x7FFF } else { offered };
let m = if mismatched > 0xFFFF {
0xFFFF
} else {
mismatched
};
OPENED_DETAIL_LIVE | (o << 16) | m
}
/// Unpack a live OPENED diagnostic word → `(offered, mismatched)`; `None` when the driver
/// never stamped [`OPENED_DETAIL_LIVE`] (a pre-detail driver — the field carries nothing).
#[must_use]
pub const fn unpack_opened_detail(detail: u32) -> Option<(u32, u32)> {
if detail & OPENED_DETAIL_LIVE == 0 {
return None;
}
Some(((detail >> 16) & 0x7FFF, detail & 0xFFFF))
}
/// The shared metadata header (host-created, mapped by both sides). Atomic fields (`magic`, `latest`,
/// `generation`) are accessed via each side's own atomic view over the mapping; this is the layout.
#[repr(C)]
@@ -650,10 +731,46 @@ pub mod gamepad {
pub _reserved1: [u8; 20],
}
/// Virtual DualSense / DualShock 4 shared section (256 B). The host writes the `0x01`-style HID
/// input report into `input`; the driver feeds it to game `READ_REPORT`s and publishes a game's
/// `0x02` output (rumble / lightbar / player-LEDs / adaptive triggers) into `output`, bumping
/// `out_seq`. `device_type` selects the HID identity ([`DEVTYPE_DUALSENSE`] / [`DEVTYPE_DUALSHOCK4`]).
/// The legacy (pre-ring) [`PadShm`] size. Old binaries on either side were built against a
/// 256-byte layout; every field they know sits below this offset, and the ring extension keeps
/// bytes `0..256` byte-identical. Pagefile-backed sections are page-granular, so a view of
/// either generation's size maps against either generation's section — but a driver must
/// still be able to fall back to mapping this size if the full-size map is ever refused
/// (see `pf_umdf_util::ChannelConfig::min_data_size`).
pub const PAD_SHM_LEGACY_SIZE: usize = 256;
/// Output-report ring depth. 8 slots at the host's ~4 ms poll tolerates a sustained 2 kHz
/// writer — double any real HID output rate.
pub const OUT_RING_LEN: u32 = 8;
pub const OUT_RING_LEN_USIZE: usize = OUT_RING_LEN as usize;
/// One slot of the lossless output-report ring: the report bytes as the game wrote them
/// (report id first), with the exact length — unlike the legacy latest-report slot, whose
/// fixed 64-byte copy can carry a stale tail from a previous longer report.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug)]
pub struct OutSlot {
/// Valid bytes in `data` (0..=64). `0` = never written.
pub len: u32,
pub data: [u8; 64],
}
/// Virtual DualSense / DualShock 4 shared section (1024 B; bytes `0..256` are the v2 legacy
/// layout verbatim — [`PAD_SHM_LEGACY_SIZE`]). The host writes the `0x01`-style HID input
/// report into `input`; the driver feeds it to game `READ_REPORT`s and publishes a game's
/// `0x02` output (rumble / lightbar / player-LEDs / adaptive triggers) twice: into the legacy
/// latest-report `output` slot (bumping `out_seq` — every host generation reads this), and,
/// when the host stamped `out_ring_ver`, into the lossless `out_ring` (bumping `ring_head`).
/// The ring exists because the single slot COALESCES: a rumble-stop report overwritten by an
/// LED/trigger report inside one host poll window was gone forever — the confirmed stuck-rumble
/// path (`design/rumble-root-fix.md` §A). `device_type` selects the HID identity
/// ([`DEVTYPE_DUALSENSE`] / [`DEVTYPE_DUALSHOCK4`]).
///
/// Version posture: this is a TAIL extension negotiated by zeroed-reserved capability fields,
/// deliberately NOT a [`GAMEPAD_PROTO_VERSION`] bump — the bootstrap fails CLOSED on a version
/// mismatch (no pad at all), which is the wrong failure mode for a feedback-quality fix. An
/// old driver never reads the new fields; an old host never stamps `out_ring_ver`, so a new
/// driver stays on the legacy slot against it.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug)]
pub struct PadShm {
@@ -678,7 +795,20 @@ pub mod gamepad {
/// The pad index this section serves (host-stamped before the magic) — see
/// [`XusbShm::pad_index`]. Carved from v1 reserved space (v2).
pub pad_index: u32,
pub _reserved1: [u8; 100],
/// Host-stamped `1` at section creation ⇔ "this section carries the `out_ring` region and
/// the host drains it". The section starts zeroed and an old host never writes it, so `0`
/// tells a new driver to stay legacy-only. Carved from v2 reserved space (v2.1).
pub out_ring_ver: u32,
/// Driver-bumped (AFTER writing `out_ring[ring_head % OUT_RING_LEN]`) count of reports
/// ever published to the ring — the host's drain cursor compares against its own tail and
/// detects overflow by `head - tail > OUT_RING_LEN`. Same publish-then-bump store order as
/// `out_seq` (the host's Acquire load orders the reads). Carved from v2 reserved space
/// (v2.1).
pub ring_head: u32,
pub _reserved1: [u8; 92],
/// The lossless output-report ring (v2.1) — see the struct docs and [`OutSlot`].
pub out_ring: [OutSlot; OUT_RING_LEN_USIZE],
pub _reserved2: [u8; 224],
}
// Offsets are the wire contract the shipped drivers already read by hand — pin every one. A failing
@@ -704,7 +834,7 @@ pub mod gamepad {
assert!(offset_of!(XusbShm, driver_heartbeat) == 36);
assert!(offset_of!(XusbShm, pad_index) == 40);
assert!(size_of::<PadShm>() == 256);
assert!(size_of::<PadShm>() == 1024);
assert!(offset_of!(PadShm, magic) == 0);
assert!(offset_of!(PadShm, input) == 8);
assert!(offset_of!(PadShm, out_seq) == 72);
@@ -713,6 +843,11 @@ pub mod gamepad {
assert!(offset_of!(PadShm, driver_proto) == 144);
assert!(offset_of!(PadShm, driver_heartbeat) == 148);
assert!(offset_of!(PadShm, pad_index) == 152);
// v2.1 ring extension — everything below PAD_SHM_LEGACY_SIZE is the v2 layout verbatim.
assert!(offset_of!(PadShm, out_ring_ver) == 156);
assert!(offset_of!(PadShm, ring_head) == 160);
assert!(offset_of!(PadShm, out_ring) == PAD_SHM_LEGACY_SIZE);
assert!(size_of::<OutSlot>() == 68);
assert!(size_of::<PadBootstrap>() == 32);
assert!(offset_of!(PadBootstrap, magic) == 0);
@@ -725,6 +860,106 @@ pub mod gamepad {
};
}
/// Virtual-pointer shared-memory layout (host ↔ the UMDF HID-mouse minidriver `pf_mouse`).
///
/// Why a virtual mouse exists at all: with no pointing device present (a headless Windows host —
/// no dongle attached), win32k reports the cursor as absent (`SM_MOUSEPRESENT` = 0) and DWM never
/// composites a cursor into the pf-vdisplay frame, so a streamed desktop has an invisible pointer
/// even though `SendInput` moves it. A resident HID mouse devnode makes Windows always consider a
/// pointer present — the Sunshine/Parsec-class fix. Injection stays `SendInput`; the report path
/// below exists for validation (`vmouse-spike`) and as the future higher-fidelity route.
///
/// The channel is the **sealed pad channel** verbatim (`design/gamepad-channel-sealing.md`): the
/// same [`gamepad::PadBootstrap`] mailbox handshake (and therefore the same
/// [`gamepad::GAMEPAD_PROTO_VERSION`] lockstep), a mouse-specific mailbox name
/// ([`mouse_boot_name`]) and DATA magic, and `pad_index` validation (a single resident mouse =
/// index 0). Reusing the handshake means `pf-umdf-util`'s audited `ChannelClient`/`PadChannel`
/// serve the mouse unchanged.
pub mod mouse {
use alloc::string::String;
use bytemuck::{Pod, Zeroable};
/// Mouse DATA-section magic ("PFMO" LE) — distinct from the pad magics so a cross-wired
/// delivery fails validation.
pub const MOUSE_MAGIC: u32 = 0x4F4D_4650;
/// `Global\pfmouse-boot-<index>` — the virtual mouse's bootstrap mailbox
/// ([`crate::gamepad::PadBootstrap`]).
pub fn mouse_boot_name(index: u8) -> String {
alloc::format!("Global\\pfmouse-boot-{index}")
}
/// HID identity both sides report/expect ("PF" / "MO" — an obviously-virtual identity; no
/// software matches on it, unlike the pads' cloned Sony/Valve ids).
pub const MOUSE_VID: u16 = 0x5046;
pub const MOUSE_PID: u16 = 0x4D4F;
pub const MOUSE_VER: u16 = 0x0100;
/// The one input report (id `0x01`): `[id, buttons(5 bits), x_lo, x_hi, y_lo, y_hi, wheel,
/// pan]` — absolute X/Y over `0..=`[`MOUSE_ABS_MAX`], relative wheel/pan.
pub const MOUSE_REPORT_ID: u8 = 0x01;
pub const MOUSE_REPORT_LEN: usize = 8;
/// Logical maximum of the absolute X/Y axes (15-bit, the HID-descriptor convention).
pub const MOUSE_ABS_MAX: u16 = 0x7FFF;
/// Build the 8-byte input report. Pure so the byte layout is unit-tested on every dev machine
/// (the driver workspace is `panic = "abort"` and hosts no test harness); the driver only
/// ferries these bytes, it never builds them.
#[must_use]
pub fn input_report(buttons: u8, x: u16, y: u16, wheel: i8, pan: i8) -> [u8; MOUSE_REPORT_LEN] {
let x = x.min(MOUSE_ABS_MAX);
let y = y.min(MOUSE_ABS_MAX);
[
MOUSE_REPORT_ID,
buttons & 0x1F,
(x & 0xFF) as u8,
(x >> 8) as u8,
(y & 0xFF) as u8,
(y >> 8) as u8,
wheel as u8,
pan as u8,
]
}
/// Virtual-mouse shared section (64 B). The host writes an input report then bumps `in_seq`
/// (Release); the driver's timer Acquire-loads `in_seq` and completes a pended `READ_REPORT`
/// with the fresh report — event-driven like a real mouse, so an idle section generates NO
/// HID traffic (a constant report stream would read as user activity to the OS).
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug)]
pub struct MouseShm {
pub magic: u32,
/// Bumped by the host AFTER `report` is in place (Release) — the driver's new-input
/// trigger. `0` = nothing published yet.
pub in_seq: u32,
/// The latest HID input report (id [`MOUSE_REPORT_ID`], [`MOUSE_REPORT_LEN`] bytes).
pub report: [u8; MOUSE_REPORT_LEN],
/// Written by the driver's timer while attached: [`crate::gamepad::GAMEPAD_PROTO_VERSION`]
/// (the mouse channel rides the gamepad handshake). `0` = no driver attached — the host
/// health check keys off it.
pub driver_proto: u32,
/// Bumped by the driver's timer each tick — liveness (advances whether or not input flows).
pub driver_heartbeat: u32,
/// The device index this section serves (host-stamped before the magic; the driver
/// validates it against its devnode Location — same fail-closed check as the pads).
pub pad_index: u32,
pub _reserved: [u8; 36],
}
// Offsets are the cross-process wire contract — pin every one (same discipline as `gamepad`).
const _: () = {
use core::mem::{offset_of, size_of};
assert!(size_of::<MouseShm>() == 64);
assert!(offset_of!(MouseShm, magic) == 0);
assert!(offset_of!(MouseShm, in_seq) == 4);
assert!(offset_of!(MouseShm, report) == 8);
assert!(offset_of!(MouseShm, driver_proto) == 16);
assert!(offset_of!(MouseShm, driver_heartbeat) == 20);
assert!(offset_of!(MouseShm, pad_index) == 24);
};
}
#[cfg(test)]
mod tests {
use super::*;
@@ -758,6 +993,27 @@ mod tests {
assert_eq!(t.pack(), (7u64 << 40) | (42u64 << 8) | 3u64);
}
#[test]
fn opened_detail_roundtrips_and_saturates() {
use frame::{pack_opened_detail, unpack_opened_detail, OPENED_DETAIL_LIVE};
// Zero counters still stamp LIVE — "attached, nothing offered yet" is information.
assert_eq!(pack_opened_detail(0, 0), OPENED_DETAIL_LIVE);
assert_eq!(unpack_opened_detail(pack_opened_detail(0, 0)), Some((0, 0)));
// Roundtrip within range.
assert_eq!(
unpack_opened_detail(pack_opened_detail(1234, 567)),
Some((1234, 567))
);
// Saturation at each counter's width (15-bit offered, 16-bit mismatched).
assert_eq!(
unpack_opened_detail(pack_opened_detail(u32::MAX, u32::MAX)),
Some((0x7FFF, 0xFFFF))
);
// A pre-detail driver's field (any value without the LIVE bit) carries no information.
assert_eq!(unpack_opened_detail(0), None);
assert_eq!(unpack_opened_detail(0x7FFF_FFFF), None);
}
#[test]
fn shared_header_is_pod_and_64_bytes() {
let mut h = frame::SharedHeader::zeroed();
@@ -889,6 +1145,27 @@ mod tests {
assert_eq!(bytes[32..40], 0x2000u64.to_le_bytes());
}
#[test]
fn update_modes_request_roundtrips_and_versions_cohere() {
let req = control::UpdateModesRequest {
session_id: 42,
width: 2560,
height: 1409, // deliberately arbitrary — the in-place path serves window-drag modes
refresh_hz: 120,
_reserved: 0,
};
let bytes = bytemuck::bytes_of(&req);
assert_eq!(bytes.len(), 24);
assert_eq!(
*bytemuck::from_bytes::<control::UpdateModesRequest>(bytes),
req
);
assert_eq!(bytes[8..12], 2560u32.to_le_bytes());
// The compat window: v4 is additive over v3, so the host floor stays one below.
assert_eq!(PROTOCOL_VERSION, 4);
assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3);
}
#[test]
fn gamepad_names_and_magics_are_stable() {
assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0");
@@ -930,6 +1207,7 @@ mod tests {
control::IOCTL_GET_INFO,
control::IOCTL_CLEAR_ALL,
control::IOCTL_SET_FRAME_CHANNEL,
control::IOCTL_UPDATE_MODES,
];
for (i, a) in all.iter().enumerate() {
for b in &all[i + 1..] {
@@ -980,6 +1258,25 @@ mod tests {
assert!((360..=440).contains(&back), "min decoded {back} millinits");
}
#[test]
fn mouse_report_and_names_are_stable() {
assert_eq!(mouse::mouse_boot_name(0), "Global\\pfmouse-boot-0");
// "PFMO" little-endian, and never colliding with a pad magic (cross-wire validation).
assert_eq!(mouse::MOUSE_MAGIC.to_le_bytes(), *b"PFMO");
assert_ne!(mouse::MOUSE_MAGIC, gamepad::XUSB_MAGIC);
assert_ne!(mouse::MOUSE_MAGIC, gamepad::PAD_MAGIC);
// The 8-byte report layout the driver ferries and the host builds.
let r = mouse::input_report(0b0000_0101, 0x1234, 0x7FFF, -3, 7);
assert_eq!(r, [0x01, 0x05, 0x34, 0x12, 0xFF, 0x7F, 0xFD, 0x07]);
// Clamps: axes to the 15-bit logical max, buttons to the declared 5.
let r = mouse::input_report(0xFF, 0xFFFF, 0, 0, 0);
assert_eq!((r[1], r[2], r[3]), (0x1F, 0xFF, 0x7F));
// A zeroed section reads as "nothing published" (in_seq 0) — the driver's idle state.
let shm = mouse::MouseShm::zeroed();
assert_eq!(shm.in_seq, 0);
assert_eq!(bytemuck::bytes_of(&shm).len(), 64);
}
#[test]
fn guid_is_not_sudovda() {
const SUDOVDA: u128 = 0xE5BC_C234_1E0C_418A_A0D4_EF8B_7501_414D;
+74
View File
@@ -0,0 +1,74 @@
# Hardware/software video encode (plan §7 / §W6): the per-vendor backends (NVENC, VAAPI, AMF, QSV,
# Vulkan-Video, PyroWave, openh264) behind one `Encoder` trait + `open_video` selector, extracted
# from the host so it depends on the shared frame vocabulary (pf-frame) rather than living inside
# the orchestrator. Speaks pf-frame (CapturedFrame/PixelFormat/dxgi identity) and pf-zerocopy
# (CUDA), never pf-capture — the capture→encode edge is one-way (plan §2.4).
[package]
name = "pf-encode"
version.workspace = true
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
description = "punktfunk host video encode: NVENC/VAAPI/AMF/QSV/Vulkan-Video/PyroWave/openh264 backends behind one Encoder trait."
publish = false
[dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
pf-frame = { path = "../pf-frame" }
pf-gpu = { path = "../pf-gpu" }
pf-host-config = { path = "../pf-host-config" }
pf-zerocopy = { path = "../pf-zerocopy" }
anyhow = "1"
tracing = "0.1"
[dev-dependencies]
# A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`).
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
# Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms.
openh264 = "0.9"
[target.'cfg(target_os = "linux")'.dependencies]
# libavcodec (NVENC libav + VAAPI backends). `ffmpeg-sys-next` auto-detects the FFmpeg version.
ffmpeg-next = "8"
libc = "0.2"
# Vulkan bindings for the raw Vulkan-Video encode + PyroWave compute backends (feature-gated below;
# the dep stays unconditional to mirror the host's Linux target — unused-but-declared is harmless).
ash = "0.38"
# `libnvidia-encode.so.1` is dlopen'd at runtime for the direct-SDK NVENC/CUDA backend.
libloading = "0.8"
# Direct-SDK NVENC (raw `sys::nvEncodeAPI` types; entry points resolved at runtime). `ci-check` =
# vendored bindings, no CUDA toolkit at build.
nvidia-video-codec-sdk = { version = "0.4", features = ["ci-check"], optional = true }
# PyroWave (opt-in wired-LAN wavelet codec) — vendored codec + bindgen'd C API, only under `pyrowave`.
pyrowave-sys = { path = "../pyrowave-sys", optional = true }
[target.'cfg(target_os = "windows")'.dependencies]
# NVENC (direct SDK, D3D11 input) + the shared D3D11/DXGI vocabulary via pf-frame.
nvidia-video-codec-sdk = { version = "0.4", features = ["ci-check"], optional = true }
# AMD (AMF) + Intel (QSV) hardware encode via libavcodec (behind `amf-qsv`; link-imports FFmpeg).
ffmpeg-next = { version = "8", optional = true }
# `libnvidia-encode`/`nvEncodeAPI64.dll` resolved at runtime; the NVENC status→cause table dlopen.
libloading = "0.8"
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
"Win32_Storage_FileSystem",
"Win32_System_LibraryLoader",
"Win32_System_Threading",
] }
[features]
default = []
# NVENC hardware encode (Linux CUDA + Windows D3D11); entry points resolved at runtime.
nvenc = ["dep:nvidia-video-codec-sdk"]
# AMD (AMF) + Intel (QSV) hardware encode on Windows via libavcodec.
amf-qsv = ["dep:ffmpeg-next"]
# Raw Vulkan-Video HEVC/AV1 encode on Linux (reuses the `ash` bindings; no new dep).
vulkan-encode = []
# PyroWave — the opt-in wired-LAN intra-only wavelet codec (Linux encode backend).
pyrowave = ["dep:pyrowave-sys"]
@@ -1,11 +1,11 @@
//! The encoder contract (plan §7, Tier 1): the [`Encoder`] trait plus the plain-data value types its
//! signatures use — [`EncodedFrame`], [`Codec`], [`ChromaFormat`], [`EncoderCaps`] — and the
//! dimension/VBV helpers [`validate_dimensions`] and [`vbv_frames_env`]. Backend selection, the
//! capability probes that mirror it, and `Codec::host_wire_caps` stay in the parent [`crate::encode`]
//! facade, which re-exports this module (`pub(crate) use codec::*;`) so every `crate::encode::*` path
//! capability probes that mirror it, and `Codec::host_wire_caps` stay in the parent the `pf-encode` crate root
//! facade, which re-exports this module (`pub(crate) use codec::*;`) so every `crate::*` path
//! is unchanged.
use crate::capture::CapturedFrame;
use anyhow::Result;
use pf_frame::CapturedFrame;
/// An encoded access unit (one NAL/AU) to hand to `punktfunk_core` for FEC + packetization.
/// `data` is in-band Annex-B (the encoder is opened without a global header), so each
@@ -94,7 +94,7 @@ impl Codec {
}
/// Lowercase stats/console label (`"h264"` / `"hevc"` / `"av1"`) — the codec string seeded into
/// the web console's session meta ([`crate::stats_recorder::StatsRecorder::register_session`]).
/// the web console's session meta (the host `stats_recorder::StatsRecorder::register_session`).
pub fn label(self) -> &'static str {
match self {
Codec::H264 => "h264",
@@ -108,7 +108,7 @@ impl Codec {
/// H.264 is always 8-bit (High10 is neither an NVENC nor a VCN encode mode — negotiation
/// never asks), and PyroWave's wavelet path ingests 8-bit. `true` here is only the
/// *codec-level* gate: the active GPU/backend must still pass
/// [`can_encode_10bit`](crate::encode::can_encode_10bit) before the host negotiates 10-bit.
/// [`can_encode_10bit`](crate::can_encode_10bit) before the host negotiates 10-bit.
pub fn supports_10bit(self) -> bool {
matches!(self, Codec::H265 | Codec::Av1)
}
@@ -311,7 +311,7 @@ impl Codec {
}
/// The codec's *spec* top level/tier bitrate (bits/s) — the usual boundary at which NVENC
/// starts rejecting `avcodec_open2` with EINVAL. NOT a hard cap: [`open_video`](crate::encode::
/// starts rejecting `avcodec_open2` with EINVAL. NOT a hard cap: [`open_video`](crate::
/// open_video) probes the actual GPU ceiling by stepping DOWN from the requested bitrate only on
/// EINVAL, and uses this purely as the first step-down candidate (so a card that accepts more —
/// an RTX 5070 Ti does >1 Gbps HEVC where a 4090 caps at ~800 Mbps — is never clamped to it).
@@ -3,7 +3,7 @@
//! (`encode/windows/ffmpeg_win.rs`) — so the byte-identical pieces live once (plan §2.2, the Tier-2
//! gap). Free functions and consts over borrowed handles; nothing here is per-frame `dyn`,
//! allocating, or on the zero-copy ingest path.
use crate::encode::EncodedFrame;
use crate::EncodedFrame;
use anyhow::{Context, Result};
use ffmpeg_next as ffmpeg;
use ffmpeg_next::ffi; // = ffmpeg_sys_next
@@ -54,7 +54,7 @@ pub(crate) fn apply_low_latency_rc(video: &mut encoder::video::Video, fps: u32,
video.set_bit_rate(bitrate_bps as usize);
video.set_max_bit_rate(bitrate_bps as usize);
video.set_max_b_frames(0);
let vbv_bits = ((bitrate_bps as f64 / fps.max(1) as f64) * crate::encode::vbv_frames_env())
let vbv_bits = ((bitrate_bps as f64 / fps.max(1) as f64) * crate::vbv_frames_env())
.clamp(1.0, i32::MAX as f64);
// SAFETY: `video` wraps a freshly-allocated `AVCodecContext` we hold by value and have not opened
// yet; `as_mut_ptr()` returns that non-null, aligned, exclusively-owned context. Writing the plain
@@ -12,12 +12,12 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{ChromaFormat, Codec, EncodedFrame, Encoder};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use ffmpeg::format::Pixel;
use ffmpeg::util::frame::Video as VideoFrame;
use ffmpeg::{codec, encoder, Dictionary};
use ffmpeg_next as ffmpeg;
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use std::os::raw::c_int;
use std::ptr;
@@ -347,7 +347,7 @@ impl NvencEncoder {
// hwdevice/hwframes contexts and set `pix_fmt = CUDA` on the raw encoder context
// *before* open (NVENC derives the device from `hw_frames_ctx`).
let cuda_hw = if cuda {
let cu_ctx = crate::zerocopy::cuda::context().context("shared CUDA context")?;
let cu_ctx = pf_zerocopy::cuda::context().context("shared CUDA context")?;
// SAFETY: `CudaHw::new` (an `unsafe fn`) requires libav initialized (the `ffmpeg::init()`
// above ran) and a valid `CUcontext`; `cu_ctx` is the shared importer context from
// `zerocopy::cuda::context()?`, non-null on the `Ok` path. `nvenc_pixel` is a valid `Pixel`
@@ -722,12 +722,7 @@ impl NvencEncoder {
/// device pointer with a bounded table, so a fresh pointer every frame would thrash/overflow
/// it — the pool recycles a small set of pointers. The extra copy is device-local (~8 MB at
/// 1080p, sub-millisecond on the GPU) and keeps the host fully off the pixel path.
fn submit_cuda(
&mut self,
buf: &crate::zerocopy::DeviceBuffer,
pts: i64,
idr: bool,
) -> Result<()> {
fn submit_cuda(&mut self, buf: &pf_zerocopy::DeviceBuffer, pts: i64, idr: bool) -> Result<()> {
let frames_ref = self
.cuda
.as_ref()
@@ -735,7 +730,7 @@ impl NvencEncoder {
.frames_ref;
// The device→device copy below uses our shared context directly; make it current on the
// encode thread (ffmpeg pushes its own around the pool alloc, so order is fine).
crate::zerocopy::cuda::make_current().context("CUDA context current (encode thread)")?;
pf_zerocopy::cuda::make_current().context("CUDA context current (encode thread)")?;
// SAFETY: `frames_ref` is the non-null CUDA frames ctx from `self.cuda` (unwrapped via
// `.context(..)?` above), and the shared CUDA context was just made current on THIS thread
// (`make_current()?`), the precondition for the device-pointer copies below.
@@ -770,11 +765,11 @@ impl NvencEncoder {
let copy_res = if buf.yuv444 {
let dsts = core::array::from_fn(|i| {
(
(*f).data[i] as crate::zerocopy::cuda::CUdeviceptr,
(*f).data[i] as pf_zerocopy::cuda::CUdeviceptr,
(*f).linesize[i] as usize,
)
});
crate::zerocopy::cuda::copy_yuv444_to_device(buf, dsts)
pf_zerocopy::cuda::copy_yuv444_to_device(buf, dsts)
} else if self.want_444 {
ffi::av_frame_free(&mut f);
bail!(
@@ -783,15 +778,15 @@ impl NvencEncoder {
CPU 4:4:4 path on this compositor"
);
} else if buf.is_nv12() {
let y_ptr = (*f).data[0] as crate::zerocopy::cuda::CUdeviceptr;
let y_ptr = (*f).data[0] as pf_zerocopy::cuda::CUdeviceptr;
let y_pitch = (*f).linesize[0] as usize;
let uv_ptr = (*f).data[1] as crate::zerocopy::cuda::CUdeviceptr;
let uv_ptr = (*f).data[1] as pf_zerocopy::cuda::CUdeviceptr;
let uv_pitch = (*f).linesize[1] as usize;
crate::zerocopy::cuda::copy_nv12_to_device(buf, y_ptr, y_pitch, uv_ptr, uv_pitch)
pf_zerocopy::cuda::copy_nv12_to_device(buf, y_ptr, y_pitch, uv_ptr, uv_pitch)
} else {
let dst_ptr = (*f).data[0] as crate::zerocopy::cuda::CUdeviceptr;
let dst_ptr = (*f).data[0] as pf_zerocopy::cuda::CUdeviceptr;
let dst_pitch = (*f).linesize[0] as usize;
crate::zerocopy::cuda::copy_device_to_device(buf, dst_ptr, dst_pitch)
pf_zerocopy::cuda::copy_device_to_device(buf, dst_ptr, dst_pitch)
};
if let Err(e) = copy_res {
ffi::av_frame_free(&mut f);
@@ -827,7 +822,7 @@ impl Drop for NvencEncoder {
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode HEVC **4:4:4** (Range
/// Extensions). Opens a tiny real `hevc_nvenc` 4:4:4 session — the exact path [`NvencEncoder::open`]
/// takes for a live 4:4:4 stream — and reports whether it succeeded. HEVC-only; the result is cached
/// by the caller ([`crate::encode::can_encode_444`]). A GPU/driver/ffmpeg without RExt 4:4:4 fails
/// by the caller ([`crate::can_encode_444`]). A GPU/driver/ffmpeg without RExt 4:4:4 fails
/// the open here, so the host resolves the session to 4:2:0 before the Welcome (honest downgrade).
pub fn probe_can_encode_444(codec: Codec) -> bool {
if codec != Codec::H265 {
@@ -36,9 +36,9 @@ use super::nvenc_core::{
};
use super::nvenc_status;
use super::{ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
use crate::capture::{CapturedFrame, FramePayload};
use crate::zerocopy::cuda::{self, InputSurface};
use anyhow::{anyhow, bail, Context, Result};
use pf_frame::{CapturedFrame, FramePayload};
use pf_zerocopy::cuda::{self, InputSurface};
use std::collections::VecDeque;
use std::ffi::c_void;
use std::ptr;
@@ -321,7 +321,7 @@ impl NvencCudaEncoder {
#[allow(clippy::too_many_arguments)]
pub fn open(
codec: Codec,
_format: crate::capture::PixelFormat,
_format: pf_frame::PixelFormat,
width: u32,
height: u32,
fps: u32,
@@ -1010,23 +1010,23 @@ impl Encoder for NvencCudaEncoder {
let is_idr = flags != 0 || opening;
let mastering_sei = self
.hdr_meta
.map(|m| crate::hdr::hevc_mastering_display_sei(&m));
.map(|m| pf_frame::hdr::hevc_mastering_display_sei(&m));
let cll_sei = self
.hdr_meta
.map(|m| crate::hdr::hevc_content_light_level_sei(&m));
.map(|m| pf_frame::hdr::hevc_content_light_level_sei(&m));
let mut sei: Vec<nv::NV_ENC_SEI_PAYLOAD> = Vec::new();
if is_idr && self.hdr {
if let Some(p) = mastering_sei.as_ref() {
sei.push(nv::NV_ENC_SEI_PAYLOAD {
payloadSize: p.len() as u32,
payloadType: crate::hdr::SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME,
payloadType: pf_frame::hdr::SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME,
payload: p.as_ptr() as *mut u8,
});
}
if let Some(p) = cll_sei.as_ref() {
sei.push(nv::NV_ENC_SEI_PAYLOAD {
payloadSize: p.len() as u32,
payloadType: crate::hdr::SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO,
payloadType: pf_frame::hdr::SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO,
payload: p.as_ptr() as *mut u8,
});
}
@@ -1253,8 +1253,8 @@ impl Drop for NvencCudaEncoder {
#[cfg(test)]
mod tests {
use super::*;
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use crate::zerocopy::cuda::DeviceBuffer;
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use pf_zerocopy::cuda::DeviceBuffer;
fn nv12_frame(w: u32, h: u32, i: u32) -> CapturedFrame {
// Content is uninitialized device memory — NVENC encodes it fine; this smoke test asserts the
@@ -1281,7 +1281,7 @@ mod tests {
fn nvenc_cuda_smoke_rfi_anchor() {
const W: u32 = 1280;
const H: u32 = 720;
crate::zerocopy::cuda::make_current().expect("shared CUDA context current");
pf_zerocopy::cuda::make_current().expect("shared CUDA context current");
let mut enc = NvencCudaEncoder::open(
Codec::H265,
@@ -1358,7 +1358,7 @@ mod tests {
fn nvenc_cuda_yuv444() {
const W: u32 = 1280;
const H: u32 = 720;
crate::zerocopy::cuda::make_current().expect("shared CUDA context current");
pf_zerocopy::cuda::make_current().expect("shared CUDA context current");
let mut enc = NvencCudaEncoder::open(
Codec::H265,
PixelFormat::Yuv444,
@@ -1403,7 +1403,7 @@ mod tests {
fn nvenc_cuda_reconfigure_no_idr() {
const W: u32 = 1280;
const H: u32 = 720;
crate::zerocopy::cuda::make_current().expect("shared CUDA context current");
pf_zerocopy::cuda::make_current().expect("shared CUDA context current");
let mut enc = NvencCudaEncoder::open(
Codec::H265,
PixelFormat::Nv12,
@@ -1510,7 +1510,7 @@ mod tests {
fn nvenc_cuda_codec_switch_reopen() {
const W: u32 = 1280;
const H: u32 = 720;
crate::zerocopy::cuda::make_current().expect("shared CUDA context current");
pf_zerocopy::cuda::make_current().expect("shared CUDA context current");
for (leg, codec) in [
Codec::H265,
Codec::Av1,
@@ -1552,7 +1552,7 @@ mod tests {
fn nvenc_cuda_dirty_teardown_reopen() {
const W: u32 = 1280;
const H: u32 = 720;
crate::zerocopy::cuda::make_current().expect("shared CUDA context current");
pf_zerocopy::cuda::make_current().expect("shared CUDA context current");
for round in 0..3 {
let mut enc = open_h265();
for f in 0..4u32 {
@@ -1581,7 +1581,7 @@ mod tests {
fn nvenc_cuda_open_failure_diagnosis_and_recovery() {
const W: u32 = 1280;
const H: u32 = 720;
crate::zerocopy::cuda::make_current().expect("shared CUDA context current");
pf_zerocopy::cuda::make_current().expect("shared CUDA context current");
try_api().expect("nvenc api");
let shared = cuda::context().expect("shared ctx");
@@ -24,11 +24,11 @@
// Every unsafe block in this module carries a `// SAFETY:` proof (parent module enforces it).
use super::vk_util::{color_range, find_mem, import_rgb_dmabuf, make_plain_image, pixel_to_vk};
use crate::capture::{CapturedFrame, FramePayload};
use crate::encode::{EncodedFrame, Encoder, EncoderCaps};
use crate::{EncodedFrame, Encoder, EncoderCaps};
use anyhow::{bail, Context, Result};
use ash::vk;
use ash::vk::Handle as _;
use pf_frame::{CapturedFrame, FramePayload};
use pyrowave_sys as pw;
use std::collections::VecDeque;
use std::os::fd::AsRawFd;
@@ -637,10 +637,7 @@ impl PyroWaveEncoder {
/// Records the small upload (only when the bitmap `serial` changed) + layout transition into
/// `cmd`, ahead of the CSC dispatch that samples binding 3. Encode is synchronous, so the single
/// shared image never races a prior frame; the first use transitions it to SHADER_READ_ONLY.
unsafe fn prep_cursor(
&mut self,
cursor: Option<&crate::capture::CursorOverlay>,
) -> Result<[i32; 4]> {
unsafe fn prep_cursor(&mut self, cursor: Option<&pf_frame::CursorOverlay>) -> Result<[i32; 4]> {
let dev = self.device.clone();
let cmd = self.cmd;
let img = self.cursor_img;
@@ -748,7 +745,7 @@ impl PyroWaveEncoder {
/// Import a dmabuf with per-buffer caching — same policy as `vulkan_video.rs::import_cached`.
unsafe fn import_cached(
&mut self,
d: &crate::capture::DmabufFrame,
d: &pf_frame::DmabufFrame,
cw: u32,
ch: u32,
) -> Result<(vk::Image, vk::ImageView, bool)> {
@@ -1303,7 +1300,7 @@ impl Drop for PyroWaveEncoder {
#[cfg(test)]
mod tests {
use super::*;
use crate::capture::PixelFormat;
use pf_frame::PixelFormat;
fn cpu_frame(w: u32, h: u32, pts_ns: u64, fill: [u8; 4]) -> CapturedFrame {
let mut buf = vec![0u8; (w * h * 4) as usize];
@@ -23,11 +23,11 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{Codec, EncodedFrame, Encoder};
use crate::capture::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use ffmpeg::format::Pixel;
use ffmpeg::{codec, encoder, Dictionary};
use ffmpeg_next as ffmpeg;
use pf_frame::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat};
use std::ffi::{CStr, CString};
use std::os::fd::AsRawFd;
use std::os::raw::c_int;
@@ -44,13 +44,11 @@ const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
(a as u32) | ((b as u32) << 8) | ((c as u32) << 16) | ((d as u32) << 24)
}
/// The render node a VAAPI/DRM device should open, from [`crate::gpu::linux_render_node`]: a
/// The render node a VAAPI/DRM device should open, from [`pf_gpu::linux_render_node`]: a
/// matched web-console GPU preference pins it, else `PUNKTFUNK_RENDER_NODE`, else the single-GPU
/// default.
fn render_node() -> CString {
let p = crate::gpu::linux_render_node()
.to_string_lossy()
.into_owned();
let p = pf_gpu::linux_render_node().to_string_lossy().into_owned();
CString::new(p).unwrap_or_else(|_| CString::new("/dev/dri/renderD128").unwrap())
}
@@ -563,7 +561,7 @@ impl DmabufInner {
fps: u32,
bitrate_bps: u64,
) -> Result<Self> {
let drm_fourcc = crate::zerocopy::drm_fourcc(format)
let drm_fourcc = pf_frame::drm_fourcc(format)
.ok_or_else(|| anyhow!("no DRM fourcc for {format:?} (VAAPI zero-copy)"))?;
let node = render_node();
// SAFETY: libav is initialized (`VaapiEncoder::open` ran `ffmpeg::init()` before
@@ -809,7 +807,7 @@ impl DmabufInner {
// Sampled breakdown of this synchronous submit under PUNKTFUNK_PERF: push = descriptor
// build + buffersrc (the per-frame DRM→VA import happens inside hwmap on the pull path),
// pull = buffersink (VPP CSC + any sync), send = avcodec_send_frame. One line per ~2 s.
let sample = crate::config::config().perf && self.frames % 120 == 0;
let sample = pf_host_config::config().perf && self.frames % 120 == 0;
self.frames += 1;
let t0 = std::time::Instant::now();
let t_push: std::time::Duration;
@@ -3,9 +3,9 @@
//! when the PyroWave backend arrived so the two don't fork copies.
// Every unsafe block carries a `// SAFETY:` proof (parent module enforces it).
use crate::capture::PixelFormat;
use anyhow::Result;
use ash::vk;
use pf_frame::PixelFormat;
pub(crate) fn color_range(layer: u32) -> vk::ImageSubresourceRange {
vk::ImageSubresourceRange {
@@ -74,7 +74,7 @@ pub(crate) unsafe fn import_rgb_dmabuf(
device: &ash::Device,
ext_fd: &ash::khr::external_memory_fd::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
d: &crate::capture::DmabufFrame,
d: &pf_frame::DmabufFrame,
cw: u32,
ch: u32,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
@@ -11,10 +11,10 @@
#![allow(clippy::too_many_arguments)]
use super::vk_util::{color_range, find_mem, make_plain_image, make_view, pixel_to_vk};
use crate::capture::{CapturedFrame, FramePayload};
use crate::encode::{Codec, EncodedFrame, Encoder, EncoderCaps};
use crate::{Codec, EncodedFrame, Encoder, EncoderCaps};
use anyhow::{bail, Context, Result};
use ash::vk;
use pf_frame::{CapturedFrame, FramePayload};
use std::collections::VecDeque;
use std::ffi::c_void;
use std::os::fd::AsRawFd;
@@ -729,7 +729,7 @@ impl VulkanVideoEncoder {
&mut self,
slot: usize,
compute_cmd: vk::CommandBuffer,
cursor: Option<&crate::capture::CursorOverlay>,
cursor: Option<&pf_frame::CursorOverlay>,
) -> Result<[i32; 4]> {
let dev = self.device.clone();
let img = self.frames[slot].cursor_img;
@@ -837,7 +837,7 @@ impl VulkanVideoEncoder {
/// Import a packed-RGB dmabuf as a SAMPLED VkImage (explicit DRM modifier). Caller destroys.
unsafe fn import_dmabuf(
&self,
d: &crate::capture::DmabufFrame,
d: &pf_frame::DmabufFrame,
cw: u32,
ch: u32,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
@@ -850,7 +850,7 @@ impl VulkanVideoEncoder {
/// true only on a first import (caller uses UNDEFINED old-layout to preserve modifier-tiled data).
unsafe fn import_cached(
&mut self,
d: &crate::capture::DmabufFrame,
d: &pf_frame::DmabufFrame,
cw: u32,
ch: u32,
) -> Result<(vk::Image, vk::ImageView, bool)> {
@@ -2680,8 +2680,8 @@ unsafe fn build_parameters_av1(
#[cfg(test)]
mod tests {
use super::{build_h265_rps_s0, pick_recovery_slot, VulkanVideoEncoder};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use crate::encode::{Codec, Encoder};
use crate::{Codec, Encoder};
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
/// The RFI anchor picker: newest resident wire strictly older than the loss; empty/newer
/// slots never qualify.
@@ -2761,7 +2761,7 @@ mod tests {
/// the reference-slot RFI end-to-end; returns the AUs. Wire frame [`SMOKE_LOST`] is "lost", one
/// normal P referencing it is still encoded (the in-flight window), then frame [`SMOKE_ANCHOR`]
/// is the clean recovery anchor referencing pre-loss frame 3 (no IDR).
fn run_smoke(codec: Codec) -> Vec<crate::encode::EncodedFrame> {
fn run_smoke(codec: Codec) -> Vec<crate::EncodedFrame> {
let env_dim = |k: &str, d: u32| {
std::env::var(k)
.ok()
@@ -2782,7 +2782,7 @@ mod tests {
[120, 200, 80, 255],
[80, 120, 200, 255],
];
let mut aus: Vec<crate::encode::EncodedFrame> = Vec::new();
let mut aus: Vec<crate::EncodedFrame> = Vec::new();
for (i, c) in colors.iter().enumerate() {
if i == SMOKE_ANCHOR {
// The client reports wire frame SMOKE_LOST lost → the next frame must re-anchor
@@ -2836,7 +2836,7 @@ mod tests {
/// concealment the client's freeze hides) and NONE at the anchor — a complaint about the
/// anchor's reference (frame 3 / POC 3) means reference retention regressed and the "clean"
/// re-anchor ships corruption.
fn dump_smoke(aus: &[crate::encode::EncodedFrame], ext: &str) {
fn dump_smoke(aus: &[crate::EncodedFrame], ext: &str) {
let Ok(home) = std::env::var("HOME") else {
return;
};
@@ -125,7 +125,7 @@ pub(super) unsafe fn apply_low_latency_config(cfg: &mut nv::NV_ENC_CONFIG, c: Lo
// when the GPU advertises custom-VBV support — else keep the preset default.
if c.custom_vbv {
// ~1-frame VBV by default; PUNKTFUNK_VBV_FRAMES scales it (parity with AMF/VAAPI/QSV).
let vbv = ((c.bitrate as f64 / c.fps.max(1) as f64) * crate::encode::vbv_frames_env())
let vbv = ((c.bitrate as f64 / c.fps.max(1) as f64) * crate::vbv_frames_env())
.clamp(1.0, u32::MAX as f64) as u32;
cfg.rcParams.vbvBufferSize = vbv;
cfg.rcParams.vbvInitialDelay = vbv;
@@ -12,7 +12,6 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{EncodedFrame, Encoder};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use anyhow::{bail, ensure, Context, Result};
use openh264::encoder::{
BitRate, Complexity, Encoder as Oh264, EncoderConfig, FrameRate, FrameType, IntraFramePeriod,
@@ -20,6 +19,7 @@ use openh264::encoder::{
};
use openh264::formats::YUVSlices;
use openh264::OpenH264API;
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use std::collections::VecDeque;
pub struct OpenH264Encoder {
@@ -258,7 +258,7 @@ fn num_threads() -> u16 {
#[cfg(test)]
mod tests {
use super::*;
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
/// The BT.709 limited-range anchor points: reference white → (235,128,128), black →
/// (16,128,128), pure red's Cr must hit the positive extreme 240 (it does exactly:
@@ -47,8 +47,8 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use std::collections::VecDeque;
use std::ffi::c_void;
use std::ptr;
@@ -1334,7 +1334,7 @@ impl AmfEncoder {
/// same shape every backend ships. Shared by [`apply_static_props`](Self::apply_static_props)
/// and [`Encoder::reconfigure_bitrate`] so a dynamic retarget rescales the buffer it opened with.
fn vbv_bits(&self, bps: u64) -> i64 {
((bps as f64 / self.fps.max(1) as f64) * crate::encode::vbv_frames_env())
((bps as f64 / self.fps.max(1) as f64) * crate::vbv_frames_env())
.clamp(1.0, i32::MAX as f64) as i64
}
@@ -1777,7 +1777,7 @@ fn probe_can_encode_on(device: &ID3D11Device, codec: Codec) -> bool {
/// encoder at 10-bit (Main10 profile / `*ColorBitDepth` 10, P010 input)? The driver rejects the
/// profile/depth props on VCN generations that can't encode them, so a successful tiny `Init` is
/// the honest per-codec answer — read *before* the Welcome by
/// [`crate::encode::can_encode_10bit`] so the negotiated bit depth matches what the session's
/// [`crate::can_encode_10bit`] so the negotiated bit depth matches what the session's
/// encoder will really open. H.264 is always `false` (High10 is not a VCN mode — the session
/// open bails on it too).
pub fn probe_can_encode_10bit(codec: Codec) -> bool {
@@ -1881,8 +1881,8 @@ fn selected_adapter_device() -> Option<ID3D11Device> {
// `D3D11CreateDevice` (explicit adapter + UNKNOWN driver type, or NULL adapter + HARDWARE)
// fills `device` only on success. Everything drops with its COM wrapper.
unsafe {
let adapter: Option<IDXGIAdapter1> = crate::win_adapter::resolve_render_adapter_luid()
.and_then(|luid| {
let adapter: Option<IDXGIAdapter1> =
pf_gpu::resolve_render_adapter_luid().and_then(|luid| {
let factory: IDXGIFactory4 = CreateDXGIFactory1().ok()?;
factory.EnumAdapterByLuid(luid).ok()
});
@@ -2785,7 +2785,7 @@ mod tests {
height: h,
pts_ns: 1 + i as u64,
format: fmt,
payload: FramePayload::D3d11(crate::capture::dxgi::D3d11Frame {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
}),
@@ -2856,8 +2856,8 @@ mod tests {
);
drop(native);
let mut ffmpeg = crate::encode::ffmpeg_win::FfmpegWinEncoder::open(
crate::encode::ffmpeg_win::WinVendor::Amf,
let mut ffmpeg = crate::ffmpeg_win::FfmpegWinEncoder::open(
crate::ffmpeg_win::WinVendor::Amf,
Codec::H265,
PixelFormat::Nv12,
w,
@@ -2970,7 +2970,7 @@ mod tests {
height: h,
pts_ns: base + i as u64,
format: PixelFormat::Nv12,
payload: FramePayload::D3d11(crate::capture::dxgi::D3d11Frame {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
}),
@@ -3111,7 +3111,7 @@ mod tests {
height: h,
pts_ns: 1 + i as u64,
format: PixelFormat::P010,
payload: FramePayload::D3d11(crate::capture::dxgi::D3d11Frame {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
}),
@@ -3258,7 +3258,7 @@ mod tests {
height: h,
pts_ns: i,
format: PixelFormat::Nv12,
payload: FramePayload::D3d11(crate::capture::dxgi::D3d11Frame {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
}),
@@ -41,11 +41,11 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{ChromaFormat, Codec, EncodedFrame, Encoder};
use crate::capture::{dxgi::D3d11Frame, CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use ffmpeg::format::Pixel;
use ffmpeg::{codec, encoder, Dictionary};
use ffmpeg_next as ffmpeg;
use pf_frame::{dxgi::D3d11Frame, CapturedFrame, FramePayload, PixelFormat};
use std::os::raw::{c_int, c_uint, c_void};
use std::ptr;
use windows::core::Interface;
@@ -122,7 +122,7 @@ impl WinVendor {
/// open-failure fallback only catches *setup* errors; a derive that opens but maps wrong would
/// corrupt silently, so it stays opt-in per the probe-never-assume rule).
fn zerocopy_enabled(vendor: WinVendor) -> bool {
crate::config::config()
pf_host_config::config()
.zerocopy
.unwrap_or(matches!(vendor, WinVendor::Amf))
}
@@ -41,8 +41,8 @@ use super::nvenc_core::{
};
use super::nvenc_status;
use super::{ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
use crate::capture::{CapturedFrame, FramePayload, PixelFormat};
use anyhow::{anyhow, bail, Context, Result};
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use std::collections::{HashMap, VecDeque};
use std::ffi::c_void;
use std::ptr;
@@ -321,7 +321,7 @@ fn retrieve_loop(
work_rx: mpsc::Receiver<RetrieveJob>,
done_tx: mpsc::Sender<RetrieveDone>,
) {
crate::native::boost_thread_priority(false);
pf_frame::thread_qos::boost_thread_priority(false);
while let Ok(job) = work_rx.recv() {
// SAFETY: `job.event` is one of the auto-reset events `init_session` created and
// registered for exactly this session, and `job.bs` one of its pool bitstreams; both stay
@@ -1250,23 +1250,23 @@ impl Encoder for NvencD3d11Encoder {
let is_idr = flags != 0 || opening;
let mastering_sei = self
.hdr_meta
.map(|m| crate::hdr::hevc_mastering_display_sei(&m));
.map(|m| pf_frame::hdr::hevc_mastering_display_sei(&m));
let cll_sei = self
.hdr_meta
.map(|m| crate::hdr::hevc_content_light_level_sei(&m));
.map(|m| pf_frame::hdr::hevc_content_light_level_sei(&m));
let mut sei: Vec<nv::NV_ENC_SEI_PAYLOAD> = Vec::new();
if is_idr && self.hdr {
if let Some(p) = mastering_sei.as_ref() {
sei.push(nv::NV_ENC_SEI_PAYLOAD {
payloadSize: p.len() as u32,
payloadType: crate::hdr::SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME,
payloadType: pf_frame::hdr::SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME,
payload: p.as_ptr() as *mut u8,
});
}
if let Some(p) = cll_sei.as_ref() {
sei.push(nv::NV_ENC_SEI_PAYLOAD {
payloadSize: p.len() as u32,
payloadType: crate::hdr::SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO,
payloadType: pf_frame::hdr::SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO,
payload: p.as_ptr() as *mut u8,
});
}
@@ -1568,7 +1568,7 @@ impl Drop for NvencD3d11Encoder {
}
/// Probe whether the active NVIDIA GPU can encode HEVC **4:4:4** (`NV_ENC_CAPS_SUPPORT_YUV444_ENCODE`).
/// HEVC-only; the result is cached by the caller ([`crate::encode::can_encode_444`]) and read *before*
/// HEVC-only; the result is cached by the caller ([`crate::can_encode_444`]) and read *before*
/// the Welcome so the host advertises the chroma it can really encode (honest downgrade to 4:2:0 on a
/// card without it). See [`probe_encode_cap`] for the throwaway-session mechanics.
pub fn probe_can_encode_444(codec: Codec) -> bool {
@@ -1580,7 +1580,7 @@ pub fn probe_can_encode_444(codec: Codec) -> bool {
/// Probe whether the active NVIDIA GPU can encode `codec` at **10-bit**
/// (`NV_ENC_CAPS_SUPPORT_10BIT_ENCODE` against the codec's own GUID — HEVC Main10 / AV1 10-bit).
/// The result is cached by the caller ([`crate::encode::can_encode_10bit`]) and read *before* the
/// The result is cached by the caller ([`crate::can_encode_10bit`]) and read *before* the
/// Welcome so the negotiated bit depth — and the HDR label derived from it — matches what NVENC
/// will really emit. The session-open path re-checks the same cap as a belt-and-braces guard
/// ([`NvencD3d11Encoder::probe_caps`]'s 8-bit fallback).
@@ -1622,8 +1622,8 @@ fn probe_encode_cap(codec: Codec, cap: nv::NV_ENC_CAPS) -> bool {
// Probe on the SELECTED render adapter — the GPU the session will actually encode on
// (web-console preference / PUNKTFUNK_RENDER_ADAPTER / max VRAM). The OS default adapter
// (NULL) can be the *other* GPU on a hybrid box, answering for hardware we won't use.
let adapter: Option<IDXGIAdapter1> = crate::win_adapter::resolve_render_adapter_luid()
.and_then(|luid| {
let adapter: Option<IDXGIAdapter1> =
pf_gpu::resolve_render_adapter_luid().and_then(|luid| {
let factory: IDXGIFactory4 = CreateDXGIFactory1().ok()?;
factory.EnumAdapterByLuid(luid).ok()
});
@@ -1692,7 +1692,7 @@ fn probe_encode_cap(codec: Codec, cap: nv::NV_ENC_CAPS) -> bool {
#[cfg(test)]
mod tests {
use super::*;
use crate::capture::{dxgi::D3d11Frame, CapturedFrame, FramePayload};
use pf_frame::{dxgi::D3d11Frame, CapturedFrame, FramePayload};
use windows::Win32::Graphics::Direct3D11::{
D3D11_BIND_RENDER_TARGET, D3D11_SUBRESOURCE_DATA, D3D11_TEXTURE2D_DESC, D3D11_USAGE_DEFAULT,
};
@@ -1760,7 +1760,7 @@ mod tests {
}
}
let adapter = adapter.expect("no hardware DXGI adapter");
let (device, _ctx) = crate::capture::dxgi::make_device(&adapter).expect("make_device");
let (device, _ctx) = pf_frame::dxgi::make_device(&adapter).expect("make_device");
let bytes = probe_pattern(W as usize, H as usize);
let init = D3D11_SUBRESOURCE_DATA {
@@ -1860,7 +1860,7 @@ mod tests {
}
}
let adapter = adapter.expect("no hardware DXGI adapter");
let (device, _ctx) = crate::capture::dxgi::make_device(&adapter).expect("make_device");
let (device, _ctx) = pf_frame::dxgi::make_device(&adapter).expect("make_device");
let bytes = probe_pattern(W as usize, H as usize);
let init = D3D11_SUBRESOURCE_DATA {
@@ -2,21 +2,27 @@
//! B-frames off. The backend is per-GPU: NVENC on NVIDIA (`*_nvenc`, accepts `bgr0` and does
//! RGB→YUV on the GPU, so no host-side CSC) and VAAPI on AMD/Intel (`*_vaapi`; the CPU-input
//! fallback swscales RGB→NV12, the zero-copy path imports the capture dmabuf straight into a
//! VA surface). One [`Encoder`] trait, selected in [`open_video`].
//! VA surface). One [`Encoder`] trait, selected in [`open_video`]. Extracted into a subsystem crate
//! (plan §W6): depends on the shared frame vocabulary (`pf-frame`) + zero-copy plumbing
//! (`pf-zerocopy`), never on capture — the capture→encode edge is one-way.
// Scaffold: some backend paths + trait defaults are defined ahead of the per-feature build that
// uses them (mirrors the host crate root's allow before the extraction).
#![allow(dead_code)]
// Every unsafe block in this module tree carries a `// SAFETY:` proof; enforce it (unsafe-proof
// program). As a parent module this also covers the child modules (encode::windows/linux::*).
// program). As a parent module this also covers the child modules (windows/linux backends).
#![deny(clippy::undocumented_unsafe_blocks)]
use crate::capture::{CapturedFrame, PixelFormat};
use anyhow::Result;
use pf_frame::{CapturedFrame, PixelFormat};
#[path = "enc/codec.rs"]
mod codec;
pub(crate) use codec::*;
pub use codec::*;
impl Codec {
/// The `quic` codec bitfield the host can currently **emit** on the punktfunk/1 native path,
/// given the resolved encode backend — the same GPU-aware advertisement GameStream builds for
/// Moonlight ([`crate::gamestream::serverinfo`]), in `quic::CODEC_*` bits. The GPU-less software
/// Moonlight (the host `gamestream::serverinfo`), in `quic::CODEC_*` bits. The GPU-less software
/// encoder (openh264) produces H.264 only; the probed backends (Linux VAAPI, Windows AMF/QSV)
/// advertise exactly what the GPU encodes ([`vaapi_codec_support`] / [`windows_codec_support`] —
/// AV1 encode is narrow, an old iGPU might lack HEVC); NVENC keeps the Moonlight-validated
@@ -30,12 +36,12 @@ impl Codec {
// client explicitly prefers it (resolve_codec ignores the bit in its ladder). Advertised
// whenever the backend could open: AMD/Intel capture hands raw dmabufs it imports
// directly, and an NVIDIA-auto host's PyroWave sessions flip capture to CPU RGB
// per-session instead ([`crate::session_plan::SessionPlan::output_format`]) — the EGL→CUDA
// per-session instead (the host `session_plan::SessionPlan::output_format`) — the EGL→CUDA
// frames the `auto` GPU path would deliver are NVENC-only. Only a software/GPU-less pref
// keeps the bit off (no Vulkan device to open).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
let pyro = if !matches!(
crate::config::config().encoder_pref.as_str(),
pf_host_config::config().encoder_pref.as_str(),
"software" | "sw" | "openh264"
) {
punktfunk_core::quic::CODEC_PYROWAVE
@@ -53,7 +59,7 @@ impl Codec {
#[cfg(target_os = "linux")]
{
if matches!(
crate::config::config().encoder_pref.as_str(),
pf_host_config::config().encoder_pref.as_str(),
"software" | "sw" | "openh264"
) {
return punktfunk_core::quic::CODEC_H264;
@@ -83,7 +89,7 @@ impl Codec {
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
{
let _ = GPU_SUPERSET;
match crate::config::config().encoder_pref.as_str() {
match pf_host_config::config().encoder_pref.as_str() {
"software" | "sw" | "openh264" => punktfunk_core::quic::CODEC_H264,
_ => punktfunk_core::quic::CODEC_HEVC,
}
@@ -128,24 +134,24 @@ pub fn open_video(
// mirroring its dispatch, which went stale the moment a backend gained an internal fallback
// (the default-on Vulkan Video path falls back to VAAPI on a failed open, and a dispatch
// mirror would report "vaapi" for every Vulkan session or vice versa). The GPU identity is the
// same selection the capturer was created on ([`crate::gpu::selected_gpu`]). Dropping the
// same selection the capturer was created on ([`pf_gpu::selected_gpu`]). Dropping the
// returned encoder ends the record, so the live count is correct by construction.
let gpu = if backend == "software" {
crate::gpu::ActiveGpu {
pf_gpu::ActiveGpu {
id: String::new(),
name: "CPU (openh264)".into(),
vendor_id: 0,
backend,
}
} else {
match crate::gpu::selected_gpu() {
Some(sel) => crate::gpu::ActiveGpu {
match pf_gpu::selected_gpu() {
Some(sel) => pf_gpu::ActiveGpu {
id: sel.info.id,
name: sel.info.name,
vendor_id: sel.info.vendor_id,
backend,
},
None => crate::gpu::ActiveGpu {
None => pf_gpu::ActiveGpu {
id: String::new(),
name: "GPU".into(),
vendor_id: 0,
@@ -155,15 +161,15 @@ pub fn open_video(
};
Ok(Box::new(TrackedEncoder {
inner,
_session: crate::gpu::session_begin(gpu),
_session: pf_gpu::session_begin(gpu),
}))
}
/// Ties the [`crate::gpu`] live-session record to the encoder's lifetime; pure delegation
/// Ties the `pf_gpu` live-session record to the encoder's lifetime; pure delegation
/// otherwise.
struct TrackedEncoder {
inner: Box<dyn Encoder>,
_session: crate::gpu::ActiveSession,
_session: pf_gpu::ActiveSession,
}
impl Encoder for TrackedEncoder {
@@ -262,7 +268,7 @@ fn open_video_backend(
// AMD/Intel → VAAPI (one libavcodec backend for both). Auto-detect by default so a single
// Linux binary serves any GPU; `PUNKTFUNK_ENCODER` forces a specific backend (and surfaces
// its errors crisply instead of silently trying the other).
let pref = crate::config::config().encoder_pref.as_str();
let pref = pf_host_config::config().encoder_pref.as_str();
// AMD/Intel opener. Default = libav VAAPI. With `--features vulkan-encode` +
// PUNKTFUNK_VULKAN_ENCODE, an HEVC session instead opens the raw Vulkan Video backend (real
// RFI loss recovery the VAAPI path can't express); a failed open falls back to VAAPI so the
@@ -406,11 +412,11 @@ fn open_video_backend(
// explicit PUNKTFUNK_ENCODER contradicts the GPU the pipeline sits on (e.g. `nvenc` forced
// while the web-console preference pins the Intel iGPU) — the open below will then fail on
// a wrong-vendor device; say why up front instead of leaving an opaque encoder error.
if let Some(sel) = crate::gpu::selected_gpu() {
if let Some(sel) = pf_gpu::selected_gpu() {
let mismatched = match backend {
WindowsBackend::Nvenc => sel.info.vendor_id != crate::gpu::VENDOR_NVIDIA,
WindowsBackend::Amf => sel.info.vendor_id != crate::gpu::VENDOR_AMD,
WindowsBackend::Qsv => sel.info.vendor_id != crate::gpu::VENDOR_INTEL,
WindowsBackend::Nvenc => sel.info.vendor_id != pf_gpu::VENDOR_NVIDIA,
WindowsBackend::Amf => sel.info.vendor_id != pf_gpu::VENDOR_AMD,
WindowsBackend::Qsv => sel.info.vendor_id != pf_gpu::VENDOR_INTEL,
WindowsBackend::Software => false,
};
if mismatched {
@@ -680,14 +686,14 @@ fn nvidia_present() -> bool {
}
/// The `auto` Linux backend decision, shared by [`open_video`] and [`linux_zero_copy_is_vaapi`]:
/// a manual web-console GPU preference (when that GPU is present — [`crate::gpu::manual_selection`])
/// a manual web-console GPU preference (when that GPU is present — [`pf_gpu::manual_selection`])
/// picks its vendor's backend — AMD/Intel → VAAPI on that GPU's render node, NVIDIA → NVENC (still
/// requiring the proprietary driver's device nodes; a nouveau NVIDIA GPU can't NVENC) — otherwise
/// today's NVIDIA-presence probe, unchanged.
#[cfg(target_os = "linux")]
fn linux_auto_is_vaapi() -> bool {
if let Some(g) = crate::gpu::manual_selection() {
if g.vendor_id == crate::gpu::VENDOR_NVIDIA {
if let Some(g) = pf_gpu::manual_selection() {
if g.vendor_id == pf_gpu::VENDOR_NVIDIA {
return !nvidia_present();
}
return true;
@@ -699,7 +705,7 @@ fn linux_auto_is_vaapi() -> bool {
/// packed-RGB fourcc — advertised by the capture when the pyrowave passthrough is active
/// (the VAAPI LINEAR-only policy starves it on Mutter+NVIDIA, which allocates tiled only).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
pub(crate) fn pyrowave_capture_modifiers(fourcc: u32) -> Vec<u64> {
pub fn pyrowave_capture_modifiers(fourcc: u32) -> Vec<u64> {
pyrowave::capture_modifiers(fourcc)
}
@@ -708,7 +714,7 @@ pub(crate) fn pyrowave_capture_modifiers(fourcc: u32) -> Vec<u64> {
/// passthrough for VAAPI vs the EGL→CUDA import for NVENC).
#[cfg(target_os = "linux")]
pub fn linux_zero_copy_is_vaapi() -> bool {
match crate::config::config().encoder_pref.as_str() {
match pf_host_config::config().encoder_pref.as_str() {
"nvenc" | "nvidia" | "cuda" => false,
"vaapi" | "amd" | "intel" => true,
// PyroWave ingests the raw capture dmabuf itself (Vulkan import + compute CSC) on ANY
@@ -788,7 +794,7 @@ pub fn can_encode_444(codec: Codec) -> bool {
// Cached per selected GPU (was a process-lifetime OnceLock): a web-console preference change
// re-probes on the newly selected adapter before the next Welcome.
static CACHE: OnceLock<Mutex<HashMap<String, bool>>> = OnceLock::new();
let key = crate::gpu::selection_key();
let key = pf_gpu::selection_key();
let cache = CACHE.get_or_init(|| Mutex::new(HashMap::new()));
if let Some(v) = cache.lock().unwrap().get(&key) {
return *v;
@@ -870,7 +876,7 @@ pub fn can_encode_10bit(codec: Codec) -> bool {
// Cached per (selected GPU, codec) — a web-console preference change re-probes on the newly
// selected adapter before the next Welcome, mirroring `can_encode_444`.
static CACHE: OnceLock<Mutex<HashMap<(String, &'static str), bool>>> = OnceLock::new();
let key = (crate::gpu::selection_key(), codec.label());
let key = (pf_gpu::selection_key(), codec.label());
let cache = CACHE.get_or_init(|| Mutex::new(HashMap::new()));
if let Some(v) = cache.lock().unwrap().get(&key) {
return *v;
@@ -918,13 +924,13 @@ pub fn can_encode_10bit(_codec: Codec) -> bool {
// ---------------------------------------------------------------------------------------------
// Windows backend selection (the analogue of the Linux nvidia_present / linux_zero_copy_is_vaapi
// logic). NVIDIA → NVENC, AMD → AMF, Intel → QSV; `auto` (default) reads the vendor of the
// SELECTED render adapter (crate::gpu — web-console preference / env pin / max VRAM), so the
// SELECTED render adapter (pf_gpu — web-console preference / env pin / max VRAM), so the
// backend always matches the GPU the capture ring and virtual display sit on.
// ---------------------------------------------------------------------------------------------
#[cfg(target_os = "windows")]
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum WindowsBackend {
pub enum WindowsBackend {
Nvenc,
Amf,
Qsv,
@@ -943,9 +949,9 @@ enum GpuVendor {
/// render adapter's vendor). Shared by [`open_video`] and the GameStream codec advertisement so
/// both agree.
#[cfg(target_os = "windows")]
pub(crate) fn windows_resolved_backend() -> WindowsBackend {
pub fn windows_resolved_backend() -> WindowsBackend {
// Resolved ONCE in HostConfig (Goal-1) — was re-read from PUNKTFUNK_ENCODER on every call.
match crate::config::config().encoder_pref.as_str() {
match pf_host_config::config().encoder_pref.as_str() {
"nvenc" | "hw" | "nvidia" | "cuda" => WindowsBackend::Nvenc,
"amf" | "amd" => WindowsBackend::Amf,
"qsv" | "intel" => WindowsBackend::Qsv,
@@ -961,35 +967,35 @@ pub(crate) fn windows_resolved_backend() -> WindowsBackend {
/// True if the session's resolved encode backend produces GPU-resident frames (so the capturer should
/// hand GPU surfaces straight through rather than CPU-stage them) — only the GPU-less software encoder
/// wants CPU staging. This is the single source for [`crate::capture::OutputFormat`]'s `gpu` bit:
/// wants CPU staging. This is the single source for [`pf_frame::OutputFormat`]'s `gpu` bit:
/// resolving it in `encode` and threading it *into* the capturer (rather than having `capture` re-derive
/// the backend) keeps the capture→encode dependency one-way, so the two can never disagree on whether
/// frames are GPU-resident (plan §2.4 / §W4).
#[cfg(target_os = "windows")]
pub(crate) fn resolved_backend_is_gpu() -> bool {
pub fn resolved_backend_is_gpu() -> bool {
!matches!(windows_resolved_backend(), WindowsBackend::Software)
}
/// Linux/other: every backend but the GPU-less software encoder (openh264) is GPU-resident. Config-backed
/// (mirrors `session_plan::resolve_encoder`; the NVENC vs VAAPI split is auto-detected in [`open_video`]).
#[cfg(not(target_os = "windows"))]
pub(crate) fn resolved_backend_is_gpu() -> bool {
pub fn resolved_backend_is_gpu() -> bool {
!matches!(
crate::config::config().encoder_pref.as_str(),
pf_host_config::config().encoder_pref.as_str(),
"software" | "sw" | "openh264"
)
}
/// True if the resolved encode backend can ingest a full-chroma (RGB) source and CSC it to 4:4:4 itself —
/// the *encoder* half of the 4:4:4 capture gate ([`crate::capture::capturer_supports_444`]). Only Windows
/// the *encoder* half of the 4:4:4 capture gate (the host capture `capturer_supports_444`). Only Windows
/// direct-NVENC does (measured on-glass: ARGB + `chromaFormatIDC=3` → true 4:4:4); AMF/QSV can't. On Linux
/// the 4:4:4 source is the capturer's own (portal RGB → `yuv444p`), independent of the auto-detected
/// backend, so the gate never consults this there.
#[cfg(target_os = "windows")]
pub(crate) fn resolved_backend_ingests_rgb_444() -> bool {
pub fn resolved_backend_ingests_rgb_444() -> bool {
windows_resolved_backend() == WindowsBackend::Nvenc
}
#[cfg(not(target_os = "windows"))]
pub(crate) fn resolved_backend_ingests_rgb_444() -> bool {
pub fn resolved_backend_ingests_rgb_444() -> bool {
false
}
@@ -1007,7 +1013,7 @@ pub fn windows_backend_is_probed() -> bool {
}
}
/// Detect the encode-GPU vendor from the **selected render adapter** ([`crate::gpu::selected_gpu`]:
/// Detect the encode-GPU vendor from the **selected render adapter** ([`pf_gpu::selected_gpu`]:
/// web-console preference > `PUNKTFUNK_RENDER_ADAPTER` > max VRAM) — the same adapter the capture
/// ring and the IddCx render pin sit on, so the encoder backend can never disagree with where the
/// captured frames live. The old first-DXGI-adapter scan did exactly that on hybrid boxes: adapter
@@ -1019,18 +1025,15 @@ pub fn windows_backend_is_probed() -> bool {
fn windows_gpu_vendor() -> Option<GpuVendor> {
fn by_id(vendor_id: u32) -> Option<GpuVendor> {
match vendor_id {
crate::gpu::VENDOR_NVIDIA => Some(GpuVendor::Nvidia),
crate::gpu::VENDOR_AMD => Some(GpuVendor::Amd),
crate::gpu::VENDOR_INTEL => Some(GpuVendor::Intel),
pf_gpu::VENDOR_NVIDIA => Some(GpuVendor::Nvidia),
pf_gpu::VENDOR_AMD => Some(GpuVendor::Amd),
pf_gpu::VENDOR_INTEL => Some(GpuVendor::Intel),
_ => None,
}
}
let sel = crate::gpu::selected_gpu()?;
by_id(sel.info.vendor_id).or_else(|| {
crate::gpu::enumerate()
.iter()
.find_map(|g| by_id(g.vendor_id))
})
let sel = pf_gpu::selected_gpu()?;
by_id(sel.info.vendor_id)
.or_else(|| pf_gpu::enumerate().iter().find_map(|g| by_id(g.vendor_id)))
}
/// Probe the active Windows AMF/QSV backend for its encodable codecs (cached **per (backend,
@@ -1050,7 +1053,7 @@ pub fn windows_codec_support() -> CodecSupport {
use std::sync::{Mutex, OnceLock};
static CACHE: OnceLock<Mutex<HashMap<String, CodecSupport>>> = OnceLock::new();
let backend = windows_resolved_backend();
let key = format!("{backend:?}:{}", crate::gpu::selection_key());
let key = format!("{backend:?}:{}", pf_gpu::selection_key());
let cache = CACHE.get_or_init(|| Mutex::new(HashMap::new()));
if let Some(c) = cache.lock().unwrap().get(&key) {
return *c;
@@ -1098,7 +1101,7 @@ pub fn windows_codec_support() -> CodecSupport {
/// degrading a live sibling's encode. NVENC is the only backend with hard session caps today
/// (GeForce consumer limit); AMF/QSV equivalents follow the same seam when they grow accounting.
#[cfg(target_os = "windows")]
pub(crate) fn can_open_another_session() -> bool {
pub fn can_open_another_session() -> bool {
#[cfg(feature = "nvenc")]
{
nvenc::can_open_another_session()
@@ -1111,62 +1114,65 @@ pub(crate) fn can_open_another_session() -> bool {
// Goal-1 stage 6: GPU/CPU encoders confined to `encode/windows/` (NVENC, native AMF, AMF/QSV
// ffmpeg, software) and `encode/linux/` (NVENC/CUDA + VAAPI); `#[path]` keeps the
// `crate::encode::*` module names flat.
// `crate::*` module names flat.
// Native AMF (direct SDK, design/native-amf-encoder.md): compiled unconditionally on Windows —
// no build feature, the driver-installed amfrt64.dll resolves at runtime like NVENC's DLL.
#[cfg(target_os = "windows")]
#[path = "encode/windows/amf.rs"]
#[path = "enc/windows/amf.rs"]
mod amf;
#[cfg(all(target_os = "windows", feature = "amf-qsv"))]
#[path = "encode/windows/ffmpeg_win.rs"]
#[path = "enc/windows/ffmpeg_win.rs"]
mod ffmpeg_win;
#[cfg(target_os = "linux")]
#[path = "enc/linux/mod.rs"]
mod linux;
// Direct-SDK NVENC on Linux (CUDA input; design/linux-direct-nvenc.md) — real RFI + recovery anchor
// + reset() lever the libavcodec `linux::NvencEncoder` can't express. Opt-in behind
// `PUNKTFUNK_NVENC_DIRECT` until on-glass validated; the `.so` resolves at runtime like the Windows
// path, so `--features nvenc` stays safe on a driver-less/AMD Linux box.
#[cfg(all(target_os = "windows", feature = "nvenc"))]
#[path = "encode/windows/nvenc.rs"]
#[path = "enc/windows/nvenc.rs"]
mod nvenc;
#[cfg(all(target_os = "linux", feature = "nvenc"))]
#[path = "encode/linux/nvenc_cuda.rs"]
#[path = "enc/linux/nvenc_cuda.rs"]
mod nvenc_cuda;
// Actionable `NVENCSTATUS` → cause mapping shared by both direct-NVENC backends, so a failed
// session open logs "update/reboot the driver" instead of the old misleading "(no NVIDIA GPU?)".
#[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "nvenc"))]
#[path = "encode/nvenc_status.rs"]
#[path = "enc/nvenc_status.rs"]
mod nvenc_status;
// Platform-agnostic direct-SDK NVENC glue (`NvStatusExt`/`nv_ok`, `codec_guid`) shared by both
// `nvEncodeAPI` backends — the byte-identical Tier-2 leaves (plan §2.2). Sibling of `nvenc_status`.
#[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "nvenc"))]
#[path = "encode/nvenc_core.rs"]
#[path = "enc/nvenc_core.rs"]
mod nvenc_core;
// Shared libavcodec glue (`pixel_to_av`, swscale consts) for the three libav backends — Linux
// NVENC + VAAPI and Windows AMF/QSV — so the byte-identical pieces live once (plan §2.2, Tier 2).
#[cfg(any(target_os = "linux", all(target_os = "windows", feature = "amf-qsv")))]
#[path = "enc/libav.rs"]
mod libav;
// Software (openh264) H.264 encoder — the GPU-less path on BOTH Windows and Linux (a headless /
// GPU-less test box, or a fallback when no hardware encoder is available). Platform-agnostic: it
// consumes CPU RGB `CapturedFrame`s and the statically-bundled openh264 build.
#[cfg(any(target_os = "windows", target_os = "linux"))]
#[path = "enc/sw.rs"]
mod sw;
#[cfg(target_os = "linux")]
#[path = "encode/linux/vaapi.rs"]
#[path = "enc/linux/vaapi.rs"]
mod vaapi;
// Raw Vulkan Video HEVC encode on Linux (AMD/Intel; design/linux-vulkan-video-encode.md) — real RFI
// via explicit DPB reference slots (the app owns the DPB), the open-stack twin of the direct-NVENC
// path. Does an on-GPU RGB→NV12 compute CSC since capture delivers packed-RGB dmabufs. Opt-in behind
// `PUNKTFUNK_VULKAN_ENCODE` until on-glass validated; needs `--features vulkan-encode`.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[path = "encode/linux/vulkan_video.rs"]
#[path = "enc/linux/vulkan_video.rs"]
mod vulkan_video;
// Vendored `VK_KHR_video_encode_av1` bindings (host-only) — the AV1 encode structs our pinned
// `ash 0.38.0+1.3.281` predates (finalized Vulkan 1.3.290). Copied verbatim from ash-master's
// generated code rather than bumping `ash` (which breaks the SDL/Vulkan client). Consumed by
// `vulkan_video.rs` via `super::vk_av1_encode`.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[path = "encode/linux/vk_av1_encode.rs"]
#[path = "enc/linux/vk_av1_encode.rs"]
mod vk_av1_encode;
// Small ash leaf helpers shared by the Linux Vulkan encode backends (dmabuf import, image/memory
// utilities) — extracted from `vulkan_video.rs` when the PyroWave backend arrived.
@@ -1174,13 +1180,13 @@ mod vk_av1_encode;
target_os = "linux",
any(feature = "vulkan-encode", feature = "pyrowave")
))]
#[path = "encode/linux/vk_util.rs"]
#[path = "enc/linux/vk_util.rs"]
mod vk_util;
// PyroWave — the opt-in wired-LAN intra-only wavelet codec (design/pyrowave-codec-plan.md §4.3):
// pure Vulkan compute via the vendored `pyrowave-sys`, sub-ms encode, every frame a keyframe.
// Explicit-only behind PUNKTFUNK_ENCODER=pyrowave; EXPERIMENTAL until CODEC_PYROWAVE lands.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[path = "encode/linux/pyrowave.rs"]
#[path = "enc/linux/pyrowave.rs"]
mod pyrowave;
#[cfg(test)]
+37
View File
@@ -0,0 +1,37 @@
# The shared media-pipeline vocabulary (plan §W6): the captured-frame types and pixel formats that
# both capture (producer) and encode (consumer) speak, plus the small pure helpers that ride the
# same seam — HDR static metadata, the metronomic-stall detector, per-thread scheduling QoS, and
# (Windows) the DXGI capture identity + D3D11 device creation. A leaf so pf-capture and pf-encode
# can depend on the vocabulary WITHOUT depending on each other.
[package]
name = "pf-frame"
version.workspace = true
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
description = "punktfunk host shared frame/format vocabulary: CapturedFrame, PixelFormat, HDR metadata, thread QoS, and the Windows DXGI capture identity."
publish = false
[dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
anyhow = "1"
tracing = "0.1"
[target.'cfg(target_os = "linux")'.dependencies]
# `FramePayload::Cuda` owns a zero-copy `DeviceBuffer`; `libc` for the per-thread `setpriority`.
pf-zerocopy = { path = "../pf-zerocopy" }
libc = "0.2"
[target.'cfg(target_os = "windows")'.dependencies]
# The DXGI capture identity (`WinCaptureTarget`/`D3d11Frame`/`pack_luid`/`make_device`) + the GPU
# scheduling-priority hardening `make_device` applies, and the thread-QoS `SetThreadPriority`.
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_Security",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_System_LibraryLoader",
"Win32_System_Threading",
] }
+222
View File
@@ -0,0 +1,222 @@
//! The Windows DXGI capture identity + shared D3D11 device creation (plan §W6): the capture
//! target descriptor ([`WinCaptureTarget`]), the GPU-resident captured texture ([`D3d11Frame`]),
//! the adapter-LUID packer ([`pack_luid`]), and [`make_device`] — a fresh D3D11 device/context on
//! a chosen adapter, applying the process GPU scheduling-priority hardening. Extracted from the
//! host's `capture/windows/dxgi.rs` so the capture IDD-push path, the encode D3D11 backends, and
//! pf-vdisplay all share ONE identity type + device factory (no capture↔encode↔vdisplay cycle).
//! The win32u GPU-preference hook, the HDR/video-engine converters, and the self-tests stay in the
//! capture crate — they are capture mechanics, not shared identity.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{Context, Result};
use windows::core::Interface;
use windows::Win32::Foundation::{HMODULE, LUID};
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_UNKNOWN, D3D_FEATURE_LEVEL_11_0};
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, ID3D11Device, ID3D11DeviceContext, ID3D11Texture2D,
D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_SDK_VERSION,
};
use windows::Win32::Graphics::Dxgi::{IDXGIAdapter1, IDXGIDevice, IDXGIDevice1};
#[derive(Clone)]
pub struct WinCaptureTarget {
/// Packed DXGI adapter LUID (`(HighPart << 32) | (LowPart & 0xffff_ffff)`).
pub adapter_luid: i64,
/// The output's GDI device name, e.g. `\\.\DISPLAY3`. Can CHANGE across a secure-desktop switch.
pub gdi_name: String,
/// Stable virtual-display (IddCx) target id — re-resolved to the current GDI name on every recovery.
pub target_id: u32,
/// The pf-vdisplay driver's WUDFHost pid (from the ADD reply) — the process the IDD-push capturer
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32,
}
/// A GPU-resident captured texture (future NVENC-D3D11 zero-copy path).
pub struct D3d11Frame {
pub texture: ID3D11Texture2D,
pub device: ID3D11Device,
}
// SAFETY: `D3d11Frame` owns an `ID3D11Texture2D` + `ID3D11Device`, which are COM interface pointers.
// D3D11 devices/resources use thread-safe (interlocked) COM reference counting, and the device is
// created free-threaded (`make_device` passes no `D3D11_CREATE_DEVICE_SINGLETHREADED`), so handing
// ownership of the frame to another thread — the capture→encode handoff — and releasing it there is
// sound. The value is moved, never aliased (no `Sync`), so there is no concurrent use of the
// single-threaded immediate context.
unsafe impl Send for D3d11Frame {}
pub fn pack_luid(luid: LUID) -> i64 {
((luid.HighPart as i64) << 32) | (luid.LowPart as i64 & 0xffff_ffff)
}
/// Create a fresh D3D11 device + context on a specific adapter (driver_type UNKNOWN with an explicit
/// adapter). Used at open and on every ACCESS_LOST: a device created on one desktop cannot sustain a
/// duplication on a *different* desktop (perpetual ACCESS_LOST), so the secure-desktop switch needs a
/// device made while the thread is attached to that desktop.
///
/// # Safety
/// `adapter` must be a live `IDXGIAdapter1` for the duration of the call. The fn calls the D3D11 /
/// DXGI FFI (`D3D11CreateDevice`, GPU scheduling-priority hardening) but forms no lasting alias to
/// `adapter`; the returned device/context are the sole owners of the new COM objects.
pub unsafe fn make_device(adapter: &IDXGIAdapter1) -> Result<(ID3D11Device, ID3D11DeviceContext)> {
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice")?;
let device = device.context("null D3D11 device")?;
let context = context.context("null D3D11 context")?;
// GPU scheduling hardening — the same approach Sunshine/Apollo use, reimplemented here via the
// documented D3DKMT/DXGI APIs (no GPL source copied). Our capture+encode
// shares the GPU with the streamed game; when the game saturates the GPU our process is starved of
// GPU time slices, so NVENC sits near-idle yet `lock_bitstream` waits ~20 ms for our context to be
// scheduled — capping the stream (~47 fps measured at 5K@240) and stuttering. Per-frame copy/convert
// is NOT the cause (zero-copy + thread-priority alone didn't move it); the PROCESS-level GPU
// scheduling priority class is the decisive cross-process lever. Secondary: the absolute per-device
// GPU thread priority and a 1-frame latency cap.
elevate_process_gpu_priority();
if let Ok(dxgi_dev) = device.cast::<IDXGIDevice>() {
// The absolute max GPU thread priority (0x4000001E; the same value Sunshine/Apollo use); fall back to relative +7.
if dxgi_dev.SetGPUThreadPriority(0x4000_001E).is_err()
&& dxgi_dev.SetGPUThreadPriority(7).is_err()
{
tracing::warn!("SetGPUThreadPriority failed (run as admin/SYSTEM for GPU priority)");
}
}
if let Ok(dxgi1) = device.cast::<IDXGIDevice1>() {
let _ = dxgi1.SetMaximumFrameLatency(1);
}
Ok((device, context))
}
/// Resolve the configured GPU scheduling-priority class from `PUNKTFUNK_GPU_PRIORITY_CLASS`
/// (`off|normal|high|realtime`, default high). `None` = leave it at the OS default (the `off` opt-out).
/// D3DKMT_SCHEDULINGPRIORITYCLASS: IDLE 0, BELOW_NORMAL 1, NORMAL 2, ABOVE_NORMAL 3, HIGH 4, REALTIME 5.
fn configured_gpu_priority_class() -> Option<i32> {
match std::env::var("PUNKTFUNK_GPU_PRIORITY_CLASS")
.ok()
.as_deref()
{
Some("off") => None,
Some("normal") => Some(2),
Some("realtime") => Some(5),
_ => Some(4), // HIGH — safe on NVIDIA+HAGS (realtime can freeze NVENC)
}
}
/// Enable SE_INC_BASE_PRIORITY on the CURRENT process token (best-effort) — the kernel gates the
/// HIGH/REALTIME GPU scheduling-priority bump on it. Held by SYSTEM/Administrators; a UAC-FILTERED
/// token does NOT have it, which is why `elevate_process_gpu_priority` may silently no-op in a
/// restricted service context.
unsafe fn enable_inc_base_priority() {
use windows::core::PCWSTR;
use windows::Win32::Foundation::{CloseHandle, HANDLE, LUID};
use windows::Win32::Security::{
AdjustTokenPrivileges, LookupPrivilegeValueW, LUID_AND_ATTRIBUTES,
SE_INC_BASE_PRIORITY_NAME, SE_PRIVILEGE_ENABLED, TOKEN_ADJUST_PRIVILEGES, TOKEN_PRIVILEGES,
TOKEN_QUERY,
};
use windows::Win32::System::Threading::{GetCurrentProcess, OpenProcessToken};
let mut token = HANDLE::default();
if OpenProcessToken(
GetCurrentProcess(),
TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY,
&mut token,
)
.is_ok()
{
let mut luid = LUID::default();
if LookupPrivilegeValueW(PCWSTR::null(), SE_INC_BASE_PRIORITY_NAME, &mut luid).is_ok() {
let tp = TOKEN_PRIVILEGES {
PrivilegeCount: 1,
Privileges: [LUID_AND_ATTRIBUTES {
Luid: luid,
Attributes: SE_PRIVILEGE_ENABLED,
}],
};
if AdjustTokenPrivileges(
token,
false,
Some(&tp as *const TOKEN_PRIVILEGES),
0,
None,
None,
)
.is_err()
{
tracing::warn!("could not enable SE_INC_BASE_PRIORITY for GPU priority");
}
}
let _ = CloseHandle(token);
}
}
/// Call `gdi32!D3DKMTSetProcessSchedulingPriorityClass(process, prio)` (no stable windows-rs binding —
/// loaded by name). Returns the NTSTATUS (0 = success) or `None` if the export can't be resolved. The
/// CALLING process must hold SE_INC_BASE_PRIORITY ([`enable_inc_base_priority`]) for HIGH/REALTIME; the
/// kernel checks the caller's privilege whether the target is self or a child we created.
unsafe fn d3dkmt_set_scheduling_priority_class(
process: windows::Win32::Foundation::HANDLE,
prio: i32,
) -> Option<i32> {
use windows::core::s;
use windows::Win32::Foundation::HANDLE;
use windows::Win32::System::LibraryLoader::{GetProcAddress, LoadLibraryA};
let gdi32 = LoadLibraryA(s!("gdi32.dll")).ok()?;
let p = GetProcAddress(gdi32, s!("D3DKMTSetProcessSchedulingPriorityClass"))?;
type SetPrio = unsafe extern "system" fn(HANDLE, i32) -> i32;
let f: SetPrio = std::mem::transmute(p);
Some(f(process, prio))
}
/// GPU scheduling-priority hardening — the same approach as Sunshine/Apollo, independently
/// implemented via the documented D3DKMT APIs (no GPL source copied). On a
/// GPU-saturated game our capture+encode process is starved of GPU time slices — NVENC sits ~idle but
/// `lock_bitstream` waits ~20 ms for our context to be scheduled. Elevating the PROCESS GPU scheduling
/// priority class (the strong cross-process lever — far more effective than `SetGPUThreadPriority`
/// alone, which we measured as no help) lets our brief encode preempt the game. Uses HIGH, NOT
/// realtime: realtime on NVIDIA + HAGS can freeze/crash NVENC (Apollo downgrades it for exactly this).
/// Runs once per process; best-effort. `PUNKTFUNK_GPU_PRIORITY_CLASS = off|normal|high|realtime`
/// (default high). Best-effort: silently no-ops under a UAC-filtered token (the process will not
/// hold SE_INC_BASE_PRIORITY, so the D3DKMT call is a no-op).
fn elevate_process_gpu_priority() {
use std::sync::Once;
static ONCE: Once = Once::new();
// SAFETY: the closure calls two of this module's `unsafe fn`s — `enable_inc_base_priority`
// (adjusts the current-process token; it has no caller precondition and builds all its FFI args
// locally) and `d3dkmt_set_scheduling_priority_class` (loads gdi32 by name and calls the export).
// The latter requires `process` to be a valid process handle; `GetCurrentProcess()` returns the
// current-process pseudo-handle, which is always valid and needs no close. Runs once via
// `Once::call_once`; no raw pointers are dereferenced here.
ONCE.call_once(|| unsafe {
use windows::Win32::System::Threading::GetCurrentProcess;
let Some(prio) = configured_gpu_priority_class() else {
tracing::info!("GPU process scheduling priority class left at default (off)");
return;
};
enable_inc_base_priority();
match d3dkmt_set_scheduling_priority_class(GetCurrentProcess(), prio) {
Some(0) => tracing::info!(
priority_class = prio,
"GPU process scheduling priority class set (2=normal 4=high 5=realtime)"
),
Some(st) => tracing::warn!(
status = format!("0x{st:08X}"),
"D3DKMTSetProcessSchedulingPriorityClass failed (run as admin/SYSTEM for GPU priority)"
),
None => tracing::warn!("D3DKMTSetProcessSchedulingPriorityClass export not found"),
}
});
}
+230
View File
@@ -0,0 +1,230 @@
//! The shared media-pipeline vocabulary (plan §W6): the frame + pixel-format types that capture
//! (producer) and encode (consumer) both speak, extracted into a leaf crate so `pf-capture` and
//! `pf-encode` depend on the vocabulary WITHOUT depending on each other. The GPU payloads pull
//! their heavy backends in from below: `FramePayload::Cuda` owns a [`pf_zerocopy::DeviceBuffer`],
//! `FramePayload::D3d11` a [`dxgi::D3d11Frame`].
//!
//! Alongside the vocabulary live the small pure helpers that ride the same capture-encode seam:
//! [`hdr`] (HDR static metadata / in-band SEI), [`metronome`] (the metronomic-stall detector),
//! [`thread_qos`] (per-thread scheduling QoS), [`session_tuning`] (Windows process session
//! tuning), and — on Windows — [`dxgi`] (the capture identity + D3D11 device creation).
// Unsafe-proof program: every `unsafe {}` / `unsafe impl` must carry a `// SAFETY:` proof.
#![deny(clippy::undocumented_unsafe_blocks)]
pub mod hdr;
pub mod metronome;
pub mod session_tuning;
pub mod thread_qos;
// The Windows DXGI capture identity + shared D3D11 device creation (plan §W6). Consumed by the
// capture IDD-push path, the encode D3D11 backends, and pf-vdisplay's `WinCaptureTarget`.
#[cfg(target_os = "windows")]
pub mod dxgi;
/// Packed pixel layout of a [`CapturedFrame`]. The ScreenCast portal negotiates the
/// format; on wlroots it is commonly packed `RGB` (3 bytes/pixel). The encoder maps these
/// to an NVENC-accepted input format (`rgb0`/`bgr0`/`rgba`/`bgra`), expanding 3→4 bytes
/// where needed — no host-side colour conversion.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum PixelFormat {
/// `[B,G,R,x]`, 4 bpp.
Bgrx,
/// `[R,G,B,x]`, 4 bpp.
Rgbx,
/// `[B,G,R,A]`, 4 bpp.
Bgra,
/// `[R,G,B,A]`, 4 bpp.
Rgba,
/// `[R,G,B]`, 3 bpp.
Rgb,
/// `[B,G,R]`, 3 bpp.
Bgr,
/// 10-bit RGB packed as `R10G10B10A2` (DXGI `R10G10B10A2_UNORM`), 4 bpp. The HDR capture path
/// produces this: scRGB FP16 desktop pixels are converted to BT.2020 PQ and written here, then
/// handed to NVENC as `ABGR10` for an HEVC Main10 / HDR10 encode.
Rgb10a2,
/// `NV12` (DXGI `NV12`): 8-bit BT.709 limited-range YUV 4:2:0. Produced by the D3D11 **video
/// processor** (video engine, not the 3D engine) so the per-frame colour conversion doesn't fight a
/// GPU-saturating game; handed to NVENC as `NV12` (it encodes YUV natively — no internal RGB→YUV).
Nv12,
/// `P010` (DXGI `P010`): 10-bit BT.2020 PQ limited-range YUV 4:2:0. HDR analogue of [`Nv12`]:
/// video-processor output for HEVC Main10 / HDR10, handed to NVENC as `YUV420_10BIT`.
P010,
/// Planar 8-bit YUV **4:4:4** (BT.709; range per `PUNKTFUNK_444_FULLRANGE`). Produced by the
/// Linux zero-copy worker's GPU convert for a 4:4:4 session ([`FramePayload::Cuda`] with
/// `DeviceBuffer::yuv444` — three full-res planes stacked in one allocation); NVENC encodes
/// it natively under the Range-Extensions profile. Never a CPU payload.
Yuv444,
}
impl PixelFormat {
pub fn bytes_per_pixel(self) -> usize {
match self {
PixelFormat::Rgb | PixelFormat::Bgr => 3,
// Three full-res 1-byte planes (GPU-resident only; no CPU payload carries this).
PixelFormat::Yuv444 => 3,
_ => 4,
}
}
}
/// DRM FourCC for a packed 32-bit format name (little-endian, e.g. `b"XR24"`).
#[cfg(target_os = "linux")]
const fn drm_fourcc_code(c: &[u8; 4]) -> u32 {
(c[0] as u32) | ((c[1] as u32) << 8) | ((c[2] as u32) << 16) | ((c[3] as u32) << 24)
}
/// Map a SPA/our [`PixelFormat`] to the DRM FourCC EGL expects for import. SPA byte order `BGRx`
/// ⇒ DRM `XRGB8888` (memory B,G,R,X), etc. Lives with the frame vocabulary (not in
/// `pf-zerocopy`) because it consumes [`PixelFormat`], which sits above that crate.
#[cfg(target_os = "linux")]
pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
use PixelFormat::*;
Some(match format {
Bgrx => drm_fourcc_code(b"XR24"), // DRM_FORMAT_XRGB8888
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format.
Rgb | Bgr | Rgb10a2 | Nv12 | P010 | Yuv444 => return None,
})
}
/// What a Windows capturer should produce, resolved **once** per session and passed **into**
/// `capture_virtual_output` (Goal-1 stage 5, plan §2.3/§5). Passing the format in is what lets a
/// capturer stop re-deriving the encode backend itself — it kills the
/// `capture/dxgi.rs → encode::windows_resolved_backend()` back-reference (the highest-severity coupling:
/// capture and encode could otherwise disagree on whether frames are GPU-resident). Neutral type; the
/// Linux portal capturer ignores it (it negotiates its own format with PipeWire).
#[derive(Clone, Copy, Debug)]
pub struct OutputFormat {
/// Produce GPU-resident D3D11 frames (zero-copy for a GPU encoder — NVENC/AMF/QSV) rather than CPU
/// staging. `false` **only** for the GPU-less software encoder.
pub gpu: bool,
/// HDR: the capturer converts to 10-bit (IDD-push FP16 → `P010`, or `Rgb10a2` for a 4:4:4 source).
/// `false` = 8-bit SDR.
pub hdr: bool,
/// Full-chroma 4:4:4 session: the capturer must keep full chroma. On Windows the IDD-push
/// capturer hands the **BGRA** slot through (skipping the subsampling BGRA→NV12
/// VideoConverter) so NVENC ingests full-chroma RGB and CSCs to 4:4:4 itself — measured
/// on-glass (RTX 5070 Ti): ARGB + `chromaFormatIDC=3` yields TRUE 4:4:4 and the conversion
/// follows the configured VUI matrix (BT.709 limited since the VUI is always written). On
/// Linux it forces the CPU RGB path the encoder swscales to `YUV444P`. `false` on every
/// 4:2:0 session.
pub chroma_444: bool,
}
impl OutputFormat {
/// Resolve the output format for an entry point that doesn't build a full [`SessionPlan`]
/// (`crate::session_plan`) — the GameStream + spike paths. `gpu` is the encoder's GPU-residency,
/// resolved by the caller via `pf_encode::resolved_backend_is_gpu` and passed **in** (capture
/// never re-derives the backend — the one-way capture→encode edge, plan §2.4 / §W4); `hdr` as given.
/// The native punktfunk/1 path uses `SessionPlan::output_format()` instead (it already resolved the
/// encoder), so neither path makes a capturer re-derive it.
pub fn resolve(hdr: bool, gpu: bool) -> Self {
OutputFormat {
gpu,
hdr,
// The GameStream + spike paths are always 4:2:0 (4:4:4 is punktfunk/1-native only).
chroma_444: false,
}
}
}
/// A mouse-cursor overlay to composite onto a frame at encode time (cursor-as-metadata). Rides on
/// [`CapturedFrame::cursor`] for the GPU zero-copy payloads (Cuda/Dmabuf), whose pixels never touch
/// the CPU — the encoder blends this small bitmap into its owned surface (Vulkan CSC image / CUDA
/// devbuf / VA surface). The CPU de-pad path composites the cursor inline instead, so it leaves
/// this `None`. `rgba` is `Arc` so attaching the (unchanged) bitmap to every frame is a refcount
/// bump, not a copy; `serial` bumps only when the bitmap image changes, so the encoder re-uploads
/// its small GPU texture on change and just moves a push-constant otherwise.
#[derive(Clone)]
pub struct CursorOverlay {
/// Top-left in frame pixels where the bitmap is drawn (already = reported position hotspot).
pub x: i32,
pub y: i32,
pub w: u32,
pub h: u32,
/// Straight-alpha RGBA pixels, `w*h*4` (bytes R,G,B,A).
pub rgba: std::sync::Arc<Vec<u8>>,
/// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves.
pub serial: u64,
}
/// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of
/// where they live — [`payload`](Self::payload) is either a CPU buffer (the spike/fallback path)
/// or a GPU buffer already on the device (the zero-copy path, plan §9).
pub struct CapturedFrame {
pub width: u32,
pub height: u32,
pub pts_ns: u64,
/// Pixel layout of the payload.
pub format: PixelFormat,
pub payload: FramePayload,
/// Cursor overlay to blend at encode time (GPU zero-copy payloads only); `None` when there's no
/// visible cursor or the pixels were already composited on the CPU de-pad path. See
/// [`CursorOverlay`].
pub cursor: Option<CursorOverlay>,
}
/// A captured frame still living in a single-plane packed-RGB dmabuf (the VAAPI zero-copy path).
/// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be
/// imported into a VA surface there without the compositor's buffer being closed underneath it.
/// (Content stability across the brief import window relies on the compositor's buffer pool depth,
/// same as any zero-copy capture — the VAAPI importer copies into its own NV12 surface promptly.)
#[cfg(target_os = "linux")]
pub struct DmabufFrame {
pub fd: std::os::fd::OwnedFd,
/// DRM FourCC of the packed-RGB plane (e.g. `XR24` for BGRx).
pub fourcc: u32,
/// DRM format modifier the compositor allocated (0 = LINEAR).
pub modifier: u64,
pub offset: u32,
pub stride: u32,
}
/// Where a captured frame's pixels live.
pub enum FramePayload {
/// Tightly-packed CPU pixels in `format`, `width*height*bytes_per_pixel` (no row padding).
Cpu(Vec<u8>),
/// A pitched GPU buffer (BGRA-order, on the shared CUDA context) — the NVIDIA zero-copy path.
/// The dmabuf has already been imported + copied into this owned device buffer.
#[cfg(target_os = "linux")]
Cuda(pf_zerocopy::DeviceBuffer),
/// A raw packed-RGB dmabuf — the AMD/Intel (VAAPI) zero-copy path. The encoder imports it into
/// a VA surface and does RGB→NV12 on the GPU video engine (no host CSC, no upload).
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame.
#[cfg(target_os = "windows")]
D3d11(dxgi::D3d11Frame),
}
impl CapturedFrame {
/// True if the frame's pixels are a GPU/CUDA buffer (the NVIDIA zero-copy path).
pub fn is_cuda(&self) -> bool {
#[cfg(target_os = "linux")]
{
matches!(self.payload, FramePayload::Cuda(_))
}
#[cfg(not(target_os = "linux"))]
{
false
}
}
/// True if the frame is a raw dmabuf (the VAAPI zero-copy path).
pub fn is_dmabuf(&self) -> bool {
#[cfg(target_os = "linux")]
{
matches!(self.payload, FramePayload::Dmabuf(_))
}
#[cfg(not(target_os = "linux"))]
{
false
}
}
}
@@ -17,7 +17,8 @@ use std::time::{Duration, Instant};
/// the gaps between the last [`Self::STREAK`] events are all within ±[`Self::TOLERANCE`] of their
/// mean, [`Self::note`] returns the mean period for the caller to warn with, then stays quiet for
/// [`Self::REWARN`] while the cycle persists.
pub(crate) struct Metronome {
#[derive(Default)]
pub struct Metronome {
events: VecDeque<Instant>,
last_warn: Option<Instant>,
}
@@ -32,7 +33,7 @@ impl Metronome {
/// Once warned, re-warn at most this often while the cycle persists.
const REWARN: Duration = Duration::from_secs(30);
pub(crate) fn new() -> Self {
pub fn new() -> Self {
Self {
events: VecDeque::new(),
last_warn: None,
@@ -41,7 +42,7 @@ impl Metronome {
/// Record a disturbance at `now`; `Some(mean period)` exactly when the metronomic-cycle
/// warning should fire.
pub(crate) fn note(&mut self, now: Instant) -> Option<Duration> {
pub fn note(&mut self, now: Instant) -> Option<Duration> {
if self
.events
.back()
@@ -32,6 +32,21 @@ mod imp {
fn GetCurrentProcess() -> Handle;
fn SetPriorityClass(hProcess: Handle, dwPriorityClass: u32) -> Bool;
fn SetThreadExecutionState(esFlags: u32) -> u32;
fn PowerCreateRequest(Context: *const ReasonContext) -> Handle;
fn PowerSetRequest(PowerRequest: Handle, RequestType: i32) -> Bool;
fn PowerClearRequest(PowerRequest: Handle, RequestType: i32) -> Bool;
fn CloseHandle(hObject: Handle) -> Bool;
}
/// `REASON_CONTEXT` (minwinbase.h), simple-string flavour: `Version` (ULONG), `Flags` (DWORD),
/// then the union collapses to `SimpleReasonString` (LPWSTR) under
/// `POWER_REQUEST_CONTEXT_SIMPLE_STRING` — same size/alignment as the C layout (4+4, 8-aligned
/// pointer).
#[repr(C)]
struct ReasonContext {
version: u32,
flags: u32,
simple_reason: *const u16,
}
#[link(name = "dwmapi")]
extern "system" {
@@ -46,6 +61,61 @@ mod imp {
const ES_CONTINUOUS: u32 = 0x8000_0000;
const ES_SYSTEM_REQUIRED: u32 = 0x0000_0001;
const ES_DISPLAY_REQUIRED: u32 = 0x0000_0002;
const POWER_REQUEST_CONTEXT_VERSION: u32 = 0; // DIAGNOSTIC_REASON_VERSION
const POWER_REQUEST_CONTEXT_SIMPLE_STRING: u32 = 0x0000_0001;
const POWER_REQUEST_DISPLAY_REQUIRED: i32 = 0;
const POWER_REQUEST_SYSTEM_REQUIRED: i32 = 1;
const INVALID_HANDLE_VALUE: isize = -1;
/// RAII display+system availability request (`PowerRequestDisplayRequired`, visible in
/// `powercfg /requests`) — the service-grade "someone is watching this screen" assertion,
/// held for a capture session so the console cannot drop into display-off mid-stream. This is
/// object-lifetime (unlike the thread-bound `ES_*` flags in [`on_hot_thread`], which the OS
/// reverts at thread exit), so a capturer can hold it across whatever threads serve the
/// session. PREVENTION only: no power request turns an already-off display back ON — that
/// wake needs input, which is the virtual-mouse compose kick's job.
pub struct DisplayWakeRequest(Handle);
// SAFETY: the wrapped power-request HANDLE is a kernel object handle — a plain opaque value
// that any thread may use; this type never aliases it (set at new, cleared+closed at drop).
unsafe impl Send for DisplayWakeRequest {}
impl DisplayWakeRequest {
/// Create + set the request. `None` when the kernel refuses (best-effort — the caller
/// streams without the assertion, exactly the pre-existing behavior).
pub fn new() -> Option<DisplayWakeRequest> {
let reason: Vec<u16> = "punktfunk streaming session\0".encode_utf16().collect();
let ctx = ReasonContext {
version: POWER_REQUEST_CONTEXT_VERSION,
flags: POWER_REQUEST_CONTEXT_SIMPLE_STRING,
simple_reason: reason.as_ptr(),
};
// SAFETY: `ctx` (and the reason buffer it points into) outlives the call, which copies
// the string into the kernel object; the returned handle is owned here and released in
// Drop. PowerSetRequest takes the just-created handle + a plain enum value.
unsafe {
let h = PowerCreateRequest(&ctx);
if h.is_null() || h as isize == INVALID_HANDLE_VALUE {
return None;
}
PowerSetRequest(h, POWER_REQUEST_DISPLAY_REQUIRED);
PowerSetRequest(h, POWER_REQUEST_SYSTEM_REQUIRED);
Some(DisplayWakeRequest(h))
}
}
}
impl Drop for DisplayWakeRequest {
fn drop(&mut self) {
// SAFETY: `self.0` is the owned, still-open power-request handle (created in `new`,
// dropped exactly once); clear + close are plain handle calls.
unsafe {
PowerClearRequest(self.0, POWER_REQUEST_DISPLAY_REQUIRED);
PowerClearRequest(self.0, POWER_REQUEST_SYSTEM_REQUIRED);
CloseHandle(self.0);
}
}
}
static PROCESS_TUNED: OnceLock<()> = OnceLock::new();
@@ -94,7 +164,7 @@ mod imp {
}
#[cfg(target_os = "windows")]
pub use imp::on_hot_thread;
pub use imp::{on_hot_thread, DisplayWakeRequest};
/// No-op on non-Windows (Linux uses `setpriority` nice + CUDA stream priority instead — see
/// `native::boost_thread_priority` and `zerocopy::cuda`).
@@ -1,7 +1,7 @@
//! Per-thread OS scheduling QoS for the native data plane (plan §W1 — carved out of the [`super`]
//! module). The capture/encode and send threads raise their own priority so a CPU-saturating game
//! can't deschedule them; the GameStream path and the direct-NVENC send thread reach this the same
//! way (`crate::native::boost_thread_priority`).
//! Per-thread OS scheduling QoS for the data plane (plan §W1/§W6 — now in the shared `pf-frame`
//! leaf). The capture/encode and send threads raise their own priority so a CPU-saturating game
//! can't deschedule them; the native, GameStream, and direct-NVENC send threads all reach this the
//! same way (`pf_frame::thread_qos::boost_thread_priority`).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
@@ -14,7 +14,7 @@
/// uncapped GPU-saturating title (e.g. CS2 direct on a virtual output, not capped by gamescope) is
/// also a CPU hog and can deschedule our submit threads. `critical` → highest non-realtime class
/// (the capture+encode loop); otherwise above-normal (the send/relay thread).
pub(crate) fn boost_thread_priority(critical: bool) {
pub fn boost_thread_priority(critical: bool) {
// Windows host-process/thread session tuning (timer 1ms, DWM MMCSS, HIGH class once; MMCSS +
// keep-display-awake per thread). No-op off Windows. Both stream threads call us, so this covers
// capture/encode (critical) and send (non-critical).
+28
View File
@@ -0,0 +1,28 @@
# GPU vendor/adapter detection + selection + the live-session record, extracted into a leaf crate so
# every subsystem crate (pf-encode, pf-capture, pf-vdisplay) can consult the selected GPU WITHOUT
# depending on the orchestrator (plan §W6). Self-contained: reads config + writes the pref store.
[package]
name = "pf-gpu"
version.workspace = true
edition = "2021"
license = "MIT OR Apache-2.0"
description = "punktfunk host GPU vendor/adapter enumeration, selection preference, and active-session accounting."
publish = false
[dependencies]
pf-host-config = { path = "../pf-host-config" }
pf-paths = { path = "../pf-paths" }
anyhow = "1"
tracing = "0.1"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
[target.'cfg(windows)'.dependencies]
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
] }
[dev-dependencies]
tempfile = "3"
@@ -28,14 +28,14 @@ use std::path::PathBuf;
use std::sync::{Mutex, OnceLock};
/// PCI vendor ids of the GPU vendors the encode backends know (NVENC / AMF / QSV, VAAPI on Linux).
pub(crate) const VENDOR_NVIDIA: u32 = 0x10DE;
pub(crate) const VENDOR_AMD: u32 = 0x1002;
pub(crate) const VENDOR_INTEL: u32 = 0x8086;
pub const VENDOR_NVIDIA: u32 = 0x10DE;
pub const VENDOR_AMD: u32 = 0x1002;
pub const VENDOR_INTEL: u32 = 0x8086;
/// Platform handle of an enumerated GPU — how the pipeline actually addresses it. Not part of the
/// stable identity (Windows LUIDs are per-boot; a render node can renumber across kernel updates).
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub(crate) struct GpuHandle {
pub struct GpuHandle {
/// DXGI `AdapterLuid` of this adapter (this boot only).
#[cfg(target_os = "windows")]
pub luid_low: u32,
@@ -48,7 +48,7 @@ pub(crate) struct GpuHandle {
/// One hardware GPU as enumerated on this host.
#[derive(Clone, Debug)]
pub(crate) struct GpuInfo {
pub struct GpuInfo {
/// Stable identifier for the API/UI: `"{vendor:04x}-{device:04x}-{occurrence}"`. Occurrence
/// disambiguates identical cards (two of the same model) by enumeration order among their
/// twins — the best available tiebreaker (PCI order), imperfect but honest.
@@ -65,7 +65,7 @@ pub(crate) struct GpuInfo {
}
/// Lowercase vendor tag for the API (`nvidia` / `amd` / `intel` / `other`).
pub(crate) fn vendor_tag(vendor_id: u32) -> &'static str {
pub fn vendor_tag(vendor_id: u32) -> &'static str {
match vendor_id {
VENDOR_NVIDIA => "nvidia",
VENDOR_AMD => "amd",
@@ -93,6 +93,9 @@ impl GpuInfo {
/// Assign the stable `id` + `occurrence` fields after enumeration (occurrence = index among
/// same-(vendor,device) twins, in inventory order — Windows sorts the inventory by LUID first so
/// twin numbering is stable for the boot, see [`enumerate`]).
// Called only by the Linux/Windows `enumerate()` arms; the stub `enumerate()` on other targets
// (macOS dev host) doesn't, so it's dead there.
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn assign_ids(gpus: &mut [GpuInfo]) {
for i in 0..gpus.len() {
let occ = gpus[..i]
@@ -127,7 +130,7 @@ mod adapter_type {
/// True when these bits describe an adapter that can never be the render/encode GPU:
/// indirect-display, software, or anything without render support.
pub(crate) fn hidden(bits: u32) -> bool {
pub fn hidden(bits: u32) -> bool {
bits & INDIRECT_DISPLAY_DEVICE != 0
|| bits & SOFTWARE_DEVICE != 0
|| bits & RENDER_SUPPORTED == 0
@@ -172,7 +175,7 @@ mod kmt {
/// The `D3DKMT_ADAPTERTYPE` bits for the adapter with this LUID, `None` when the kernel
/// query fails (callers fail open — better a listed twin than a hidden real GPU).
pub(crate) fn adapter_type_bits(luid_low: u32, luid_high: i32) -> Option<u32> {
pub fn adapter_type_bits(luid_low: u32, luid_high: i32) -> Option<u32> {
// SAFETY: every pointer handed to the three D3DKMT calls addresses a stack local that
// outlives the call; NTSTATUS >= 0 is success. The kernel handle is closed on every
// path that opened it, including a failed query.
@@ -208,7 +211,7 @@ mod kmt {
/// Other platforms (the macOS dev/test host build): empty — the endpoints still exist, they just
/// report no GPUs.
#[cfg(target_os = "windows")]
pub(crate) fn enumerate() -> Vec<GpuInfo> {
pub fn enumerate() -> Vec<GpuInfo> {
use windows::Win32::Graphics::Dxgi::{
CreateDXGIFactory1, IDXGIFactory1, DXGI_ADAPTER_FLAG_SOFTWARE,
};
@@ -281,7 +284,7 @@ pub(crate) fn enumerate() -> Vec<GpuInfo> {
}
#[cfg(target_os = "linux")]
pub(crate) fn enumerate() -> Vec<GpuInfo> {
pub fn enumerate() -> Vec<GpuInfo> {
let mut nodes: Vec<String> = std::fs::read_dir("/dev/dri")
.map(|rd| {
rd.filter_map(|e| e.ok())
@@ -331,7 +334,7 @@ pub(crate) fn enumerate() -> Vec<GpuInfo> {
}
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
pub(crate) fn enumerate() -> Vec<GpuInfo> {
pub fn enumerate() -> Vec<GpuInfo> {
Vec::new()
}
@@ -343,7 +346,7 @@ pub(crate) fn enumerate() -> Vec<GpuInfo> {
/// `Manual` (an explicit GPU chosen in the web console).
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub(crate) enum GpuMode {
pub enum GpuMode {
#[default]
Auto,
Manual,
@@ -351,7 +354,7 @@ pub(crate) enum GpuMode {
/// Stable identity of the manually preferred GPU (see [`GpuInfo::id`] for why not LUID/index).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub(crate) struct PreferredGpu {
pub struct PreferredGpu {
pub vendor_id: u32,
pub device_id: u32,
#[serde(default)]
@@ -364,7 +367,7 @@ pub(crate) struct PreferredGpu {
/// The persisted GPU preference (`<config>/gpu-settings.json`).
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub(crate) struct GpuPreference {
pub struct GpuPreference {
#[serde(default)]
pub mode: GpuMode,
/// `Some` when `mode == Manual` (kept when switching back to Auto so the console can offer
@@ -376,7 +379,7 @@ pub(crate) struct GpuPreference {
/// The preference store: in-memory current value + its JSON file. Mirrors `native_pairing`'s
/// persistence discipline (private dir, secret-file temp write + atomic rename, in-memory
/// rollback if the disk write fails).
pub(crate) struct GpuPrefStore {
pub struct GpuPrefStore {
path: PathBuf,
cur: Mutex<GpuPreference>,
}
@@ -409,10 +412,10 @@ impl GpuPrefStore {
/// succeeds, so a full disk can't leave memory and file disagreeing.
pub fn set(&self, pref: GpuPreference) -> Result<()> {
if let Some(dir) = self.path.parent() {
crate::gamestream::create_private_dir(dir)?;
pf_paths::create_private_dir(dir)?;
}
let tmp = self.path.with_extension("json.tmp");
crate::gamestream::write_secret_file(&tmp, &serde_json::to_vec_pretty(&pref)?)?;
pf_paths::write_secret_file(&tmp, &serde_json::to_vec_pretty(&pref)?)?;
std::fs::rename(&tmp, &self.path)?;
*self.cur.lock().unwrap() = pref;
Ok(())
@@ -420,13 +423,11 @@ impl GpuPrefStore {
}
/// The process-wide preference store (config-dir file), loaded once on first access — the same
/// global-accessor shape as [`crate::config::config`], because selection happens deep inside
/// global-accessor shape as [`pf_host_config::config`], because selection happens deep inside
/// capture/encode setup where no app state is threaded.
pub(crate) fn prefs() -> &'static GpuPrefStore {
pub fn prefs() -> &'static GpuPrefStore {
static STORE: OnceLock<GpuPrefStore> = OnceLock::new();
STORE.get_or_init(|| {
GpuPrefStore::load_from(crate::gamestream::config_dir().join("gpu-settings.json"))
})
STORE.get_or_init(|| GpuPrefStore::load_from(pf_paths::config_dir().join("gpu-settings.json")))
}
// ---------------------------------------------------------------------------------------------
@@ -435,7 +436,7 @@ pub(crate) fn prefs() -> &'static GpuPrefStore {
/// Why a GPU was selected — surfaced by the mgmt API so the console can explain the decision.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum PickSource {
pub enum PickSource {
/// The operator's manual preference matched a present GPU.
Preference,
/// `PUNKTFUNK_RENDER_ADAPTER` substring matched.
@@ -460,7 +461,7 @@ impl PickSource {
/// A resolved selection: the GPU the next session's pipeline will be created on, and why.
#[derive(Clone, Debug)]
pub(crate) struct SelectedGpu {
pub struct SelectedGpu {
pub info: GpuInfo,
pub source: PickSource,
}
@@ -468,7 +469,7 @@ pub(crate) struct SelectedGpu {
/// Find the manually preferred GPU in the inventory. Match order: exact stable identity
/// (vendor, device, occurrence) → same model (vendor, device; a twin renumbered) → exact name
/// (ids changed across a driver/firmware quirk but the marketing name survived).
pub(crate) fn find_preferred(gpus: &[GpuInfo], want: &PreferredGpu) -> Option<usize> {
pub fn find_preferred(gpus: &[GpuInfo], want: &PreferredGpu) -> Option<usize> {
gpus.iter()
.position(|g| {
g.vendor_id == want.vendor_id
@@ -491,7 +492,7 @@ pub(crate) fn find_preferred(gpus: &[GpuInfo], want: &PreferredGpu) -> Option<us
/// the index into `gpus` plus the reason. `None` only when `gpus` is empty. A set-but-unmatched
/// env substring falls through to max-VRAM (same outcome as env unset — deliberately more robust
/// than the old `resolve_render_adapter_luid`, which returned *no* adapter on a stale substring).
pub(crate) fn pick(
pub fn pick(
gpus: &[GpuInfo],
pref: &GpuPreference,
env_substr: Option<&str>,
@@ -534,10 +535,10 @@ pub(crate) fn pick(
/// the encoder-vendor dispatch both consume, so capture, encode, and the advertisement agree by
/// construction. Pure query — callers log (this runs per serverinfo poll).
#[cfg(target_os = "windows")]
pub(crate) fn selected_gpu() -> Option<SelectedGpu> {
pub fn selected_gpu() -> Option<SelectedGpu> {
let gpus = enumerate();
let pref = prefs().get();
let env = crate::config::config()
let env = pf_host_config::config()
.render_adapter
.clone()
.filter(|s| !s.is_empty());
@@ -553,7 +554,7 @@ pub(crate) fn selected_gpu() -> Option<SelectedGpu> {
/// owns the VAAPI render node. (The *authoritative* Linux switches stay in `encode::open_video` /
/// [`linux_render_node`] — this is the console's view of them.)
#[cfg(target_os = "linux")]
pub(crate) fn selected_gpu() -> Option<SelectedGpu> {
pub fn selected_gpu() -> Option<SelectedGpu> {
let gpus = enumerate();
let pref = prefs().get();
let mut preference_missing = false;
@@ -595,14 +596,14 @@ pub(crate) fn selected_gpu() -> Option<SelectedGpu> {
}
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
pub(crate) fn selected_gpu() -> Option<SelectedGpu> {
pub fn selected_gpu() -> Option<SelectedGpu> {
None
}
/// The manually preferred GPU, only when `mode == Manual` **and** it is currently present.
/// The Linux encode dispatch consults this (auto mode keeps today's NVIDIA-presence behavior
/// exactly).
pub(crate) fn manual_selection() -> Option<GpuInfo> {
pub fn manual_selection() -> Option<GpuInfo> {
let pref = prefs().get();
if pref.mode != GpuMode::Manual {
return None;
@@ -616,7 +617,7 @@ pub(crate) fn manual_selection() -> Option<GpuInfo> {
/// The VAAPI/DRM render node for this host: matched manual preference > `PUNKTFUNK_RENDER_NODE`
/// (a deliberate live env read — see `config.rs` module docs) > `/dev/dri/renderD128`.
#[cfg(target_os = "linux")]
pub(crate) fn linux_render_node() -> PathBuf {
pub fn linux_render_node() -> PathBuf {
if let Some(g) = manual_selection() {
if let Some(node) = g.handle.render_node {
return node;
@@ -639,7 +640,7 @@ fn linux_nvidia_present() -> bool {
/// A cache key that changes whenever the *selection* changes (preference edits included), for the
/// per-GPU probe caches (`can_encode_444`, `windows_codec_support`) that were process-lifetime
/// `OnceLock`s back when selection was env-only.
pub(crate) fn selection_key() -> String {
pub fn selection_key() -> String {
match selected_gpu() {
Some(sel) => {
#[cfg(target_os = "windows")]
@@ -664,7 +665,7 @@ pub(crate) fn selection_key() -> String {
/// What a live session encodes on — the console's "currently used GPU".
#[derive(Clone, Debug)]
pub(crate) struct ActiveGpu {
pub struct ActiveGpu {
/// Stable id of the GPU ([`GpuInfo::id`]; empty for the CPU/software path) so a UI can match
/// it against the inventory.
pub id: String,
@@ -684,7 +685,7 @@ static ACTIVE: Mutex<Option<ActiveState>> = Mutex::new(None);
/// RAII marker for one live encode session; dropping it decrements the session count. Held by the
/// encoder wrapper `open_video` returns, so the count is correct by construction (every successful
/// open is paired with a drop).
pub(crate) struct ActiveSession(());
pub struct ActiveSession(());
impl Drop for ActiveSession {
fn drop(&mut self) {
@@ -698,7 +699,7 @@ impl Drop for ActiveSession {
/// Record a session opening on `gpu`. Concurrent sessions share one GPU (the Windows pipeline is
/// single-GPU by construction; Linux sessions share the selection), so the latest record wins and
/// a counter tracks liveness.
pub(crate) fn session_begin(gpu: ActiveGpu) -> ActiveSession {
pub fn session_begin(gpu: ActiveGpu) -> ActiveSession {
let mut st = ACTIVE.lock().unwrap_or_else(|e| e.into_inner());
let sessions = st.as_ref().map(|s| s.sessions).unwrap_or(0) + 1;
*st = Some(ActiveState { gpu, sessions });
@@ -707,7 +708,7 @@ pub(crate) fn session_begin(gpu: ActiveGpu) -> ActiveSession {
/// The GPU live sessions encode on + how many sessions hold it. `Some` with `sessions == 0` means
/// "last used, idle now" — the mgmt API distinguishes the two.
pub(crate) fn active() -> Option<(ActiveGpu, u32)> {
pub fn active() -> Option<(ActiveGpu, u32)> {
ACTIVE
.lock()
.unwrap_or_else(|e| e.into_inner())
@@ -913,3 +914,39 @@ mod tests {
}
}
}
/// Pick the render GPU LUID the Windows pipeline is created on: the IDD-push capturer's
/// shared-texture ring, the IddCx `SET_RENDER_ADAPTER` pin, and (via the captured frame's device)
/// NVENC/AMF/QSV all follow this one decision — see [`selected_gpu`] for the precedence (operator
/// preference > `PUNKTFUNK_RENDER_ADAPTER` substring > max `DedicatedVideoMemory`). A configured
/// preference that doesn't match a present GPU falls back to auto selection (with a warning) rather
/// than returning `None`, so a stale preference never stops the host from streaming.
///
/// Lives here (not in a host module) so BOTH the capture and encode subsystem crates depend on it
/// as a peer of GPU selection instead of the orchestrator — the plan's `windows/adapter.rs`, folded
/// into `pf-gpu` (plan §W6). It was historically the SudoVDA backend's, then the host's
/// `win_adapter.rs`; the LUID-shaped view of [`selected_gpu`] plus the per-decision logging.
#[cfg(target_os = "windows")]
pub fn resolve_render_adapter_luid() -> Option<windows::Win32::Foundation::LUID> {
match selected_gpu() {
Some(sel) => {
tracing::info!(
adapter = sel.info.name,
vram_mb = sel.info.vram_bytes / (1024 * 1024),
source = sel.source.tag(),
"render adapter selected"
);
if sel.source == PickSource::PreferenceMissing {
tracing::warn!(
"the preferred GPU is not present — auto-selected the adapter above \
(fix or clear the preference in the web console)"
);
}
Some(sel.info.luid())
}
None => {
tracing::warn!("no suitable render adapter found for SET_RENDER_ADAPTER");
None
}
}
}
+12
View File
@@ -0,0 +1,12 @@
# The process-wide host configuration global (HostConfig + the config() OnceLock), extracted into a
# leaf crate so every subsystem crate (pf-encode, pf-capture, pf-vdisplay, pf-gpu) can read config
# WITHOUT depending on the orchestrator (plan §W6 — config parked above its consumers). Pure std.
[package]
name = "pf-host-config"
version.workspace = true
edition = "2021"
license = "MIT OR Apache-2.0"
description = "Process-wide punktfunk host configuration (env-parsed HostConfig behind a OnceLock)."
publish = false
[dependencies]
@@ -12,7 +12,7 @@
//! capture/topology/encoder decision.
//!
//! **What is deliberately NOT here (and must stay a live `env::var` read):**
//! - **Runtime-mutated session vars.** On Linux, [`crate::vdisplay::apply_session_env`] rewrites the process
//! - **Runtime-mutated session vars.** On Linux, `crate::vdisplay::apply_session_env` rewrites the process
//! env on *every connect* so one host follows a Bazzite box across Gaming↔Desktop: `WAYLAND_DISPLAY`,
//! `XDG_CURRENT_DESKTOP`, `XDG_RUNTIME_DIR`, `DBUS_SESSION_BUS_ADDRESS`, and the *derived* `PUNKTFUNK_*`
//! vars `INPUT_BACKEND`, `GAMESCOPE_SESSION`/`GAMESCOPE_NODE`, `KWIN_VIRTUAL_PRIMARY`,
+65
View File
@@ -0,0 +1,65 @@
# Input injection (plan §W6): the per-OS injector backends (wlroots virtual-input, KWin fake_input,
# libei/reis, gamescope-EI on Linux; SendInput on Windows) + the virtual-gamepad HID stack (DualSense/
# DualShock4/Switch Pro/Steam Controller/Deck over uhid/usbip and the Windows UMDF drivers), extracted
# into a subsystem crate. Consumes punktfunk_core::input (the neutral GamepadEvent/InputEvent vocabulary,
# moved to core in W5) and the pf-driver-proto wire contract; NEVER reaches the orchestrator (the one
# gamescope-EI socket path is the shared pf-paths contract, not a vdisplay reach-in).
[package]
name = "pf-inject"
version = "0.12.0"
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
description = "punktfunk host input injection: per-OS keyboard/mouse injectors + the virtual-gamepad HID backends behind one InputInjector trait."
publish = false
[dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
pf-driver-proto = { path = "../pf-driver-proto" }
pf-host-config = { path = "../pf-host-config" }
pf-paths = { path = "../pf-paths" }
# The Windows gamepad-channel bootstrap reuses the IDD-push WUDFHost verification + the resident-mouse
# compose-kick hook (both live in pf-capture).
pf-capture = { path = "../pf-capture" }
anyhow = "1"
tracing = "0.1"
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
parking_lot = "0.12"
# The RemoteDesktop portal for the libei injector on KWin/GNOME (headless grant via kde-authorized).
ashpd = { version = "0.13", features = ["remote_desktop"] }
# Input injection into headless Sway via the wlroots virtual-input Wayland protocols.
wayland-client = "0.31"
wayland-protocols-wlr = { version = "0.3", features = ["client"] }
wayland-protocols-misc = { version = "0.3", features = ["client"] }
wayland-protocols = { version = "0.32", features = ["client"] }
# Codegen for KDE's `org_kde_kwin_fake_input` (vendored in `protocols/fake-input.xml`); the generated
# interface tables reference `wayland-backend`.
wayland-scanner = "0.31"
wayland-backend = "0.3"
# libei (EI sender) for the portable input path on KWin/GNOME (RemoteDesktop portal) + gamescope-EI.
reis = { version = "0.6.1", features = ["tokio"] }
futures-util = "0.3"
tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
# Builds/validates the xkb keymap uploaded to the virtual keyboard + tracks modifier state.
xkbcommon = "0.8"
# Vendored + trimmed usbip server core — presents a virtual Steam Deck over USB/IP for Steam Input.
usbip-sim = { path = "../punktfunk-host/vendor/usbip-sim" }
[target.'cfg(target_os = "windows")'.dependencies]
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_Security",
"Win32_Security_Authorization",
"Win32_Devices_DeviceAndDriverInstallation",
"Win32_Devices_Enumeration_Pnp",
# SwDeviceCreate's SW_DEVICE_CREATE_INFO references DEVPROPKEY (Properties).
"Win32_Devices_Properties",
"Win32_System_Memory",
"Win32_System_IO",
"Win32_System_StationsAndDesktops",
"Win32_System_Threading",
"Win32_UI_Input_KeyboardAndMouse",
"Win32_UI_WindowsAndMessaging",
] }
@@ -1,6 +1,6 @@
//! Per-pad dedup for the rich HID-output feedback plane (0xCD), carved out of `dualsense_proto`
//! (plan §W4 — it is device-agnostic, shared by the DualSense/DS4/Deck managers via
//! [`crate::inject::uhid_manager`], not DualSense-specific). A game bundles rumble + lightbar +
//! [`crate::uhid_manager`], not DualSense-specific). A game bundles rumble + lightbar +
//! LEDs + adaptive triggers into one output report, so a merely-rumbling pad re-sends unchanged
//! rich state every report; this forwards only genuine changes (one-shot pulses always fire).
@@ -17,7 +17,7 @@ use super::dualsense_proto::{
DS_EDGE_PRODUCT, DS_FEATURE_CALIBRATION, DS_FEATURE_FIRMWARE, DS_INPUT_REPORT_LEN, DS_PRODUCT,
DS_TOUCH_H, DS_TOUCH_W, DS_VENDOR, DUALSENSE_EDGE_RDESC, DUALSENSE_RDESC,
};
use crate::inject::uhid_manager::{PadFeedback, PadProto, UhidManager};
use crate::uhid_manager::{PadFeedback, PadProto, UhidManager};
use anyhow::{Context, Result};
use punktfunk_core::quic::RichInput;
use std::fs::{File, OpenOptions};
@@ -232,13 +232,13 @@ impl Drop for DualSensePad {
pub struct DsLinuxProto {
/// Fallback policy for the Steam back grips a client may send (the DualSense has no back-button
/// HID slot). `PUNKTFUNK_STEAM_REMAP=paddles=…`; default drop.
remap: crate::inject::steam_remap::RemapConfig,
remap: crate::steam_remap::RemapConfig,
}
impl Default for DsLinuxProto {
fn default() -> DsLinuxProto {
DsLinuxProto {
remap: crate::inject::steam_remap::RemapConfig::from_env(),
remap: crate::steam_remap::RemapConfig::from_env(),
}
}
}
@@ -268,7 +268,7 @@ impl PadProto for DsLinuxProto {
fn merge_frame(&self, prev: &DsState, f: &punktfunk_core::input::GamepadFrame) -> DsState {
// Steam back grips have no DualSense slot — fold them onto standard buttons per the
// configured policy (default drop) so they aren't silently lost.
let buttons = crate::inject::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let buttons = crate::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let mut s = DsState::from_gamepad(
buttons,
f.ls_x,
@@ -303,9 +303,14 @@ impl PadProto for DsLinuxProto {
PadFeedback {
rumble: fb.rumble,
hidout: fb.hidout,
// Linux hid-playstation reliably surfaces the game's rumble stop, so this backend does
// not need the abandoned-rumble force-off — stays untracked (see `PadFeedback`).
game_drove: None,
// Rumble-plane liveness (arms the shared abandoned-rumble force-off). evdev-FF games
// going through hid-playstation get their stops surfaced reliably, but Steam Input
// drives this pad over hidraw DIRECTLY — the same abandonment semantics as a Windows
// game, so the same watchdog applies. SDL-class writers re-assert a held level every
// ~2 s (inside the idle window), and a writer that goes silent on a latched level is
// cut exactly as real firmware decay would cut it on a physical pad.
rumble_drove: Some(fb.rumble.is_some()),
resync: false,
}
}
}
@@ -392,9 +397,14 @@ impl PadProto for DsEdgeLinuxProto {
PadFeedback {
rumble: fb.rumble,
hidout: fb.hidout,
// Linux hid-playstation reliably surfaces the game's rumble stop, so this backend does
// not need the abandoned-rumble force-off — stays untracked (see `PadFeedback`).
game_drove: None,
// Rumble-plane liveness (arms the shared abandoned-rumble force-off). evdev-FF games
// going through hid-playstation get their stops surfaced reliably, but Steam Input
// drives this pad over hidraw DIRECTLY — the same abandonment semantics as a Windows
// game, so the same watchdog applies. SDL-class writers re-assert a held level every
// ~2 s (inside the idle window), and a writer that goes silent on a latched level is
// cut exactly as real firmware decay would cut it on a physical pad.
rumble_drove: Some(fb.rumble.is_some()),
resync: false,
}
}
}
@@ -18,7 +18,7 @@ use super::dualshock4_proto::{
parse_ds4_output, serialize_state, Ds4Feedback, DS4_INPUT_REPORT_LEN, DS4_PRODUCT, DS4_TOUCH_H,
DS4_TOUCH_W, DS4_VENDOR,
};
use crate::inject::uhid_manager::{PadFeedback, PadProto, UhidManager};
use crate::uhid_manager::{PadFeedback, PadProto, UhidManager};
use anyhow::{Context, Result};
use punktfunk_core::quic::{HidOutput, RichInput};
use std::fs::{File, OpenOptions};
@@ -302,13 +302,13 @@ impl Drop for DualShock4Pad {
pub struct Ds4LinuxProto {
/// Fallback policy for the Steam back grips a client may send (the DS4 has no back-button HID
/// slot). `PUNKTFUNK_STEAM_REMAP=paddles=…`; default drop.
remap: crate::inject::steam_remap::RemapConfig,
remap: crate::steam_remap::RemapConfig,
}
impl Default for Ds4LinuxProto {
fn default() -> Ds4LinuxProto {
Ds4LinuxProto {
remap: crate::inject::steam_remap::RemapConfig::from_env(),
remap: crate::steam_remap::RemapConfig::from_env(),
}
}
}
@@ -338,7 +338,7 @@ impl PadProto for Ds4LinuxProto {
fn merge_frame(&self, prev: &DsState, f: &punktfunk_core::input::GamepadFrame) -> DsState {
// Steam back grips have no DS4 slot — fold them onto standard buttons per the configured
// policy (default drop) so they aren't silently lost.
let buttons = crate::inject::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let buttons = crate::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let mut s = DsState::from_gamepad(
buttons,
f.ls_x,
@@ -378,7 +378,11 @@ impl PadProto for Ds4LinuxProto {
.map(|(r, g, b)| HidOutput::Led { pad: idx, r, g, b })
.into_iter()
.collect(),
game_drove: None,
// Rumble-plane liveness (arms the shared abandoned-rumble force-off) — see the Linux
// DualSense backend for the hidraw-writer rationale; `parse_ds4_output` gates rumble
// on flag0 bit0 the same way.
rumble_drove: Some(fb.rumble.is_some()),
resync: false,
}
}
}
@@ -18,13 +18,12 @@
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use crate::gamestream::gamepad;
use crate::inject::pad_slots::PadSlots;
use crate::pad_slots::PadSlots;
use anyhow::{bail, Result};
use punktfunk_core::input::{GamepadFrame, MAX_PADS};
use punktfunk_core::input::{gamepad, GamepadFrame, MAX_PADS};
use std::collections::HashMap;
use std::os::fd::{AsRawFd, OwnedFd};
use std::time::Instant;
use std::time::{Duration, Instant};
// ioctls (x86_64).
const UI_DEV_CREATE: libc::c_ulong = 0x5501;
@@ -264,14 +263,80 @@ struct Effect {
replay_ms: u16,
}
/// One virtual X-Box-360 pad backed by a uinput device.
pub struct VirtualPad {
fd: OwnedFd,
/// The force-feedback half of a virtual pad — the game-side effect table plus the mixdown policy
/// (finite-replay expiry + the abandoned-INFINITE-effect force-off), split from [`VirtualPad`] so
/// the policy is pure and unit-testable without a live uinput fd.
struct FfState {
effects: HashMap<i16, Effect>,
next_effect_id: i16,
gain: u32,
/// Last `(low, high)` reported, to dedup.
last_mix: (u16, u16),
/// When a game last touched the FF plane (upload / erase / play / stop / gain). An
/// infinite-replay effect still playing past the shared idle window against this is a residual
/// the game abandoned (kernel auto-erase only covers a game whose fd CLOSED) — finite effects
/// are untouched: their declared replay deadline is the contract, exactly as a real pad honors
/// it. SDL-class writers re-play held rumble every ~2 s, refreshing this clock.
last_activity: Instant,
}
impl FfState {
fn new() -> FfState {
FfState {
effects: HashMap::new(),
next_effect_id: 0,
gain: 0xFFFF,
last_mix: (0, 0),
last_activity: Instant::now(),
}
}
/// The game touched the FF plane — refresh the abandoned-effect clock.
fn note_activity(&mut self) {
self.last_activity = Instant::now();
}
/// Mix: sum playing effects (expiring finished ones, force-stopping abandoned infinite ones),
/// scale by gain. Returns the new `(low, high)` only when it changed since the last call.
fn mix(&mut self, now: Instant, idle: Option<Duration>) -> Option<(u16, u16)> {
let stale = idle.is_some_and(|t| now.duration_since(self.last_activity) >= t);
let (mut strong, mut weak) = (0u32, 0u32);
for e in self.effects.values_mut() {
let Some(deadline) = e.playing else { continue };
match deadline {
Some(d) if now >= d => e.playing = None,
// An infinite-replay effect the game stopped driving (no FF traffic for the whole
// idle window) — the alive-but-abandoned case the kernel's close-time auto-erase
// cannot see. Stop it once; a later EV_FF play re-arms it (and refreshes the
// clock). Mirrors the XUSB/UHID abandoned-rumble force-off.
None if stale => {
tracing::info!(
strong = e.strong,
weak = e.weak,
"rumble: stale infinite FF effect (game stopped driving the pad) — forcing off"
);
e.playing = None;
}
_ => {
strong = strong.saturating_add(e.strong as u32);
weak = weak.saturating_add(e.weak as u32);
}
}
}
// Linux FF: strong = low-frequency (big) motor, weak = high-frequency motor.
let low = ((strong.min(0xFFFF) * self.gain) >> 16) as u16;
let high = ((weak.min(0xFFFF) * self.gain) >> 16) as u16;
(self.last_mix != (low, high)).then(|| {
self.last_mix = (low, high);
(low, high)
})
}
}
/// One virtual X-Box-360 pad backed by a uinput device.
pub struct VirtualPad {
fd: OwnedFd,
ff: FfState,
}
impl VirtualPad {
@@ -370,10 +435,7 @@ impl VirtualPad {
Ok(VirtualPad {
fd,
effects: HashMap::new(),
next_effect_id: 0,
gain: 0xFFFF,
last_mix: (0, 0),
ff: FfState::new(),
})
}
@@ -461,6 +523,7 @@ impl VirtualPad {
unsafe { std::ptr::read_unaligned(buf.as_ptr() as *const InputEventRaw) };
match (ev.type_, ev.code) {
(EV_UINPUT, UI_FF_UPLOAD) => {
self.ff.note_activity();
// SAFETY: `UinputFfUpload` is `#[repr(C)]` over integers (`u32`, `i32`) and two
// `FfEffect`s (integers + `[u8; 32]`); all-zero is a valid bit pattern for every field
// (no bool/NonZero/enum/reference niche), so `zeroed` yields a fully-initialized valid
@@ -470,13 +533,13 @@ impl VirtualPad {
if ioctl_ptr(raw, UI_BEGIN_FF_UPLOAD, &mut up, "UI_BEGIN_FF_UPLOAD").is_ok() {
let mut e = up.effect;
if e.id == -1 {
e.id = self.next_effect_id;
self.next_effect_id = self.next_effect_id.wrapping_add(1);
e.id = self.ff.next_effect_id;
self.ff.next_effect_id = self.ff.next_effect_id.wrapping_add(1);
}
if e.type_ == FF_RUMBLE {
let strong = u16::from_ne_bytes([e.u[0], e.u[1]]);
let weak = u16::from_ne_bytes([e.u[2], e.u[3]]);
let slot = self.effects.entry(e.id).or_insert(Effect {
let slot = self.ff.effects.entry(e.id).or_insert(Effect {
strong: 0,
weak: 0,
playing: None,
@@ -492,20 +555,25 @@ impl VirtualPad {
}
}
(EV_UINPUT, UI_FF_ERASE) => {
self.ff.note_activity();
// SAFETY: `UinputFfErase` is `#[repr(C)]` over three integer fields (`u32`, `i32`,
// `u32`); all-zero is a valid bit pattern for each, so `zeroed` produces a fully-valid
// initialized value — `request_id` is set below and `effect_id` filled by the ioctl.
let mut er: UinputFfErase = unsafe { std::mem::zeroed() };
er.request_id = ev.value as u32;
if ioctl_ptr(raw, UI_BEGIN_FF_ERASE, &mut er, "UI_BEGIN_FF_ERASE").is_ok() {
self.effects.remove(&(er.effect_id as i16));
self.ff.effects.remove(&(er.effect_id as i16));
er.retval = 0;
let _ = ioctl_ptr(raw, UI_END_FF_ERASE, &mut er, "UI_END_FF_ERASE");
}
}
(EV_FF, FF_GAIN) => self.gain = (ev.value as u32).min(0xFFFF),
(EV_FF, FF_GAIN) => {
self.ff.note_activity();
self.ff.gain = (ev.value as u32).min(0xFFFF);
}
(EV_FF, code) => {
if let Some(e) = self.effects.get_mut(&(code as i16)) {
self.ff.note_activity();
if let Some(e) = self.ff.effects.get_mut(&(code as i16)) {
e.playing = if ev.value != 0 {
Some((e.replay_ms > 0).then(|| {
Instant::now()
@@ -520,26 +588,8 @@ impl VirtualPad {
}
}
// Mix: sum playing effects (expiring finished ones), scale by gain.
let now = Instant::now();
let (mut strong, mut weak) = (0u32, 0u32);
for e in self.effects.values_mut() {
if let Some(deadline) = e.playing {
if deadline.is_some_and(|d| now >= d) {
e.playing = None;
} else {
strong = strong.saturating_add(e.strong as u32);
weak = weak.saturating_add(e.weak as u32);
}
}
}
// Linux FF: strong = low-frequency (big) motor, weak = high-frequency motor.
let low = ((strong.min(0xFFFF) * self.gain) >> 16) as u16;
let high = ((weak.min(0xFFFF) * self.gain) >> 16) as u16;
(self.last_mix != (low, high)).then(|| {
self.last_mix = (low, high);
(low, high)
})
self.ff
.mix(Instant::now(), crate::uhid_manager::rumble_idle_timeout())
}
}
@@ -728,3 +778,87 @@ mod tests {
);
}
}
#[cfg(test)]
mod ff_state_tests {
use super::*;
/// The default idle window the shared hatch resolves to when the env is unset.
const IDLE: Option<Duration> = Some(Duration::from_millis(2500));
/// `gain` is 0xFFFF (not a true 1.0 multiplier), so a magnitude loses 1 LSB in the mixdown.
fn scaled(v: u16) -> u16 {
((v as u32 * 0xFFFF) >> 16) as u16
}
fn ff_with(effect: Effect) -> FfState {
let mut ff = FfState::new();
ff.effects.insert(0, effect);
ff
}
#[test]
fn abandoned_infinite_effect_is_forced_off_after_idle_window() {
let mut ff = ff_with(Effect {
strong: 0x8000,
weak: 0,
playing: Some(None),
replay_ms: 0,
});
let now = Instant::now();
assert_eq!(ff.mix(now, IDLE), Some((scaled(0x8000), 0)));
assert_eq!(ff.mix(now, IDLE), None); // unchanged level dedups, still playing
// The game goes silent on the FF plane past the idle window: cut, exactly once.
ff.last_activity = now - Duration::from_millis(2600);
assert_eq!(ff.mix(now, IDLE), Some((0, 0)));
assert_eq!(ff.mix(now, IDLE), None); // already off — no repeat
}
#[test]
fn finite_effect_honors_its_replay_deadline_not_the_idle_window() {
let now = Instant::now();
let mut ff = ff_with(Effect {
strong: 0x4000,
weak: 0,
playing: Some(Some(now + Duration::from_secs(10))),
replay_ms: 10_000,
});
// FF plane long stale, but the effect declared a finite replay — the declared duration is
// the contract (a real pad honors it too), so it keeps playing…
ff.last_activity = now - Duration::from_secs(60);
assert_eq!(ff.mix(now, IDLE), Some((scaled(0x4000), 0)));
// …and expires at its own deadline.
assert_eq!(ff.mix(now + Duration::from_secs(11), IDLE), Some((0, 0)));
}
#[test]
fn replay_after_cut_rearms_the_effect() {
let now = Instant::now();
let mut ff = ff_with(Effect {
strong: 0x8000,
weak: 0,
playing: Some(None),
replay_ms: 0,
});
assert_eq!(ff.mix(now, IDLE), Some((scaled(0x8000), 0)));
ff.last_activity = now - Duration::from_millis(3000);
assert_eq!(ff.mix(now, IDLE), Some((0, 0)));
// The game plays the effect again — an FF event refreshes the clock and re-arms playback.
ff.last_activity = now;
ff.effects.get_mut(&0).unwrap().playing = Some(None);
assert_eq!(ff.mix(now, IDLE), Some((scaled(0x8000), 0)));
}
#[test]
fn disabled_watchdog_never_cuts() {
let now = Instant::now();
let mut ff = ff_with(Effect {
strong: 0x8000,
weak: 0,
playing: Some(None),
replay_ms: 0,
});
ff.last_activity = now - Duration::from_secs(600);
assert_eq!(ff.mix(now, None), Some((scaled(0x8000), 0)));
}
}

Some files were not shown because too many files have changed in this diff Show More