Compare commits

..

39 Commits

Author SHA1 Message Date
enricobuehler efbcc4cdb6 fix(packaging): ship packages@unom.io as the maintainer, not noreply@anthropic.com
android-screenshots / screenshots (push) Successful in 4m1s
ci / rust (push) Successful in 26m27s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m54s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m15s
decky / build-publish (push) Successful in 20s
apple / swift (push) Successful in 1m21s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m37s
android / android (push) Successful in 14m8s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m58s
deb / build-publish (push) Successful in 10m58s
ci / web (push) Successful in 54s
apple / screenshots (push) Successful in 6m19s
ci / docs-site (push) Successful in 1m6s
ci / bench (push) Successful in 5m16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / deploy-docs (push) Successful in 9s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
release / apple (push) Successful in 10m54s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
deb / build-publish-host (push) Successful in 10m40s
linux-client-screenshots / screenshots (push) Successful in 7m14s
web-screenshots / screenshots (push) Successful in 2m49s
flatpak / build-publish (push) Failing after 8m19s
windows-host / package (push) Successful in 16m34s
arch / build-publish (push) Successful in 12m31s
Every Linux package declared its maintainer as `noreply@anthropic.com` — the
git co-author trailer address, copied out of commit metadata into PACKAGE
metadata, where it is user-visible:

  * deb:   `Maintainer:` in the host/client/web/scripting control files, plus
           the generated debian/changelog trailer  (`dpkg -s punktfunk`)
  * rpm:   all three %changelog entries                (`rpm -q --changelog`)
  * arch:  the PKGBUILD Maintainer comment

So installed packages named Anthropic as the maintainer of this project, and
pointed anyone with a packaging problem at an address that discards mail.

packages@unom.io is the project's real packaging identity — it's already the
EdDSA key (AF245C506F4E4763, "punktfunk packages") that sign-rpms.sh signs
every RPM with, and it's what packaging/rpm/README and the bootc Containerfile
document. The RPMs were therefore SIGNED by packages@unom.io while DECLARING
noreply@anthropic.com; those now agree.

Checked the other publisher surfaces — the Inno installer (AppPublisher=unom),
the flatpak metainfo (developer id io.unom) and the decky package.json were
already correct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 11:21:05 +02:00
enricobuehler 168040005a fix(packaging/rpm): correct the bogus %changelog weekday (Jul 17 2026 was a Friday)
rpmbuild reported "bogus date in %changelog: Thu Jul 17 2026 ... - 0.0.1-3"
among its errors on the Fedora builds. 2026-07-17 was a Friday. The other two
entries (Mon Jun 15, Wed Jun 10) check out.

Surfaced underneath the pf-capture compile failure in the same job, so it
would have become the next rpm blocker once that was fixed rather than
showing up on its own.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 11:18:57 +02:00
enricobuehler 99a1d1bed6 fix(pf-capture): don't require a libspa that exports SPA_VIDEO_TRANSFER_SMPTE2084
ci / web (push) Successful in 49s
ci / docs-site (push) Successful in 1m0s
apple / swift (push) Successful in 1m14s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
ci / bench (push) Successful in 6m46s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 38s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
apple / screenshots (push) Successful in 6m13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m7s
deb / build-publish (push) Successful in 11m12s
arch / build-publish (push) Successful in 12m8s
deb / build-publish-host (push) Successful in 11m58s
android / android (push) Successful in 13m36s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m48s
ci / rust (push) Successful in 26m51s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m1s
docker / deploy-docs (push) Successful in 29s
The GNOME 50 HDR format offer took the PQ transfer id straight from
pw::spa::sys, which only exists on libspa new enough to carry the
BT2020_10/SMPTE2084/ARIB_STD_B67 block. Ubuntu 24.04 (noble) — the .deb host
builder — ships older headers, so bindgen emitted no such constant and the
host failed to compile there:

    error[E0425]: cannot find value `SPA_VIDEO_TRANSFER_SMPTE2084`
                  in crate `pw::spa::sys`

This never showed up locally or on the Linux CI: both run a current PipeWire,
where the binding is present. It broke deb.yml's build-publish-host job, so
v0.14.0 published its client .debs but no host .deb.

Spell the id out (14) instead. It's wire ABI, not a private detail — SPA
mirrors GStreamer's GstVideoTransferFunction and that block was added as a
unit, so the value is the same on every libspa that has the symbol. On one
that doesn't, PipeWire fails to intersect the offer and the session
negotiates SDR, which is what an HDR-incapable host should do anyway (the
path needs GNOME 50+ regardless).

A test pins our value against pw::spa::sys wherever the symbol exists, so a
renumbered enum fails loudly instead of silently mis-tagging the transfer
function. It only builds where tests are compiled — the .deb/.rpm builders
run plain `cargo build`, so it can't reintroduce the failure it guards.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 11:17:15 +02:00
enricobuehler 2123e4e580 fix(host/rtsp): scope the HDR mut allow so Windows clippy stays green
deb / build-publish-host (push) Failing after 7m33s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 50s
apple / swift (push) Successful in 1m21s
ci / bench (push) Successful in 6m13s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m5s
release / apple (push) Successful in 10m36s
android-screenshots / screenshots (push) Successful in 3m18s
decky / build-publish (push) Successful in 16s
windows-host / package (push) Successful in 16m30s
apple / screenshots (push) Successful in 6m19s
deb / build-publish (push) Successful in 9m26s
ci / rust (push) Successful in 24m57s
android / android (push) Successful in 13m31s
arch / build-publish (push) Successful in 13m29s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m41s
web-screenshots / screenshots (push) Successful in 2m57s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m51s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m10s
linux-client-screenshots / screenshots (push) Successful in 7m27s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 7m23s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 7m9s
`let mut hdr` tripped -D unused_mut on Windows: the only reassignment is the
GNOME colour-mode probe below it, which is #[cfg(target_os = "linux")]. Dropping
`mut` would break the Linux build, so allow unused_mut on non-Linux only — the
lint still fires on Linux if that probe is ever removed.

Second Windows-only breakage in this release that the Linux CI cannot see (see
also the ffmpeg_win swscale match): `cargo clippy --workspace` on the Linux
runner never compiles cfg(windows) code, and the Windows job builds only
punktfunk-host + punktfunk-tray.

Verified: cargo clippy --release -p punktfunk-host --features nvenc,amf-qsv,qsv
-- -D warnings on 192.168.1.173.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:36:32 +02:00
enricobuehler 232b41e88b fix(pf-encode/windows): cover the Linux HDR pixel formats in the win swscale match
The GNOME 50 HDR work added PixelFormat::X2Rgb10 / X2Bgr10 but only taught the
Linux encoders about them. `sws_src` in the Windows-gated ffmpeg_win.rs matches
PixelFormat exhaustively, so the Windows host stopped compiling:

    error[E0004]: non-exhaustive patterns: `X2Rgb10` and `X2Bgr10` not covered
      --> crates\pf-encode\src\enc\windows\ffmpeg_win.rs:132:14

Linux CI never caught it — the file is cfg(windows), so `cargo clippy
--workspace` on the Linux runner never compiles it.

Both are Linux-only screencast formats (the Windows HDR path stays
Rgb10a2/P010, per the PixelFormat docs), so they join the existing bail arm.
Spelled out rather than folded into a `_` catch-all so the next PixelFormat
addition breaks this match again on purpose.

Verified: cargo check --workspace --all-targets --features nvenc,amf-qsv on the
Windows box (192.168.1.173).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:30:33 +02:00
enricobuehler 59b766fb6c chore(release): bump workspace version to 0.14.0
android / android (push) Successful in 15m52s
flatpak / build-publish (push) Failing after 8m8s
apple / screenshots (push) Successful in 6m24s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m8s
ci / rust (push) Successful in 19m20s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m48s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m47s
docker / deploy-docs (push) Successful in 23s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m2s
audit / bun-audit (push) Successful in 13s
ci / web (push) Successful in 54s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m52s
apple / swift (push) Successful in 1m15s
ci / docs-site (push) Successful in 1m7s
audit / cargo-audit (push) Successful in 2m7s
windows-host / package (push) Failing after 5m28s
ci / bench (push) Successful in 6m10s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
deb / build-publish-host (push) Failing after 6m8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 38s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
deb / build-publish (push) Successful in 9m29s
release / apple (push) Successful in 10m7s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 7m23s
arch / build-publish (push) Successful in 14m31s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m52s
MINOR, not patch: 31 commits since v0.13.0 carry 10 features, including a new
/api/v1/plugins surface with a regenerated OpenAPI spec + SDK client. Since
sdk-publish pushes this version to consumers, a patch bump would understate a
new API surface for anyone pinning ~0.13.

Headline work: GNOME 50 HDR screencast capture + Linux Main10 encode, PyroWave
4:4:4 + HDR end to end (Linux/Windows/Apple), the console-hosted plugin UI
surface, and the pyrowave perf work that lifted the 2.5 Gbps wall. Notable
fixes: the RSA host-identity keygen that broke every fresh Windows install on
0.13.0, and SDR negotiation on the Windows IDD-push path.

pf-clipboard / pf-inject / pf-vdisplay were still hardcoding 0.12.0 — the three
the 22b352c1 sweep missed. Switched to version.workspace = true so the next
bump stays one line. (fec-rs, pf-driver-proto and usbip-sim are deliberately
versioned independently and stay put.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:25:27 +02:00
enricobuehler 00f759ec72 style(pf-encode): clear the clippy gate on the HDR/PyroWave additions
`cargo clippy --workspace --all-targets --locked -- -D warnings` was red on
main — three lints landed with the GNOME 50 HDR + PyroWave 4:4:4 work:

* pyrowave_wire.rs: `aw / 2 >> level` tripped clippy::precedence. Rust already
  binds `/` tighter than `>>`, so this always parsed as `(aw / 2) >> level`
  (subband dim at half res, then one halving per DWT level) — the parens are
  purely explicit, no change in behaviour.
* linux/mod.rs: `probe_can_encode_10bit` sat after `mod hdr_tests`
  (clippy::items_after_test_module) — moved above the test module, unchanged.

Lint-only; no functional change. fmt/clippy/test all green afterwards.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:23:47 +02:00
enricobuehler c131603f0d fix(apple): point pairing copy at the web console's real port (47992, HTTPS)
Every pairing surface in the Apple client still told users to open the web
console on port 3000 — the pre-move port. The console has served :47992 for a
long time, so anyone following the on-screen instructions hit a dead port and
had no way to approve the device or read the PIN.

Seven strings across ContentView, SessionModel and PairSheet (iOS/tvOS/macOS
share them). The two that spell out a full URL also said `http://`; the console
is HTTPS-only (host's self-signed cert), so they now read
`https://<host>:47992`, matching the phrasing in the host README, install docs
and the deb/rpm packaging notes.

Copy-only — no behavioural change; a repo-wide sweep found no other stale :3000.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 10:17:36 +02:00
enricobuehler a83acf3ee1 style(fmt): rustfmt the GNOME 50 HDR files under the pinned toolchain
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 1m2s
apple / swift (push) Successful in 1m15s
decky / build-publish (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
windows-host / package (push) Failing after 5m25s
ci / bench (push) Successful in 6m55s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
apple / screenshots (push) Successful in 6m11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
deb / build-publish-host (push) Failing after 7m52s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m42s
ci / rust (push) Failing after 9m20s
docker / deploy-docs (push) Successful in 23s
deb / build-publish (push) Successful in 10m3s
arch / build-publish (push) Successful in 12m4s
android / android (push) Successful in 17m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 10m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m38s
`feat(hdr): GNOME 50 HDR screencast capture` (0e977817) landed with rustfmt
drift — six files were not clean under the pinned 1.96.0 toolchain, so
`cargo fmt --all --check` (ci.yml "Format") is red on main. Pure whitespace/
wrapping from `cargo fmt --all`; no semantic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 10:09:34 +02:00
enricobuehler 3ae5f222d3 fix(hdr): honor SDR negotiation on the Windows host IDD-push path
A Windows client with HDR off (video_caps=0) negotiates bit_depth=8, but the
H.26x IDD-push capture composition FOLLOWED the virtual display's live "Use
HDR" state rather than the negotiated depth. On a display whose advanced color
was on — leftover from a prior 10-bit session on a reused monitor, the driver
default, or the host's global toggle — the capturer emitted P010 and NVENC
stamped HEVC Main10 + BT.2020 PQ from the pixel format alone (the in-band HDR
upgrade). That 10-bit PQ stream reached a client that advertised SDR-only; on a
client whose monitor is HDR-capable but has "Use HDR" off, the PQ landed on an
SDR desktop and blew out.

Make client_10bit authoritative for the composition depth on BOTH codecs,
mirroring the PyroWave branch that already did this:

* setup: force advanced color OFF for any !client_10bit session (was
  pyrowave-only), settling before the ring is sized;
* display_hdr gates on client_10bit alone, so an SDR-negotiated session
  composes SDR even if a physical display forces HDR;
* the descriptor poller re-asserts SDR against a mid-session "Use HDR" flip for
  any SDR-negotiated session (an HDR-negotiated H.26x session still follows
  flips both ways — its encoder re-inits on the depth change).

Only Punktfunk's virtual display is touched, never the host's real desktop.

Not yet on-glass; the client-side robustness fix (presenter should tone-map PQ
on an SDR-mode display instead of taking an HDR10 swapchain) is a follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 10:01:17 +02:00
enricobuehler 0e977817f9 feat(hdr): GNOME 50 HDR screencast capture + Linux Main10 encode
ci / docs-site (push) Successful in 58s
ci / rust (push) Failing after 1m3s
ci / web (push) Successful in 1m4s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
windows-host / package (push) Failing after 5m38s
ci / bench (push) Successful in 6m12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m38s
apple / screenshots (push) Successful in 6m25s
docker / deploy-docs (push) Successful in 29s
deb / build-publish-host (push) Failing after 7m30s
deb / build-publish (push) Successful in 11m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m59s
arch / build-publish (push) Successful in 11m57s
android / android (push) Successful in 15m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m46s
GNOME 50 (Mutter MR 4928, PipeWire >= 1.6) added HDR screen sharing for
monitor streams: 10-bit PQ formats (xRGB_210LE/xBGR_210LE) with MANDATORY
BT.2020 + SMPTE-2084 colorimetry props, advertised while the mirrored
monitor is in BT.2100 colour mode. Wire the Linux host into it end-to-end
on the GameStream desktop-mirror path (PUNKTFUNK_VIDEO_SOURCE=portal):

* pf-frame: PixelFormat::X2Rgb10/X2Bgr10 (DRM XR30/XB30; X2Bgr10 is the
  Windows Rgb10a2 layout) + fourccs.
* pf-capture: want_hdr portal offer — HDR-only LINEAR-dmabuf pods with
  MANDATORY PQ/BT.2020 props (SHM excluded: Mutter's SHM record path
  paints 8-bit ARGB32 regardless of format; tiled excluded: the EGL
  de-tile blit is 8-bit RGBA8), negotiated-colorimetry parse, generic
  HDR10 hdr_meta(), packed-10-bit CPU cursor blend, a process-wide SDR
  downgrade latch on negotiation timeout, and a DisplayConfig BT.2100
  colour-mode probe (gnome_hdr_monitor_active).
* pf-encode: libav NVENC X2RGB10->P010 swscale (BT.2020 limited) ->
  HEVC Main10 / 10-bit AV1 with PQ VUI; VAAPI 10-bit on both paths (CPU
  P010 upload + dmabuf XR30 scale_vaapi p010/bt2020); can_encode_10bit
  now probes for real on Linux; 10-bit sessions route around the
  8-bit-only Vulkan-video/direct-NVENC backends.
* GameStream: host_hdr_capable() Linux arm, live monitor-HDR check at
  RTSP honor time, capturer-pool reuse keyed on HDR-ness, gs_bit_depth
  covers the new formats. New `punktfunk-host hdr-probe` diagnostic and
  a PUNKTFUNK_SPIKE_HDR spike lever.
* Native plane stays honestly 8-bit via capturer_supports_hdr(): Mutter
  RecordVirtual streams are SDR-only upstream (GNOME 50 and 51-dev), so
  virtual-display sources cannot deliver HDR yet.

Validated on the RTX 5070 Ti (GNOME 50.3 / PipeWire 1.6.8): the Main10
probes pass and the ignored nvenc_hdr10_smoke GPU test emits an IDR that
ffprobe reads as Main 10 / yuv420p10le / bt2020nc / smpte2084 / limited.
Live HDR capture negotiation still needs an HDR monitor on glass; VAAPI
10-bit needs the AMD box.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 09:30:55 +02:00
enricobuehler 4f64125025 docs: add a Plugins page (ROM Manager + Playnite) with install steps
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m22s
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 6m28s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 59s
ci / bench (push) Successful in 5m29s
ci / rust (push) Failing after 7m23s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m1s
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
docker / deploy-docs (push) Successful in 14s
arch / build-publish (push) Successful in 17m59s
android / android (push) Successful in 18m13s
New docs-site page documenting the two first-party plugins and how to
install them: the shared `bun add @punktfunk/plugin-*` + runner-enable
recipe, then ROM Manager (ROM roots + art) and Playnite (the host plugin
plus the Punktfunk Sync .pext exporter). Registered in the nav after
"automation" and cross-linked from the Events & hooks page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 01:31:08 +02:00
enricobuehler a8c8b1bb13 fix(apple): detect macOS HDR via potential EDR headroom, not current
apple / swift (push) Successful in 1m14s
release / apple (push) Successful in 9m12s
ci / web (push) Successful in 52s
apple / screenshots (push) Successful in 6m21s
ci / docs-site (push) Successful in 1m10s
ci / rust (push) Failing after 6m24s
ci / bench (push) Successful in 5m14s
decky / build-publish (push) Successful in 17s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
arch / build-publish (push) Successful in 11m58s
deb / build-publish (push) Successful in 12m27s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
deb / build-publish-host (push) Successful in 9m38s
android / android (push) Successful in 20m55s
windows-host / package (push) Successful in 19m4s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9m45s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9m20s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
docker / deploy-docs (push) Successful in 27s
`maximumExtendedDynamicRangeColorComponentValue` is the CURRENTLY-allocated EDR
headroom, which macOS hands out on demand — an idle SDR desktop reads 1.0 even
with an HDR display enabled and active, so gating HDR advertisement on it means
an HDR monitor (e.g. Samsung G95SC) never gets advertised at connect time. Use
`maximumPotentialExtendedDynamicRangeColorComponentValue`, the mode-independent
capability (the macOS analogue of the tvOS/iOS gates). Also point the Xcode
scheme's LaunchAction at Release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 01:17:10 +02:00
enricobuehler 75474dcc90 fix(pyrowave): guard 4:4:4 modes that overflow the rate controller's block index
The vendored rate controller packs its wavelet block index into 16 bits
(RDOperation.block_offset_saving), so a mode whose 32x32-block count exceeds
u16::MAX wraps inside the controller and corrupts the bitstream — ~8K-class
4:4:4 territory. Compute the exact count (`block_count_32x32`, the counting walk
of upstream init_block_meta, pinned against the validated Apple WaveletLayout)
and expose `pyrowave_mode_fits_rdo`; the negotiator downgrades such a session to
4:2:0 before the Welcome (the honest-downgrade channel), and both encoders
refuse outright if one slips through rather than emit a wrapped stream.

Vendor patches: 0002-rdo-saving-clamp (analyze_rate_control.comp clamps the
saving accumulation to the target, same overrun class as 0001; slangmosh.hpp
regenerated), 0003-devel-encode-16bit-read (devel tool y4m 16-bit plane reads;
tool-only, kept so the vendored source stays honest).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 01:16:51 +02:00
enricobuehler ac0e73321c perf(pyrowave): elevated GPU scheduling + global-priority encode queue
apple / screenshots (push) Successful in 6m18s
ci / web (push) Successful in 1m25s
ci / docs-site (push) Successful in 1m5s
android / android (push) Successful in 13m2s
arch / build-publish (push) Successful in 12m39s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
ci / bench (push) Successful in 5m36s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
windows-host / package (push) Successful in 16m23s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m8s
deb / build-publish-host (push) Successful in 10m16s
deb / build-publish (push) Successful in 12m30s
ci / rust (push) Successful in 19m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m20s
docker / deploy-docs (push) Successful in 23s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 24m32s
apple / swift (push) Successful in 1m17s
PyroWave's wavelet encode runs on the GPU's compute/shader cores, so a GPU-bound
game starves it: submit spikes from ~2 ms to ~15 ms under a 95%+ game load and
the stream fps collapses. NVENC is immune (separate encoder ASIC). Two levers to
let the encode get scheduled ahead of the game's rendering:

- Windows process GPU scheduling: D3DKMTSetProcessSchedulingPriorityClass, env
  PUNKTFUNK_GPU_PRIORITY = off|above-normal|high (default)|realtime. Best-effort,
  once per process, non-fatal on refusal (enc/windows/pyrowave.rs).
- Global-priority Vulkan encode queue (Granite patch 0005): request a
  VK_KHR_global_priority queue (PYROWAVE_QUEUE_PRIORITY = off|high|realtime,
  default realtime), downgrading REALTIME→HIGH→none on NOT_PERMITTED so a refused
  class never regresses the encoder to HEVC.

HONEST STATUS: on an RTX 4090 / Windows / WDDM neither moved the ~15 ms spikes —
the graphics-vs-compute preemption granularity is the wall, not the priority
level. Kept because both are correct, harmless (graceful fallback), and may help
other GPUs/drivers. For a GPU-saturated game the working levers are reducing the
encode's GPU cost (4:2:0/8-bit) or H.265; PyroWave holds full rate on the desktop
and in games that leave the GPU headroom.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 01:02:01 +02:00
enricobuehler dc20e4452e fix(pyrowave): anchor frame-driven pacing to arrival, hold full framerate
The frame-driven capture loop set its rate-limit floor (`earliest`) relative to
`next`, which is re-based to the instant AFTER submit(). For an async encoder
(NVENC) submit() returns in ~0, so that anchor is ~frame-arrival and the loop
correctly waits for the next vsync — full framerate. But PyroWave's encode is
SYNCHRONOUS (~2 ms inline in submit()), so the anchor lands ~2 ms late every
frame: the loop misses the next arrival and samples one interval behind, making
the period `interval + encode`. That capped a 240 Hz source at ~158 fps (and a
360 Hz request at ~200) with the link and the encoder both idle — no drops.

Anchor the floor to this frame's arrival (`t_cap`) instead. The synchronous
encode now overlaps the interval rather than stacking onto it; the ≥0.9×interval
spacing from the last grab still caps the rate at ~1.11× target. No-op for async
NVENC (t_cap ≈ post-submit there), which is why H.26x already held full rate.
Measured on-glass (5120x1440@240, RTX 4090 host, macOS client): desktop now
holds 240 fps. Also reduces latency (samples fresher frames).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 01:01:59 +02:00
enricobuehler 017b083e32 feat(library): serve provider entries' local art via the host art proxy
deb / build-publish-host (push) Successful in 9m32s
deb / build-publish (push) Successful in 12m29s
windows-host / package (push) Successful in 16m0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 8m48s
arch / build-publish (push) Successful in 19m57s
android / android (push) Successful in 22m41s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 21m30s
ci / rust (push) Successful in 29m59s
docker / deploy-docs (push) Successful in 23s
ci / web (push) Successful in 1m0s
apple / swift (push) Successful in 1m17s
ci / docs-site (push) Successful in 1m22s
decky / build-publish (push) Successful in 43s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
ci / bench (push) Successful in 6m34s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
apple / screenshots (push) Successful in 6m25s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m27s
A provider plugin that runs on the host (e.g. the Playnite sync plugin) can
now reconcile cover art as an on-host FILE PATH instead of an inlined data
URL. `GET /library` rewrites such local-file art into `/library/art/<id>/
<kind>` proxy URLs (the same relative-proxy shape Steam art already uses),
and the art proxy serves the bytes from the stored path. Moonlight's
/appasset proxy reads the local file too.

This is what lets a large Playnite library sync with covers: the reconcile
payload carries paths, not bytes, so it no longer blows past the mgmt API's
request-body limit and scales to thousands of titles.

Cross-platform; local-path detection is Windows-shaped (Playnite is
Windows-only). Unit-tested (compiles + 5/5 art tests on Linux).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 00:53:40 +02:00
enricobuehler e5eec51a78 docs(pyrowave): document 4:4:4 + HDR and add an interactive bitrate calculator
decky / build-publish (push) Successful in 20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 38s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 56s
arch / build-publish (push) Successful in 12m17s
deb / build-publish (push) Successful in 9m20s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m18s
docker / deploy-docs (push) Successful in 26s
android / android (push) Successful in 16m30s
deb / build-publish-host (push) Successful in 12m43s
ci / rust (push) Successful in 24m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m38s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m39s
ci / web (push) Successful in 48s
apple / swift (push) Successful in 1m22s
ci / docs-site (push) Successful in 54s
ci / bench (push) Successful in 5m55s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
apple / screenshots (push) Successful in 6m31s
- New "4:4:4 and HDR" section: what each mode buys, the ~1.6× / +15% bitrate
  cost, that HDR needs a Windows host today (Linux capture has no HDR source),
  and that the two combine (~1.9× the 4:2:0 SDR rate).
- Interactive <BitrateCalculator> (registered in the MDX components map):
  resolution (presets or custom), frame rate, 4:2:0/4:4:4, SDR/HDR -> the
  estimated Automatic pin, bits/pixel, per-frame size, and which link tier it
  needs. Formula mirrors the host's resolve_bitrate_kbps_for exactly.
- Expanded the bandwidth table with 120 Hz rows; note the big modes want 5/10 GbE.
- Document PUNKTFUNK_PYROWAVE_MAX_MBPS (cap the open-loop pin on a constrained
  link) in configuration.md.
- pyrowave.md -> .mdx so the page can host the component.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 19:28:28 +02:00
enricobuehler 9fe9c451dc perf(pyrowave): pool encoder scratch buffers + fix client parser O(n²) — lift the 2.5 Gbps wall
ci / web (push) Successful in 47s
ci / docs-site (push) Successful in 1m7s
apple / swift (push) Successful in 1m17s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
ci / bench (push) Successful in 5m51s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m51s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
release / apple (push) Successful in 9m11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m6s
deb / build-publish (push) Successful in 12m55s
android / android (push) Successful in 13m15s
docker / deploy-docs (push) Successful in 25s
deb / build-publish-host (push) Successful in 13m7s
arch / build-publish (push) Successful in 16m4s
windows-host / package (push) Successful in 16m19s
apple / screenshots (push) Successful in 6m34s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m47s
ci / rust (push) Successful in 24m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m31s
Real-world PyroWave streaming maxed ~2.5 Gbps with sagging fps while raw
transport does 4.8. Root-caused to serial per-frame paths at BOTH ends
(the transport was never the limit); this fixes the two dominant ones.

Host (vendored shim, patch 0004): pyrowave_encoder_encode_gpu_synchronous
allocated four Vulkan buffers (meta + bitstream, Device + CachedHost) on
EVERY frame. At 240 fps with MB-scale bitstreams that per-frame allocator
churn stalled the encode itself. Pool them on the encoder and reuse across
frames (recreate only on a size grow); the sizes are session-fixed, so it
is pure reuse after frame 1. On an RTX 4090 the 5120x1440 submit+fence-wait
drops ~15 ms -> ~1 ms, i.e. the host serial ceiling goes 64 -> 1025 fps
(444+HDR 44 -> 614). Safe under the synchronous encode model; re-validated
by pyrowave_win_smoke (Windows) and pyrowave_smoke/_444 (Linux). Applies to
both host encoder paths (they share the shim).

Client (Apple Metal decoder): WaveletBitstream.parse reserved the payload
buffer per packet (reserveCapacity(count + words), an exact realloc each of
~3000 packets/frame => O(n²)) and copied word-by-word. Reserve once up
front and memcpy each packet's coefficients in one shot (all Apple
platforms are little-endian, so the wire's LE u32s land verbatim; memcpy is
alignment-free). 5.44 ms -> 0.055 ms per 1.44 MB frame (25x); byte-identical
(parser unit tests + golden-frame PSNR unchanged).

Also:
- native.rs: PUNKTFUNK_PYROWAVE_MAX_MBPS caps PyroWave's open-loop Automatic
  bitrate pin for hosts on a constrained link (unset => no cap; an explicit
  client rate bypasses it). The pin is all-intra + ABR-off, so at a high
  pixel rate it can outrun the fabric (4:4:4+HDR 5120x1440@240 pins ~5.3
  Gbps, over a 5 GbE link) and the overshoot just becomes loss.
- pf-encode caps(): report the real opened chroma instead of a hardcoded
  4:2:0 default, so a genuine 4:4:4 session no longer trips the spurious
  "encoder chroma disagrees with the negotiated Welcome" warn. Also fix a
  latent Windows reset() that rebuilt at 4:2:0 for a 4:4:4 session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 19:09:23 +02:00
enricobuehler b89dbfa979 feat(gamescope): end dedicated stream on Steam game exit + auto --steam
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m12s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
ci / bench (push) Successful in 6m57s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m51s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
apple / screenshots (push) Successful in 6m13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m1s
deb / build-publish (push) Successful in 9m37s
docker / deploy-docs (push) Successful in 29s
deb / build-publish-host (push) Successful in 9m29s
windows-host / package (push) Successful in 16m24s
arch / build-publish (push) Successful in 17m40s
android / android (push) Successful in 18m22s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m39s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m52s
ci / rust (push) Successful in 26m16s
Dedicated single-game sessions now end cleanly when the launched game
quits, and Steam launches get gamescope's Steam integration without an
operator env knob.

End-on-exit: the existing APP_EXITED close only fired when gamescope's
PipeWire node vanished, which never happens for Steam — the nested
`steam` is the resident singleton client and stays up after a game
quits, so gamescope (and its node) never die and the stream sat on a
hidden Steam session forever. Add a Steam AppId reaper watcher
(pf-vdisplay .../gamescope/discovery.rs): steam_appid_from_launch()
parses steam://rungameid/<id>; wait_for_steam_game_exit() waits for the
game to start (<=300s grace) then exit (3s confirm); steam_game_running()
scans same-uid /proc for Steam's launch reaper matching both the
`SteamLaunch` and `AppId=<id>` argv tokens (exact-match; reaper lifetime
== game lifetime, so shader precompile can't false-trigger). The host
spawns a pf1-gamewatch thread for nested Steam launches that closes the
connection with APP_EXITED (launcher clients return to their library)
and sets quit/stop; cancelled via stop if the session ends first.
Non-Steam nested launches keep the node-death path (gamescope's child
IS the game).

Flags: auto-enable `--steam` whenever the launch is a Steam launch (was
only the global PUNKTFUNK_GAMESCOPE_STEAM knob, default-off) — in-game
overlay / Steam+X / gamepad-UI nav for Steam titles with no operator
config. New opt-in PUNKTFUNK_GAMESCOPE_GRAB_CURSOR adds
`--force-grab-cursor` for a real game launch (FPS mouselook); default
OFF because it forces relative mode, which breaks absolute-pointer
games/menus.

Verified: fmt clean; clippy -D warnings clean on the three crates;
pf-vdisplay 64/0 (incl. new steam-appid parse test); punktfunk-host
builds + 186/0; reaper /proc detection smoke-tested (detect, exact
non-match, gone-after-exit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 16:24:34 +02:00
enricobuehler 770994b7aa fix(web): tighten mobile padding so content isn't too narrow
apple / swift (push) Successful in 1m15s
apple / screenshots (push) Successful in 6m8s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m3s
ci / bench (push) Successful in 5m28s
arch / build-publish (push) Successful in 11m41s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 36s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 39s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m33s
android / android (push) Successful in 16m39s
windows-host / package (push) Successful in 16m15s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
docker / deploy-docs (push) Successful in 24s
deb / build-publish (push) Successful in 8m59s
ci / rust (push) Successful in 19m26s
deb / build-publish-host (push) Successful in 10m5s
The main content wrapper used p-6 (24px) on mobile while the mobile top bar uses
px-4 (16px) — an inconsistent, oversized side gutter that ate into the usable
width on phones. Drop it to a 16px side gutter (px-4) on mobile, matching the top
bar; sm+ padding is unchanged. Also make the shared Card padding responsive
(p-4 on mobile → p-6 from sm up) so card content isn't double-inset on small
screens. Desktop layout is identical.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 13:50:14 +02:00
enricobuehler 3ff1973d7f feat(pyrowave): Apple Metal 4:4:4 + HDR decode, EDR present — self-configured in-band
docker / deploy-docs (push) Successful in 11s
flatpak / build-publish (push) Successful in 6m25s
deb / build-publish (push) Successful in 9m7s
windows-host / package (push) Successful in 16m12s
deb / build-publish-host (push) Successful in 9m31s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m41s
ci / rust (push) Successful in 25m11s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m59s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m45s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m2s
apple / swift (push) Successful in 1m15s
release / apple (push) Successful in 9m36s
apple / screenshots (push) Successful in 6m43s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 50s
android / android (push) Successful in 13m41s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 14m23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
ci / bench (push) Successful in 5m58s
Phases 4+5 of design/pyrowave-444-hdr.md. The Metal decoder needs NO new ABI:
every frame's sequence header carries chroma (444) and, since the Phase-3
stamps, the PQ/BT.2020 bits — so the decoder self-configures per session.

Decoder: WaveletLayout grows the 4:4:4 block space (chroma runs the full
pyramid like luma — no level-0 skip, no early half-res emit; the Metal
kernels were already chroma-agnostic, only the dispatch structure changes);
the parser accepts chroma_resolution=444, reads the PQ transfer bit, and
lifts the even-dims rule for 444; the plane ring allocates full-res chroma
and r16Unorm for PQ streams; CSC rows switch to depth-10 MSB-packed.

Presenter: planar HDR passthrough reuses pf_frag_planar on an rgba16Float
drawable (itur_2100_PQ + EDR metadata interpret the samples — same split as
pf_frag/pf_frag_hdr), plus a new pf_frag_planar_tm PQ->SDR tone-map (shared
pqToSdr tail refactored out of pf_frag_hdr_tv) for tvOS-without-headroom AND
macOS WINDOWED sessions, whose IOSurface present path (the DCP-panic
mitigation) is BGRA8-only. SessionModel stops stripping the HDR/10-bit/444
caps on the PyroWave opt-in.

New golden: au-dense444 + upstream's own 4:4:4 reference planes (regenerated
via the extended pyrowave_dump_golden); Metal decode matches at 64-67 dB
(420 fixtures re-verify 77-88 dB). Full Apple suite 157 tests green on a
real M-series GPU. Docs updated: the 8-bit-SDR-only wording is gone, the
Windows host is no longer 'on the roadmap', bpp scaling documented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:21:23 +02:00
enricobuehler 188edde2b3 feat(pyrowave): Windows host HDR + 4:4:4, Rust client HDR present
Phase 3 of design/pyrowave-444-hdr.md. A PyroWave session now negotiates HDR
(10-bit) and 4:4:4 on a Windows host exactly like HEVC/AV1, and the Linux
client presents it through the real HDR10 path.

Host (Windows): BgraToYuvPlanes becomes mode-aware — SDR/BGRA and HDR/scRGB
variants at half- or full-res chroma. The HDR passes reuse HdrP010Converter's
exact colour math (scRGB -> PQ BT.2020 limited studio codes, verified by
hdr_p010_selftest) but write P010-style MSB-packed codes into two separate
shareable R16_UNORM/R16G16_UNORM textures; chroma keeps the pyrowave family's
centre-sited 2x2 box. idd_push pins the composition to the NEGOTIATED depth
(SDR sessions force advanced color off as before; 10-bit sessions enable it
and ride the FP16 ring), and the descriptor poller re-asserts that state
instead of following display flips the fixed-format encoder can't. The
encoder imports 8/16-bit planes per session and stamps the sequence header's
BT.2020/PQ/matrix bits on HDR (stamp_color_bits, extending 574e3e4e's range
stamp); supports_10bit/can_encode_10bit/can_encode_444 gates open (HDR
Windows-only — Linux capture has no HDR source).

Client: the plane ring becomes R16_UNORM for 10-bit sessions (with a
STORAGE_IMAGE format probe), the planar CSC pass joins the HDR10 swapchain
rebuild (set_hdr_mode previously destroyed it without rebuilding — latent),
st.hdr follows frame.color.is_pq(), and the planar push constants carry
depth-10 MSB-packed rows + the PQ tonemap mode, identical to the NV12 arm.

Verified: .173 (RTX 4090) deploy-config clippy + fmt + wire tests + the
extended pyrowave_win_smoke (10-case {SDR,HDR}x{420,444} matrix incl. R16
imports and header stamps); .21 (RTX 5070 Ti) clippy across 4 crates, host
186 tests, client/presenter/encode tests, both Linux GPU smokes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:21:06 +02:00
enricobuehler 4861824e7d feat(pyrowave): Linux 4:4:4 encode — per-pixel CSC, full-res chroma, gate open
ci / web (push) Successful in 49s
ci / docs-site (push) Successful in 55s
ci / bench (push) Successful in 6m44s
arch / build-publish (push) Successful in 12m8s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
android / android (push) Successful in 13m26s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
deb / build-publish (push) Successful in 12m31s
windows-host / package (push) Successful in 16m21s
deb / build-publish-host (push) Successful in 15m0s
apple / swift (push) Successful in 1m23s
apple / screenshots (push) Successful in 6m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m3s
ci / rust (push) Successful in 29m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m18s
docker / deploy-docs (push) Successful in 26s
Phase 2 of design/pyrowave-444-hdr.md. can_encode_444(PyroWave) now returns
true on Linux, so a client that advertises VIDEO_CAP_444 (the 4:4:4 setting)
and prefers PyroWave negotiates a full-chroma wavelet session end to end
(the client decoder side landed in 5eb930e7).

- rgb2yuv444.comp: the 4:4:4 twin of rgb2yuv.comp — one invocation per pixel,
  full-res interleaved RG8 CbCr, no box filter/siting, byte-identical BT.709
  limited coefficients; compiled .spv committed (glslangValidator -V, matches
  the existing shader's toolchain).
- Encoder: chroma-conditional pyrowave create (open + reset), full-res chroma
  plane + views, per-pixel dispatch, 4:2:0-only even-dims check.
- Tests: decode oracle grows a 4:4:4 mode (YUV444P CPU readback);
  pyrowave_smoke_444 round-trips plane means AND drives the busy test card at
  the ~2.6 bpp operating point asserting in-budget + run-to-run deterministic
  AU sizes — the exact regime that silently corrupted before the vendored
  payload_data fix (patches/0001), so this doubles as its regression test.

Verified on .21 (RTX 5070 Ti): clippy -D warnings, host tests, and both GPU
smokes (pyrowave_smoke + pyrowave_smoke_444) green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 12:43:34 +02:00
enricobuehler 5eb930e71d feat(pyrowave): negotiation plumbing for 4:4:4 + HDR — thread chroma/depth/ColorInfo end to end
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m49s
ci / web (push) Successful in 58s
ci / docs-site (push) Successful in 1m10s
arch / build-publish (push) Successful in 11m34s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m43s
ci / bench (push) Successful in 5m21s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m9s
deb / build-publish-host (push) Successful in 9m20s
android / android (push) Successful in 22m17s
flatpak / build-publish (push) Successful in 6m12s
ci / rust (push) Successful in 19m3s
deb / build-publish (push) Successful in 12m40s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 9m8s
apple / swift (push) Successful in 1m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m17s
docker / deploy-docs (push) Successful in 11s
apple / screenshots (push) Successful in 6m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 23m5s
windows-host / package (push) Successful in 16m27s
Phase 1 of design/pyrowave-444-hdr.md. No behavior change yet: the handshake's
4:4:4 gate now admits PyroWave (probe = can_encode_444(codec), capture gate
inherently satisfied — the wavelet path always ingests an RGB source and does
its own CSC), but can_encode_444 stays false for PyroWave until the per-OS
full-res-chroma CSC variants land (Phase 2 Linux, Phase 3 Windows), so every
session still resolves 4:2:0/8-bit.

- Both host encoders take the negotiated ChromaFormat (bail on 444 for now);
  the PUNKTFUNK_ENCODER=pyrowave lab override pins 4:2:0.
- Bitrate: the automatic ~1.6 bpp pin resolves AFTER depth+chroma and scales
  x1.625 for 4:4:4 / x1.15 for 10-bit (factors from the Phase-0 fixture
  matrix); the mid-stream mode-switch re-resolve threads the session's values.
- Client: PyroWaveDecoder builds its plane ring (full-res chroma when 444) and
  creates the upstream decoder from the negotiated chroma, keeps chroma fixed
  across mid-stream resizes, drops the even-dims requirement for 444, and
  returns the negotiated Welcome ColorInfo as the frame colour contract
  instead of hardcoded BT.709 (the wavelet bitstream has no VUI).

Verified on .21 (RTX 5070 Ti): clippy -D warnings (host+client+encode), host
186 tests, client + pf-encode tests, fmt, and the pyrowave_smoke GPU
round-trip through the patched vendored lib (97cf15e3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 12:37:12 +02:00
enricobuehler 97cf15e3b7 fix(pyrowave-sys): vendor patch — size the encoder's payload_data staging for 4:4:4
ci / rust (push) Successful in 18m56s
deb / build-publish-host (push) Successful in 12m29s
apple / swift (push) Successful in 1m21s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 25m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 25m25s
apple / screenshots (push) Successful in 6m28s
docker / deploy-docs (push) Successful in 24s
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m31s
ci / bench (push) Successful in 5m59s
arch / build-publish (push) Successful in 11m36s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
android / android (push) Successful in 17m0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
deb / build-publish (push) Successful in 9m0s
Upstream sizes the raw quantized-payload worst-case buffer at
aligned_width*aligned_height*2 bytes: real headroom for 4:2:0's 1.5
samples/px, but half of what 4:4:4's 3 samples/px can produce. Busy 4:4:4
content overruns the buffer on the GPU and corrupts the adjacent meta/bucket
suballocations — nondeterministic corrupt bitstreams and encoder crashes at
ANY target bitrate (smooth content never trips it, which is why 4:2:0 and
simple 4:4:4 both look fine). Found by the Phase-0 measurement matrix for
design/pyrowave-444-hdr.md; fix validated alone via upstream's own devel
tools on the RTX 5070 Ti: deterministic byte-identical outputs across runs,
1080p + 4K, 8- and 16-bit, PSNR at the expected operating points.

Patch lives in crates/pyrowave-sys/patches/ and vendor-pyrowave.sh now
re-applies patches on re-vendor; PUNKTFUNK-VENDOR.txt records it. Upstream
report to follow. No wire/ABI change; 4:2:0 sizing unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 12:16:56 +02:00
enricobuehler 94533bb071 feat(scripting): discover scoped @punktfunk/plugin-* packages
apple / swift (push) Successful in 1m24s
apple / screenshots (push) Successful in 6m27s
windows-host / package (push) Failing after 15m41s
ci / web (push) Successful in 1m33s
ci / docs-site (push) Successful in 1m26s
android / android (push) Successful in 14m3s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
ci / bench (push) Successful in 7m40s
deb / build-publish (push) Successful in 8m58s
deb / build-publish-host (push) Successful in 9m32s
arch / build-publish (push) Successful in 17m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m33s
ci / rust (push) Successful in 28m32s
docker / deploy-docs (push) Failing after 12s
The runner only found unscoped `punktfunk-plugin-*` packages. An unscoped
package can't be split across registries, which forced plugins to bundle their
own copy of @punktfunk/host + effect (the plugin's Gitea registry has no
`effect`; `--registry <gitea>` 404s it). Discovering the scoped
`@punktfunk/plugin-*` convention lets a plugin resolve cleanly from one scope
map and depend on @punktfunk/host + effect as SHARED (hoisted) deps instead of
bundling — no per-plugin duplication. Additive: unscoped names still work.

Bumps @punktfunk/host to 0.1.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 12:12:11 +02:00
enricobuehler f407f41855 fix(apple): present windowed macOS PyroWave via IOSurface layer contents — swapID panic survives glass pacing
ci / bench (push) Successful in 7m3s
deb / build-publish (push) Successful in 9m22s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
arch / build-publish (push) Successful in 12m22s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
android / android (push) Successful in 13m43s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
deb / build-publish-host (push) Successful in 12m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m34s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m41s
ci / rust (push) Successful in 28m55s
apple / swift (push) Successful in 1m20s
release / apple (push) Successful in 9m32s
ci / web (push) Successful in 46s
ci / docs-site (push) Successful in 1m5s
apple / screenshots (push) Successful in 6m28s
docker / deploy-docs (push) Successful in 26s
The DCP "mismatched swapID's" kernel panic reproduced on a 240 Hz Mac
Studio with stage-3 glass pacing active: a fully serialized,
one-in-flight present stream still races WindowServer's own swap
submissions. So the mitigation has to change the MECHANISM, not the
rate — the CAMetalLayer image queue itself is the racing path in a
composited (windowed) session.

Windowed PyroWave now presents the way video players do: the planar
CSC renders into a pooled IOSurface (4 × BGRA8, in-use-aware LRU
reuse) and the render thread hands it to a plain CALayer's `contents`
on main inside an ordinary CATransaction. WindowServer treats that as
normal layer damage on its own composite cadence — no out-of-band
image-queue swaps to race. Fullscreen keeps the CAMetalLayer path
(direct scanout, no compositing, no panic reports); the hosting view
pushes the window's composited state on every layout, and flipping
modes just covers/uncovers the metal layer (no black flash).

VT codecs keep the metal path everywhere: no panic reports there, and
their HDR/EDR presentation has no surface-contents equivalent wired.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 11:51:31 +02:00
enricobuehler 574e3e4e3f fix(pyrowave): signal ycbcr_range=LIMITED in the sequence header
docker / deploy-docs (push) Successful in 23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m8s
apple / swift (push) Successful in 1m23s
apple / screenshots (push) Successful in 6m20s
ci / docs-site (push) Successful in 1m10s
ci / web (push) Successful in 1m35s
android / android (push) Successful in 13m8s
windows-host / package (push) Successful in 16m16s
ci / bench (push) Successful in 5m21s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m28s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
arch / build-publish (push) Successful in 18m52s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
deb / build-publish-host (push) Successful in 9m38s
deb / build-publish (push) Successful in 12m35s
ci / rust (push) Successful in 19m12s
pyrowave's encoder fills the BitstreamSequenceHeader with `= {}` and its C API
offers no way to set colour/range, so it signals ycbcr_range=0=FULL — but both
host CSCs (rgb2yuv.comp on Linux, BgraToYuvPlanes on Windows) always emit BT.709
LIMITED Y'CbCr (black = Y'16). A client that honours the VUI (the Apple wavelet
decoder reads bit 30 of word1) then skips the limited→full expansion and shows
washed-out, raised blacks — reported on both Linux and Windows hosts.

Patch the range bit HONEST (mark_limited_range in the shared pyrowave_wire, called
by both encoders after packetize). Clients that hardcode limited (the Vulkan
video_pyrowave path) are unaffected, and pyrowave's own decode ignores the flag
(raw Y'CbCr reconstruction). No client rebuild needed. Unit-tested + asserted in
pyrowave_win_smoke; the smoke decode still round-trips 100/180/60.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 11:39:44 +02:00
enricobuehler ebd9967547 feat(pyrowave): Windows host encoder — separate-plane zero-copy D3D11→Vulkan
Wire PyroWave into the Windows host (design/pyrowave-windows-host-zerocopy.md).
Before this a macOS client + Windows host that both selected PyroWave silently ran
HEVC: the host never advertised CODEC_PYROWAVE and open_video_backend bailed.

Approach (zero-copy, no GPU→CPU→GPU): pyrowave owns its own Vulkan device
(create_device_by_compat, by render-GPU vendor/device-id — NOT LUID, invalid in
Session 0). The capturer runs a BGRA→YUV BT.709-limited CSC (matching rgb2yuv.comp)
into TWO SEPARATE shareable plane textures — full-res R8 Y + half-res R8G8 CbCr —
which the encoder imports into pyrowave's device. Separate single/two-component
textures import reliably on NVIDIA at any size; a single planar NV12 import does NOT
(the vendored interop test: "only very specific resource sizes" — confirmed on-glass:
1024² fine, 720p/1080p/1440p garbage). A shared D3D11 fence, signalled after the CSC,
is imported as a Vulkan timeline semaphore so the wavelet read is ordered after it.

- pf-encode: enc/windows/pyrowave.rs (Encoder impl, two-plane import + Linux-style
  plane views); host_wire_caps advertises CODEC_PYROWAVE on Windows when the backend
  isn't Software; open_video_backend routes a negotiated PyroWave session first;
  pyrowave-sys on the Windows target; interop confirmed at open → clean HEVC fallback.
- pf-encode: shared, unit-tested enc/pyrowave_wire.rs (single source of truth for the
  client-facing AU framing); Linux encoder uses it too.
- pf-capture: dxgi.rs BgraToYuvPlanes CSC; idd_push.rs pyrowave mode — forces the
  virtual display SDR (the VideoProcessor can't ingest the FP16 HDR ring), a
  two-plane shareable out-ring, a shared fence passed every frame (so a rebuilt
  encoder re-imports it). Threaded via OutputFormat::pyrowave.
- pf-frame: D3d11Frame::pyro carries the CbCr plane + fence; OutputFormat::pyrowave.

Verified on .173 (RTX 4090): full-host build + clippy -D warnings (nvenc,amf-qsv) +
fmt --all --check; pyrowave_wire unit tests; pyrowave_win_smoke GPU test round-trips
distinct Y/Cb/Cr (100/180/60) exactly at 1024²/720p/1080p/1440p; Stage-0 interop
validated in the real Session-0 service context on-glass. Deployed to the box.
Owed: final on-glass picture/latency confirmation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 11:39:44 +02:00
enricobuehler 1e7c18b2c8 fix(packaging): install punktfunk-host on Ubuntu 24.04 LTS via a noble builder that bundles FFmpeg 8
ci / web (push) Successful in 58s
ci / docs-site (push) Successful in 52s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 5m31s
arch / build-publish (push) Successful in 11m54s
apple / screenshots (push) Successful in 6m29s
android / android (push) Successful in 14m53s
deb / build-publish (push) Successful in 9m6s
deb / build-publish-host (push) Successful in 12m9s
ci / rust (push) Successful in 19m5s
docker / deploy-docs (push) Successful in 26s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m55s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 23m46s
The host .deb was built on the Ubuntu 26.04 rust-ci image, so dpkg-shlibdeps
baked in `Depends: libavcodec62` (FFmpeg 8) and a glibc-2.41 floor — making it
uninstallable on Ubuntu 24.04 LTS (FFmpeg 6.1 / libavcodec60, glibc 2.39; apt
reports the deps as "too recent"). The source floor (ffmpeg-next 8, libavcodec
>=61 APIs) means a straight 24.04 rebuild would fail too.

Build the host on Ubuntu 24.04 instead — lowering the glibc floor to 2.39 so one
binary runs on 24.04 -> 26.04 — and bundle a from-source LGPL FFmpeg 8 into the
package so it no longer depends on the distro libav*. Everything else the host
links is soname-compatible on 24.04 (opus is vendored via cmake; NVENC/libcuda
are dlopen-only, never link-time), and the only FFmpeg encoders used are
*_nvenc / *_vaapi (software H.264 fallback is the BSD-2 openh264 crate, not
FFmpeg libx264), so an LGPL build keeps the bundle license-clean.

- ci/rust-ci-noble.Dockerfile (new): ubuntu:24.04 builder; nv-codec-headers +
  FFmpeg 8 (--enable-nvenc --enable-vaapi, shared) -> /opt/ffmpeg; PKG_CONFIG_PATH.
- packaging/debian/build-deb.sh: BUNDLE_FFMPEG=1 copies libav*/libsw*/libpostproc
  into /usr/lib/punktfunk-host, patchelf-sets the rpath ($ORIGIN per-lib + binary
  --force-rpath), feeds them to dpkg-shlibdeps (captures libva2/libdrm2), and drops
  the libav* sonames from Depends. Normal (non-bundle) path unchanged.
- .gitea/workflows/deb.yml: split into build-publish (client/web/scripting on the
  26.04 image) and build-publish-host (noble image, BUNDLE_FFMPEG=1); parallel,
  identical version step, same apt distribution -> one universal host .deb.
- .gitea/workflows/docker.yml: build+push punktfunk-rust-ci-noble.
- packaging/debian/README.md: document the 24.04 LTS path + bundled local build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 11:30:19 +02:00
enricobuehler 9aebc3f251 fix(apple): default macOS PyroWave sessions to glass-gated present pacing — DCP swapID kernel-panic mitigation
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 16s
ci / bench (push) Successful in 7m29s
docker / deploy-docs (push) Successful in 26s
release / apple (push) Successful in 10m9s
arch / build-publish (push) Successful in 13m2s
android / android (push) Successful in 14m9s
deb / build-publish (push) Successful in 16m41s
apple / screenshots (push) Successful in 6m26s
ci / rust (push) Successful in 20m34s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m8s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 54s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
apple / swift (push) Successful in 1m15s
decky / build-publish (push) Successful in 20s
Windowed PyroWave sessions were kernel-panicking Macs ("mismatched
swapID's" @UnifiedPipeline.cpp, WindowServer → AppleMobileDisp*-DCP).
The panic is an Apple DCP bug (SketchUp/Unity/240Hz-monitor reports hit
the same signature), but our stage-2 arrival pacing feeds the trigger
pattern: displaySyncEnabled=false out-of-band presents arriving faster
than the compositor latches them in a composited (windowed) session.
PyroWave makes that pattern routine — near-instant Metal wavelet decode
turns network clumps into same-millisecond present bursts, and it is
the codec that sustains stream rates above panel refresh.

Mitigation: PyroWave sessions on macOS now default to the stage-3
PresentGate (one presented-but-undisplayed swap in flight, serialized
on the on-glass callback, 100 ms stale backstop) — the racing pattern
cannot occur. Explicit stage2/stage3 picks (setting or
PUNKTFUNK_PRESENTER) still win, so the arrival-pacing A/B stays honest.
VideoToolbox codecs keep arrival pacing (decode latency spaces their
presents; no panic reports there).

PresenterChoice gains an `explicit` resolver (nil = no user selection)
so the codec-conditional default only applies when the user hasn't
picked a stage; pacing selection is a testable helper carrying the
rationale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 11:17:48 +02:00
enricobuehler 7f639f7cf5 docs: link every client in the clients-page chooser + correctness sweep
apple / swift (push) Successful in 1m16s
apple / screenshots (push) Successful in 6m29s
ci / web (push) Successful in 1m11s
ci / docs-site (push) Successful in 56s
android / android (push) Successful in 13m53s
arch / build-publish (push) Successful in 12m20s
decky / build-publish (push) Successful in 19s
ci / bench (push) Successful in 5m50s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 59s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5m45s
docker / deploy-docs (push) Successful in 29s
deb / build-publish (push) Successful in 11m25s
ci / rust (push) Successful in 24m23s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m16s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m8s
The "Which should I use?" table on the clients page listed most client
names as bold text, not links — only the Decky plugin and pf-webos were
clickable — so the client references appeared broken. Link each client to
its section (or dedicated page), and fix a stale Windows headless command.

Repo-wide docs correctness/staleness pass against the code:
- steam-deck: client-not-found -> flatpak-not-found (the real backend code)
- install: host cert is punktfunk-host-windows_<ver>.cer, not ..._setup.cer
- configuration: GPU_PRIORITY_CLASS default is auto; 10BIT/444 are default-on
- how-it-works/index: GameStream/Moonlight is opt-in (--gamestream)
- roadmap: clipboard sync is shipped, not planned
- install-client: MSIX/cert artifacts are arch-suffixed (_x64/_arm64)
- requirements: fix garbled 22H2/IddCx sentence
- status: Linux encode also covers AMD/Intel (VAAPI/Vulkan Video)
- automation: add the plugins.changed event
- windows-host: note the optional bundled VB-CABLE virtual mic
- sway: PUNKTFUNK_COMPOSITOR=hyprland is a wlroots-family alias
- running-as-a-service: punktfunk-probe is a source-build-only dev tool

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 09:50:29 +02:00
enricobuehler 2c0aee3979 chore(plugins): regenerate OpenAPI spec + SDK client for /api/v1/plugins
ci / web (push) Successful in 1m11s
ci / docs-site (push) Successful in 1m13s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 32s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
ci / bench (push) Successful in 6m40s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
apple / screenshots (push) Successful in 6m26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9m52s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m57s
deb / build-publish (push) Successful in 13m13s
windows-host / package (push) Successful in 15m54s
arch / build-publish (push) Successful in 17m25s
android / android (push) Successful in 18m9s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m26s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m50s
ci / rust (push) Successful in 27m22s
docker / deploy-docs (push) Successful in 23s
Regenerated api/openapi.json (host built on Linux) with the plugins tag —
registerPlugin/listPlugins/deregisterPlugin/getPluginUiCredential + the
PluginRegistration/PluginSummary/PluginUi/PluginUiPublic/UiCredential schemas —
and the SDK's generated client. Drift test + 44 mgmt/events host tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 09:48:37 +02:00
enricobuehler ec84b30eae feat(plugins): console-hosted plugin UI surface (host registry + SDK servePluginUi + console proxy/nav)
Implements planning/design/plugin-ui-surface.md (U1-U3):

- host: in-memory lease-based plugin registry (mgmt/plugins.rs) — PUT/GET/DELETE
  /api/v1/plugins + GET /plugins/{id}/ui-credential; bearer+loopback only (not on
  the mTLS read-only allowlist); plugins.changed event; port-only registration
  (proxy always dials 127.0.0.1); secret never in the listing.
- sdk: servePluginUi — loopback ephemeral bind + per-boot secret + constant-time
  check + /__health + static/SPA-fallback + register/renew(30s)/deregister via
  pf.request (skew-proof, D7). Example + tests.
- console: /plugin-ui/{id}/** reverse proxy (server-side secret injection, cookie
  strip, SSE streaming, stale-secret 401-retry) + credential cache; BFF denylist
  for the credential endpoint; dynamic Plugins nav (desktop + mobile) fed by a
  polled list; iframe-in-shell page with health probe, offline card, open-in-tab,
  deep-link sync. Dev-mode /plugin-ui middleware in vite.config.ts.

OpenAPI regen for the new endpoints follows in the next commit (built on Linux).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 09:48:37 +02:00
enricobuehler d579cd318e ci(windows-host): lint host + tray in --release so clippy reuses the release build
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m13s
docker / deploy-docs (push) Successful in 12s
ci / rust (push) Successful in 19m29s
deb / build-publish (push) Successful in 11m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m54s
apple / swift (push) Successful in 1m22s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m34s
windows-host / package (push) Successful in 16m22s
apple / screenshots (push) Successful in 6m18s
ci / web (push) Successful in 1m26s
ci / docs-site (push) Successful in 1m38s
android / android (push) Successful in 13m16s
ci / bench (push) Successful in 6m0s
arch / build-publish (push) Successful in 11m57s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 17s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
The clippy step ran in the default (debug) profile, compiling the whole
dep tree into a second target dir (C:\t\debug) — a second full build of
openh264-sys2's vendored C++ (pf-encode's software-H.264 fallback) on top
of the release copy the Build steps already produced. That second cc-rs
cl.exe fan-out tips the self-hosted runner into
`cabac_decoder.cpp: fatal error C1069 (cannot read compiler command line)`
— environmental disk/temp exhaustion, not a source error: the identical
file compiles fine in the release build minutes earlier, and openh264 is
untouched here. Linting in --release reuses the release build-script
artifacts (no openh264 rebuild) and keeps everything in one C:\t\release
tree. Matches pf-vkhdr-layer's clippy, which already runs --release.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 02:12:43 +02:00
enricobuehler bb755ef7d2 fix(gamestream): generate the RSA host identity via the rsa crate, not rcgen (ring can't RSA-keygen)
apple / screenshots (push) Successful in 6m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m54s
android / android (push) Successful in 13m24s
windows-host / package (push) Successful in 18m37s
deb / build-publish (push) Successful in 12m42s
arch / build-publish (push) Successful in 16m12s
ci / rust (push) Successful in 27m16s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m29s
docker / deploy-docs (push) Successful in 27s
ci / web (push) Successful in 1m22s
apple / swift (push) Successful in 1m27s
ci / docs-site (push) Successful in 1m29s
decky / build-publish (push) Successful in 26s
ci / bench (push) Successful in 6m16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
The workspace went ring-only in 0.13.0 (aws-lc-sys breaks Windows CI), and
`ring` can sign with an existing RSA key but cannot *generate* one —
rcgen's ring backend returns `KeyGenerationUnavailable` for
`generate_for(&PKCS_RSA_SHA256)`. So `ServerIdentity::{load_or_create,
ephemeral}` panics with "rcgen RSA keygen / There is no support for
generating keys for the given algorithm" on any host without an existing
`cert.pem`. This is the shared trust root for both the GameStream TLS cert
and the QUIC identity clients pin, so a *fresh* 0.13.0 install can't start
the host at all (existing dev boxes survive only because they already have
a cert on disk); in CI every test that builds a `ServerIdentity` (all of
`mgmt::tests`, `gamestream::nvhttp`, and the `native::tests` synthetic host
via `serve()`) failed — 30 failures from this one cause.

Moonlight requires an RSA-2048 identity, so generate the key with the
pure-Rust `rsa` crate (already a dependency for the pairing signer) and
hand its PKCS#8 PEM to rcgen, whose ring backend *can* load and self-sign
with an externally supplied RSA key (`from_pkcs8_pem_and_sign_algo` →
`RsaKeyPair::from_pkcs8`). Return that same PEM so it stays byte-identical
to what `from_pems` re-parses.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 01:59:28 +02:00
enricobuehler 08a397cf08 fix(transport): treat Windows WSAENOBUFS as a transient send drop, not a stream teardown
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 7m18s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m14s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 9m17s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 57s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 21s
apple / swift (push) Successful in 1m20s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
ci / bench (push) Successful in 7m7s
flatpak / build-publish (push) Failing after 8m7s
docker / deploy-docs (push) Successful in 22s
release / apple (push) Successful in 9m3s
windows-host / package (push) Failing after 11m41s
deb / build-publish (push) Successful in 13m34s
ci / rust (push) Failing after 14m2s
arch / build-publish (push) Successful in 14m59s
android / android (push) Successful in 16m45s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m16s
apple / screenshots (push) Successful in 6m38s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 7m16s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m38s
`is_transient_io` only recognized `ENOBUFS` on unix; on Windows the
`#[cfg(not(unix))]` arm returned `false`, so `WSAENOBUFS` (10055) — which
Rust maps to `ErrorKind::Uncategorized`, missing the `WouldBlock` arm —
propagated out of the USO `send_gso` path through `Session::send_sealed`
and killed the session with `native::stream` "send failed — stopping
stream". A high-bitrate keyframe burst (one `WSASendMsg` USO super-buffer
is up to ~512 segments) momentarily exhausts the socket send buffer / AFD
non-paged pool; it's a lossy drop that FEC + the next frame recover,
exactly like the unix `ath11k` `ENOBUFS` case the classifier already
handles. Fires independently of client platform (Windows/macOS clients
both saw the crash) because it's the Windows *host's* send socket.

Add a `#[cfg(windows)]` arm matching `WSAENOBUFS` plus the
`WSAENET*`/`WSAEHOST*` network-path family (the Windows counterparts of
the droppable unix set), and extend the classifier test with per-platform
raw-errno coverage so the Windows CI runner exercises the 10055 path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 01:43:15 +02:00
enricobuehler 11cca33300 fix(host/hooks): honor hand-edits to hooks.json without a restart
android / android (push) Successful in 19m27s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m51s
windows-host / package (push) Successful in 18m48s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m58s
ci / rust (push) Failing after 14m6s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 58s
apple / swift (push) Successful in 1m21s
decky / build-publish (push) Successful in 20s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 5m56s
apple / screenshots (push) Successful in 6m15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8m35s
docker / deploy-docs (push) Successful in 27s
deb / build-publish (push) Successful in 12m33s
arch / build-publish (push) Successful in 18m31s
docs/automation.md promises 'changes apply immediately, no restart' for BOTH
config paths, but the store loaded hooks.json once (OnceLock) — only PUT
/api/v1/hooks applied live, and a hand-edited file silently did nothing until
the next host start. Since the file is the accessible path most users will
take, make it real: get() re-stats the file per event (mtime + length) and
re-reads it on change, with the same lenient contract as startup (missing =
no hooks, invalid = disabled loudly). set() records the identity it wrote so
the API path doesn't trigger a spurious reload.

Validated live on the 0.13.0 host (.21): full connect/disconnect lifecycle
fires exec hooks (client.*/session.*/stream.*) — the reload gap was found
setting that test up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 23:53:25 +02:00
139 changed files with 14054 additions and 2574 deletions
+119 -18
View File
@@ -1,8 +1,18 @@
# Build the punktfunk-host and punktfunk-client .debs and publish them to Gitea's Debian
# package registry, so Ubuntu boxes get new builds via `apt update && apt upgrade`. Runs
# inside the same Ubuntu 26.04 rust-ci builder image as ci.yml, so dpkg-shlibdeps pins the
# runtime lib package names (libavcodec62, libpipewire-0.3-0t64, …) to exactly what the
# target boxes run.
# Build the punktfunk .debs and publish them to Gitea's Debian package registry, so Ubuntu
# boxes get new builds via `apt update && apt upgrade`. Two jobs, both publishing to the same
# apt distribution/component:
#
# build-publish — client + web + scripting, on the Ubuntu 26.04 rust-ci image (the client
# needs 24.04-absent libs: SDL3, GTK4 ≥ 4.20).
# build-publish-host — the HOST, on the Ubuntu 24.04 rust-ci-noble image with a from-source
# FFmpeg 8 BUNDLED into the .deb. This lowers the host's glibc floor to 2.39
# and removes the hard `Depends: libavcodec62`, so the ONE host .deb installs
# on Ubuntu 24.04 LTS through 26.04. (A 26.04-built host .deb is uninstallable
# on 24.04 — the reason this job exists; see packaging/debian/README.md.)
#
# Both compute VERSION identically (scripts/ci/pf-version.sh is deterministic per commit), so the
# host and client packages always share a version line. The release-attach helpers are race-safe
# (ensure_release create-or-fetch; upsert_asset only conflicts on same-name), so the jobs run parallel.
#
# Registry (public, unom org): https://git.unom.io/unom/-/packages
# Box setup (once): see packaging/debian/README.md
@@ -83,22 +93,15 @@ jobs:
key: cargo-target-v3-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-target-v3-${{ env.rustc }}-
- name: Build release host + client
- name: Build release clients
env:
PUNKTFUNK_BUILD_VERSION: ${{ env.VERSION }} # stamped into the binary (build.rs)
PUNKTFUNK_BUILD_VERSION: ${{ env.VERSION }} # stamped into the binaries (build.rs)
run: |
git config --global --add safe.directory "$PWD"
# punktfunk-client-session is the Vulkan/Skia streamer the shell execs for a connect —
# both client binaries must ship (build-client-deb.sh installs both).
# --features punktfunk-host/nvenc: the direct-SDK NVENC path (real RFI + recovery anchor on
# Linux NVIDIA; design/linux-direct-nvenc.md). AMD/Intel-safe — NVENC/CUDA is dlopen'd at
# runtime (no link-time dep; identical DT_NEEDED to a plain build), and the encoder is only
# constructed for a CUDA capture frame + PUNKTFUNK_NVENC_DIRECT, never on VAAPI hosts.
# --features punktfunk-host/vulkan-encode: the AMD/Intel twin — raw VK_KHR_video_encode_h265
# with real RFI (design/linux-vulkan-video-encode.md). Pure Rust ash (no new lib / link dep);
# default on for HEVC (PUNKTFUNK_VULKAN_ENCODE=0 → libav VAAPI), failed open falls back to VAAPI.
cargo build --release --locked --features punktfunk-host/nvenc,punktfunk-host/vulkan-encode \
-p punktfunk-host -p punktfunk-client-linux -p punktfunk-client-session
# both client binaries must ship (build-client-deb.sh installs both). The HOST is built
# separately in the build-publish-host job (Ubuntu 24.04 image + bundled FFmpeg 8).
cargo build --release --locked -p punktfunk-client-linux -p punktfunk-client-session
- name: Build + smoke-boot web console (bun preset)
# Gate the .deb on a real bun boot: the punktfunk-web .deb runs the Nitro `bun` preset
@@ -128,7 +131,7 @@ jobs:
- name: Build .debs
run: |
export PATH="$HOME/.bun/bin:$PATH"
VERSION="$VERSION" bash packaging/debian/build-deb.sh
# host .deb is built in build-publish-host (Ubuntu 24.04 image); this job ships the rest.
VERSION="$VERSION" bash packaging/debian/build-client-deb.sh
# Reuse CI's bun for the vendored runtime (matches the amd64 runner) instead of downloading.
VERSION="$VERSION" BUN_BIN="$(command -v bun || true)" bash packaging/debian/build-web-deb.sh
@@ -168,3 +171,101 @@ jobs:
for DEB in dist/*.deb; do
upsert_asset "$RID" "$DEB"
done
# ---------------------------------------------------------------------------------------------
# The HOST .deb — built on Ubuntu 24.04 (noble) with a from-source FFmpeg 8 BUNDLED, so it installs
# on Ubuntu 24.04 LTS through 26.04 (see the file header + packaging/debian/README.md). Runs in
# parallel with build-publish and publishes to the SAME distribution/component; the version step is
# byte-identical so host and clients share a version line.
build-publish-host:
runs-on: ubuntu-24.04
container:
image: git.unom.io/unom/punktfunk-rust-ci-noble:latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- name: Version + channel
run: |
git config --global --add safe.directory "$PWD"
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; DIST=stable ;;
*) V="${PF_BASE}~ci${GITHUB_RUN_NUMBER}.g${SHORT}"; DIST=canary ;;
esac
echo "VERSION=$V" >> "$GITHUB_ENV"
echo "DISTRIBUTION=$DIST" >> "$GITHUB_ENV"
echo "host package version $V -> apt distribution '$DIST'"
# dpkg-shlibdeps/dpkg-deb + patchelf are baked into rust-ci-noble; python3 is for the
# release-attach helper. Re-install defensively so the job stays green against the PREVIOUS
# image on the same push (docker.yml bootstrap lag) — a no-op once the image ships them.
- name: dpkg-dev + patchelf + python3
run: |
apt-get update
apt-get install -y --no-install-recommends dpkg-dev patchelf python3
- name: Cache keys
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
- uses: actions/cache@v4
with:
path: |
/usr/local/cargo/registry
/usr/local/cargo/git
key: cargo-home-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-home-
- uses: actions/cache@v4
with:
path: target
# Own key: this target dir is built against 24.04's glibc/toolchain and must NOT share
# ci.yml's 26.04 target cache (mixing would poison both).
key: cargo-target-noble-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-target-noble-v1-${{ env.rustc }}-
- name: Build release host
env:
PUNKTFUNK_BUILD_VERSION: ${{ env.VERSION }} # stamped into the binary (build.rs)
run: |
git config --global --add safe.directory "$PWD"
# Same features the old combined build used: --nvenc (direct-SDK NVENC, real RFI on NVIDIA;
# NVENC/CUDA is dlopen'd — no link dep, so this image needs no libcuda stub) + --vulkan-encode
# (raw VK_KHR_video_encode_h265 on AMD/Intel, pure ash). punktfunk-tray also ships in the host
# .deb (build-deb.sh builds+installs it). ffmpeg-sys-next links the image's bundled FFmpeg 8
# via PKG_CONFIG_PATH (set in rust-ci-noble).
cargo build --release --locked --features punktfunk-host/nvenc,punktfunk-host/vulkan-encode \
-p punktfunk-host -p punktfunk-tray
- name: Build host .deb (FFmpeg bundled)
# BUNDLE_FFMPEG=1 copies the image's /opt/ffmpeg libav* into the package and repoints the
# binary's rpath, so there is no `Depends: libavcodec62` to block install on 24.04. FFMPEG_PREFIX
# defaults to /opt/ffmpeg (the image's ENV also sets it).
run: |
VERSION="$VERSION" BUNDLE_FFMPEG=1 bash packaging/debian/build-deb.sh
- name: Publish to the Gitea apt registry
env:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
for DEB in dist/*.deb; do
echo "uploading $DEB"
NAME=$(dpkg-deb -f "$DEB" Package)
VER=$(dpkg-deb -f "$DEB" Version)
ARCH=$(dpkg-deb -f "$DEB" Architecture)
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
"https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/$NAME/$VER/$ARCH" || true
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$DEB" \
"https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/upload"
done
echo "published host to $OWNER/debian $DISTRIBUTION/$COMPONENT"
- name: Attach the host .deb to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v')
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
. scripts/ci/gitea-release.sh
RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
for DEB in dist/*.deb; do
upsert_asset "$RID" "$DEB"
done
+6
View File
@@ -39,6 +39,12 @@ jobs:
- image: punktfunk-rust-ci
dockerfile: ci/rust-ci.Dockerfile
context: ci
# Ubuntu 24.04 LTS host builder: same purpose as rust-ci but lowers the host .deb's glibc
# floor to 2.39 and bundles a from-source FFmpeg 8, so the package installs on 24.04 LTS
# (rust-ci's 26.04 build is uninstallable there). Consumed by deb.yml's build-publish-host job.
- image: punktfunk-rust-ci-noble
dockerfile: ci/rust-ci-noble.Dockerfile
context: ci
- image: punktfunk-fedora-rpm
dockerfile: ci/fedora-rpm.Dockerfile
context: ci
+11 -2
View File
@@ -145,9 +145,18 @@ jobs:
- name: Clippy (host + tray, Windows)
shell: pwsh
# First-ever Windows lint coverage for the host (Linux CI never lints the windows-cfg code).
# --release is REQUIRED, not just faster: a default (debug) clippy compiles the whole dep tree
# into a SECOND target dir (C:\t\debug), which means a second full build of openh264-sys2's
# vendored C++ (the software-H.264 fallback in pf-encode) on top of the release copy the Build
# steps above already produced. That second cc-rs `cl.exe` fan-out tips this runner over into
# `cabac_decoder.cpp: fatal error C1069 (cannot read compiler command line)` — an environmental
# disk/temp exhaustion, NOT a source error (the identical file compiles fine in the release
# build minutes earlier). Linting in release reuses those native build-script artifacts (no
# openh264 rebuild), and keeps everything in one C:\t\release tree. Same reason
# pf-vkhdr-layer's clippy below runs --release.
run: |
cargo clippy -p punktfunk-host --features nvenc,amf-qsv,qsv -- -D warnings; if ($LASTEXITCODE) { throw "host clippy" }
cargo clippy -p punktfunk-tray -- -D warnings; if ($LASTEXITCODE) { throw "tray clippy" }
cargo clippy --release -p punktfunk-host --features nvenc,amf-qsv,qsv -- -D warnings; if ($LASTEXITCODE) { throw "host clippy" }
cargo clippy --release -p punktfunk-tray -- -D warnings; if ($LASTEXITCODE) { throw "tray clippy" }
- name: Build + lint the HDR Vulkan layer (pf-vkhdr-layer)
shell: pwsh
Generated
+27 -27
View File
@@ -2159,7 +2159,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.13.0"
version = "0.14.0"
[[package]]
name = "lazy_static"
@@ -2264,7 +2264,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"bindgen",
"cmake",
@@ -2299,7 +2299,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"punktfunk-core",
]
@@ -2788,7 +2788,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2808,7 +2808,7 @@ dependencies = [
[[package]]
name = "pf-client-core"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ash",
@@ -2832,7 +2832,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.12.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2850,7 +2850,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ash",
@@ -2871,7 +2871,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ash",
@@ -2894,7 +2894,7 @@ dependencies = [
[[package]]
name = "pf-ffvk"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"ash",
"bindgen",
@@ -2903,7 +2903,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"libc",
@@ -2915,7 +2915,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"pf-host-config",
@@ -2929,11 +2929,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.13.0"
version = "0.14.0"
[[package]]
name = "pf-inject"
version = "0.12.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2961,14 +2961,14 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ash",
@@ -2983,7 +2983,7 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.12.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3013,7 +3013,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"pf-paths",
@@ -3025,7 +3025,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ash",
@@ -3221,7 +3221,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"android_logger",
"jni",
@@ -3237,7 +3237,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3253,7 +3253,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3268,7 +3268,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3287,7 +3287,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"aes-gcm",
"bytes",
@@ -3318,7 +3318,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"aes",
"aes-gcm",
@@ -3400,7 +3400,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3414,7 +3414,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"anyhow",
"ksni",
@@ -3437,7 +3437,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.13.0"
version = "0.14.0"
dependencies = [
"bindgen",
"cmake",
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.13.0"
version = "0.14.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+330 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.13.0"
"version": "0.14.0"
},
"paths": {
"/api/v1/clients": {
@@ -1931,6 +1931,184 @@
}
}
},
"/api/v1/plugins": {
"get": {
"tags": [
"plugins"
],
"summary": "List registered plugins",
"description": "The live plugin directory (lease not expired), sorted by title. **Secret-free**: each entry\nreports its id, title, optional version, and — for plugins that serve one — a UI descriptor\n(loopback port + icon). The console renders these as nav entries and proxies to the port; it\nfetches the secret separately, server-side.",
"operationId": "listPlugins",
"responses": {
"200": {
"description": "Live plugin registrations",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/PluginSummary"
}
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/plugins/{id}": {
"put": {
"tags": [
"plugins"
],
"summary": "Register or renew a plugin",
"description": "Upserts the plugin's directory entry and renews its lease (TTL 90 s). Idempotent: a plugin PUTs\nthis every ~30 s while it runs. The optional `ui` block declares a loopback UI surface the console\nwill proxy and add to its nav. Emits `plugins.changed` when an operator-visible field changed\n(first registration, restart, or re-scan) — a pure renewal is silent.",
"operationId": "registerPlugin",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The plugin id (its `definePlugin` name: `[a-z][a-z0-9-]*`)",
"required": true,
"schema": {
"type": "string"
}
}
],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PluginRegistration"
}
}
},
"required": true
},
"responses": {
"204": {
"description": "Registered / renewed"
},
"400": {
"description": "Invalid id or registration",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
},
"delete": {
"tags": [
"plugins"
],
"summary": "Deregister a plugin",
"description": "The clean-shutdown path: removes the plugin's directory entry immediately (the SDK helper calls\nthis from its scope finalizer on `SIGTERM`). Emits `plugins.changed` when a live entry was\nremoved. Idempotent — deleting an unknown/expired id is a no-op `204`.",
"operationId": "deregisterPlugin",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The plugin id",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"204": {
"description": "Deregistered (or already absent)"
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/plugins/{id}/ui-credential": {
"get": {
"tags": [
"plugins"
],
"summary": "Fetch a plugin UI's proxy credential",
"description": "Returns `{port, secret}` for a live plugin's loopback UI — the console proxy's server-side lookup.\nBearer + loopback only (like every mutation), and additionally excluded from the console's browser\npassthrough: the secret never reaches a browser.",
"operationId": "getPluginUiCredential",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The plugin id",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "The proxy credential",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/UiCredential"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No live plugin with that id, or it serves no UI",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/session": {
"delete": {
"tags": [
@@ -3294,6 +3472,25 @@
}
}
},
{
"type": "object",
"required": [
"id",
"kind"
],
"properties": {
"id": {
"type": "string",
"description": "The plugin whose registration changed (registered, restarted, deregistered, or\nlease-expired). A consumer re-reads `GET /api/v1/plugins` for the new set."
},
"kind": {
"type": "string",
"enum": [
"plugins.changed"
]
}
}
},
{
"type": "object",
"required": [
@@ -4099,6 +4296,116 @@
"gamestream"
]
},
"PluginRegistration": {
"type": "object",
"description": "Register/renew body for `PUT /plugins/{id}`.",
"required": [
"title"
],
"properties": {
"title": {
"type": "string",
"description": "Human-readable title for the console nav entry (164 chars; control chars stripped)."
},
"ui": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/PluginUi",
"description": "Present iff the plugin serves a UI surface. A registration with no `ui` is a liveness/phone-book\nentry only (e.g. a future runner-management listing) and grows no nav entry."
}
]
},
"version": {
"type": [
"string",
"null"
],
"description": "Optional plugin version, purely informational (≤32 chars)."
}
}
},
"PluginSummary": {
"type": "object",
"description": "One entry in `GET /plugins`. **Never carries the secret** — the browser learns a plugin exists\nand has a UI, nothing that lets it reach the plugin directly (it goes through the console proxy).",
"required": [
"id",
"title"
],
"properties": {
"id": {
"type": "string"
},
"title": {
"type": "string"
},
"ui": {
"oneOf": [
{
"type": "null"
},
{
"$ref": "#/components/schemas/PluginUiPublic"
}
]
},
"version": {
"type": [
"string",
"null"
]
}
}
},
"PluginUi": {
"type": "object",
"description": "A plugin's UI surface as it registers it. Carries the secret — this shape is only ever a request\nbody, never a response ([`PluginUiPublic`] is the secret-free view).",
"required": [
"port",
"secret"
],
"properties": {
"icon": {
"type": [
"string",
"null"
],
"description": "Optional lucide icon name for the console nav entry (`^[a-z0-9-]{1,48}$`)."
},
"port": {
"type": "integer",
"format": "int32",
"description": "The **loopback** port the plugin serves its UI on. The host and console only ever dial\n`127.0.0.1:<port>`; a registration can never carry a hostname.",
"minimum": 0
},
"secret": {
"type": "string",
"description": "Per-boot shared secret the console proxy must present (as `Authorization: Bearer`) on every\nrequest to the plugin's UI server. Rotated whenever the plugin restarts."
}
}
},
"PluginUiPublic": {
"type": "object",
"description": "The secret-free view of a plugin's UI surface — what [`list_plugins`] returns to the browser.",
"required": [
"port"
],
"properties": {
"icon": {
"type": [
"string",
"null"
]
},
"port": {
"type": "integer",
"format": "int32",
"minimum": 0
}
}
},
"PortMap": {
"type": "object",
"description": "Every port a client integration may need (Moonlight derives the stream ports from the\nHTTP base; a control pane should not have to).",
@@ -4710,6 +5017,24 @@
"primary",
"exclusive"
]
},
"UiCredential": {
"type": "object",
"description": "`GET /plugins/{id}/ui-credential` — the console proxy's server-side lookup (bearer + loopback).\nThis is the only endpoint that returns a secret; the console BFF denylists it from the browser.",
"required": [
"port",
"secret"
],
"properties": {
"port": {
"type": "integer",
"format": "int32",
"minimum": 0
},
"secret": {
"type": "string"
}
}
}
},
"securitySchemes": {
@@ -4772,6 +5097,10 @@
{
"name": "hooks",
"description": "Operator hooks: commands and webhooks fired on lifecycle events (fire-and-forget — hooks observe, never veto)"
},
{
"name": "plugins",
"description": "Plugin directory: running `punktfunk-plugin-*` processes register a lease and, optionally, a loopback UI the web console proxies and adds to its nav"
}
]
}
+85
View File
@@ -0,0 +1,85 @@
# LTS builder for the punktfunk HOST .deb — Ubuntu 24.04 (noble), the current Ubuntu LTS.
#
# WHY THIS EXISTS (see packaging/debian/README.md → "Ubuntu 24.04 LTS"):
# The default builder (ci/rust-ci.Dockerfile) is Ubuntu 26.04, so the host .deb it produces bakes
# in a glibc 2.41 floor and a hard `Depends: libavcodec62, …` (FFmpeg 8). Ubuntu 24.04 LTS ships
# glibc 2.39 and FFmpeg 6.1 (libavcodec60), so that .deb is uninstallable there — apt reports the
# deps as "too recent". Building the host on 24.04 instead lowers the glibc floor to 2.39 (the
# binary then runs on 24.04 → 26.04), and the ONE library 24.04 is too old for — FFmpeg — is built
# from source here and BUNDLED into the .deb (packaging/debian/build-deb.sh, BUNDLE_FFMPEG=1), so
# the package no longer depends on the distro's libav* at all. Everything else the host links
# (PipeWire, Wayland, xkbcommon, GL/EGL/GBM, Vulkan; opus is vendored via cmake) is soname-compatible
# on 24.04, so this ONE universal host .deb replaces the 26.04-built one for every Ubuntu user.
#
# libcuda is deliberately NOT provided: the host dlopen's libcuda.so.1 at runtime (pf-zerocopy /
# pf-encode) and never link-imports it, so — unlike the full-workspace rust-ci image, which builds
# tests that DO link a cuda stub — this host-only build needs no NVIDIA driver package. NVENC/EGL
# come from whatever driver the target runs, out of band.
#
# Rebuilt+pushed by .gitea/workflows/docker.yml (matrix: punktfunk-rust-ci-noble); consumed by the
# `build-publish-host` job in .gitea/workflows/deb.yml. Bootstrap: like rust-ci, the first deb.yml
# run after this image is added uses the image from a PRIOR docker.yml push — seed it once manually
# (docker build -f ci/rust-ci-noble.Dockerfile -t … ci && docker push) before the host job can run.
FROM ubuntu:24.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
# toolchain + bindgen; nodejs runs the JS actions (checkout/cache); unzip for the rustup installer's deps
build-essential clang libclang-dev pkg-config cmake git curl ca-certificates nodejs unzip \
# .deb assembly: dpkg-shlibdeps/dpkg-deb; patchelf repoints the binary's rpath at the bundled FFmpeg
dpkg-dev patchelf \
# FFmpeg 8 build deps: nasm (asm), VAAPI (libva/libdrm) so the built libav* keep the AMD/Intel
# encode backend the host auto-selects; zlib (libavformat). NVENC needs only headers (below), dlopen'd.
nasm libva-dev libdrm-dev zlib1g-dev \
# host link deps present on 24.04 with sonames compatible up to 26.04
libpipewire-0.3-dev libwayland-dev libxkbcommon-dev \
libgl-dev libegl-dev libgbm-dev libvulkan-dev \
&& rm -rf /var/lib/apt/lists/*
# --- FFmpeg 8 from source -> /opt/ffmpeg (shared libs + .pc files) ----------------------------------
# libavcodec.so.62, matching the 26.04 line's soname so the host behaves identically. This is an
# LGPL build (no --enable-gpl / --enable-nonfree) so bundling the .so's into an MIT/Apache .deb stays
# license-clean — LGPL's relink clause is satisfied by dynamic linking, and the only encoders the host
# calls (h264/hevc/av1 _nvenc + _vaapi, plus scale_vaapi/hwmap filters; software H.264 fallback is the
# BSD-2 openh264 crate, NOT FFmpeg libx264) are all LGPL-compatible.
# Sourced from the official FFmpeg GitHub mirror by release tag, NOT ffmpeg.org: the CI build network
# can't reach ffmpeg.org (curl times out) but reaches github.com fine. The `nX.Y` tag pins the version
# (n8.0 -> libavcodec 62); bump it to move FFmpeg. Immutable-tag clone, so no separate checksum needed.
ARG FFMPEG_TAG=n8.0
# nv-codec-headers must MATCH the FFmpeg version: its `master` is NVENC SDK 13, which renamed
# NV_ENC_CLOCK_TIMESTAMP_SET.countingType -> countingTypeLSB and won't compile against FFmpeg 8.0's
# nvenc.c. Pin the last SDK-12 tag (has the field FFmpeg 8.0 expects). Bump alongside FFMPEG_TAG.
ARG NVHDR_TAG=n12.2.72.0
RUN set -eux; \
# nv-codec-headers: the NVENC/NVDEC headers FFmpeg's --enable-nvenc needs (headers only, no lib —
# the driver is dlopen'd at runtime). Installs ffnvcodec.pc under /usr/local/lib/pkgconfig.
git clone --depth 1 --branch "$NVHDR_TAG" https://github.com/FFmpeg/nv-codec-headers.git /tmp/nvhdr; \
make -C /tmp/nvhdr install PREFIX=/usr/local; \
git clone --depth 1 --branch "$FFMPEG_TAG" https://github.com/FFmpeg/FFmpeg.git /tmp/ffmpeg; \
cd /tmp/ffmpeg; \
PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure \
--prefix=/opt/ffmpeg \
--enable-shared --disable-static \
--disable-doc --disable-programs --disable-debug \
--enable-nvenc --enable-vaapi \
--extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/lib; \
make -j"$(nproc)"; make install; \
cd /; rm -rf /tmp/ffmpeg /tmp/nvhdr; \
# sanity: the soname we expect to bundle (libavcodec.so.62 on FFmpeg 8)
test -e /opt/ffmpeg/lib/libavcodec.so.62
# ffmpeg-sys-next discovers FFmpeg via pkg-config; point it at the bundled build. PKG_CONFIG_PATH is
# PREPENDED to pkg-config's default dirs (not a replacement — that's PKG_CONFIG_LIBDIR), so PipeWire /
# Wayland / libva / … still resolve from the system. FFMPEG_PREFIX is read by build-deb.sh's bundler.
ENV PKG_CONFIG_PATH=/opt/ffmpeg/lib/pkgconfig \
FFMPEG_PREFIX=/opt/ffmpeg
# Toolchain shared across CI users (jobs may run as different uids).
ENV RUSTUP_HOME=/usr/local/rustup \
CARGO_HOME=/usr/local/cargo \
PATH=/usr/local/cargo/bin:$PATH
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
| sh -s -- -y --no-modify-path --profile minimal \
--component rustfmt,clippy \
&& chmod -R a+w "$RUSTUP_HOME" "$CARGO_HOME" \
&& rustc --version && cargo clippy --version && cargo fmt --version
@@ -30,7 +30,7 @@
shouldAutocreateTestPlan = "YES">
</TestAction>
<LaunchAction
buildConfiguration = "Debug"
buildConfiguration = "Release"
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
launchStyle = "0"
@@ -292,7 +292,7 @@ struct ContentView: View {
Button("Cancel", role: .cancel) {}
} message: { req in
Text("\(req.host.displayName) requires pairing. Request access and approve this "
+ "device in the host's web console (port 3000 → Pairing) — no PIN needed. Or "
+ "device in the host's web console (port 47992 → Pairing) — no PIN needed. Or "
+ "pair with the 4-digit PIN it can display.")
}
// One "Connection failed" surface for every home screen (touch grid, gamepad launcher) and
@@ -335,7 +335,7 @@ struct ContentView: View {
Button("Cancel", role: .cancel) { model.disconnect() }
} message: { req in
Text("Approve \u{201C}\(localDeviceName)\u{201D} in \(req.host.displayName)'s web "
+ "console (port 3000 → Pairing). This device connects automatically once you "
+ "console (port 47992 → Pairing). This device connects automatically once you "
+ "approve it — no need to reconnect.")
}
// Informational deep-link outcome (unknown host / already streaming). Not an error.
@@ -210,7 +210,13 @@ final class SessionModel: ObservableObject {
// metadata we apply (Step 2) when it IS HDR.
let displayHDR: Bool = {
#if os(macOS)
return (NSScreen.main?.maximumExtendedDynamicRangeColorComponentValue ?? 1.0) > 1.0
// POTENTIAL, not current, headroom: `maximumExtendedDynamicRangeColorComponentValue`
// is the CURRENTLY-ALLOCATED headroom, which macOS hands out on demand on an idle
// SDR desktop it reads 1.0 even with HDR enabled and active (external HDR displays
// like the Samsung G95SC allocate EDR only when content asks). Gating on it means an
// HDR monitor never gets advertised at connect time. `maximumPotential` is the
// mode-independent capability (the macOS analogue of the tvOS/iOS gates below).
return (NSScreen.main?.maximumPotentialExtendedDynamicRangeColorComponentValue ?? 1.0) > 1.0
#elseif os(tvOS)
// NOT the EDR headroom here: on tvOS that reflects the CURRENT output mode, and
// Apple's recommended setup runs an SDR home screen with Match Content an
@@ -264,14 +270,11 @@ final class SessionModel: ObservableObject {
// PyroWave (wired LAN) is a pure opt-in: picking it in the codec setting both
// advertises the bit and prefers it the host never auto-selects it, and the
// picker only offers it when the Metal decode probe passed (simdgroup floor A13;
// every M-series Mac and the ATV 4K gen 3 pass). The codec is 8-bit 4:2:0 SDR
// BT.709 by contract, so the opt-in also drops the HDR/10-bit/4:4:4 caps for this
// session HDR sessions stay HEVC/AV1 (plan §4.7).
// every M-series Mac and the ATV 4K gen 3 pass). The decoder self-configures from
// the per-frame sequence header (4:2:0/4:4:4, SDR/PQ design/pyrowave-444-hdr.md),
// so the session keeps the user's HDR/10-bit/4:4:4 caps exactly like HEVC/AV1.
if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported {
videoCodecs |= PunktfunkConnection.codecPyroWave
videoCaps &= ~(PunktfunkConnection.videoCap10Bit
| PunktfunkConnection.videoCapHDR
| PunktfunkConnection.videoCap444)
}
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
@@ -335,7 +338,7 @@ final class SessionModel: ObservableObject {
// operator didn't approve it before the host's park window elapsed (or
// the host was unreachable).
self.errorMessage = "\(host.displayName) didn't let this device in. "
+ "Approve it in the host's web console (port 3000 → Pairing), then "
+ "Approve it in the host's web console (port 47992 → Pairing), then "
+ "request access again — the request expires after a few minutes."
} else {
self.errorMessage = pin != nil
@@ -1,4 +1,4 @@
// PIN pairing sheet. The host shows the pairing PIN in its web console (port 3000
// PIN pairing sheet. The host shows the pairing PIN in its web console (port 47992
// Pairing; also printed in the host's log when armed via --allow-pairing); the user
// types it here. The ceremony is SPAKE2, so a wrong PIN buys an
// attacker exactly one online guess for the user a typo just means "try again" (the
@@ -45,7 +45,7 @@ struct PairSheet: View {
#if os(tvOS)
VStack(spacing: 24) {
Text("The PIN is shown in the host's web console "
+ "(http://<host>:3000 → Pairing). "
+ "(https://<host>:47992 → Pairing). "
+ "Pairing verifies both sides at once — no fingerprint comparison "
+ "needed.")
.font(.geist(22, relativeTo: .callout)) // TV-legible (system callout is ~25 there)
@@ -118,7 +118,7 @@ struct PairSheet: View {
.foregroundStyle(.tint)
} footer: {
Text("The PIN is shown in the host's web console "
+ "(http://<host>:3000 → Pairing). "
+ "(https://<host>:47992 → Pairing). "
+ "Pairing verifies both sides at once — no fingerprint "
+ "comparison needed.")
.font(.geist(12, relativeTo: .caption))
@@ -210,7 +210,7 @@ struct PairSheet: View {
onPaired(fingerprint)
dismiss()
case .failure(PunktfunkClientError.wrongPIN):
errorText = "Wrong PIN — check the host's web console (port 3000) "
errorText = "Wrong PIN — check the host's web console (port 47992) "
+ "and try again."
case .failure(PunktfunkClientError.rejected(let rejection)):
// The host answered and said why (not armed / rate-limited / armed for
@@ -14,6 +14,9 @@
#if canImport(Metal) && canImport(QuartzCore)
import CoreGraphics
import CoreVideo
#if os(macOS)
import IOSurface
#endif
import Metal
import QuartzCore
import os
@@ -193,13 +196,11 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
// display-referred SDR. (When the display IS in an HDR mode — requested per session via
// AVDisplayManager, see StreamViewIOS — tvOS presents pf_frag_hdr's PQ passthrough instead:
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]],
constant CscUniform& csc [[buffer(0)]]) {
// YCbCr → full-range PQ RGB via the per-frame rows (as pf_frag_hdr).
float3 pq = sampleRgb(lumaTex, chromaTex, in.uv, csc);
// ST 2084 EOTF: PQ code value → linear light, 1.0 = 10,000 nits.
// The shared PQ→display-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
// EOTF → 203-nit-anchored scene light → BT.2020→709 primaries → extended-Reinhard rolloff →
// BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map — the
// latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only.
static inline float3 pqToSdr(float3 pq) {
const float m1 = 2610.0/16384.0;
const float m2 = 78.84375;
const float c1 = 3424.0/4096.0;
@@ -207,20 +208,49 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
const float c3 = 18.6875;
float3 p = pow(pq, 1.0/m2);
float3 lin = pow(max(p - c1, 0.0) / (c2 - c3 * p), 1.0/m1);
// Scene-referred with diffuse white at 1.0 (the same 203-nit anchor the EDR path uses).
float3 t = lin * (10000.0/203.0);
// BT.2020 → BT.709 primaries while still linear; negatives are out-of-gamut, floor them.
float3 t709 = float3(
dot(t, float3( 1.6605, -0.5876, -0.0728)),
dot(t, float3(-0.1246, 1.1329, -0.0083)),
dot(t, float3(-0.0182, -0.1006, 1.1187)));
t709 = max(t709, 0.0);
// Extended Reinhard: 1.0 stays put, the 1000-nit knee lands at display white, above rolls off.
const float w = 1000.0/203.0;
float3 mapped = saturate(t709 * (1.0 + t709 / (w * w)) / (1.0 + t709));
// BT.709 OETF — the same encoding the SDR stream arrives in, so both paths present alike.
float3 e = select(1.099 * pow(mapped, 0.45) - 0.099, 4.5 * mapped, mapped < 0.018);
return float4(e, 1.0);
return e;
}
fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]],
constant CscUniform& csc [[buffer(0)]]) {
// YCbCr → full-range PQ RGB via the per-frame rows (as pf_frag_hdr), then the tail.
return float4(pqToSdr(sampleRgb(lumaTex, chromaTex, in.uv, csc)), 1.0);
}
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
// fold in depth-10 MSB packing) → PQ RGB → the shared SDR tail. Used when a PQ pyrowave
// stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions
// (the IOSurface present path — the DCP-panic mitigation — is BGRA8). The passthrough planar
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math — the
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> cbTex [[texture(1)]],
texture2d<float> crTex [[texture(2)]],
constant CscUniform& csc [[buffer(0)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
#ifdef PF_BILINEAR_LUMA
float lumaY = lumaTex.sample(s, in.uv).r;
#else
float lumaY = catmullRomLuma(lumaTex, s, in.uv);
#endif
float2 cuv = chromaUV(lumaTex, cbTex, in.uv);
float3 yuv = float3(lumaY, cbTex.sample(s, cuv).r, crTex.sample(s, cuv).r);
float3 pq = saturate(float3(dot(csc.r0.xyz, yuv) + csc.r0.w,
dot(csc.r1.xyz, yuv) + csc.r1.w,
dot(csc.r2.xyz, yuv) + csc.r2.w));
return float4(pqToSdr(pq), 1.0);
}
"""
@@ -228,6 +258,51 @@ public final class MetalVideoPresenter {
/// The layer the hosting view installs (as a sublayer) and sizes to its bounds.
public let layer: CAMetalLayer
#if os(macOS)
/// The WINDOWED-mode PyroWave present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it), fed IOSurfaces via `contents` inside ordinary CATransactions.
///
/// Why this exists the macOS DCP KERNEL PANIC ("mismatched swapID's" @UnifiedPipeline.cpp,
/// WindowServer dies, machine reboots): out-of-band CAMetalLayer image-queue swaps into a
/// COMPOSITED (windowed) session race WindowServer's own swap submissions on high-refresh
/// displays, and the race survives glass pacing a fully serialized one-in-flight present
/// stream still panicked a 240 Hz Mac Studio (2026-07-18, twice). So in windowed mode we stop
/// using the image queue entirely and present the way video players do: render the planar CSC
/// into an IOSurface pool and swap `contents` on main WindowServer treats it as ordinary
/// damage on its own composite cadence, coalescing faster-than-refresh updates instead of
/// latching queue swaps mid-cycle. Fullscreen keeps the CAMetalLayer path (direct-scanout
/// promotion, no compositing, no panic reports). Contents updates are transparent to the
/// layer below when nil, so flipping modes just covers/uncovers the metal layer.
public let surfaceLayer: CALayer = {
let l = CALayer()
l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout
l.isOpaque = true
l.actions = ["contents": NSNull(), "bounds": NSNull(), "position": NSNull()]
return l
}()
/// One IOSurface-backed render target of the windowed present pool. All pool state is
/// RENDER-THREAD confined; only the immutable surface refs cross to main (contents swap).
private struct SurfaceSlot {
let surface: IOSurfaceRef
let texture: MTLTexture
/// Monotonic use stamp the reuse picker takes the least-recently-rendered free slot.
var seq: UInt64 = 0
}
private var surfacePool: [SurfaceSlot] = []
private var surfacePoolSize: CGSize = .zero
private var surfaceSeq: UInt64 = 0
/// Index of the slot most recently handed to the layer never rewritten next, even if its
/// use count already dropped (the compositor may still be scanning out the previous frame).
private var lastHandedOff: Int?
/// Staged (under `stagingLock`, like every cross-thread input): the hosting view's windowed
/// vs fullscreen state, pushed from main via `setSurfacePresents`. Drained in `renderPlanar`.
private var surfacePresentsStaged = false
/// Render-thread copy, so pool teardown happens exactly once on a mode flip.
private var surfacePresentsActive = false
#endif
private let device: MTLDevice
private let queue: MTLCommandQueue
/// SDR (BT.709 8-bit bgra8) and HDR (BT.2020 PQ 10-bit rgba16Float) pipelines. Selected per
@@ -239,6 +314,11 @@ public final class MetalVideoPresenter {
private let pipelineHDRToneMap: MTLRenderPipelineState?
/// PyroWave's 3-plane SDR path (pf_frag_planar bgra8) see `renderPlanar`.
private let pipelinePlanar: MTLRenderPipelineState
/// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour
/// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm
/// bgra8; tvOS without headroom + macOS windowed IOSurface presents).
private let pipelinePlanarHDR: MTLRenderPipelineState
private let pipelinePlanarToneMap: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache?
/// The PyroWave Metal decoder records on the presenter's device + queue: one device means
@@ -289,6 +369,8 @@ public final class MetalVideoPresenter {
let pipelineHDR: MTLRenderPipelineState
let pipelineHDRToneMap: MTLRenderPipelineState?
let pipelinePlanar: MTLRenderPipelineState
let pipelinePlanarHDR: MTLRenderPipelineState
let pipelinePlanarToneMap: MTLRenderPipelineState
do {
// DEBUG A/B lever: PUNKTFUNK_BILINEAR_LUMA=1 compiles the shader with Catmull-Rom OFF
// (plain bilinear luma) by prepending a #define ahead of the source. Default (unset) is
@@ -326,8 +408,18 @@ public final class MetalVideoPresenter {
let planar = MTLRenderPipelineDescriptor()
planar.vertexFunction = vtx
planar.fragmentFunction = library.makeFunction(name: "pf_frag_planar")
planar.colorAttachments[0].pixelFormat = .bgra8Unorm // PyroWave is 8-bit SDR
planar.colorAttachments[0].pixelFormat = .bgra8Unorm
pipelinePlanar = try device.makeRenderPipelineState(descriptor: planar)
let planarHdr = MTLRenderPipelineDescriptor()
planarHdr.vertexFunction = vtx
planarHdr.fragmentFunction = library.makeFunction(name: "pf_frag_planar")
planarHdr.colorAttachments[0].pixelFormat = .rgba16Float // PQ passthrough
pipelinePlanarHDR = try device.makeRenderPipelineState(descriptor: planarHdr)
let planarTm = MTLRenderPipelineDescriptor()
planarTm.vertexFunction = vtx
planarTm.fragmentFunction = library.makeFunction(name: "pf_frag_planar_tm")
planarTm.colorAttachments[0].pixelFormat = .bgra8Unorm
pipelinePlanarToneMap = try device.makeRenderPipelineState(descriptor: planarTm)
} catch {
return nil
}
@@ -368,6 +460,7 @@ public final class MetalVideoPresenter {
return MetalVideoPresenter(
device: device, queue: queue, pipelineSDR: pipelineSDR, pipelineHDR: pipelineHDR,
pipelineHDRToneMap: pipelineHDRToneMap, pipelinePlanar: pipelinePlanar,
pipelinePlanarHDR: pipelinePlanarHDR, pipelinePlanarToneMap: pipelinePlanarToneMap,
textureCache: textureCache, layer: layer)
}
@@ -375,6 +468,8 @@ public final class MetalVideoPresenter {
device: MTLDevice, queue: MTLCommandQueue, pipelineSDR: MTLRenderPipelineState,
pipelineHDR: MTLRenderPipelineState, pipelineHDRToneMap: MTLRenderPipelineState?,
pipelinePlanar: MTLRenderPipelineState,
pipelinePlanarHDR: MTLRenderPipelineState,
pipelinePlanarToneMap: MTLRenderPipelineState,
textureCache: CVMetalTextureCache, layer: CAMetalLayer
) {
self.device = device
@@ -383,6 +478,8 @@ public final class MetalVideoPresenter {
self.pipelineHDR = pipelineHDR
self.pipelineHDRToneMap = pipelineHDRToneMap
self.pipelinePlanar = pipelinePlanar
self.pipelinePlanarHDR = pipelinePlanarHDR
self.pipelinePlanarToneMap = pipelinePlanarToneMap
self.textureCache = textureCache
self.layer = layer
}
@@ -493,6 +590,18 @@ public final class MetalVideoPresenter {
stagingLock.unlock()
}
#if os(macOS)
/// Park the windowed-vs-fullscreen present routing (MAIN thread the hosting view pushes its
/// window state on every layout). true = PyroWave frames present via `surfaceLayer` contents
/// (the DCP swapID-panic mitigation see `surfaceLayer`); false = the CAMetalLayer path.
/// Applied by the render thread on the next frame, like every other staged value here.
public func setSurfacePresents(_ on: Bool) {
stagingLock.lock()
surfacePresentsStaged = on
stagingLock.unlock()
}
#endif
/// Draw one decoded frame to the next drawable and present it. RENDER THREAD (Stage2Pipeline's;
/// `nextDrawable()` may block up to a frame that wait belongs here, never on main). `isHDR`
/// selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
@@ -588,12 +697,51 @@ public final class MetalVideoPresenter {
) -> Bool {
stagingLock.lock()
let targetFromLayout = drawableTarget
#if os(macOS)
let surfaceMode = surfacePresentsStaged
#endif
stagingLock.unlock()
configure(hdr: false)
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path;
// macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader).
#if os(macOS)
configure(hdr: planes.pq && !surfaceMode)
#else
configure(hdr: planes.pq)
#endif
var csc = planes.csc
#if os(macOS)
if surfaceMode != surfacePresentsActive {
surfacePresentsActive = surfaceMode
presenterLog.info(
"stage2: windowed surface presents \(surfaceMode ? "ON" : "OFF", privacy: .public) (PyroWave DCP-panic mitigation)")
if !surfaceMode {
// Back to the metal path (fullscreen): drop the pool at 5K it holds >100 MB,
// and re-entering windowed mode rebuilds it in one frame.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
}
if surfaceMode {
return renderPlanarToSurface(
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
}
#endif
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
// 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps
// in-shader instead (the pipeline must match the drawable's pixel format).
#if os(tvOS)
let planarPassthrough = hdrActive && hdrPassthroughActive
#else
let planarPassthrough = hdrActive
#endif
let planarPipeline: MTLRenderPipelineState =
planes.pq
? (planarPassthrough ? pipelinePlanarHDR : pipelinePlanarToneMap)
: pipelinePlanar
return encodePresent(
decodedSize: CGSize(width: planes.width, height: planes.height),
targetFromLayout: targetFromLayout, pipeline: pipelinePlanar,
targetFromLayout: targetFromLayout, pipeline: planarPipeline,
presentAtMediaTime: presentAtMediaTime, onPresented: onPresented,
// The ring textures stay valid by ring depth; retaining them here also pins the
// slot's set until the sample completes (mirrors the biplanar keep-alive).
@@ -606,6 +754,118 @@ public final class MetalVideoPresenter {
}
}
#if os(macOS)
/// The windowed-mode present tail (see `surfaceLayer` for why this path exists): render the
/// planar CSC into a pooled IOSurface and hand it to `surfaceLayer.contents` on MAIN inside a
/// plain CATransaction an ordinary damaged-layer update on WindowServer's own composite
/// cadence, no CAMetalLayer image-queue swap anywhere. `presentAtMediaTime` doesn't apply
/// (the compositor paces); `onPresented` fires after the contents swap is committed, stamped
/// with CLOCK_REALTIME then the closest observable analogue of "reached glass" here (the
/// composite follows within a refresh, so the meters' display stage reads slightly optimistic).
private func renderPlanarToSurface(
_ planes: WaveletPlanes, targetFromLayout: CGSize, csc: inout CscUniform,
onPresented: ((Int64?) -> Void)?
) -> Bool {
let decodedSize = CGSize(width: planes.width, height: planes.height)
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
ensureSurfacePool(size: targetSize)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let keepAlive: [Any] = [planes.y, planes.cb, planes.cr]
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // ring textures pinned until the GPU finished sampling
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
onPresented?(
Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size` 4 BGRA8 IOSurface render targets (one on glass, one queued
/// in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the pool empty;
/// the caller returns false and the ring's putBack + display-link retry take over.
private func ensureSurfacePool(size: CGSize) {
guard size != surfacePoolSize else { return }
surfacePool.removeAll()
surfacePoolSize = size
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// 256-byte row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerRow = ((w * 4) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: 4,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one
/// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule
/// the present and the on-glass callback.
@@ -47,6 +47,9 @@ struct WaveletLayout {
let width: Int
let height: Int
/// Full-res chroma (4:4:4): chroma components get the full band set including level 0,
/// exactly like luma upstream `init_block_meta` with `Chroma444`.
let chroma444: Bool
let alignedWidth: Int
let alignedHeight: Int
/// blockMeta[component][level][band] = (blockOffset32x32, blockStride32x32); -1 offset =
@@ -59,9 +62,10 @@ struct WaveletLayout {
func levelWidth(_ level: Int) -> Int { (alignedWidth / 2) >> level }
func levelHeight(_ level: Int) -> Int { (alignedHeight / 2) >> level }
init(width: Int, height: Int) {
init(width: Int, height: Int, chroma444: Bool) {
self.width = width
self.height = height
self.chroma444 = chroma444
let align = { (v: Int) in
max((v + Self.alignment - 1) & ~(Self.alignment - 1), Self.minimumImageSize)
}
@@ -78,7 +82,7 @@ struct WaveletLayout {
let ah = alignedHeight
for level in stride(from: Self.decompositionLevels - 1, through: 0, by: -1) {
for component in 0..<3 {
if level == 0 && component != 0 { continue } // 4:2:0: no top-level chroma
if level == 0 && component != 0 && !chroma444 { continue } // 4:2:0: no top-level chroma
for band in (level == Self.decompositionLevels - 1 ? 0 : 1)..<4 {
let levelW = (aw / 2) >> level
let levelH = (ah / 2) >> level
@@ -108,6 +112,9 @@ struct ParsedWaveletFrame {
var decodedBlocks: Int
/// VUI bits from the sequence header (BitstreamSequenceHeader).
var bt2020: Bool
/// PQ transfer HDR session: 16-bit studio-code planes + EDR present (the host stamps
/// this bit iff the session negotiated 10-bit the depth is coupled to the transfer).
var pq: Bool
var fullRange: Bool
/// The frame's YCbCrRGB signal for the presenter's planar CSC. PyroWave today is always
@@ -131,6 +138,12 @@ enum WaveletBitstream {
/// decoding upstream's `decoded_blocks > total/2` partial rule).
static func parse(au: Data, chunkAligned: Bool, windowSize: Int) -> ParsedWaveletFrame? {
var state = ParseState()
// Reserve the coefficient buffer ONCE, up front. Every packet's payload is a slice of the
// AU, so `au.count / 4` words is a tight upper bound reserving it here lets the per-packet
// appends stay amortized O(1). (Reserving per packet forces Swift to allocate the exact new
// size each time, turning the walk O(n²) invisible on the tiny golden fixtures, but ~5 ms
// per 1.4 MB frame on a real 5120x1440 stream.)
state.payload.reserveCapacity(au.count / 4)
let ok = au.withUnsafeBytes { (raw: UnsafeRawBufferPointer) -> Bool in
guard let base = raw.baseAddress?.assumingMemoryBound(to: UInt8.self) else {
return false
@@ -203,6 +216,7 @@ enum WaveletBitstream {
var totalBlocks = 0
var decodedBlocks = 0
var bt2020 = false
var pq = false
var fullRange = false
var sawSOF = false
@@ -220,22 +234,27 @@ enum WaveletBitstream {
// siting[31].
let code = (word1 >> 24) & 0x3
guard code == 0 else { return false } // only START_OF_FRAME is defined
let chromaRes = (word1 >> 26) & 1
guard chromaRes == 0 else { return false } // host contract: 4:2:0
let chroma444 = (word1 >> 26) & 1 != 0
let w = Int(word0 & 0x3fff) + 1
let h = Int((word0 >> 14) & 0x3fff) + 1
guard w >= 2, h >= 2, w % 2 == 0, h % 2 == 0 else { return false }
guard w >= 2, h >= 2, chroma444 || (w % 2 == 0 && h % 2 == 0) else {
return false
}
if sawSOF {
// One frame, one geometry a second SOF must agree.
guard layout?.width == w, layout?.height == h else { return false }
guard layout?.width == w, layout?.height == h,
layout?.chroma444 == chroma444
else { return false }
} else {
sawSOF = true
let l = WaveletLayout(width: w, height: h)
let l = WaveletLayout(width: w, height: h, chroma444: chroma444)
layout = l
offsets = [UInt32](repeating: .max, count: l.blockCount32)
payload.reserveCapacity(64 * 1024 / 4)
totalBlocks = Int(word1 & 0xff_ffff)
bt2020 = (word1 >> 29) & 1 != 0
// transfer_function bit: PQ an HDR session (16-bit studio-code
// planes by the negotiated coupling design/pyrowave-444-hdr.md).
pq = (word1 >> 28) & 1 != 0
fullRange = (word1 >> 30) & 1 == 0 // YCBCR_RANGE_FULL = 0
}
pos += 8
@@ -252,9 +271,15 @@ enum WaveletBitstream {
if offsets[blockIndex] == .max {
offsets[blockIndex] = UInt32(payload.count)
decodedBlocks += 1
payload.reserveCapacity(payload.count + payloadWords)
for w in 0..<payloadWords {
payload.append(loadWord(base, pos + w * 4))
// Bulk-copy the packet's coefficient words in one memcpy rather than
// word-by-word. All Apple platforms are little-endian, so the wire's LE
// u32s land in the [UInt32] buffer verbatim; memcpy has no alignment
// requirement, so a non-word-aligned `base + pos` is fine. `reserveCapacity`
// up in `parse` keeps the grow amortized O(1).
let dstWord = payload.count
payload.append(contentsOf: repeatElement(0, count: payloadWords))
payload.withUnsafeMutableBytes { dst in
_ = memcpy(dst.baseAddress! + dstWord * 4, base + pos, payloadWords * 4)
}
}
} else if layout != nil {
@@ -280,7 +305,7 @@ enum WaveletBitstream {
return ParsedWaveletFrame(
layout: layout, offsets: offsets, payload: payload,
totalBlocks: totalBlocks, decodedBlocks: decodedBlocks,
bt2020: bt2020, fullRange: fullRange)
bt2020: bt2020, pq: pq, fullRange: fullRange)
}
}
}
@@ -293,6 +318,8 @@ public struct WaveletPlanes: @unchecked Sendable {
public let cb: MTLTexture
public let cr: MTLTexture
public let csc: CscUniform
/// PQ (HDR) stream: the presenter picks the HDR/tone-map planar pipeline + EDR config.
public let pq: Bool
public var width: Int { y.width }
public var height: Int { y.height }
}
@@ -351,6 +378,8 @@ public final class MetalWaveletDecoder {
private var slots: [Slot] = []
private var nextSlot = 0
/// The ring's plane format facts (from the last SOF): PQ 16-bit UNORM planes.
private var hdr16 = false
/// The current geometry (from the last SOF that built the resources) the pump reports
/// decoded-size changes to the resize overlay from this. PUMP THREAD.
@@ -409,8 +438,9 @@ public final class MetalWaveletDecoder {
au: au, chunkAligned: chunkAligned, windowSize: windowSize)
else { return false }
if layout?.width != frame.layout.width || layout?.height != frame.layout.height {
guard rebuild(layout: frame.layout) else { return false }
if layout?.width != frame.layout.width || layout?.height != frame.layout.height
|| layout?.chroma444 != frame.layout.chroma444 || hdr16 != frame.pq {
guard rebuild(layout: frame.layout, hdr16: frame.pq) else { return false }
}
guard let layout, !slots.isEmpty else { return false }
@@ -450,7 +480,7 @@ public final class MetalWaveletDecoder {
dequant.setBuffer(slot.payload, offset: 0, index: 1)
for level in 0..<WaveletLayout.decompositionLevels {
for component in 0..<3 {
if level == 0 && component != 0 { continue } // 4:2:0
if level == 0 && component != 0 && !layout.chroma444 { continue } // 4:2:0
for band in (level == WaveletLayout.decompositionLevels - 1 ? 0 : 1)..<4 {
let meta = layout.blockMeta[component][level][band]
let w = layout.levelWidth(level)
@@ -489,15 +519,20 @@ public final class MetalWaveletDecoder {
let grid = MTLSize(width: (rx + 15) / 16, height: (ry + 15) / 16, depth: 1)
let group = MTLSize(width: 64, height: 1, depth: 1)
if inputLevel == 0 {
// 4:2:0: the final full-res pass is luma only (chroma finished at level 1).
// Final full-res pass: luma only in 4:2:0 (chroma finished at level 1); all
// three components in 4:4:4 (chroma runs the full pyramid like luma).
idwt.setComputePipelineState(idwtShiftPipeline)
idwt.setTexture(coefficients[0][0], index: 0)
idwt.setTexture(slot.y, index: 1)
idwt.dispatchThreadgroups(grid, threadsPerThreadgroup: group)
let components = layout.chroma444 ? 3 : 1
for component in 0..<components {
idwt.setTexture(coefficients[component][0], index: 0)
let out = component == 0 ? slot.y : (component == 1 ? slot.cb : slot.cr)
idwt.setTexture(out, index: 1)
idwt.dispatchThreadgroups(grid, threadsPerThreadgroup: group)
}
} else {
for component in 0..<3 {
idwt.setTexture(coefficients[component][inputLevel], index: 0)
if component != 0 && inputLevel == 1 {
if component != 0 && inputLevel == 1 && !layout.chroma444 {
// 4:2:0 chroma emits its final half-res plane one level early.
idwt.setComputePipelineState(idwtShiftPipeline)
idwt.setTexture(component == 1 ? slot.cb : slot.cr, index: 1)
@@ -513,7 +548,9 @@ public final class MetalWaveletDecoder {
let planes = WaveletPlanes(
y: slot.y, cb: slot.cb, cr: slot.cr,
csc: CscRows.rows(frame.cscSignal, depth: 8, msbPacked: false))
csc: CscRows.rows(
frame.cscSignal, depth: frame.pq ? 10 : 8, msbPacked: frame.pq),
pq: frame.pq)
cmd.addCompletedHandler { buffer in
completion(buffer.error == nil ? planes : nil)
}
@@ -524,9 +561,9 @@ public final class MetalWaveletDecoder {
/// (Re)allocate every size-dependent resource for `layout`'s geometry. Also the mid-stream
/// resize path: a Reconfigure shows up here as new SOF dims.
private func rebuild(layout newLayout: WaveletLayout) -> Bool {
private func rebuild(layout newLayout: WaveletLayout, hdr16 newHdr16: Bool) -> Bool {
waveletLog.info(
"pyrowave: building decoder \(newLayout.width)x\(newLayout.height) (aligned \(newLayout.alignedWidth)x\(newLayout.alignedHeight), \(newLayout.blockCount32) blocks)")
"pyrowave: building decoder \(newLayout.width)x\(newLayout.height) (aligned \(newLayout.alignedWidth)x\(newLayout.alignedHeight), \(newLayout.blockCount32) blocks, \(newLayout.chroma444 ? "4:4:4" : "4:2:0", privacy: .public)\(newHdr16 ? " HDR16" : "", privacy: .public))")
var coeff: [[MTLTexture]] = []
var lls: [[MTLTexture]] = []
for component in 0..<3 {
@@ -560,19 +597,22 @@ public final class MetalWaveletDecoder {
var newSlots: [Slot] = []
for i in 0..<Self.ringDepth {
let planeFormat: MTLPixelFormat = newHdr16 ? .r16Unorm : .r8Unorm
let plane = { (w: Int, h: Int, name: String) -> MTLTexture? in
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .r8Unorm, width: w, height: h, mipmapped: false)
pixelFormat: planeFormat, width: w, height: h, mipmapped: false)
desc.usage = [.shaderRead, .shaderWrite]
desc.storageMode = .private
let t = self.device.makeTexture(descriptor: desc)
t?.label = name
return t
}
let cw = newLayout.chroma444 ? newLayout.width : newLayout.width / 2
let ch = newLayout.chroma444 ? newLayout.height : newLayout.height / 2
guard
let y = plane(newLayout.width, newLayout.height, "pyrowave Y[\(i)]"),
let cb = plane(newLayout.width / 2, newLayout.height / 2, "pyrowave Cb[\(i)]"),
let cr = plane(newLayout.width / 2, newLayout.height / 2, "pyrowave Cr[\(i)]"),
let cb = plane(cw, ch, "pyrowave Cb[\(i)]"),
let cr = plane(cw, ch, "pyrowave Cr[\(i)]"),
let offsets = device.makeBuffer(
length: max(newLayout.blockCount32 * 4, 4), options: .storageModeShared),
let payload = device.makeBuffer(length: 64 * 1024, options: .storageModeShared)
@@ -585,6 +625,7 @@ public final class MetalWaveletDecoder {
slots = newSlots
nextSlot = 0
layout = newLayout
hdr16 = newHdr16
return true
}
@@ -42,12 +42,20 @@ enum PresenterChoice: Equatable {
/// leftover DEBUG "stage1" value silently maps to the default rather than reviving the
/// freeze-prone fallback.
static func resolve(setting: String?, env: String?, allowStage1: Bool) -> PresenterChoice {
explicit(setting: setting, env: env, allowStage1: allowStage1) ?? platformDefault
}
/// The user's EXPLICIT stage selection, nil when they haven't made one (unset/unknown values,
/// and a release build's gated "stage1"). Split from `resolve` so a codec-conditional default
/// (see `SessionPresenter.pacing`) can apply only when the user hasn't picked a stage an
/// explicit "stage2" must stay a faithful A/B of arrival pacing.
static func explicit(setting: String?, env: String?, allowStage1: Bool) -> PresenterChoice? {
let raw = env.flatMap { $0.isEmpty ? nil : $0 } ?? setting
switch raw {
case "stage1": return allowStage1 ? .stage1 : platformDefault
case "stage1": return allowStage1 ? .stage1 : nil
case "stage2": return .stage2
case "stage3": return .stage3
default: return platformDefault
default: return nil
}
}
@@ -66,10 +74,41 @@ enum PresenterChoice: Equatable {
}
final class SessionPresenter {
/// Present pacing for this session. Stage-3 always means glass gating; under the stage-2
/// default, macOS PyroWave sessions ALSO get glass gating a kernel-panic mitigation, not a
/// latency tweak. macOS's DCP panics ("mismatched swapID's" @UnifiedPipeline.cpp, the whole
/// machine dies) when WindowServer's swap submissions race, and the reliable trigger is
/// out-of-band CAMetalLayer presents (displaySyncEnabled=false mandatory for us, see
/// MetalVideoPresenter's init) arriving faster than the compositor latches them in a
/// COMPOSITED (windowed) session. Arrival pacing does exactly that with PyroWave: the wavelet
/// decode is near-instant Metal compute, so a network clump of frames presents within the
/// same millisecond, and PyroWave is the codec that sustains stream rates above the panel's
/// refresh. The glass gate admits one presented-but-undisplayed swap at a time (serialized on
/// the on-glass callback, 100 ms stale backstop), which removes the racing pattern outright;
/// frames the panel couldn't have shown anyway coalesce in the newest-wins ring. An explicit
/// stage-2 pick (setting/env) still forces arrival pacing that A/B lever must stay honest.
/// VideoToolbox codecs keep arrival pacing: decode latency spaces their presents, and years
/// of stage-2 defaults there predate any panic report.
static func pacing(
for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec
) -> PresentPacing {
if choice == .stage3 { return .glass }
#if os(macOS)
if explicit == nil, codec == .pyrowave { return .glass }
#endif
return .arrival
}
private var pump: StreamPump?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer?
#if os(macOS)
/// The windowed-mode PyroWave present target (sibling above `metalLayer`) and the last
/// routing pushed to the pipeline see `setComposited`. Main-thread only, like all of this.
private var surfaceLayer: CALayer?
private var surfacePresentsActive = false
#endif
private var connection: PunktfunkConnection?
/// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's
/// `onDecodedSize` new-mode-IDR callback). Used for the aspect-fit in `layout` in preference to
@@ -112,20 +151,29 @@ final class SessionPresenter {
#else
let allowStage1 = false
#endif
let choice = PresenterChoice.resolve(
let explicit = PresenterChoice.explicit(
setting: UserDefaults.standard.string(forKey: DefaultsKey.presenter),
env: ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENTER"],
allowStage1: allowStage1)
let choice = explicit ?? PresenterChoice.platformDefault
if choice != .stage1,
let pipeline = Stage2Pipeline(
endToEndMeter: endToEndMeter, decodeMeter: decodeMeter,
displayMeter: displayMeter,
pacing: choice == .stage3 ? .glass : .arrival) {
pacing: Self.pacing(
for: choice, explicit: explicit, codec: connection.videoCodec)) {
let metal = pipeline.layer
// The opaque metal layer composites OVER the AVSampleBufferDisplayLayer base, which
// sits idle (un-enqueued) in stage-2. contentsScale + frame are set in layout().
baseLayer.addSublayer(metal)
metalLayer = metal
#if os(macOS)
// The windowed-PyroWave present target sits ABOVE the metal layer: transparent (nil
// contents) while the metal path presents, covering it while surface presents run.
baseLayer.addSublayer(pipeline.surfaceLayer)
surfaceLayer = pipeline.surfaceLayer
surfacePresentsActive = false
#endif
stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
// (frame arrival is see Stage2Pipeline's header). timestamptargetTimestamp is the
@@ -224,6 +272,12 @@ final class SessionPresenter {
CATransaction.setDisableActions(true)
metalLayer.contentsScale = contentsScale
metalLayer.frame = snapped
#if os(macOS)
// The surface present target mirrors the metal layer's geometry exactly its IOSurfaces
// are sized to the same snapped pixel rect, so the contents composite is a 1:1 blit too.
surfaceLayer?.contentsScale = contentsScale
surfaceLayer?.frame = snapped
#endif
CATransaction.commit()
// Hand the resulting pixel size to the render thread (it must not read layer geometry
// cross-thread) this is what the presenter sizes its drawable to. Uses the SNAPPED size so
@@ -251,6 +305,31 @@ final class SessionPresenter {
contentSize = size
}
#if os(macOS)
/// Route presents for the window's composited state (MAIN thread the view pushes it on
/// every layout, which fullscreen transitions always trigger). PyroWave sessions in a
/// COMPOSITED (windowed) session present via `surfaceLayer` contents instead of the
/// CAMetalLayer image queue the DCP "mismatched swapID's" kernel-panic mitigation (see
/// `MetalVideoPresenter.surfaceLayer`; the metal-swap race survives glass pacing, so pacing
/// alone was not enough). VT codecs keep the metal path: no panic reports there, and their
/// HDR/EDR presentation has no surface-contents equivalent wired.
func setComposited(_ composited: Bool) {
guard let stage2, let connection else { return }
let wantsSurface = composited && connection.videoCodec == .pyrowave
guard wantsSurface != surfacePresentsActive else { return }
surfacePresentsActive = wantsSurface
stage2.setSurfacePresents(wantsSurface)
if !wantsSurface {
// Uncover the metal layer NOW (its last drawable is still attached, so fullscreen
// entry shows the previous frame until the next present no black flash).
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer?.contents = nil
CATransaction.commit()
}
}
#endif
/// Stop the active pump/pipeline ( one poll timeout; stage-2 joins its pump) and detach the
/// stage-2 layer + link. Does not close the connection that stays with whoever owns it.
/// Idempotent.
@@ -264,6 +343,11 @@ final class SessionPresenter {
stage2 = nil
metalLayer?.removeFromSuperlayer()
metalLayer = nil
#if os(macOS)
surfaceLayer?.removeFromSuperlayer()
surfaceLayer = nil
surfacePresentsActive = false
#endif
connection = nil
}
@@ -122,6 +122,11 @@ private final class VsyncClock: @unchecked Sendable {
/// coalesce in the newest-wins ring. Freshness is preserved by DROPPING stale frames before
/// present instead of queueing them behind the display the hidden queue latency becomes
/// explicit, correct frame drops.
///
/// macOS PyroWave sessions default to `glass` even though the platform default is stage-2: burst
/// presents into a composited (windowed) layer are the trigger pattern for the macOS DCP
/// "mismatched swapID's" KERNEL PANIC, and the one-in-flight gate removes that pattern see
/// `SessionPresenter.pacing` for the full rationale.
public enum PresentPacing: Sendable {
case arrival
case glass
@@ -625,6 +630,18 @@ public final class Stage2Pipeline {
presenter.setDrawableTarget(size)
}
#if os(macOS)
/// The windowed-mode PyroWave present target (see `MetalVideoPresenter.surfaceLayer` the
/// DCP swapID-panic mitigation). The hosting view installs it as a sibling above `layer`.
public var surfaceLayer: CALayer { presenter.surfaceLayer }
/// Forward the windowed-vs-fullscreen present routing (MAIN thread see
/// `MetalVideoPresenter.setSurfacePresents`).
public func setSurfacePresents(_ on: Bool) {
presenter.setSurfacePresents(on)
}
#endif
/// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen`
/// read). tvOS flips HDR presentation between PQ passthrough and the in-shader tone-map on
/// it; see `MetalVideoPresenter.setDisplayHeadroom`.
@@ -692,6 +692,11 @@ public final class StreamLayerView: NSView {
/// the view's physical-pixel size (bounds backing), so a window resize / retina move follows.
private func layoutPresenter() {
presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1)
// Present routing tracks the window's composited state (fullscreen transitions always
// re-layout, so this stays current): windowed PyroWave presents via surface contents
// the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited). A view
// not yet in a window counts as composited (the safe default).
presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false))
// Feed the follower only once in a window (backing scale is real then) and with real
// bounds a pre-window layout would report point-sized dimensions.
if window != nil, bounds.width > 0, bounds.height > 0 {
@@ -79,5 +79,44 @@ final class PresentPacingTests: XCTestCase {
XCTAssertEqual(
PresenterChoice.resolve(setting: nil, env: "stage1", allowStage1: false), .stage2)
}
/// `explicit` is nil exactly when `resolve` would fall back to the platform default the
/// distinction the codec-conditional pacing default rides on.
func testPresenterChoiceExplicitIsNilWithoutASelection() {
XCTAssertNil(PresenterChoice.explicit(setting: nil, env: nil, allowStage1: true))
XCTAssertNil(PresenterChoice.explicit(setting: "garbage", env: nil, allowStage1: true))
XCTAssertNil(PresenterChoice.explicit(setting: "stage1", env: nil, allowStage1: false))
XCTAssertEqual(
PresenterChoice.explicit(setting: "stage2", env: nil, allowStage1: true), .stage2)
XCTAssertEqual(
PresenterChoice.explicit(setting: nil, env: "stage3", allowStage1: true), .stage3)
}
// MARK: - Session pacing (the macOS PyroWave swapID-panic mitigation)
/// macOS PyroWave sessions under the DEFAULT stage-2 choice must get glass pacing (the
/// one-in-flight gate is the "mismatched swapID's" kernel-panic mitigation); an EXPLICIT
/// stage-2 pick must stay a faithful arrival-pacing A/B. Elsewhere the default is unchanged.
func testPacingDefaultsPyroWaveToGlassOnMacOS() {
#if os(macOS)
XCTAssertEqual(
SessionPresenter.pacing(for: .stage2, explicit: nil, codec: .pyrowave), .glass,
"defaulted macOS PyroWave must serialize presents (swapID-panic mitigation)")
XCTAssertEqual(
SessionPresenter.pacing(for: .stage2, explicit: .stage2, codec: .pyrowave), .arrival,
"an explicit stage-2 pick must keep arrival pacing (honest A/B)")
#else
XCTAssertEqual(
SessionPresenter.pacing(for: .stage2, explicit: nil, codec: .pyrowave), .arrival)
#endif
// Non-PyroWave defaults keep arrival pacing under stage-2 everywhere.
XCTAssertEqual(
SessionPresenter.pacing(for: .stage2, explicit: nil, codec: .hevc), .arrival)
// Stage-3 means glass regardless of codec or how it was chosen.
XCTAssertEqual(
SessionPresenter.pacing(for: .stage3, explicit: .stage3, codec: .hevc), .glass)
XCTAssertEqual(
SessionPresenter.pacing(for: .stage3, explicit: nil, codec: .pyrowave), .glass)
}
}
#endif
@@ -57,7 +57,7 @@ final class PyroWaveParserTests: XCTestCase {
func testLayoutMatchesUpstreamBlockSpace() {
// init_block_meta's walk for 256x144 (aligned 256x160): level extents halve from
// 128x80; per (comp,level,band) count32 = ceil(ceil(w/8)/4) * ceil(ceil(h/8)/4).
let layout = WaveletLayout(width: width, height: height)
let layout = WaveletLayout(width: width, height: height, chroma444: false)
XCTAssertEqual(layout.alignedWidth, 256)
XCTAssertEqual(layout.alignedHeight, 160)
XCTAssertEqual(layout.levelWidth(0), 128)
@@ -85,7 +85,7 @@ final class PyroWaveParserTests: XCTestCase {
}
func testDenseParseFillsOffsetsAndCountsBlocks() throws {
let layout = WaveletLayout(width: width, height: height)
let layout = WaveletLayout(width: width, height: height, chroma444: false)
var au = sof(totalBlocks: 4)
au += packet(blockIndex: 0)
au += packet(blockIndex: 3)
@@ -273,6 +273,17 @@ final class PyroWaveGoldenTests: XCTestCase {
try assertMatchesReference(decoded, prefix: "ref-chunked")
}
/// 4:4:4: the chroma components run the full pyramid like luma (no level-0 skip, no
/// early half-res emit) the layout + dispatch structure Phase 4 added
/// (design/pyrowave-444-hdr.md). The fixture comes from the 4:4:4 host encoder; the
/// reference is upstream's own 4:4:4 decode (full-res chroma planes).
func testDense444GoldenFrame() throws {
try XCTSkipIf(!MetalWaveletDecoder.supported, "no capable Metal device")
let au = try fixture("au-dense444")
let decoded = try decode(au: au, chunkAligned: false, windowSize: 0)
try assertMatchesReference(decoded, prefix: "ref-dense444")
}
/// Phase-4 partial delivery: zero a mid-AU window (a lost shard) the frame must still
/// decode (blocks > half) and stay recognizably the same picture (holes reconstruct as
/// localized blur, not garbage).
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+76 -6
View File
@@ -254,6 +254,55 @@ pub struct ZeroCopyPolicy {
pub fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
true
}
/// Whether the **native-plane** capturer (a compositor virtual output) can deliver an HDR (10-bit
/// PQ/BT.2020) source on this platform — the capture-side gate the punktfunk/1 handshake consults
/// before negotiating 10-bit (mirroring [`capturer_supports_444`]).
///
/// Linux: `false`. GNOME 50 added HDR **screen sharing** for *monitor* streams only — Mutter's
/// `RecordVirtual` virtual-monitor streams advertise 8-bit BGRx/BGRA exclusively (still true on
/// the GNOME 51 dev branch), and virtual outputs report no BT2020/PQ colour capabilities, so they
/// can't be flipped into HDR mode via DisplayConfig either. The Linux HDR path that DOES exist —
/// the GNOME 50+ portal **monitor mirror** (`open_portal_monitor` with `want_hdr`) — is gated
/// separately by the GameStream plane (`host_hdr_capable` + the live monitor colour-mode probe).
#[cfg(target_os = "linux")]
pub fn capturer_supports_hdr() -> bool {
false
}
/// Windows: the IDD-push capturer proactively enables advanced colour and delivers P010/Rgb10a2.
#[cfg(target_os = "windows")]
pub fn capturer_supports_hdr() -> bool {
true
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn capturer_supports_hdr() -> bool {
false
}
/// Process-wide latch: a `want_hdr` portal capture failed to negotiate the HDR (10-bit PQ) offer —
/// the compositor never accepted it (monitor left HDR mode between the probe and the negotiation,
/// NVIDIA EGL not listing LINEAR for XR30, a pre-50 Mutter…). Later sessions consult
/// [`hdr_capture_failed`] and fall back to the SDR offer instead of re-running the same doomed
/// 10-second negotiation timeout on every reconnect. Sticky until host restart (matching the
/// zero-copy downgrade latches); the log line at latch time says so.
#[cfg(target_os = "linux")]
static HDR_CAPTURE_FAILED: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(false);
#[cfg(target_os = "linux")]
pub fn hdr_capture_failed() -> bool {
HDR_CAPTURE_FAILED.load(std::sync::atomic::Ordering::Relaxed)
}
#[cfg(target_os = "linux")]
pub(crate) fn note_hdr_capture_failed() {
if !HDR_CAPTURE_FAILED.swap(true, std::sync::atomic::Ordering::Relaxed) {
tracing::warn!(
"HDR capture negotiation failed — this host will offer SDR capture for the rest of \
the process lifetime (restart the host after fixing the monitor's HDR mode to retry)"
);
}
}
#[cfg(target_os = "windows")]
pub fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
@@ -316,16 +365,28 @@ pub use idd_push::verify_is_wudfhost;
#[cfg(target_os = "linux")]
#[path = "linux/mod.rs"]
mod linux;
// The GNOME BT.2100 colour-mode probe — the host's capture-side gate for offering HDR on the
// portal monitor path (see `open_portal_monitor`'s `want_hdr`).
#[cfg(target_os = "linux")]
pub use linux::gnome_hdr_monitor_active;
#[cfg(target_os = "windows")]
#[path = "windows/synthetic_nv12.rs"]
pub mod synthetic_nv12;
/// Open the Linux xdg-ScreenCast portal capturer for a client-sized monitor. `anchored` drives
/// ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits that grant headlessly. The
/// [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
/// ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits that grant headlessly.
/// `want_hdr` offers the GNOME 50+ HDR formats (10-bit PQ/BT.2020 dmabufs) instead of the SDR
/// set — pass it only when the mirrored monitor is actually in HDR mode (the host probes
/// DisplayConfig) or the negotiation runs into its 10 s timeout and latches the SDR downgrade.
/// The [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
#[cfg(target_os = "linux")]
pub fn open_portal_monitor(anchored: bool, policy: ZeroCopyPolicy) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::open(anchored, policy).map(|c| Box::new(c) as Box<dyn Capturer>)
pub fn open_portal_monitor(
anchored: bool,
want_hdr: bool,
policy: ZeroCopyPolicy,
) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::open(anchored, want_hdr && !hdr_capture_failed(), policy)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// Open the Linux portal capturer bound to an already-created virtual output's PipeWire node. The
@@ -365,9 +426,18 @@ pub fn open_idd_push(
preferred: Option<(u32, u32, u32)>,
client_10bit: bool,
want_444: bool,
pyrowave: bool,
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(target, preferred, client_10bit, want_444, keepalive, sender)
.map(|c| Box::new(c) as Box<dyn Capturer>)
idd_push::IddPushCapturer::open(
target,
preferred,
client_10bit,
want_444,
pyrowave,
keepalive,
sender,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
+373 -20
View File
@@ -62,6 +62,13 @@ pub struct PortalCapturer {
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
/// rebuild retries on the CPU offer instead of failing identically forever.
vaapi_dmabuf: bool,
/// This capture ran the HDR (10-bit PQ/BT.2020 dmabuf) offer — see [`Self::open`]'s
/// `want_hdr`. Read by the negotiation-timeout diagnosis (a failed HDR offer latches the
/// process-wide SDR downgrade) and by [`hdr_meta`](Capturer::hdr_meta).
hdr_offer: bool,
/// Set once the stream negotiated one of the 10-bit PQ formats (`param_changed`), i.e. frames
/// really are PQ/BT.2020 — drives [`hdr_meta`](Capturer::hdr_meta).
hdr_negotiated: Arc<AtomicBool>,
/// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis.
node_id: u32,
/// Stops the PipeWire loop on teardown (sent in `Drop`). Without it a dropped or failed
@@ -80,8 +87,9 @@ pub struct PortalCapturer {
impl PortalCapturer {
/// `anchored` drives ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits the
/// RemoteDesktop grant and never raises a separate ScreenCast dialog; `false` uses a plain
/// ScreenCast session (wlroots, which has no RemoteDesktop portal).
pub fn open(anchored: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> {
/// ScreenCast session (wlroots, which has no RemoteDesktop portal). `want_hdr` offers the
/// GNOME 50+ HDR formats (10-bit PQ/BT.2020, dmabuf-only) instead of the SDR set.
pub fn open(anchored: bool, want_hdr: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> {
// Portal handshake (async) on its own thread; hands back the PW fd + node id.
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
thread::Builder::new()
@@ -102,11 +110,12 @@ impl PortalCapturer {
};
tracing::info!(
node_id,
want_hdr,
"ScreenCast portal session started; connecting PipeWire"
);
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok(
spawn_pipewire(Some(fd), node_id, None, true, false, policy)?
spawn_pipewire(Some(fd), node_id, None, true, false, want_hdr, policy)?
.into_capturer(node_id, None),
)
}
@@ -135,12 +144,15 @@ impl PortalCapturer {
want_444,
"connecting PipeWire to virtual output"
);
// Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit
// BGRx/BGRA exclusively, GNOME 50 and 51-dev alike) — never run the HDR offer here.
Ok(spawn_pipewire(
remote_fd,
node_id,
preferred_mode,
allow_zerocopy,
want_444,
false,
policy,
)?
.into_capturer(node_id, Some(keepalive)))
@@ -160,6 +172,10 @@ struct PwHandles {
/// This capture will offer LINEAR-dmabuf-only for the VAAPI passthrough (see
/// [`PortalCapturer::vaapi_dmabuf`]).
vaapi_dmabuf: bool,
/// This capture ran the HDR offer (see [`PortalCapturer::hdr_offer`]).
hdr_offer: bool,
/// See [`PortalCapturer::hdr_negotiated`].
hdr_negotiated: Arc<AtomicBool>,
quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>,
}
@@ -177,6 +193,8 @@ impl PwHandles {
broken: self.broken,
stall_since: None,
vaapi_dmabuf: self.vaapi_dmabuf,
hdr_offer: self.hdr_offer,
hdr_negotiated: self.hdr_negotiated,
node_id,
quit: Some(self.quit),
join: Some(self.join),
@@ -199,6 +217,10 @@ fn spawn_pipewire(
// 4:4:4 session: tiled dmabufs convert to planar YUV444 on the GPU (`ImportKind::Tiled444`)
// instead of NV12/RGB, so the session stays zero-copy at full chroma.
want_444: bool,
// HDR session (GNOME 50+ monitor mirror): offer ONLY the 10-bit PQ/BT.2020 formats as
// LINEAR dmabufs (SHM can't carry them — Mutter's SHM record path paints 8-bit ARGB32
// regardless of the negotiated format, and the tiled EGL de-tile blit is 8-bit).
want_hdr: bool,
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6).
policy: ZeroCopyPolicy,
@@ -213,17 +235,29 @@ fn spawn_pipewire(
let streaming_cb = streaming.clone();
let broken = Arc::new(AtomicBool::new(false));
let broken_cb = broken.clone();
let hdr_negotiated = Arc::new(AtomicBool::new(false));
let hdr_negotiated_cb = hdr_negotiated.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope.
let (quit_tx, quit_rx) = ::pipewire::channel::channel::<()>();
let zerocopy = allow_zerocopy && pf_zerocopy::enabled();
// HDR cannot ride the SHM path (see `want_hdr` above): under PUNKTFUNK_FORCE_SHM the HDR
// offer is dropped — SDR capture, loudly.
let force_shm = std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() == Ok("1");
let want_hdr = if want_hdr && force_shm {
tracing::warn!(
"HDR capture requested but PUNKTFUNK_FORCE_SHM=1 — the SHM path is 8-bit only; \
offering SDR"
);
false
} else {
want_hdr
};
// Mirror of the thread's `vaapi_passthrough` decision (deterministic from here: on a VAAPI
// backend the EGL→CUDA importer is never built) — kept on the capturer so `next_frame`'s
// negotiation-timeout branch knows a failed negotiation was the LINEAR-dmabuf offer.
let vaapi_dmabuf = zerocopy
&& std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() != Ok("1")
&& policy.backend_is_vaapi;
let vaapi_dmabuf = zerocopy && !force_shm && policy.backend_is_vaapi;
let join = thread::Builder::new()
.name("punktfunk-pipewire".into())
.spawn(move || {
@@ -235,8 +269,10 @@ fn spawn_pipewire(
negotiated_cb,
streaming_cb,
broken_cb,
hdr_negotiated_cb,
zerocopy,
want_444,
want_hdr,
preferred,
quit_rx,
policy,
@@ -252,6 +288,8 @@ fn spawn_pipewire(
streaming,
broken,
vaapi_dmabuf,
hdr_offer: want_hdr,
hdr_negotiated,
quit: quit_tx,
join,
})
@@ -354,6 +392,26 @@ impl Capturer for PortalCapturer {
fn set_active(&self, active: bool) {
self.active.store(active, Ordering::Relaxed);
}
/// Generic HDR10 mastering metadata once the stream negotiated a 10-bit PQ format. Mutter
/// exposes no per-monitor mastering volume through the screencast, so this is the standard
/// HDR10 default block (BT.2020 primaries, D65 white, 1000 / 0.005 cd/m², CLL unknown) — the
/// same fallback Windows uses when a display reports nothing. The native stream loop prefers
/// the client display's own volume when the client sent one (`Hello::display_hdr`).
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
if !self.hdr_negotiated.load(Ordering::Relaxed) {
return None;
}
Some(punktfunk_core::quic::HdrMeta {
// ST.2086 order G, B, R; (x, y) chromaticity in 1/50000 units.
display_primaries: [[8500, 39850], [6550, 2300], [35400, 14600]],
white_point: [15635, 16450], // D65
max_display_mastering_luminance: 10_000_000, // 1000 cd/m² (0.0001 units)
min_display_mastering_luminance: 50, // 0.005 cd/m²
max_cll: 0,
max_fall: 0,
})
}
}
impl PortalCapturer {
@@ -372,6 +430,20 @@ impl PortalCapturer {
or capture never started)",
self.node_id
))
} else if self.hdr_offer {
// The HDR (10-bit PQ dmabuf) offer was never accepted — the monitor left HDR
// mode between the probe and the negotiation, the compositor pre-dates the
// GNOME 50 HDR formats, or its allocator can't do LINEAR for XR30/XB30.
// Latch the process-wide SDR downgrade so the next session (Moonlight
// auto-reconnects) negotiates SDR instead of re-running this same timeout.
super::note_hdr_capture_failed();
Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored monitor in \
HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \
to stream SDR",
self.node_id
))
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
// The LINEAR-dmabuf-only offer (VAAPI passthrough default) was never accepted.
// Latch the process-wide downgrade so the encode loop's pipeline rebuild
@@ -416,6 +488,87 @@ impl Drop for PortalCapturer {
}
}
/// Whether any monitor of the live GNOME session is currently in BT.2100 (HDR) colour mode — the
/// precondition for Mutter's monitor screencast advertising the 10-bit PQ formats (GNOME 50+;
/// Mutter only appends the HDR formats while the mirrored monitor's colour state is BT.2020+PQ).
/// Queried over the session bus: `DisplayConfig.GetCurrentState`, monitor property
/// `"color-mode" == 1` (`META_COLOR_MODE_BT2100`). `false` on any error — not GNOME, a pre-48
/// Mutter without colour modes, no monitors — so callers fall back to the honest SDR offer.
/// Blocking (one D-Bus round-trip on a fresh connection); call from control-plane threads only.
pub fn gnome_hdr_monitor_active() -> bool {
use ashpd::zbus;
// GetCurrentState reply: (serial, monitors, logical_monitors, properties); each monitor is
// (spec(ssss), modes a(siiddada{sv}), properties a{sv}) — "color-mode" lives in the monitor
// properties.
type Mode = (
String,
i32,
i32,
f64,
f64,
Vec<f64>,
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
);
type Monitor = (
(String, String, String, String),
Vec<Mode>,
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
);
type LogicalMonitor = (
i32,
i32,
f64,
u32,
bool,
Vec<(String, String, String, String)>,
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
);
type State = (
u32,
Vec<Monitor>,
Vec<LogicalMonitor>,
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
);
let probe = || -> Result<bool> {
// zbus is built async-only here (ashpd's tokio integration) — run the one round-trip on
// a throwaway current-thread runtime; this is a control-plane call, never per-frame.
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.context("build tokio runtime")?;
rt.block_on(async {
let conn = zbus::Connection::session().await.context("session bus")?;
let reply = conn
.call_method(
Some("org.gnome.Mutter.DisplayConfig"),
"/org/gnome/Mutter/DisplayConfig",
Some("org.gnome.Mutter.DisplayConfig"),
"GetCurrentState",
&(),
)
.await
.context("DisplayConfig.GetCurrentState")?;
let (_serial, monitors, _logical, _props): State = reply
.body()
.deserialize()
.context("parse GetCurrentState")?;
Ok(monitors.iter().any(|(_spec, _modes, props)| {
props
.get("color-mode")
.and_then(|v| u32::try_from(v).ok())
.is_some_and(|mode| mode == 1) // META_COLOR_MODE_BT2100
}))
})
};
match probe() {
Ok(hdr) => hdr,
Err(e) => {
tracing::debug!(error = %format!("{e:#}"), "GNOME HDR colour-mode probe failed — SDR");
false
}
}
}
/// Pick the ScreenCast cursor mode from what the backend advertises (`AvailableCursorModes`),
/// preferring **cursor-as-metadata**: the compositor keeps its cheap hardware cursor plane and
/// ships the pointer as PipeWire `SPA_META_Cursor` metadata (position + an occasional bitmap),
@@ -669,6 +822,10 @@ mod pipewire {
VideoFormat::RGBA => PixelFormat::Rgba,
VideoFormat::RGB => PixelFormat::Rgb,
VideoFormat::BGR => PixelFormat::Bgr,
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
VideoFormat::xBGR_210LE => PixelFormat::X2Bgr10,
_ => return None,
})
}
@@ -732,6 +889,9 @@ mod pipewire {
/// irrecoverably gone for this stream — the import worker died, or tiled imports failed
/// [`IMPORT_FAIL_POISON`] times in a row.
broken: Arc<AtomicBool>,
/// Set when the negotiated format is one of the 10-bit PQ formats (`param_changed`) —
/// read by [`PortalCapturer::hdr_meta`](super::PortalCapturer).
hdr_negotiated: Arc<AtomicBool>,
/// Consecutive tiled-import failures (reset on success); see [`IMPORT_FAIL_POISON`].
import_fail_streak: u32,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer,
@@ -886,6 +1046,91 @@ mod pipewire {
serialize_pod(obj)
}
/// Build one GNOME 50+ HDR format pod: `format` (xRGB_210LE / xBGR_210LE) as a LINEAR-only
/// dmabuf with **MANDATORY** BT.2020 primaries + SMPTE ST.2084 (PQ) transfer-function props —
/// the exact colorimetry Mutter's monitor stream advertises while the mirrored monitor is in
/// HDR mode (its HDR pods carry the same props MANDATORY, so both sides must speak them for
/// the intersection to exist; an SDR or pre-50 producer can never match this pod).
///
/// LINEAR-only because every 10-bit consumer we have reads the buffer without a de-tile pass:
/// the CPU path mmaps it, and the VAAPI passthrough imports it into a VA surface. The tiled
/// EGL de-tile blit renders into an 8-bit `GL_RGBA8` texture — it would silently crush the
/// depth — so tiled modifiers are deliberately NOT advertised (a zero-copy 10-bit de-tile is
/// the follow-up). SHM is excluded entirely: Mutter's SHM record path paints 8-bit ARGB32
/// regardless of the negotiated format.
/// `SPA_VIDEO_TRANSFER_SMPTE2084` (PQ) — spelled out rather than taken from `pw::spa::sys`
/// because libspa only grew the constant with the BT2020_10/SMPTE2084/ARIB_STD_B67 block, and
/// the distro builders (Ubuntu 24.04 noble for the .deb) ship headers predating it — bindgen
/// then emits no such constant and the host fails to compile there, even though the code never
/// runs on those systems (the HDR path needs GNOME 50+).
///
/// 14 is the enum's position in `spa/param/video/color.h` and is wire ABI, not a private
/// detail: SPA mirrors GStreamer's `GstVideoTransferFunction`, where that block was added
/// together, so the value is identical on every libspa that has the symbol at all. On one that
/// doesn't, PipeWire simply fails to intersect this format offer and the session negotiates
/// SDR — the same outcome as not offering HDR.
const SPA_VIDEO_TRANSFER_SMPTE2084: u32 = 14;
fn build_hdr_dmabuf_format(
format: VideoFormat,
preferred: Option<(u32, u32, u32)>,
) -> Result<Vec<u8>> {
let (dw, dh, dhz) = preferred.unwrap_or((1920, 1080, 60));
use pw::spa::param::format::{FormatProperties, MediaSubtype, MediaType};
let mut obj = pw::spa::pod::object!(
pw::spa::utils::SpaTypes::ObjectParamFormat,
pw::spa::param::ParamType::EnumFormat,
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format),
pw::spa::pod::property!(
FormatProperties::VideoSize,
Choice,
Range,
Rectangle,
pw::spa::utils::Rectangle {
width: dw,
height: dh
},
pw::spa::utils::Rectangle {
width: 1,
height: 1
},
pw::spa::utils::Rectangle {
width: 8192,
height: 8192
}
),
pw::spa::pod::property!(
FormatProperties::VideoFramerate,
Choice,
Range,
Fraction,
pw::spa::utils::Fraction { num: dhz, denom: 1 },
pw::spa::utils::Fraction { num: 0, denom: 1 },
pw::spa::utils::Fraction { num: 240, denom: 1 }
),
);
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Long(0), // DRM_FORMAT_MOD_LINEAR
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_transferFunction,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(SPA_VIDEO_TRANSFER_SMPTE2084)),
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorPrimaries,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_PRIMARIES_BT2020,
)),
});
serialize_pod(obj)
}
/// The default (shm/CPU-path) format offer: raw video in any encoder-mappable layout, any
/// size, any framerate (0/1 = variable allowed — gamescope fixates exactly that).
fn build_default_format_obj(preferred: Option<(u32, u32, u32)>) -> pw::spa::pod::Object {
@@ -1157,6 +1402,54 @@ mod pipewire {
})
}
/// Alpha-blend the cached cursor bitmap into a packed 10-bit (`X2Rgb10`/`X2Bgr10`) CPU frame:
/// unpack each u32, blend the 8-bit cursor channels scaled to 10 bits (`v<<2 | v>>6`), repack.
/// The frame samples are PQ-encoded, so like the 8-bit gamma-space blend this is a display-
/// referred approximation — fine for a cursor. `r_shift` is the R channel's bit offset (20 for
/// x:R:G:B, 0 for x:B:G:R); G is always at 10 and B mirrors R.
fn composite_cursor_rgb10(
tight: &mut [u8],
w: usize,
h: usize,
r_shift: u32,
cursor: &CursorState,
) {
let b_shift = 20 - r_shift; // 0 or 20 — the opposite end from R
let (bw, bh) = (cursor.bw as i32, cursor.bh as i32);
for cy in 0..bh {
let dy = cursor.y + cy;
if dy < 0 || dy as usize >= h {
continue;
}
for cx in 0..bw {
let dx = cursor.x + cx;
if dx < 0 || dx as usize >= w {
continue;
}
let s = ((cy * bw + cx) as usize) * 4;
let a = cursor.rgba[s + 3] as u32;
if a == 0 {
continue;
}
// 8-bit cursor channel → 10-bit (replicate the top bits into the bottom).
let up10 = |v: u8| ((v as u32) << 2) | ((v as u32) >> 6);
let (sr, sg, sb) = (
up10(cursor.rgba[s]),
up10(cursor.rgba[s + 1]),
up10(cursor.rgba[s + 2]),
);
let di = (dy as usize * w + dx as usize) * 4;
let px = u32::from_le_bytes(tight[di..di + 4].try_into().unwrap());
let blend = |dst: u32, src: u32| (src * a + dst * (255 - a)) / 255;
let dr = blend((px >> r_shift) & 0x3ff, sr);
let dg = blend((px >> 10) & 0x3ff, sg);
let db = blend((px >> b_shift) & 0x3ff, sb);
let out = (px & 0xc000_0000) | (dr << r_shift) | (dg << 10) | (db << b_shift);
tight[di..di + 4].copy_from_slice(&out.to_le_bytes());
}
}
}
/// Alpha-blend the cached cursor bitmap into the tightly-packed CPU frame at its latched
/// position. Cheap: a straight-alpha blit over at most ~256×256 pixels, clipped to the frame —
/// the whole point of cursor-as-metadata (no forced full-frame composite on the producer).
@@ -1170,6 +1463,12 @@ mod pipewire {
if !cursor.visible || cursor.rgba.is_empty() {
return;
}
// The packed 10-bit HDR layouts blend via bit unpack/repack, not byte offsets.
match fmt {
PixelFormat::X2Rgb10 => return composite_cursor_rgb10(tight, w, h, 20, cursor),
PixelFormat::X2Bgr10 => return composite_cursor_rgb10(tight, w, h, 0, cursor),
_ => {}
}
let Some((ri, gi, bi, bpp)) = dst_offsets(fmt) else {
return;
};
@@ -1344,7 +1643,10 @@ mod pipewire {
// through to the shm de-pad copy below.
let mut gpu_import_broken = false;
if let (Some(importer), Some(fmt)) = (ud.importer.as_mut(), ud.format) {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
// Defense-in-depth: the 10-bit PQ formats must never enter the EGL→CUDA import (its
// de-tile blit is 8-bit RGBA8 — silent depth loss). An HDR offer never builds the
// importer, so this gate only matters if those invariants ever drift apart.
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf && !fmt.is_hdr_rgb10() {
let plane = pf_zerocopy::DmabufPlane {
fd: datas[0].fd(),
offset: datas[0].chunk().offset(),
@@ -1604,9 +1906,13 @@ mod pipewire {
negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
broken: Arc<AtomicBool>,
hdr_negotiated: Arc<AtomicBool>,
zerocopy: bool,
// 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert.
want_444: bool,
// HDR session: offer ONLY the 10-bit PQ/BT.2020 formats as LINEAR dmabufs (see
// `build_hdr_dmabuf_format`); the SDR offers are not built at all.
want_hdr: bool,
preferred: Option<(u32, u32, u32)>,
quit_rx: pw::channel::Receiver<()>,
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
@@ -1645,7 +1951,10 @@ mod pipewire {
// succeed and produce CUDA payloads the VAAPI encoder must reject. Also skipped once
// repeated worker deaths latched the import off (a wedged GPU stack must not crash-loop).
let backend_is_vaapi = policy.backend_is_vaapi;
let mut importer = if zerocopy && !backend_is_vaapi {
// HDR never builds the EGL→CUDA importer: its de-tile blit renders into 8-bit RGBA8,
// which would silently crush the 10-bit depth. The HDR consumers are the CPU mmap path
// (LINEAR de-pad → X2Rgb10 CPU frames) and the VAAPI raw-dmabuf passthrough.
let mut importer = if zerocopy && !backend_is_vaapi && !want_hdr {
if pf_zerocopy::gpu_import_disabled() {
tracing::warn!(
"zero-copy GPU import disabled after repeated import-worker deaths — using CPU path"
@@ -1755,6 +2064,7 @@ mod pipewire {
negotiated,
streaming,
broken,
hdr_negotiated,
import_fail_streak: 0,
importer,
vaapi_passthrough,
@@ -1822,12 +2132,20 @@ mod pipewire {
let sz = ud.info.size();
ud.format = map_format(ud.info.format());
ud.modifier = ud.info.modifier();
// HDR: the 10-bit PQ formats are only ever offered with MANDATORY BT.2020/PQ
// colorimetry props, so a 10-bit negotiation IS an HDR negotiation — but log
// what the producer actually fixated for diagnosis.
let hdr = ud.format.is_some_and(|f| f.is_hdr_rgb10());
ud.hdr_negotiated.store(hdr, Ordering::Relaxed);
tracing::info!(
width = sz.width,
height = sz.height,
spa_format = ?ud.info.format(),
mapped = ?ud.format,
modifier = ud.modifier,
hdr,
transfer_function = ud.info.transfer_function(),
color_primaries = ud.info.color_primaries(),
"pipewire format negotiated"
);
if ud.format.is_none() {
@@ -2029,20 +2347,36 @@ mod pipewire {
// (offering shm too makes the compositor pick shm). The modifier list is advertised with
// DONT_FIXATE so the compositor's allocator chooses one; we re-emit the fixated format in
// `param_changed` (the two-step DMA-BUF handshake). Otherwise offer the multi-format shm
// pod and let MAP_BUFFERS map it.
let shm_values = serialize_pod(obj)?;
let (dmabuf_values, buffers_values) = if want_dmabuf {
(
Some(build_dmabuf_format(&modifiers, preferred)?),
Some(build_dmabuf_buffers()?),
)
// pod and let MAP_BUFFERS map it. An HDR session replaces ALL of this with the two 10-bit
// PQ pods (LINEAR dmabuf, MANDATORY colorimetry — see `build_hdr_dmabuf_format`): offering
// SDR alongside would make the producer pick its earlier-listed SDR format, and the
// negotiation-timeout path latches the process-wide SDR downgrade if nothing matches.
let format_pods: Vec<Vec<u8>> = if want_hdr {
tracing::info!(
"HDR capture: offering xRGB_210LE/xBGR_210LE LINEAR dmabufs with MANDATORY \
BT.2020 + SMPTE-2084 (PQ) colorimetry (GNOME 50+ monitor stream)"
);
vec![
build_hdr_dmabuf_format(VideoFormat::xRGB_210LE, preferred)?,
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
]
} else if want_dmabuf {
vec![build_dmabuf_format(&modifiers, preferred)?]
} else {
vec![serialize_pod(obj)?]
};
let buffers_values = if want_hdr || want_dmabuf {
// Dmabuf-only. For HDR this is load-bearing beyond zero-copy: Mutter's SHM record
// path paints 8-bit ARGB32 regardless of the negotiated format, so a MemFd buffer
// under a 10-bit format would carry mislabeled bytes.
Some(build_dmabuf_buffers()?)
} else if force_shm {
// True SHM: exclude DmaBuf so Mutter MUST download (glReadPixels orders against render).
(None, Some(build_shm_only_buffers()?))
Some(build_shm_only_buffers()?)
} else {
// CPU path still accepts mappable dmabufs (gamescope offers only those once its
// modifier-bearing format pod wins the intersection).
(None, Some(build_mappable_buffers()?))
Some(build_mappable_buffers()?)
};
// Ask for cursor-as-metadata on every path (harmless if the producer can't supply it): the
@@ -2050,9 +2384,8 @@ mod pipewire {
// compositor keeps its cheap hardware cursor plane (see `choose_cursor_mode`).
let cursor_meta = build_cursor_meta_param()?;
let mut byte_slices: Vec<&[u8]> = Vec::new();
match &dmabuf_values {
Some(d) => byte_slices.push(d),
None => byte_slices.push(&shm_values),
for pod in &format_pods {
byte_slices.push(pod);
}
if let Some(b) = &buffers_values {
byte_slices.push(b);
@@ -2076,4 +2409,24 @@ mod pipewire {
mainloop.run();
Ok(())
}
#[cfg(test)]
mod tests {
/// Pin our hand-written PQ transfer id against the real libspa binding. We can't take the
/// constant from `pw::spa::sys` directly (older distro headers don't export it — see
/// [`super::SPA_VIDEO_TRANSFER_SMPTE2084`]), so assert the two agree wherever the symbol
/// DOES exist. Any libspa that renumbers the enum fails this instead of silently tagging
/// the HDR offer with the wrong transfer function.
///
/// Only builds where tests are compiled — the .deb/.rpm builders run plain `cargo build`,
/// so this never reintroduces the compile failure it exists to prevent.
#[test]
fn pq_transfer_id_matches_libspa() {
assert_eq!(
super::SPA_VIDEO_TRANSFER_SMPTE2084,
super::pw::spa::sys::SPA_VIDEO_TRANSFER_SMPTE2084,
"libspa renumbered spa_video_transfer_function — update the hardcoded PQ id"
);
}
}
}
+238 -15
View File
@@ -12,7 +12,7 @@
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
pub use pf_frame::dxgi::{make_device, pack_luid, D3d11Frame, WinCaptureTarget};
pub use pf_frame::dxgi::{make_device, pack_luid, D3d11Frame, PyroFrameShare, WinCaptureTarget};
use anyhow::{bail, Context, Result};
use std::ffi::c_void;
@@ -466,6 +466,229 @@ impl HdrP010Converter {
}
}
/// PyroWave LUMA pass PS — full-res, writes Y to a separate `R8_UNORM` texture. BT.709 limited from
/// the 8-bit sRGB (gamma) BGRA slot, BYTE-IDENTICAL to the Linux `rgb2yuv.comp` `lumaY` (so the
/// wavelet client — whose golden fixtures come from that shader — decodes the same colours). `Load`
/// (texelFetch) reads the exact source texel: RTV pixel (x,y) → source texel (x,y).
const PYRO_Y_PS: &str = r"
Texture2D<float4> tx : register(t0);
float main(float4 pos : SV_POSITION) : SV_TARGET {
float3 c = tx.Load(int3(int2(pos.xy), 0)).rgb;
return 16.0/255.0 + 0.1826*c.r + 0.6142*c.g + 0.0620*c.b;
}
";
/// PyroWave CHROMA pass PS — half-res, writes interleaved (Cb,Cr) to a separate `R8G8_UNORM` texture.
/// **2×2 box average** (centre-sited) of the four luma-block RGB texels, then BT.709 limited Cb/Cr —
/// BYTE-IDENTICAL to `rgb2yuv.comp` (which averages `(c00+c10+c01+c11)*0.25` then U/V), so the chroma
/// siting matches the client's decoder. Even dimensions guarantee the 2×2 block is in-bounds.
const PYRO_UV_PS: &str = r"
Texture2D<float4> tx : register(t0);
float2 main(float4 pos : SV_POSITION) : SV_TARGET {
int2 p = int2(pos.xy) * 2;
float3 c00 = tx.Load(int3(p, 0)).rgb;
float3 c10 = tx.Load(int3(p + int2(1,0), 0)).rgb;
float3 c01 = tx.Load(int3(p + int2(0,1), 0)).rgb;
float3 c11 = tx.Load(int3(p + int2(1,1), 0)).rgb;
float3 a = (c00 + c10 + c01 + c11) * 0.25;
float u = 128.0/255.0 - 0.1006*a.r - 0.3386*a.g + 0.4392*a.b;
float v = 128.0/255.0 + 0.4392*a.r - 0.3989*a.g - 0.0403*a.b;
return float2(u, v);
}
";
/// PyroWave 4:4:4 CHROMA pass PS — FULL-res, per-pixel (no box filter, no siting), the Windows twin
/// of the Linux `rgb2yuv444.comp` chroma math.
const PYRO_UV444_PS: &str = r"
Texture2D<float4> tx : register(t0);
float2 main(float4 pos : SV_POSITION) : SV_TARGET {
float3 c = tx.Load(int3(int2(pos.xy), 0)).rgb;
float u = 128.0/255.0 - 0.1006*c.r - 0.3386*c.g + 0.4392*c.b;
float v = 128.0/255.0 + 0.4392*c.r - 0.3989*c.g - 0.0403*c.b;
return float2(u, v);
}
";
/// Shared HLSL for the PyroWave **HDR** passes: scRGB FP16 → PQ-encoded BT.2020 → 10-bit studio
/// codes MSB-packed into 16-bit UNORM — the SAME colour math as [`HDR_P010_COMMON`] (verified by
/// `hdr_p010_selftest`), restated over `Load`ed texels so the pyrowave passes stay texel-exact like
/// their SDR twins. The wavelet client decodes these planes with the same CSC rows as the P010 path.
const PYRO_HDR_COMMON: &str = r"
Texture2D<float4> tx : register(t0);
static const float3x3 BT709_TO_BT2020 = {
0.627403914, 0.329283038, 0.043313048,
0.069097292, 0.919540405, 0.011362303,
0.016391439, 0.088013308, 0.895595253
};
float3 pq_oetf(float3 L) {
const float m1 = 0.1593017578125;
const float m2 = 78.84375;
const float c1 = 0.8359375;
const float c2 = 18.8515625;
const float c3 = 18.6875;
float3 Lp = pow(saturate(L), m1);
return pow((c1 + c2 * Lp) / (1.0 + c3 * Lp), m2);
}
float3 scrgb_to_pq2020_rgb(float3 scrgb) {
float3 nits = max(scrgb, 0.0) * 80.0;
return pq_oetf(mul(BT709_TO_BT2020, nits) / 10000.0);
}
static const float KR = 0.2627;
static const float KG = 0.6780;
static const float KB = 0.0593;
float y_unorm(float3 pq) {
float y = KR * pq.r + KG * pq.g + KB * pq.b;
float code = clamp(64.0 + 876.0 * y, 64.0, 940.0);
return (code * 64.0) / 65535.0;
}
float2 cbcr_unorm(float3 pq) {
float y = KR * pq.r + KG * pq.g + KB * pq.b;
float cbc = clamp(512.0 + 896.0 * (pq.b - y) / 1.8814, 64.0, 960.0);
float crc = clamp(512.0 + 896.0 * (pq.r - y) / 1.4746, 64.0, 960.0);
return float2((cbc * 64.0) / 65535.0, (crc * 64.0) / 65535.0);
}
";
/// PyroWave HDR LUMA pass PS — full-res, writes PQ Y studio codes to an `R16_UNORM` texture.
const PYRO_HDR_Y_PS: &str = r"
#include_common
float main(float4 pos : SV_POSITION) : SV_TARGET {
float3 pq = scrgb_to_pq2020_rgb(tx.Load(int3(int2(pos.xy), 0)).rgb);
return y_unorm(pq);
}
";
/// PyroWave HDR 4:2:0 CHROMA pass PS — half-res, centre-sited 2×2 box in scRGB-LINEAR space (the
/// pyrowave family's siting, matching the SDR pass + `rgb2yuv.comp`, NOT the P010 path's
/// left-cositing), then PQ + studio Cb/Cr into an `R16G16_UNORM` texture.
const PYRO_HDR_UV_PS: &str = r"
#include_common
float2 main(float4 pos : SV_POSITION) : SV_TARGET {
int2 p = int2(pos.xy) * 2;
float3 a = max(tx.Load(int3(p, 0)).rgb, 0.0);
float3 b = max(tx.Load(int3(p + int2(1,0), 0)).rgb, 0.0);
float3 c = max(tx.Load(int3(p + int2(0,1), 0)).rgb, 0.0);
float3 d = max(tx.Load(int3(p + int2(1,1), 0)).rgb, 0.0);
float3 pq = scrgb_to_pq2020_rgb((a + b + c + d) * 0.25);
return cbcr_unorm(pq);
}
";
/// PyroWave HDR 4:4:4 CHROMA pass PS — full-res, per-pixel.
const PYRO_HDR_UV444_PS: &str = r"
#include_common
float2 main(float4 pos : SV_POSITION) : SV_TARGET {
float3 pq = scrgb_to_pq2020_rgb(tx.Load(int3(int2(pos.xy), 0)).rgb);
return cbcr_unorm(pq);
}
";
/// scRGB/BGRA → **separate** YUV planes for the PyroWave wavelet encoder: a full-res Y texture + a
/// (half- or full-res) interleaved CbCr texture (design/pyrowave-windows-host-zerocopy.md +
/// design/pyrowave-444-hdr.md). SDR mode reads the BGRA slot and writes BT.709-limited 8-bit planes
/// (`R8_UNORM`/`R8G8_UNORM`), byte-identical to the Linux `rgb2yuv(444).comp`; HDR mode reads the
/// scRGB FP16 slot and writes P010-style 10-bit studio codes MSB-packed into 16-bit planes
/// (`R16_UNORM`/`R16G16_UNORM`), colour math identical to [`HdrP010Converter`]. The wavelet encoder
/// imports the two SEPARATE textures into its own Vulkan device — the NVIDIA D3D11→Vulkan import of
/// a single *planar* NV12 texture is unreliable at arbitrary sizes, whereas simple single/
/// two-component textures import reliably. The caller owns the two textures + their RTVs (shareable,
/// per out-ring slot); this only records the passes.
pub(crate) struct BgraToYuvPlanes {
vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader,
ps_uv: ID3D11PixelShader,
/// Full-res chroma pass (4:4:4) — the chroma viewport skips the /2.
chroma444: bool,
}
impl BgraToYuvPlanes {
pub(crate) unsafe fn new(device: &ID3D11Device, hdr: bool, chroma444: bool) -> Result<Self> {
let (y_src, uv_src) = match (hdr, chroma444) {
(false, false) => (PYRO_Y_PS.to_string(), PYRO_UV_PS.to_string()),
(false, true) => (PYRO_Y_PS.to_string(), PYRO_UV444_PS.to_string()),
(true, false) => (
PYRO_HDR_Y_PS.replace("#include_common", PYRO_HDR_COMMON),
PYRO_HDR_UV_PS.replace("#include_common", PYRO_HDR_COMMON),
),
(true, true) => (
PYRO_HDR_Y_PS.replace("#include_common", PYRO_HDR_COMMON),
PYRO_HDR_UV444_PS.replace("#include_common", PYRO_HDR_COMMON),
),
};
let vsb = compile_shader(HDR_VS, s!("main"), s!("vs_5_0"))?;
let yb = compile_shader(&y_src, s!("main"), s!("ps_5_0"))?;
let uvb = compile_shader(&uv_src, s!("main"), s!("ps_5_0"))?;
let mut vs = None;
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
let mut ps_y = None;
device.CreatePixelShader(&yb, None, Some(&mut ps_y))?;
let mut ps_uv = None;
device.CreatePixelShader(&uvb, None, Some(&mut ps_uv))?;
Ok(Self {
vs: vs.context("pyro vs")?,
ps_y: ps_y.context("pyro y ps")?,
ps_uv: ps_uv.context("pyro uv ps")?,
chroma444,
})
}
/// Convert `src_srv` (BGRA slot for SDR / scRGB FP16 slot for HDR, WxH) → `y_rtv` (full-res Y
/// texture) + `cbcr_rtv` (half- or full-res CbCr texture per the constructed mode). Two opaque
/// passes; `w`/`h` are the full luma dims (even for 4:2:0).
#[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn convert(
&self,
ctx: &ID3D11DeviceContext,
src_srv: &ID3D11ShaderResourceView,
y_rtv: &ID3D11RenderTargetView,
cbcr_rtv: &ID3D11RenderTargetView,
w: u32,
h: u32,
) -> Result<()> {
ctx.OMSetBlendState(None, None, 0xffff_ffff); // opaque overwrite
ctx.VSSetShader(&self.vs, None);
ctx.PSSetShaderResources(0, Some(&[Some(src_srv.clone())]));
ctx.IASetInputLayout(None);
ctx.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
// LUMA pass: full-res → the R8 Y texture.
ctx.RSSetViewports(Some(&[D3D11_VIEWPORT {
TopLeftX: 0.0,
TopLeftY: 0.0,
Width: w as f32,
Height: h as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
}]));
ctx.OMSetRenderTargets(Some(&[Some(y_rtv.clone())]), None);
ctx.PSSetShader(&self.ps_y, None);
ctx.Draw(3, 0);
ctx.OMSetRenderTargets(Some(&[None]), None);
// CHROMA pass: half-res (4:2:0) or full-res (4:4:4) → the CbCr texture.
let (cw, ch) = if self.chroma444 {
(w, h)
} else {
(w / 2, h / 2)
};
ctx.RSSetViewports(Some(&[D3D11_VIEWPORT {
TopLeftX: 0.0,
TopLeftY: 0.0,
Width: cw as f32,
Height: ch as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
}]));
ctx.OMSetRenderTargets(Some(&[Some(cbcr_rtv.clone())]), None);
ctx.PSSetShader(&self.ps_uv, None);
ctx.Draw(3, 0);
ctx.OMSetRenderTargets(Some(&[None]), None);
ctx.PSSetShaderResources(0, Some(&[None]));
Ok(())
}
}
/// f64 reference for the P010 colour math — the EXACT analogue of the HLSL in [`HDR_P010_COMMON`].
/// Input is one scRGB pixel (linear, Rec.709 primaries, 1.0 = 80 nits, may be >1 for HDR). Output is
/// the 10-bit studio-range (Y, Cb, Cr) codes the shader should produce for a flat (constant) block.
@@ -829,8 +1052,7 @@ use windows::Win32::Graphics::Direct3D11::{
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_COLOR_SPACE_RGB_FULL_G10_NONE_P709, DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G2084_LEFT_P2020, DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709,
DXGI_RATIONAL,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709, DXGI_RATIONAL,
};
/// D3D11 **Video Processor** colour/format converter — runs on the GPU's dedicated VIDEO engine, NOT
@@ -846,12 +1068,17 @@ pub(crate) struct VideoConverter {
}
impl VideoConverter {
/// A BGRA/FP16-RGB → **NV12 (BT.709 limited SDR)** video-engine converter. `scrgb_input` picks
/// the input colour space: `false` = 8-bit sRGB `BGRA` (the SDR ring); `true` = FP16 scRGB
/// linear (the HDR ring, used by a PyroWave session that tone-maps the HDR desktop down to the
/// 8-bit wavelet stream). The output is always studio-range BT.709 NV12 — the P010/BT.2020 HDR
/// path is [`HdrP010Converter`]'s job, never this one.
pub(crate) unsafe fn new(
device: &ID3D11Device,
context: &ID3D11DeviceContext,
width: u32,
height: u32,
hdr: bool,
scrgb_input: bool,
) -> Result<Self> {
let vdev: ID3D11VideoDevice = device.cast().context("device -> ID3D11VideoDevice")?;
let vctx: ID3D11VideoContext1 = context.cast().context("context -> ID3D11VideoContext1")?;
@@ -876,19 +1103,15 @@ impl VideoConverter {
.CreateVideoProcessor(&enumr, 0)
.context("CreateVideoProcessor")?;
// Full-range RGB in → studio-range YUV out. HDR: scRGB linear (G10) → BT.2020 PQ (G2084).
// SDR: sRGB (G22) → BT.709 (G22).
let (in_cs, out_cs) = if hdr {
(
DXGI_COLOR_SPACE_RGB_FULL_G10_NONE_P709,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G2084_LEFT_P2020,
)
// Full-range RGB in → studio-range BT.709 NV12 out. Input gamma follows the ring format:
// scRGB linear (G10) for the FP16 HDR ring, sRGB (G22) for the 8-bit BGRA SDR ring. The
// output is always BT.709 SDR (the video processor tone-maps the scRGB case).
let in_cs = if scrgb_input {
DXGI_COLOR_SPACE_RGB_FULL_G10_NONE_P709
} else {
(
DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709,
)
DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
};
let out_cs = DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709;
vctx.VideoProcessorSetStreamColorSpace1(&vp, 0, in_cs);
vctx.VideoProcessorSetOutputColorSpace1(&vp, out_cs);
// One frame in, one frame out — no interpolation/auto-processing.
+425 -36
View File
@@ -19,7 +19,10 @@
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::dxgi::{make_device, D3d11Frame, HdrP010Converter, VideoConverter, WinCaptureTarget};
use super::dxgi::{
make_device, BgraToYuvPlanes, D3d11Frame, HdrP010Converter, PyroFrameShare, VideoConverter,
WinCaptureTarget,
};
use super::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{bail, Context, Result};
use pf_driver_proto::{control, frame};
@@ -33,13 +36,16 @@ use windows::Win32::Foundation::{
HANDLE, INVALID_HANDLE_VALUE, LUID, POINT, WAIT_OBJECT_0,
};
use windows::Win32::Graphics::Direct3D11::{
ID3D11Device, ID3D11DeviceContext, ID3D11ShaderResourceView, ID3D11Texture2D,
D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE, D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX,
D3D11_RESOURCE_MISC_SHARED_NTHANDLE, D3D11_TEXTURE2D_DESC, D3D11_USAGE_DEFAULT,
ID3D11Device, ID3D11Device5, ID3D11DeviceContext, ID3D11DeviceContext4, ID3D11Fence,
ID3D11RenderTargetView, ID3D11ShaderResourceView, ID3D11Texture2D, D3D11_BIND_RENDER_TARGET,
D3D11_BIND_SHADER_RESOURCE, D3D11_FENCE_FLAG_SHARED, D3D11_RESOURCE_MISC_SHARED,
D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX, D3D11_RESOURCE_MISC_SHARED_NTHANDLE,
D3D11_TEXTURE2D_DESC, D3D11_USAGE_DEFAULT,
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_FORMAT, DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_NV12, DXGI_FORMAT_P010,
DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_SAMPLE_DESC,
DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM, DXGI_FORMAT_R16_UNORM,
DXGI_FORMAT_R8G8_UNORM, DXGI_FORMAT_R8_UNORM, DXGI_SAMPLE_DESC,
};
use windows::Win32::Graphics::Dxgi::{
CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4, IDXGIKeyedMutex, IDXGIResource1,
@@ -142,6 +148,18 @@ struct HostSlot {
srv: ID3D11ShaderResourceView,
}
/// One PyroWave output-ring slot: the two SEPARATE shareable plane textures the wavelet encoder
/// imports (design/pyrowave-windows-host-zerocopy.md) plus their RTVs (the [`BgraToYuvPlanes`] CSC
/// renders into them). Y is full-res `R8_UNORM`, CbCr is half-res `R8G8_UNORM`; both are
/// `SHARED | SHARED_NTHANDLE`. Rotated per frame like `out_ring` so encode N and convert N+1 touch
/// different textures.
struct PyroOutSlot {
y: ID3D11Texture2D,
y_rtv: ID3D11RenderTargetView,
cbcr: ID3D11Texture2D,
cbcr_rtv: ID3D11RenderTargetView,
}
/// RAII guard over an [`IDXGIKeyedMutex`]: [`acquire`](Self::acquire) does `AcquireSync(key, timeout)`,
/// `Drop` does `ReleaseSync(key)`. So the lock is released even if the work between acquire and the end
/// of the guard's scope `?`-returns or panics — the "leak the keyed-mutex lock → stall the driver on
@@ -375,10 +393,12 @@ pub struct IddPushCapturer {
/// display's HDR mode flipped). Stamped into the header + each delivery so the driver re-attaches
/// (and so stale-ring publishes are rejected).
generation: u32,
/// The CLIENT's advertised 10-bit capability (= negotiated `bit_depth >= 10`). Only used at `open`
/// to PROACTIVELY enable advanced color (so a 10-bit client gets HDR without a manual toggle); it
/// does NOT gate the per-frame conversion — that follows the display, like the WGC path (clients
/// under-report 10-bit yet all decode Main10 + auto-detect PQ from the VUI).
/// The CLIENT's advertised 10-bit capability (= negotiated `bit_depth >= 10`). Gates the
/// composition depth: a 10-bit client PROACTIVELY enables advanced color at `open` (HDR without a
/// manual toggle); an SDR-only client forces it OFF and the descriptor poller PINS it there, so a
/// client that advertised SDR ("HDR off") is never handed the in-band PQ upgrade the pixel-format-
/// driven encoder would otherwise stamp from an HDR composition. (An HDR-negotiated H.26x session
/// still follows a host-side "Use HDR" flip; all clients decode Main10 + auto-detect PQ from the VUI.)
client_10bit: bool,
/// The DISPLAY's CURRENT HDR state (from `advanced_color_enabled`) — the user can flip "Use HDR" in
/// Windows mid-session. Drives the ring format (HDR → FP16 surfaces, SDR → BGRA) and the conversion.
@@ -391,6 +411,32 @@ pub struct IddPushCapturer {
/// While the display is HDR this is overridden to the P010 path (no 10-bit 4:4:4 source):
/// the stream honestly downgrades to 4:2:0 — the encoder's caps cross-check reports it.
want_444: bool,
/// A PyroWave (wavelet) session (design/pyrowave-windows-host-zerocopy.md +
/// design/pyrowave-444-hdr.md). When set, frames come from the separate-plane `pyro_ring`
/// (shareable Y + CbCr textures the mode-aware [`BgraToYuvPlanes`] CSC writes) and a **shared
/// fence** is signalled after each convert, so the pyrowave encoder zero-copy-imports the two
/// textures into its own Vulkan device ordered after the D3D11 convert. The composition is
/// PINNED to the negotiated depth: SDR sessions force advanced color OFF (8-bit BGRA → R8
/// planes), 10-bit sessions enable it like H.26x (scRGB FP16 → R16 studio-code planes);
/// `want_444` sizes the chroma plane full-res.
pyrowave: bool,
/// PyroWave: the shared D3D11 timeline fence (created lazily on the first frame, `SHARED` flag).
/// The capturer `Signal`s it after each frame's GPU convert; the encoder's Vulkan side waits it.
pyro_fence: Option<ID3D11Fence>,
/// PyroWave: the fence's persistent shared NT handle (raw), passed on EVERY frame. The encoder
/// DUPLICATEs + imports it as a Vulkan timeline semaphore whenever it has none (first frame or
/// after an encoder rebuild), so this original stays valid across rebuilds.
pyro_fence_handle: Option<isize>,
/// PyroWave: the monotonically increasing fence value (one `Signal` per emitted frame).
pyro_fence_value: u64,
/// PyroWave: the separate-plane output ring (Y R8 + CbCr R8G8 shareable textures + RTVs), used
/// INSTEAD of `out_ring` for a pyrowave session. Built lazily; rebuilt on a mode change.
pyro_ring: Vec<PyroOutSlot>,
/// PyroWave: the BGRA→YUV-planes CSC (BT.709 limited, matching `rgb2yuv.comp`). Built lazily.
pyro_conv: Option<BgraToYuvPlanes>,
/// PyroWave: the last presented (Y, CbCr) textures — the repeat source (analogue of
/// `last_present` for the two-plane path).
pyro_last: Option<(ID3D11Texture2D, ID3D11Texture2D)>,
/// Off-thread display-descriptor sampler (see [`DescriptorPoller`]) — the capture loop reads
/// its snapshot instead of running CCD queries inline on the frame path.
desc_poller: DescriptorPoller,
@@ -556,18 +602,20 @@ impl IddPushCapturer {
/// virtual display); on FAILURE the keepalive is handed BACK so the caller can fall back to DDA
/// instead of tearing the display down (audit §5.1 — no more 20 s black bail). "Failure" includes the
/// driver not attaching to the ring within a few seconds (e.g. a hybrid-GPU render mismatch).
#[allow(clippy::too_many_arguments)]
pub fn open(
target: WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
client_10bit: bool,
want_444: bool,
pyrowave: bool,
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
pf_win_display::display_events::spawn_once();
match Self::open_inner(target, preferred, client_10bit, want_444, sender) {
match Self::open_inner(target, preferred, client_10bit, want_444, pyrowave, sender) {
Ok(mut me) => {
me._keepalive = keepalive;
Ok(me)
@@ -576,11 +624,13 @@ impl IddPushCapturer {
}
}
#[allow(clippy::too_many_arguments)]
fn open_inner(
target: WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
client_10bit: bool,
want_444: bool,
pyrowave: bool,
sender: crate::FrameChannelSender,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
@@ -601,6 +651,7 @@ impl IddPushCapturer {
preferred,
client_10bit,
want_444,
pyrowave,
luid,
sender.clone(),
) {
@@ -628,17 +679,27 @@ impl IddPushCapturer {
"IDD push: ring/driver render-adapter mismatch — rebinding the ring to the \
driver's reported adapter"
);
Self::open_on(target, preferred, client_10bit, want_444, drv, sender)
.context("IDD-push rebind to the driver's reported render adapter")
Self::open_on(
target,
preferred,
client_10bit,
want_444,
pyrowave,
drv,
sender,
)
.context("IDD-push rebind to the driver's reported render adapter")
}
}
}
#[allow(clippy::too_many_arguments)]
fn open_on(
target: WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
client_10bit: bool,
want_444: bool,
pyrowave: bool,
luid: LUID,
sender: crate::FrameChannelSender,
) -> Result<Self> {
@@ -663,12 +724,12 @@ impl IddPushCapturer {
}
// The driver composes the virtual display in FP16 (R16G16B16A16_FLOAT scRGB) when the display is
// in advanced-color (HDR) mode, and 8-bit BGRA otherwise (per swap_chain_processor.rs + the
// COMMIT_MODES2 colorspace/rgb_bpc log). The user can flip "Use HDR" in Windows at any time, so
// the ring format must TRACK the display's ACTUAL mode (the driver's format-guard drops a
// mismatch). We poll the live state here and on every recreate. For a 10-bit-capable client we
// PROACTIVELY enable advanced color so HDR streams without the user toggling anything; an
// SDR-only client leaves the display alone (and still gets a tone-mapped picture, never a freeze,
// if the user does enable HDR).
// COMMIT_MODES2 colorspace/rgb_bpc log). For a 10-bit-capable client we PROACTIVELY enable
// advanced color so HDR streams without the user toggling anything, then TRACK the display's
// actual mode (a mid-session "Use HDR" flip; the driver's format-guard drops a mismatch), polling
// the live state here and on every recreate. An SDR-only client instead forces advanced color OFF
// and is PINNED there (below + the descriptor poller), so the SDR negotiation is honored and the
// encoder never emits the in-band PQ upgrade to a client that asked for SDR.
// SAFETY: one block over the whole ring setup; every operation in it is sound:
// - `set_advanced_color`/`advanced_color_enabled` are `unsafe fn`s taking only a copy of the plain
// `u32` target id; they read/flip CCD display config and return owned values, borrowing nothing.
@@ -691,6 +752,49 @@ impl IddPushCapturer {
// - `header` points into the OS mapping, NOT into the `MappedSection` struct, so moving `section`
// into `me` leaves it valid (see the `MappedSection` doc comment).
unsafe {
// An SDR-NEGOTIATED session (either codec) must run on an SDR (BGRA) composition, so
// actively turn advanced color OFF — undoing any leftover HDR state from a prior 10-bit
// session on a reused/lingering monitor, the driver's default, or the host's global
// "Use HDR" — and settle before sizing the ring. Non-optional for two reasons:
// - PyroWave: its CSC reads 8-bit BGRA and the NVIDIA D3D11 VideoProcessor can't ingest
// the FP16 ring at all.
// - H.26x: off an HDR composition the capturer emits P010 and the encoder stamps
// Main10 + BT.2020 PQ from the pixel format alone (the in-band HDR upgrade), sending a
// 10-bit PQ stream to a client that advertised SDR-only ("HDR off = never send me
// 10-bit"). On a client whose monitor is HDR-capable but has "Use HDR" off, that PQ
// lands on an SDR desktop and blows out — the composition must honor the negotiation.
// An HDR-negotiated (10-bit) session instead enables HDR below and rides the FP16 scRGB
// ring (design/pyrowave-444-hdr.md Phase 3 for PyroWave; the H.26x P010 path otherwise).
if !client_10bit {
let _ = pf_win_display::win_display::set_advanced_color(target.target_id, false);
let settle = Instant::now();
while settle.elapsed() < Duration::from_millis(250) {
if pf_win_display::win_display::advanced_color_enabled(target.target_id)
== Some(false)
{
break;
}
std::thread::sleep(Duration::from_millis(25));
}
if pf_win_display::win_display::advanced_color_enabled(target.target_id)
== Some(true)
{
tracing::error!(
target = target.target_id,
pyrowave,
"IDD push: SDR session but advanced color (HDR) could NOT be turned off on the \
virtual display (a physical display forcing HDR?) PyroWave will likely fail \
its first frame; H.26x would emit PQ the SDR-only client never asked for"
);
} else {
tracing::info!(
target = target.target_id,
pyrowave,
settle_ms = settle.elapsed().as_millis() as u64,
"IDD push: SDR-negotiated session — advanced color forced OFF (SDR/BGRA composition)"
);
}
}
// If we ENABLE advanced color for a 10-bit client, trust it (the driver will compose FP16) and
// size the ring FP16 directly — don't race the advanced_color_enabled poll, which may not have
// settled within 250 ms and would size the ring SDR while the driver composes FP16 → a format
@@ -721,9 +825,14 @@ impl IddPushCapturer {
}
// A failed open-time read defaults to SDR (unless the 10-bit path enabled HDR above) —
// there is no "last known" yet; the descriptor poller corrects a wrong guess mid-session.
let display_hdr = enabled_hdr
|| pf_win_display::win_display::advanced_color_enabled(target.target_id)
.unwrap_or(false);
// An SDR-negotiated session (either codec) forced advanced color OFF above and composes
// SDR unconditionally: `client_10bit` gates HDR so a client that advertised SDR-only is
// never handed a PQ stream, even if a physical display forces HDR on (the descriptor
// poller re-asserts OFF; PyroWave's format guard/stash absorbs any lingering FP16 compose).
let display_hdr = client_10bit
&& (enabled_hdr
|| pf_win_display::win_display::advanced_color_enabled(target.target_id)
.unwrap_or(false));
// Downgrade point D (design/hdr-10bit-default-and-av1.md item 2d): the session was
// NEGOTIATED 10-bit (the client was told HDR in the Welcome), but the virtual display
// could not enable advanced color — the ring sizes SDR and the encoder will emit 8-bit
@@ -853,6 +962,13 @@ impl IddPushCapturer {
client_10bit,
display_hdr,
want_444,
pyrowave,
pyro_fence: None,
pyro_fence_handle: None,
pyro_fence_value: 0,
pyro_ring: Vec::new(),
pyro_conv: None,
pyro_last: None,
desc_poller: DescriptorPoller::spawn(
target.target_id,
DisplayDescriptor {
@@ -1128,6 +1244,16 @@ impl IddPushCapturer {
/// auto-switch, exactly as on the WGC path. HDR wins over 4:4:4 (there is no 10-bit
/// full-chroma source): the stream downgrades to 4:2:0 with a warning.
fn out_format(&self) -> (DXGI_FORMAT, PixelFormat) {
// PyroWave never uses this out-ring (it has its own separate-plane `pyro_ring`); the
// format here only labels the frame. SDR sessions label NV12 (BT.709 limited), HDR
// (negotiated 10-bit) sessions P010 — matching the studio-code planes the pyro CSC writes.
if self.pyrowave {
return if self.display_hdr {
(DXGI_FORMAT_P010, PixelFormat::P010)
} else {
(DXGI_FORMAT_NV12, PixelFormat::Nv12)
};
}
if self.display_hdr {
if self.want_444 {
warn_444_hdr_downgrade_once();
@@ -1215,6 +1341,8 @@ impl IddPushCapturer {
self.out_ring.clear(); // the output format changed → rebuild lazily at the new format
self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode
self.hdr_p010_conv = None;
self.pyro_ring.clear(); // PyroWave two-plane ring is sized → rebuild at the new mode
self.pyro_last = None;
self.out_idx = 0;
self.last_present = None;
Ok(())
@@ -1228,11 +1356,31 @@ impl IddPushCapturer {
/// only when TWO consecutive samples agree on the same new descriptor (~½ s), so a
/// single-sample transient during a topology re-probe never costs a ring recreate.
fn poll_display_hdr(&mut self) {
let (now, seq) = self.desc_poller.snapshot();
let (mut now, seq) = self.desc_poller.snapshot();
if seq == self.desc_seq {
return; // no new sample since last consume
}
self.desc_seq = seq;
// Two cases re-assert the NEGOTIATED depth instead of following a mid-session "Use HDR"
// flip — flip the display back and treat the descriptor as the negotiated state (so the ring
// is never recreated at the wrong format):
// - a PyroWave session: its encoder was opened for fixed plane formats (R8 SDR / R16 HDR),
// so it can't follow a flip the way H.26x re-inits do;
// - ANY SDR-negotiated session (`!client_10bit`, either codec): a host-side flip to HDR
// must not promote the stream to P010 PQ behind a client that advertised SDR-only.
// An HDR-negotiated H.26x session is NOT pinned — it still follows a host "Use HDR" flip in
// either direction (its encoder re-inits on the depth change).
if (self.pyrowave || !self.client_10bit) && now.hdr != self.client_10bit {
// SAFETY: `set_advanced_color` is `unsafe` (CCD DisplayConfig calls); it takes a plain
// `u32` target id + bool, forms no lasting borrow, and returns a bool.
unsafe {
let _ = pf_win_display::win_display::set_advanced_color(
self.target_id,
self.client_10bit,
);
}
now.hdr = self.client_10bit;
}
let current = DisplayDescriptor {
hdr: self.display_hdr,
width: self.width,
@@ -1281,7 +1429,8 @@ impl IddPushCapturer {
},
Usage: D3D11_USAGE_DEFAULT,
// RENDER_TARGET: the VIDEO processor (NV12) and the P010 shader passes both write here, and
// NVENC registers it as encode input — matching the WGC YUV ring.
// NVENC registers it as encode input — matching the WGC YUV ring. (PyroWave uses its own
// shareable two-plane `pyro_ring` instead, so this NVENC/AMF/QSV ring stays unshared.)
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
CPUAccessFlags: 0,
MiscFlags: 0,
@@ -1302,6 +1451,91 @@ impl IddPushCapturer {
Ok(())
}
/// PyroWave: build the separate-plane output ring (`OUT_RING` × {full-res R8 Y, half-res R8G8
/// CbCr}, both `SHARED | SHARED_NTHANDLE` + RTV) if not yet built. The wavelet encoder imports the
/// two SEPARATE textures (a single planar NV12 import is unreliable on NVIDIA); the
/// [`BgraToYuvPlanes`] CSC renders into their RTVs.
fn ensure_pyro_ring(&mut self) -> Result<()> {
if !self.pyro_ring.is_empty() {
return Ok(());
}
let (w, h) = (self.width, self.height);
// SAFETY: all D3D11 calls target `self.device`; every `&desc` is a fully-initialized stack
// struct and every `Some(&mut _)` a live out-param; `?` rejects a failed HRESULT before use.
// The created textures/RTVs belong to `self.device`.
unsafe {
let make = |dev: &ID3D11Device,
fmt: DXGI_FORMAT,
w: u32,
h: u32|
-> Result<(ID3D11Texture2D, ID3D11RenderTargetView)> {
let desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: fmt,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
CPUAccessFlags: 0,
MiscFlags: (D3D11_RESOURCE_MISC_SHARED_NTHANDLE.0
| D3D11_RESOURCE_MISC_SHARED.0) as u32,
};
let mut tex: Option<ID3D11Texture2D> = None;
dev.CreateTexture2D(&desc, None, Some(&mut tex))
.context("CreateTexture2D(pyro plane)")?;
let tex = tex.context("null pyro plane texture")?;
let mut rtv: Option<ID3D11RenderTargetView> = None;
dev.CreateRenderTargetView(&tex, None, Some(&mut rtv))
.context("CreateRenderTargetView(pyro plane)")?;
Ok((tex, rtv.context("null pyro plane rtv")?))
};
// Plane formats/geometry follow the negotiated session: 16-bit UNORM planes for an
// HDR (10-bit) session (P010-style studio codes from the pyro HDR CSC), full-res
// chroma for 4:4:4 (design/pyrowave-444-hdr.md Phase 3).
let (yf, cf) = if self.display_hdr {
(DXGI_FORMAT_R16_UNORM, DXGI_FORMAT_R16G16_UNORM)
} else {
(DXGI_FORMAT_R8_UNORM, DXGI_FORMAT_R8G8_UNORM)
};
let (cw, ch) = if self.want_444 {
(w, h)
} else {
(w / 2, h / 2)
};
for _ in 0..OUT_RING {
let (y, y_rtv) = make(&self.device, yf, w, h)?;
let (cbcr, cbcr_rtv) = make(&self.device, cf, cw, ch)?;
self.pyro_ring.push(PyroOutSlot {
y,
y_rtv,
cbcr,
cbcr_rtv,
});
}
}
Ok(())
}
/// PyroWave: build the (mode-aware) RGB→YUV-planes CSC if not yet built. The mode is
/// session-fixed: SDR/BGRA vs HDR/scRGB input, half- vs full-res chroma — the composition
/// is pinned to the negotiated depth (`poll_display_hdr`), so the converter never needs a
/// mid-session mode swap.
fn ensure_pyro_conv(&mut self) -> Result<()> {
if self.pyro_conv.is_none() {
// SAFETY: `BgraToYuvPlanes::new` compiles D3D11 shaders on `self.device`; `?` propagates
// failure before it is stored.
self.pyro_conv = Some(unsafe {
BgraToYuvPlanes::new(&self.device, self.display_hdr, self.want_444)?
});
}
Ok(())
}
/// Build the per-mode YUV converter if not already built: a VIDEO-engine BGRA→NV12 processor on an
/// SDR display, or the FP16→P010 shader on an HDR display. Both keep NVENC's RGB→YUV CSC off the SM.
/// An SDR 4:4:4 session needs NO converter — the BGRA slot passes through (see `out_format`).
@@ -1327,6 +1561,61 @@ impl IddPushCapturer {
Ok(())
}
/// PyroWave: after this frame's GPU convert, `Signal` the shared fence and return the fence
/// `(handle, value)` for the encoder — the persistent shared handle EVERY frame (the encoder
/// imports it whenever it has no timeline yet, e.g. after a mode-switch rebuild) + the
/// incrementing value. `None` for a non-PyroWave session. The fence + its shared handle are
/// created lazily on the first call. `Flush` submits the queued convert + signal so the encoder's
/// cross-API Vulkan timeline wait resolves promptly instead of blocking on a still-unsubmitted
/// signal. The caller pairs the returned fence with the frame's CbCr texture into a
/// [`PyroFrameShare`].
///
/// # Safety
/// Runs on the owning capture/encode thread that holds the immediate context; forms no lasting
/// borrow of `self`'s COM objects.
unsafe fn pyro_fence_signal(&mut self) -> Result<Option<(Option<isize>, u64)>> {
if !self.pyrowave {
return Ok(None);
}
if self.pyro_fence.is_none() {
let dev5: ID3D11Device5 = self
.device
.cast()
.context("ID3D11Device -> ID3D11Device5 (shared fence)")?;
// windows-rs returns COM interfaces via an out-param (unlike the HANDLE-returning
// CreateSharedHandle below).
let mut fence_out: Option<ID3D11Fence> = None;
dev5.CreateFence(0, D3D11_FENCE_FLAG_SHARED, &mut fence_out)
.context("CreateFence(D3D11_FENCE_FLAG_SHARED)")?;
let fence = fence_out.context("null D3D11 fence")?;
// GENERIC_ALL (0x1000_0000) — the access the pyrowave interop test hands the handle.
let handle: HANDLE = fence
.CreateSharedHandle(None, 0x1000_0000, PCWSTR::null())
.context("ID3D11Fence::CreateSharedHandle")?;
self.pyro_fence = Some(fence);
self.pyro_fence_handle = Some(handle.0 as isize);
self.pyro_fence_value = 0;
}
self.pyro_fence_value += 1;
let value = self.pyro_fence_value;
let ctx4: ID3D11DeviceContext4 = self
.context
.cast()
.context("ID3D11DeviceContext -> ID3D11DeviceContext4 (fence signal)")?;
{
let fence = self.pyro_fence.as_ref().expect("fence just created");
ctx4.Signal(fence, value)
.context("ID3D11 fence Signal after convert")?;
}
// Submit the queued convert + signal so the encoder's Vulkan timeline wait can resolve.
self.context.Flush();
// Pass the persistent shared handle EVERY frame (not once): the encoder can be rebuilt on a
// client mode-switch, and a rebuilt encoder needs to re-import the fence into its fresh Vulkan
// device. The encoder imports only when it has no timeline yet (and DUPLICATES the handle so
// this original stays valid for the next rebuild).
Ok(Some((self.pyro_fence_handle, value)))
}
fn try_consume(&mut self) -> Result<Option<CapturedFrame>> {
self.log_driver_status_once();
// Follow the display: a "Use HDR" flip recreates the ring at the matching format.
@@ -1391,13 +1680,34 @@ impl IddPushCapturer {
if seq == self.last_seq || slot >= self.slots.len() {
return Ok(None);
}
self.ensure_out_ring()?;
// Build the converter BEFORE acquiring the slot so nothing between Acquire and Release can
// `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot).
self.ensure_converter()?;
// Build the ring + converter BEFORE acquiring the slot so nothing between Acquire and Release
// can `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot).
// PyroWave uses its OWN two-plane ring (`pyro_ring`); everything else the single NV12/BGRA ring.
let i = self.out_idx;
let out = self.out_ring[i].clone();
let (out, pyro_slot) = if self.pyrowave {
self.ensure_pyro_ring()?;
self.ensure_pyro_conv()?;
let s = &self.pyro_ring[i];
(
None,
Some((
s.y.clone(),
s.y_rtv.clone(),
s.cbcr.clone(),
s.cbcr_rtv.clone(),
)),
)
} else {
self.ensure_out_ring()?;
self.ensure_converter()?;
(Some(self.out_ring[i].clone()), None)
};
let (_, pf) = self.out_format();
let ring_len = if self.pyrowave {
self.pyro_ring.len()
} else {
self.out_ring.len()
};
// Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the
// ~3 ms encode — NVENC reads the host out-ring slot, not the keyed-mutex slot), so the driver gets
@@ -1414,14 +1724,30 @@ impl IddPushCapturer {
// A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing
// the slot back to the driver.
unsafe {
if self.display_hdr {
if self.pyrowave {
// PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate
// plane textures via the mode-aware CSC; the shared fence signalled just after
// (`pyro_fence_signal`) orders the encoder's cross-device Vulkan read after this
// convert. The composition format is pinned to the negotiated depth.
let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot");
if let Some(conv) = self.pyro_conv.as_ref() {
conv.convert(
&self.context,
&s.srv,
y_rtv,
cbcr_rtv,
self.width,
self.height,
)?;
}
} else if self.display_hdr {
// HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010.
if let Some(conv) = self.hdr_p010_conv.as_ref() {
conv.convert(
&self.device,
&self.context,
&s.srv,
&out,
out.as_ref().expect("out ring"),
self.width,
self.height,
)?;
@@ -1430,19 +1756,24 @@ impl IddPushCapturer {
// SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma
// RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain
// copy-engine move; the slot releases back to the driver immediately.
self.context.CopyResource(&out, &s.tex);
self.context
.CopyResource(out.as_ref().expect("out ring"), &s.tex);
} else {
// SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC.
if let Some(conv) = self.video_conv.as_ref() {
conv.convert(&s.tex, &out)?;
conv.convert(&s.tex, out.as_ref().expect("out ring"))?;
}
}
}
// `_lock` drops here → `ReleaseSync(0)`.
}
self.out_idx = (i + 1) % self.out_ring.len();
self.out_idx = (i + 1) % ring_len;
self.last_seq = seq;
self.last_present = Some((out.clone(), pf));
if let Some((y, _, cbcr, _)) = pyro_slot.as_ref() {
self.pyro_last = Some((y.clone(), cbcr.clone()));
} else {
self.last_present = Some((out.as_ref().expect("out ring").clone(), pf));
}
let now = Instant::now();
if self.recovering_since.take().is_some() {
// A fresh frame resumed → recovered. The recovery gap is self-inflicted (ring
@@ -1517,14 +1848,33 @@ impl IddPushCapturer {
}
}
self.last_fresh = now; // feeds the driver-death watch
// Build the frame. For PyroWave the encode input is the Y plane
// (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence
// after the convert above. SAFETY: on the owning capture/encode thread.
let (texture, pyro) = if let Some((y, _, cbcr, _)) = pyro_slot {
// SAFETY: on the owning capture/encode thread holding the immediate context.
let (fence_handle, fence_value) =
unsafe { self.pyro_fence_signal() }?.expect("pyrowave session signals its fence");
(
y,
Some(PyroFrameShare {
cbcr,
fence_handle,
fence_value,
}),
)
} else {
(out.expect("out ring texture"), None)
};
Ok(Some(CapturedFrame {
width: self.width,
height: self.height,
pts_ns: now_ns(),
format: pf,
payload: FramePayload::D3d11(D3d11Frame {
texture: out,
texture,
device: self.device.clone(),
pyro,
}),
cursor: None,
}))
@@ -1535,8 +1885,46 @@ impl IddPushCapturer {
// new driver frame) never re-hands a slot that may still be encoding under pipeline_depth>1 — the
// out-ring rotation IS the texture-ownership contract, and repeats must honor it too (audit §5.3).
// OUT_RING(3) > the max pipeline_depth(2) guarantees the rotated slot is not in flight.
let (src, pf) = self.last_present.clone()?;
let i = self.out_idx;
// PyroWave: copy the last Y+CbCr into a fresh two-plane slot; texture = Y, CbCr + fence in `pyro`.
if self.pyrowave {
let (src_y, src_cbcr) = self.pyro_last.clone()?;
let slot = self.pyro_ring.get(i)?;
let (dst_y, dst_cbcr) = (slot.y.clone(), slot.cbcr.clone());
// SAFETY: GPU copies on the owning thread's immediate context; src/dst are our own pyro-ring
// plane textures of identical format/size.
unsafe {
self.context.CopyResource(&dst_y, &src_y);
self.context.CopyResource(&dst_cbcr, &src_cbcr);
}
self.out_idx = (i + 1) % self.pyro_ring.len();
self.pyro_last = Some((dst_y.clone(), dst_cbcr.clone()));
// Fence the copies above so the encoder reads completed textures. SAFETY: owning thread.
let (fence_handle, fence_value) = match unsafe { self.pyro_fence_signal() } {
Ok(Some(f)) => f,
_ => {
tracing::warn!("pyrowave: fence signal failed on a repeat frame — dropping it");
return None;
}
};
return Some(CapturedFrame {
width: self.width,
height: self.height,
pts_ns: now_ns(),
format: self.out_format().1,
payload: FramePayload::D3d11(D3d11Frame {
texture: dst_y,
device: self.device.clone(),
pyro: Some(PyroFrameShare {
cbcr: dst_cbcr,
fence_handle,
fence_value,
}),
}),
cursor: None,
});
}
let (src, pf) = self.last_present.clone()?;
let dst = self.out_ring.get(i)?.clone();
// SAFETY: GPU copy on the owning thread's immediate context; src/dst are our out-ring textures of
// identical format/size (src is a previous out-ring slot; dst the next).
@@ -1553,6 +1941,7 @@ impl IddPushCapturer {
payload: FramePayload::D3d11(D3d11Frame {
texture: dst,
device: self.device.clone(),
pyro: None,
}),
cursor: None,
})
@@ -127,6 +127,7 @@ impl Capturer for SyntheticNv12Capturer {
payload: FramePayload::D3d11(D3d11Frame {
texture: self.default_tex.clone(),
device: self.device.clone(),
pyro: None,
}),
cursor: None,
})
+12
View File
@@ -299,12 +299,24 @@ fn pump(
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
let built = if connector.codec == punktfunk_core::quic::CODEC_PYROWAVE {
let mode = connector.mode();
// The wavelet bitstream has no VUI: the negotiated Welcome colour signalling IS
// the session's colour contract (BT.709 limited SDR today, BT.2020 PQ once the
// HDR leg lands), and the chroma the host resolved sizes the plane ring.
let color = crate::video::ColorDesc {
primaries: connector.color.primaries,
transfer: connector.color.transfer,
matrix: connector.color.matrix,
full_range: connector.color.full_range != 0,
};
match params.vulkan.as_ref() {
Some(vk) => Decoder::new_pyrowave(
vk,
mode.width,
mode.height,
connector.shard_payload as usize,
connector.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
color,
connector.bit_depth >= 10,
),
None => Err(anyhow::anyhow!(
"pyrowave session without a presenter device"
+6
View File
@@ -489,6 +489,9 @@ impl Decoder {
width: u32,
height: u32,
shard_payload: usize,
chroma444: bool,
color: ColorDesc,
hdr16: bool,
) -> Result<Decoder> {
Ok(Decoder {
backend: Backend::PyroWave(Box::new(crate::video_pyrowave::PyroWaveDecoder::new(
@@ -496,6 +499,9 @@ impl Decoder {
width,
height,
shard_payload,
chroma444,
color,
hdr16,
)?)),
codec_id: ffmpeg::codec::Id::HEVC,
vaapi_fails: 0,
+85 -21
View File
@@ -258,11 +258,12 @@ unsafe fn make_plane(
mem_props: &vk::PhysicalDeviceMemoryProperties,
w: u32,
h: u32,
fmt: vk::Format,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
let img = device.create_image(
&vk::ImageCreateInfo::default()
.image_type(vk::ImageType::TYPE_2D)
.format(vk::Format::R8_UNORM)
.format(fmt)
.extent(vk::Extent3D {
width: w,
height: h,
@@ -306,7 +307,7 @@ unsafe fn make_plane(
&vk::ImageViewCreateInfo::default()
.image(img)
.view_type(vk::ImageViewType::TYPE_2D)
.format(vk::Format::R8_UNORM)
.format(fmt)
.subresource_range(vk::ImageSubresourceRange {
aspect_mask: vk::ImageAspectFlags::COLOR,
base_mip_level: 0,
@@ -347,12 +348,21 @@ unsafe fn build_ring(
mem_props: &vk::PhysicalDeviceMemoryProperties,
width: u32,
height: u32,
chroma444: bool,
fmt: vk::Format,
) -> Result<Vec<PlaneSet>> {
// 4:2:0 = half-res chroma; 4:4:4 = full-res. The presenter's planar CSC samples with
// normalized UVs, so the chroma plane resolution is transparent to it.
let (cw, ch) = if chroma444 {
(width, height)
} else {
(width / 2, height / 2)
};
let mut ring: Vec<PlaneSet> = Vec::with_capacity(RING);
for _ in 0..RING {
let built = (|| -> Result<PlaneSet> {
let (y, ym, yv) = make_plane(device, mem_props, width, height)?;
let (cb, cbm, cbv) = match make_plane(device, mem_props, width / 2, height / 2) {
let (y, ym, yv) = make_plane(device, mem_props, width, height, fmt)?;
let (cb, cbm, cbv) = match make_plane(device, mem_props, cw, ch, fmt) {
Ok(p) => p,
Err(e) => {
device.destroy_image_view(yv, None);
@@ -361,7 +371,7 @@ unsafe fn build_ring(
return Err(e);
}
};
let (cr, crm, crv) = match make_plane(device, mem_props, width / 2, height / 2) {
let (cr, crm, crv) = match make_plane(device, mem_props, cw, ch, fmt) {
Ok(p) => p,
Err(e) => {
for (v, i, m) in [(yv, y, ym), (cbv, cb, cbm)] {
@@ -409,6 +419,16 @@ pub struct PyroWaveDecoder {
mem_props: vk::PhysicalDeviceMemoryProperties,
width: u32,
height: u32,
/// Session-fixed negotiated chroma ([`Welcome::chroma_format`]): 4:4:4 = full-res
/// chroma planes + `Chroma444` pyrowave decoders (the seq-header bit is
/// decoder-enforced upstream, so a mismatch fails loudly, never silently).
chroma444: bool,
/// Session colour signalling ([`Welcome::color`]): the wavelet bitstream has no VUI,
/// so the negotiated `ColorInfo` is the contract the presenter CSC configures from.
color: ColorDesc,
/// Session-fixed negotiated depth ≥10: the planes are `R16_UNORM` carrying the host's
/// P010-style studio codes (the presenter samples them with depth-10 MSB-packed rows).
hdr16: bool,
/// The wire shard payload — the parse-window size for chunk-aligned AUs (§4.4): each
/// window holds whole self-delimiting codec packets, zero-padded to the window.
wire_window: usize,
@@ -424,17 +444,20 @@ impl PyroWaveDecoder {
width: u32,
height: u32,
shard_payload: usize,
chroma444: bool,
color: ColorDesc,
hdr16: bool,
) -> Result<PyroWaveDecoder> {
if !vkd.pyrowave_decode {
bail!("presenter device lacks the PyroWave compute feature set");
}
if width % 2 != 0 || height % 2 != 0 {
if !chroma444 && (width % 2 != 0 || height % 2 != 0) {
bail!("pyrowave 4:2:0 needs even dimensions (got {width}x{height})");
}
// SAFETY: the handles in `vkd` are the presenter's live instance/device (it
// outlives the decoder — same contract the FFmpeg Vulkan backend relies on);
// `Hold` pins the reconstructed create-infos for the pyrowave device's lifetime.
unsafe { Self::new_inner(vkd, width, height, shard_payload) }
unsafe { Self::new_inner(vkd, width, height, shard_payload, chroma444, color, hdr16) }
}
unsafe fn new_inner(
@@ -442,6 +465,9 @@ impl PyroWaveDecoder {
width: u32,
height: u32,
shard_payload: usize,
chroma444: bool,
color: ColorDesc,
hdr16: bool,
) -> Result<PyroWaveDecoder> {
let static_fn = ash::StaticFn {
get_instance_proc_addr: std::mem::transmute::<usize, vk::PFN_vkGetInstanceProcAddr>(
@@ -496,7 +522,11 @@ impl PyroWaveDecoder {
device: pw_dev,
width: width as i32,
height: height as i32,
chroma: pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420,
chroma: if chroma444 {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_444
} else {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420
},
// The fragment-iDWT path is for Mali/Adreno-class mobile GPUs only.
fragment_path: false,
};
@@ -513,7 +543,27 @@ impl PyroWaveDecoder {
let mem_props = instance.get_physical_device_memory_properties(
vk::PhysicalDevice::from_raw(vkd.physical_device as u64),
);
let ring = match build_ring(&device, &mem_props, width, height) {
// 16-bit sessions decode into R16_UNORM storage planes; STORAGE_IMAGE support for
// R16_UNORM is optional in Vulkan (universal on desktop) — probe it so an exotic
// device fails with a clear message instead of a validation error.
let plane_fmt = if hdr16 {
let props = instance.get_physical_device_format_properties(
vk::PhysicalDevice::from_raw(vkd.physical_device as u64),
vk::Format::R16_UNORM,
);
if !props
.optimal_tiling_features
.contains(vk::FormatFeatureFlags::STORAGE_IMAGE)
{
pw::pyrowave_decoder_destroy(pw_dec);
pw::pyrowave_device_destroy(pw_dev);
bail!("this GPU lacks R16_UNORM STORAGE_IMAGE — cannot decode a 10-bit PyroWave session");
}
vk::Format::R16_UNORM
} else {
vk::Format::R8_UNORM
};
let ring = match build_ring(&device, &mem_props, width, height, chroma444, plane_fmt) {
Ok(r) => r,
Err(e) => {
pw::pyrowave_decoder_destroy(pw_dec);
@@ -557,6 +607,9 @@ impl PyroWaveDecoder {
mem_props,
width,
height,
chroma444,
color,
hdr16,
wire_window: shard_payload.max(64),
})
}
@@ -569,14 +622,19 @@ impl PyroWaveDecoder {
/// The old ring is RETIRED, not destroyed: the presenter / frame channel may still
/// reference its views (see [`RETIRE_HANDOVERS`]).
unsafe fn reconfigure(&mut self, width: u32, height: u32) -> Result<()> {
if width % 2 != 0 || height % 2 != 0 {
if !self.chroma444 && (width % 2 != 0 || height % 2 != 0) {
bail!("pyrowave 4:2:0 needs even dimensions (resize to {width}x{height})");
}
let dinfo = pw::pyrowave_decoder_create_info {
device: self.pw_dev,
width: width as i32,
height: height as i32,
chroma: pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420,
// Chroma is session-fixed (negotiated); a resize never changes it.
chroma: if self.chroma444 {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_444
} else {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420
},
fragment_path: false,
};
let mut new_dec: pw::pyrowave_decoder = std::ptr::null_mut();
@@ -584,7 +642,18 @@ impl PyroWaveDecoder {
pw::pyrowave_decoder_create(&dinfo, &mut new_dec),
"decoder_create (mid-stream resize)",
)?;
let new_ring = match build_ring(&self.device, &self.mem_props, width, height) {
let new_ring = match build_ring(
&self.device,
&self.mem_props,
width,
height,
self.chroma444,
if self.hdr16 {
vk::Format::R16_UNORM
} else {
vk::Format::R8_UNORM
},
) {
Ok(r) => r,
Err(e) => {
pw::pyrowave_decoder_destroy(new_dec);
@@ -886,15 +955,10 @@ impl PyroWaveDecoder {
],
width: w,
height: h,
// No VUI in the bitstream: BT.709 limited is the fixed contract with the
// host's CSC (plan §4.7 CscRows note; sequence-header signaling is a
// follow-up once the C API exposes it).
color: ColorDesc {
primaries: 1,
transfer: 1,
matrix: 1,
full_range: false,
},
// No VUI in the bitstream: the negotiated Welcome `ColorInfo` is the contract
// with the host's CSC (BT.709 limited for SDR sessions; BT.2020 PQ once the
// HDR leg lands — design/pyrowave-444-hdr.md).
color: self.color,
keyframe: true,
}))
}
+1 -1
View File
@@ -7,7 +7,7 @@
# `start`), so it stays free of platform cfg.
[package]
name = "pf-clipboard"
version = "0.12.0"
version.workspace = true
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
+8
View File
@@ -53,15 +53,23 @@ ffmpeg-next = { version = "8", optional = true }
libloading = "0.8"
# Native Intel QSV (VPL): vendored static MIT dispatcher + bindgen'd C API, only under `qsv`.
libvpl-sys = { path = "../libvpl-sys", optional = true }
# PyroWave (opt-in wired-LAN wavelet codec) — vendored codec + bindgen'd C API, only under
# `pyrowave`. The Windows backend is the NV12 zero-copy D3D11→Vulkan encoder; same crate as Linux.
pyrowave-sys = { path = "../pyrowave-sys", optional = true }
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
# SECURITY_ATTRIBUTES — the PyroWave backend's IDXGIResource1::CreateSharedHandle signature.
"Win32_Security",
"Win32_Storage_FileSystem",
"Win32_System_LibraryLoader",
"Win32_System_Threading",
# D3DKMTSetProcessSchedulingPriorityClass — raise the host's WDDM GPU scheduling priority
# above a running game so PyroWave's compute-shader encode isn't starved (enc/windows/pyrowave.rs).
"Wdk_Graphics_Direct3D",
] }
[features]
+5 -4
View File
@@ -104,13 +104,14 @@ impl Codec {
}
}
/// Whether this codec has a negotiable **10-bit** encode path (HEVC Main10 / AV1 10-bit).
/// H.264 is always 8-bit (High10 is neither an NVENC nor a VCN encode mode — negotiation
/// never asks), and PyroWave's wavelet path ingests 8-bit. `true` here is only the
/// Whether this codec has a negotiable **10-bit** encode path (HEVC Main10 / AV1 10-bit;
/// PyroWave rides 16-bit UNORM planes carrying P010-style studio codes — the wavelet is
/// depth-agnostic, design/pyrowave-444-hdr.md). H.264 is always 8-bit (High10 is neither an
/// NVENC nor a VCN encode mode — negotiation never asks). `true` here is only the
/// *codec-level* gate: the active GPU/backend must still pass
/// [`can_encode_10bit`](crate::can_encode_10bit) before the host negotiates 10-bit.
pub fn supports_10bit(self) -> bool {
matches!(self, Codec::H265 | Codec::Av1)
matches!(self, Codec::H265 | Codec::Av1 | Codec::PyroWave)
}
/// The FFmpeg NVENC encoder name (selected by name, not codec id — the latter would
+197 -38
View File
@@ -27,8 +27,8 @@ use super::libav::{
use ffmpeg::ffi; // = ffmpeg_sys_next
/// The swscale *source* pixel format for a captured packed RGB/BGR layout (the real byte order, not
/// the NVENC-padded `*0` form). Used by the 4:4:4 RGB→YUV444P conversion path. Mirrors the VAAPI
/// CPU-input mapping; YUV/10-bit inputs can't feed this path (the 4:4:4 session forces packed RGB).
/// the NVENC-padded `*0` form). Used by the CPU conversion paths: 4:4:4 RGB→YUV444P, and HDR
/// X2RGB10/X2BGR10→P010. Mirrors the VAAPI CPU-input mapping; YUV inputs can't feed this path.
fn sws_src_pixel(format: PixelFormat) -> Result<Pixel> {
Ok(match format {
PixelFormat::Bgrx => Pixel::BGRZ, // bgr0
@@ -37,8 +37,12 @@ fn sws_src_pixel(format: PixelFormat) -> Result<Pixel> {
PixelFormat::Rgba => Pixel::RGBA,
PixelFormat::Rgb => Pixel::RGB24,
PixelFormat::Bgr => Pixel::BGR24,
// The GNOME 50+ HDR capture formats (PQ/BT.2020 packed 2:10:10:10) — the HDR CPU path's
// swscale source for the X2RGB10→P010 conversion.
PixelFormat::X2Rgb10 => Pixel::X2RGB10LE,
PixelFormat::X2Bgr10 => Pixel::X2BGR10LE,
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => {
bail!("NVENC 4:4:4 CPU-input path supports packed RGB/BGR only; got {format:?}")
bail!("NVENC CPU-input conversion supports packed RGB/BGR only; got {format:?}")
}
})
}
@@ -136,6 +140,9 @@ fn nvenc_input(format: PixelFormat) -> (Pixel, bool) {
// the Windows paths; the Linux capturer never emits them. Map to BGRA so the match is
// exhaustive — unreachable here.
PixelFormat::Rgb10a2 | PixelFormat::P010 => (Pixel::BGRA, false),
// The Linux HDR capture formats never take the RGB-passthrough input: `open` intercepts
// them onto the X2RGB10→P010 swscale path before consulting this mapping (like 4:4:4).
PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => (Pixel::BGRA, false),
}
}
@@ -164,11 +171,12 @@ pub struct NvencEncoder {
frame: Option<VideoFrame>,
/// Zero-copy path: CUDA hwdevice/hwframes contexts (the encoder takes `AV_PIX_FMT_CUDA`).
cuda: Option<CudaHw>,
/// 4:4:4 CPU path only: swscale context converting the captured packed RGB/BGR → planar
/// YUV444P into [`Self::frame`], because `hevc_nvenc` only emits 4:4:4 from a YUV444 *input*
/// (RGB-in is always 4:2:0). `None` on the 4:2:0 paths AND on the zero-copy 4:4:4 path (the
/// worker's GPU convert delivers YUV444 CUDA frames). Freed in `Drop`.
sws_444: Option<*mut ffi::SwsContext>,
/// CPU CSC paths only: swscale context converting the captured packed source into
/// [`Self::frame`] — RGB/BGR → planar YUV444P for a 4:4:4 session (`hevc_nvenc` only emits
/// 4:4:4 from a YUV444 *input*; RGB-in is always 4:2:0), or X2RGB10/X2BGR10 → P010 (BT.2020
/// limited) for an HDR session. `None` on the plain RGB paths AND on the zero-copy paths (the
/// worker's GPU convert delivers ready CUDA frames). Freed in `Drop`.
sws_csc: Option<*mut ffi::SwsContext>,
/// This session opened as full-chroma 4:4:4 (FREXT) — via either input path.
want_444: bool,
src_format: PixelFormat,
@@ -191,7 +199,7 @@ pub struct NvencEncoder {
args: OpenArgs,
}
// `CudaHw` holds raw `AVBufferRef`s and `sws_444` a raw `SwsContext`; the encoder lives on a single
// `CudaHw` holds raw `AVBufferRef`s and `sws_csc` a raw `SwsContext`; the encoder lives on a single
// thread. The CPU encoder is already `Send` via ffmpeg-next; assert it for the raw fields too.
// SAFETY: `NvencEncoder` owns an ffmpeg-next `Encoder`/`VideoFrame` (already `Send`) plus a `CudaHw`
// holding raw `AVBufferRef`s and an optional raw `SwsContext`, none of which are `Send` by default.
@@ -247,14 +255,27 @@ impl NvencEncoder {
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Self> {
// TODO(hdr): Linux 10-bit parity. Unlike the Windows raw-SDK path (which upconverts 8-bit
// ARGB → Main10 via pixelBitDepthMinus8), libavcodec hevc_nvenc needs a 10-bit input pixel
// format (p010) for Main10, so it's a bigger change; deferred until a Linux GPU box is
// available to validate. The Linux host stays 8-bit for now.
if bit_depth != 8 {
// HDR / 10-bit (GNOME 50+ HDR screencast): a 10-bit session whose capture negotiated a
// packed 2:10:10:10 PQ/BT.2020 format (`X2Rgb10`/`X2Bgr10`) encodes HEVC Main10 / 10-bit
// AV1 from a P010 input frame we produce by swscale (BT.2020 limited; the PQ transfer
// rides through per-channel — BT.2020 NCL Y'CbCr *is* derived from the PQ-encoded R'G'B').
// A 10-bit request whose capture stayed SDR (HDR offer downgraded) honestly encodes 8-bit.
let want_hdr10 = bit_depth == 10 && format.is_hdr_rgb10() && codec.supports_10bit();
if bit_depth == 10 && !want_hdr10 {
tracing::warn!(
bit_depth,
"Linux NVENC 10-bit not yet wired — encoding 8-bit"
?format,
codec = codec.nvenc_name(),
"10-bit requested but the capture format/codec has no 10-bit path — encoding 8-bit"
);
}
if format.is_hdr_rgb10() && !want_hdr10 {
// A 10-bit PQ capture on an 8-bit session would be encoded with a BT.709 VUI and
// garbage bit-packing — never silently; the session must renegotiate.
bail!(
"captured 10-bit HDR frames ({format:?}) on an 8-bit/{} session — refusing to \
mislabel PQ content",
codec.nvenc_name()
);
}
// Full-chroma 4:4:4 (HEVC Range Extensions). `hevc_nvenc` only emits 4:4:4 from a YUV444
@@ -263,6 +284,11 @@ impl NvencEncoder {
// (planar-YUV444 CUDA frames — `cuda` true), or the CPU path's swscale RGB→YUV444P. Both
// feed `profile=rext`; the range follows `PUNKTFUNK_444_FULLRANGE` in both.
let want_444 = chroma.is_444() && codec == Codec::H265;
if want_444 && want_hdr10 {
// The handshake resolves 4:4:4∧10-bit down to 8-bit on Linux, so this can't happen —
// fail loudly if it ever does rather than picking one silently.
bail!("4:4:4 + 10-bit HDR is not a supported Linux NVENC combination");
}
ffmpeg::init().context("ffmpeg init")?;
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
// SAFETY: `av_log_set_level` sets libav's global integer log level; `48` (= AV_LOG_DEBUG)
@@ -274,10 +300,13 @@ impl NvencEncoder {
let av_codec = encoder::find_by_name(name)
.ok_or_else(|| anyhow!("{name} not built into libavcodec"))?;
let (rgb_pixel, rgb_expand) = nvenc_input(format);
// 4:4:4 feeds NVENC a planar YUV444P frame we produce by swscale; the ordinary path feeds the
// captured RGB straight in and lets NVENC's internal CSC subsample to 4:2:0.
// 4:4:4 feeds NVENC a planar YUV444P frame we produce by swscale; HDR feeds it a P010
// frame likewise; the ordinary path feeds the captured RGB straight in and lets NVENC's
// internal CSC subsample to 4:2:0.
let (nvenc_pixel, expand) = if want_444 {
(Pixel::YUV444P, false)
} else if want_hdr10 {
(Pixel::P010LE, false)
} else {
(rgb_pixel, rgb_expand)
};
@@ -325,7 +354,21 @@ impl NvencEncoder {
// visible win. Linux-only: the Windows path's NVENC-internal CSC range is unmeasured.
let full_range_444 =
want_444 && std::env::var("PUNKTFUNK_444_FULLRANGE").is_ok_and(|v| v.trim() == "1");
if matches!(format, PixelFormat::Nv12) || want_444 {
if want_hdr10 {
// HDR10: BT.2020 primaries + SMPTE-2084 (PQ) transfer, limited range — matches the
// swscale BT.2020 CSC below and the Windows paths' signalling. The client decoder
// auto-detects PQ from the VUI; static mastering metadata rides out-of-band.
// SAFETY: `raw = video.as_mut_ptr()` is the non-null, properly-aligned, sole-owned,
// not-yet-opened `AVCodecContext`; we set its four VUI colour enum fields to valid
// variants before `open_with`. Sole owner → no aliasing; synchronous writes.
unsafe {
let raw = video.as_mut_ptr();
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT2020;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084;
}
} else if matches!(format, PixelFormat::Nv12) || want_444 {
// SAFETY: same `video` builder — `raw = video.as_mut_ptr()` is the non-null, properly-
// aligned, sole-owned, not-yet-opened `AVCodecContext`. We set its four VUI colour enum
// fields to valid `AVColorSpace`/`AVColorRange`/`AVColorPrimaries`/`AVColorTransfer-
@@ -370,17 +413,20 @@ impl NvencEncoder {
None
};
// 4:4:4 CPU path: build the RGB→YUV444P swscale (BT.709, range per the flag; no rescale).
// Mirrors the VAAPI CPU path's RGB→NV12 scaler, but the dst is full-chroma planar 4:4:4.
// Skipped on the zero-copy path (`cuda`): the worker's GPU convert already delivers
// planar YUV444 CUDA frames — no CPU pixels exist to scale.
let sws_444 = if want_444 && !cuda {
// CPU CSC paths: build the packed-RGB → planar swscale (no rescale) into the encoder's
// input frame. Two users: 4:4:4 (RGB→YUV444P, BT.709, range per the flag) and HDR
// (X2RGB10/X2BGR10→P010, BT.2020 limited — the PQ transfer is per-channel and rides
// through the matrix untouched). Skipped on the zero-copy path (`cuda`): the worker's GPU
// convert already delivers ready CUDA frames — no CPU pixels exist to scale.
let sws_csc = if (want_444 || want_hdr10) && !cuda {
let src_av = pixel_to_av(sws_src_pixel(format)?);
let dst_av = pixel_to_av(nvenc_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated, packed-RGB-only source), the
// dst is YUV444P. The trailing filter/param pointers are null = "use defaults" (documented
// as accepted). No Rust memory is borrowed; the returned pointer is null-checked below.
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated packed-RGB source), the dst is
// YUV444P (4:4:4) or P010LE (HDR). The trailing filter/param pointers are null = "use
// defaults" (documented as accepted). No Rust memory is borrowed; the returned pointer is
// null-checked below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
@@ -388,7 +434,7 @@ impl NvencEncoder {
src_av,
width as c_int,
height as c_int,
ffi::AVPixelFormat::AV_PIX_FMT_YUV444P,
dst_av,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
@@ -396,17 +442,22 @@ impl NvencEncoder {
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→YUV444P) failed");
bail!("sws_getContext(RGB→{nvenc_pixel:?}) failed");
}
// SAFETY: `sws` is the non-null context from the call above (null-checked). The ITU-709
// coefficient table from `sws_getCoefficients` is a process-lifetime libswscale static,
// reused for src+dst matrices; `sws_setColorspaceDetails` only reads it and writes scalar
// CSC settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
// PUNKTFUNK_444_FULLRANGE experiment). No Rust memory is passed.
// SAFETY: `sws` is the non-null context from the call above (null-checked). The
// coefficient tables from `sws_getCoefficients` (ITU-709 for 4:4:4, BT.2020 NCL for HDR
// — matching the VUI written above) are process-lifetime libswscale statics, reused for
// src+dst matrices; `sws_setColorspaceDetails` only reads them and writes scalar CSC
// settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
// PUNKTFUNK_444_FULLRANGE experiment; HDR is always limited). No Rust memory is passed.
unsafe {
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
let cs = ffi::sws_getCoefficients(if want_hdr10 {
super::libav::SWS_CS_BT2020
} else {
SWS_CS_ITU709
});
let dst_range = i32::from(full_range_444);
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, dst_range, 0, 1 << 16, 1 << 16);
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, dst_range, 0, 1 << 16, 1 << 16);
}
Some(sws)
} else {
@@ -432,6 +483,12 @@ impl NvencEncoder {
// dropped on a future libavcodec.
opts.set("profile", "rext");
}
if want_hdr10 && codec == Codec::H265 {
// HEVC Main10. `hevc_nvenc` auto-selects it from the P010 input, but pin it explicitly
// so the depth is never silently dropped on a future libavcodec. (10-bit AV1 needs no
// profile — AV1 Main carries 10-bit, driven by the input format.)
opts.set("profile", "main10");
}
// Split-frame encode across both NVENC engines (GB203 has 2) when the pixel rate exceeds
// a single engine's HEVC capacity (~1 Gpix/s); e.g. 5120x1440@240 = 1.77 Gpix/s needs it,
@@ -501,7 +558,7 @@ impl NvencEncoder {
enc,
frame,
cuda: cuda_hw,
sws_444,
sws_csc,
want_444,
src_format: format,
expand,
@@ -640,7 +697,7 @@ impl NvencEncoder {
);
// 4:4:4: swscale the packed RGB straight into the planar YUV444P input frame (BT.709 limited),
// then send it — no byte-expand. The 4:2:0 RGB path (below) feeds NVENC packed RGB directly.
if let Some(sws) = self.sws_444 {
if let Some(sws) = self.sws_csc {
let frame = self
.frame
.as_mut()
@@ -810,7 +867,7 @@ impl NvencEncoder {
impl Drop for NvencEncoder {
fn drop(&mut self) {
if let Some(sws) = self.sws_444.take() {
if let Some(sws) = self.sws_csc.take() {
// SAFETY: `sws` is the non-null `SwsContext` allocated by `sws_getContext` in `open` and
// owned exclusively by this encoder (taken out of the field so it can't be freed twice).
// `sws_freeContext` frees it; nothing else references it after this single-threaded drop.
@@ -855,3 +912,105 @@ pub fn probe_can_encode_444(codec: Codec) -> bool {
unsafe { ffi::av_log_set_level(prev) };
ok
}
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode 10-bit (HEVC Main10 /
/// 10-bit AV1) from a P010 input — the exact path [`NvencEncoder::open`] takes for a live HDR
/// stream (a tiny X2RGB10-sourced, P010-input open). The result is cached by the caller
/// ([`crate::can_encode_10bit`]); a GPU/driver/ffmpeg without the 10-bit encode fails the open
/// here, so the host resolves the session to 8-bit SDR before the Welcome (honest downgrade).
pub fn probe_can_encode_10bit(codec: Codec) -> bool {
if !codec.supports_10bit() {
return false;
}
if ffmpeg::init().is_err() {
return false;
}
// Quiet ffmpeg's open error on a GPU that lacks 10-bit — the probe failing is an expected outcome.
// SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
// (no pointer args) and are always sound post-init.
let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
let ok = NvencEncoder::open(
codec,
PixelFormat::X2Rgb10,
640,
480,
30,
2_000_000,
false, // CPU input (the HDR swscale path)
10,
ChromaFormat::Yuv420,
)
.is_ok();
// SAFETY: restore the saved global log level (scalar arg, no pointers).
unsafe { ffi::av_log_set_level(prev) };
ok
}
#[cfg(test)]
mod hdr_tests {
use super::*;
/// The Linux HDR (GNOME 50 portal) encode path end-to-end on a real NVIDIA GPU: a synthetic
/// PQ-ish X2RGB10 CPU frame → swscale BT.2020 → P010 → `hevc_nvenc` Main10, drained to a real
/// AU. `#[ignore]`d (needs NVENC):
/// `cargo test -p pf-encode nvenc_hdr10_smoke -- --ignored --nocapture`
#[test]
#[ignore]
fn nvenc_hdr10_smoke() {
let (w, h) = (640u32, 480u32);
let mut enc = NvencEncoder::open(
Codec::H265,
PixelFormat::X2Rgb10,
w,
h,
30,
2_000_000,
false,
10,
ChromaFormat::Yuv420,
)
.expect("open hevc_nvenc Main10 (P010 input)");
// Packed x:R:G:B 2:10:10:10 gradient (values are treated as PQ-encoded — fine for a smoke).
let mut bytes = vec![0u8; (w * h * 4) as usize];
for y in 0..h {
for x in 0..w {
let r = (x * 1023 / w.max(1)) & 0x3ff;
let g = (y * 1023 / h.max(1)) & 0x3ff;
let b = ((x + y) * 1023 / (w + h)) & 0x3ff;
let px: u32 = (r << 20) | (g << 10) | b;
let i = ((y * w + x) * 4) as usize;
bytes[i..i + 4].copy_from_slice(&px.to_le_bytes());
}
}
let frame = CapturedFrame {
width: w,
height: h,
pts_ns: 0,
format: PixelFormat::X2Rgb10,
payload: FramePayload::Cpu(bytes),
cursor: None,
};
let mut au = None;
for _ in 0..30 {
enc.submit(&frame).expect("submit X2Rgb10 frame");
if let Some(a) = enc.poll().expect("poll") {
au = Some(a);
break;
}
}
let au = au.expect("no AU produced within 30 frames");
assert!(!au.data.is_empty(), "empty AU");
assert!(au.keyframe, "first AU should be the IDR");
println!("HDR10 smoke: first AU {} bytes (IDR)", au.data.len());
// PF_HDR_SMOKE_DUMP=/path.h265: write the Annex-B AU for external inspection —
// `ffprobe -show_streams` should report Main 10, bt2020nc/smpte2084/bt2020 colours.
if let Ok(path) = std::env::var("PF_HDR_SMOKE_DUMP") {
std::fs::write(&path, &au.data).expect("dump AU");
println!("HDR10 smoke: AU written to {path}");
}
}
}
+6 -2
View File
@@ -474,9 +474,13 @@ impl NvencCudaEncoder {
// clear reason instead of an opaque session error on the first frame.
try_api().map_err(|e| anyhow!("NVENC (Linux direct) unavailable: {e}"))?;
if bit_depth >= 10 {
// An HDR (GNOME 50 portal) session never reaches this backend: its X2RGB10 frames ride
// the CPU/dmabuf paths (no CUDA import for the 10-bit formats yet), so the dispatcher
// opens the libav P010 path instead. Reaching here 10-bit means a CUDA capture payload
// on a 10-bit session — not wired; encode 8-bit rather than mislabel.
tracing::warn!(
"Linux direct-NVENC: 10-bit requested but no P010 capture path exists yet \
(Phase 5.1) encoding 8-bit SDR"
"Linux direct-NVENC: 10-bit requested but the CUDA capture path has no 10-bit \
import yet (HDR rides the libav P010 path) encoding 8-bit SDR"
);
}
Ok(Self {
+201 -105
View File
@@ -38,6 +38,9 @@ use std::os::raw::c_char;
/// uses. PyroWave carries no VUI, so the colour contract is fixed by this shader: the Phase-2
/// client CSC must assume BT.709 limited range.
const CSC_SPV: &[u8] = include_bytes!("rgb2yuv.spv");
/// The 4:4:4 twin (`rgb2yuv444.comp`): one invocation per pixel, full-res interleaved CbCr,
/// same BT.709-limited coefficients byte-for-byte.
const CSC444_SPV: &[u8] = include_bytes!("rgb2yuv444.spv");
/// Fixed cursor-overlay texture size (px) — mirrors `vulkan_video.rs`; the shared CSC shader bounds
/// sampling by its push constant, so one allocation fits every pointer bitmap.
const CURSOR_MAX: u32 = 256;
@@ -46,13 +49,6 @@ const IMPORT_CACHE_CAP: usize = 16;
/// Headroom over the per-frame rate budget for the packetized bitstream (block headers + meta;
/// the rate controller itself never exceeds the budget).
const BS_SLACK: usize = 256 * 1024;
/// Chunked-mode window framing (§4.4): 4-byte prefix per shard-sized window.
const WINDOW_PREFIX: usize = 4;
/// Window kinds: whole packets / an oversized packet's fragments.
const WIN_PACKED: u16 = 0;
const WIN_FRAG_FIRST: u16 = 1;
const WIN_FRAG_CONT: u16 = 2;
const WIN_FRAG_LAST: u16 = 3;
/// The DRM modifiers the PyroWave device can import as a SAMPLED image of the capture's
/// packed-RGB format. The capture advertises these for the pyrowave passthrough instead of
@@ -197,6 +193,9 @@ pub struct PyroWaveEncoder {
width: u32,
height: u32,
fps: u32,
/// Session-fixed negotiated chroma: 4:4:4 = full-res RG8 chroma plane + per-pixel CSC
/// (`rgb2yuv444.comp`) + `Chroma444` pyrowave objects.
chroma444: bool,
/// Per-frame bitstream budget (hard CBR): `bitrate / (8 * fps)`.
frame_budget: usize,
/// Datagram-aligned mode (plan §4.4): packetize at this boundary and pad every codec
@@ -218,17 +217,39 @@ fn budget_for(bitrate_bps: u64, fps: u32) -> usize {
}
impl PyroWaveEncoder {
pub fn open(width: u32, height: u32, fps: u32, bitrate_bps: u64) -> Result<Self> {
if width % 2 != 0 || height % 2 != 0 {
pub fn open(
width: u32,
height: u32,
fps: u32,
bitrate_bps: u64,
chroma: crate::ChromaFormat,
) -> Result<Self> {
if !chroma.is_444() && (width % 2 != 0 || height % 2 != 0) {
bail!("pyrowave 4:2:0 needs even dimensions (got {width}x{height})");
}
if chroma.is_444() && !crate::pyrowave_mode_fits_rdo(width, height, true) {
// The negotiator downgrades these modes to 4:2:0 pre-Welcome; refuse if one
// slips through (e.g. the lab override) rather than wrap the RDO block index.
bail!(
"pyrowave 4:4:4 at {width}x{height} exceeds the rate controller's 16-bit \
block index (see pyrowave-sys patches/0002 note) use 4:2:0 at this size"
);
}
// SAFETY: `open_inner` only issues Vulkan/pyrowave calls whose preconditions it
// establishes itself (valid instance/device, correctly-chained create-infos that
// `DeviceHold` keeps alive); all handles are freshly created and owned by the result.
unsafe { Self::open_inner(width, height, fps.max(1), bitrate_bps.max(1_000_000)) }
unsafe {
Self::open_inner(
width,
height,
fps.max(1),
bitrate_bps.max(1_000_000),
chroma.is_444(),
)
}
}
unsafe fn open_inner(w: u32, h: u32, fps: u32, bitrate: u64) -> Result<Self> {
unsafe fn open_inner(w: u32, h: u32, fps: u32, bitrate: u64, chroma444: bool) -> Result<Self> {
let entry = ash::Entry::load().context("load vulkan loader")?;
let mut hold = DeviceHold {
@@ -381,7 +402,11 @@ impl PyroWaveEncoder {
device: pw_dev,
width: w as i32,
height: h as i32,
chroma: pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420,
chroma: if chroma444 {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_444
} else {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420
},
};
let mut pw_enc: pw::pyrowave_encoder = std::ptr::null_mut();
if let Err(e) = pw_check(
@@ -392,8 +417,10 @@ impl PyroWaveEncoder {
return Err(e);
}
// ---- CSC planes: full-res R8 luma + half-res RG8 chroma, storage-written by the CSC
// and sampled directly by pyrowave (R/G view swizzles synthesize Cb/Cr) ----
// ---- CSC planes: full-res R8 luma + RG8 chroma (half-res for 4:2:0, full-res for
// 4:4:4), storage-written by the CSC and sampled directly by pyrowave (R/G view
// swizzles synthesize Cb/Cr) ----
let (cw, ch) = if chroma444 { (w, h) } else { (w / 2, h / 2) };
let (y_img, y_mem, y_view) = make_plain_image(
&device,
&mem_props,
@@ -406,8 +433,8 @@ impl PyroWaveEncoder {
&device,
&mem_props,
vk::Format::R8G8_UNORM,
w / 2,
h / 2,
cw,
ch,
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::SAMPLED,
)?;
@@ -420,7 +447,11 @@ impl PyroWaveEncoder {
.address_mode_v(vk::SamplerAddressMode::CLAMP_TO_EDGE),
None,
)?;
let spv = ash::util::read_spv(&mut std::io::Cursor::new(CSC_SPV))?;
let spv = ash::util::read_spv(&mut std::io::Cursor::new(if chroma444 {
CSC444_SPV
} else {
CSC_SPV
}))?;
let shader =
device.create_shader_module(&vk::ShaderModuleCreateInfo::default().code(&spv), None)?;
let sb = |b: u32, t: vk::DescriptorType| {
@@ -566,7 +597,8 @@ impl PyroWaveEncoder {
gpu = %props.device_name_as_c_str().unwrap_or(c"?").to_string_lossy(),
mode = %format!("{w}x{h}@{fps}"),
budget_kib = frame_budget / 1024,
"PyroWave encoder open (intra-only wavelet, BT.709 limited 4:2:0)"
chroma = if chroma444 { "4:4:4" } else { "4:2:0" },
"PyroWave encoder open (intra-only wavelet, BT.709 limited)"
);
Ok(Self {
@@ -608,6 +640,7 @@ impl PyroWaveEncoder {
width: w,
height: h,
fps,
chroma444,
frame_budget,
wire_chunk: None,
bitstream: Vec::new(),
@@ -971,7 +1004,12 @@ impl PyroWaveEncoder {
0,
&pc_bytes,
);
dev.cmd_dispatch(self.cmd, (w / 2).div_ceil(8), (h / 2).div_ceil(8), 1);
// 4:2:0: one invocation per 2x2 luma block (per chroma sample); 4:4:4: per pixel.
if self.chroma444 {
dev.cmd_dispatch(self.cmd, w.div_ceil(8), h.div_ceil(8), 1);
} else {
dev.cmd_dispatch(self.cmd, (w / 2).div_ceil(8), (h / 2).div_ceil(8), 1);
}
// CSC storage writes -> pyrowave's sampled reads (images stay GENERAL — the layout
// pyrowave's GPU-buffer contract accepts without transitions).
@@ -1024,17 +1062,19 @@ impl PyroWaveEncoder {
),
// Two-component chroma image: view swizzles R/G synthesize the Cb/Cr planes
// (the documented NV12-style hand-off, pyrowave.h `pyrowave_gpu_buffers`).
// The view extent is the chroma IMAGE's own mip0 extent (it's a separate
// image, not a planar aspect): half-res for 4:2:0, full-res for 4:4:4.
plane(
self.uv_img,
w / 2,
h / 2,
if self.chroma444 { w } else { w / 2 },
if self.chroma444 { h } else { h / 2 },
rg8,
pw::VkComponentSwizzle_VK_COMPONENT_SWIZZLE_R,
),
plane(
self.uv_img,
w / 2,
h / 2,
if self.chroma444 { w } else { w / 2 },
if self.chroma444 { h } else { h / 2 },
rg8,
pw::VkComponentSwizzle_VK_COMPONENT_SWIZZLE_G,
),
@@ -1077,8 +1117,8 @@ impl PyroWaveEncoder {
// boundary by design.
let cap = self.frame_budget + BS_SLACK;
self.bitstream.resize(cap, 0);
// Chunked mode reserves 4 bytes per window for the framing prefix.
let boundary = self.wire_chunk.map(|c| c - WINDOW_PREFIX).unwrap_or(cap);
// Chunked mode reserves the 4-byte window prefix from the packetize boundary (shared helper).
let boundary = crate::pyrowave_wire::packet_boundary(self.wire_chunk, cap);
let mut n: usize = 0;
pw_check(
pw::pyrowave_encoder_compute_num_packets(self.pw_enc, boundary, &mut n),
@@ -1101,67 +1141,16 @@ impl PyroWaveEncoder {
"packetize",
)?;
packets.truncate(out_n.max(1));
let au = if let Some(chunk) = self.wire_chunk {
// Window framing (§4.4): each `chunk`-sized window opens with a 4-byte prefix
// (u16 used-length + u16 kind) and carries either WHOLE self-delimiting codec
// packets (PACKED — several small ones share a window) or one fragment of an
// oversized packet (FRAG chain — pyrowave 32×32 blocks are atomic and may
// exceed a shard). A lost shard zeroes its window (used = 0) — the receiver
// skips it and drops any fragment chain it interrupts.
let payload_max = chunk - WINDOW_PREFIX;
let mut au: Vec<u8> = Vec::with_capacity((packets.len() + 1) * chunk);
// The currently-open PACKED window: (start offset of its prefix, bytes used).
let mut open: Option<(usize, usize)> = None;
let close = |au: &mut Vec<u8>, open: &mut Option<(usize, usize)>, chunk: usize| {
if let Some((start, used)) = open.take() {
au[start..start + 2].copy_from_slice(&(used as u16).to_le_bytes());
au[start + 2..start + 4].copy_from_slice(&WIN_PACKED.to_le_bytes());
au.resize(start + chunk, 0);
}
};
for p in &packets {
let bytes = &self.bitstream[p.offset..p.offset + p.size];
if p.size <= payload_max {
let fits = open.is_some_and(|(_, used)| used + p.size <= payload_max);
if !fits {
close(&mut au, &mut open, chunk);
let start = au.len();
au.resize(start + WINDOW_PREFIX, 0);
open = Some((start, 0));
}
au.extend_from_slice(bytes);
if let Some((_, used)) = open.as_mut() {
*used += p.size;
}
} else {
// Oversized packet: its own FRAG chain of full windows.
close(&mut au, &mut open, chunk);
let mut off = 0usize;
while off < p.size {
let take = (p.size - off).min(payload_max);
let kind = if off == 0 {
WIN_FRAG_FIRST
} else if off + take == p.size {
WIN_FRAG_LAST
} else {
WIN_FRAG_CONT
};
let start = au.len();
au.resize(start + WINDOW_PREFIX, 0);
au[start..start + 2].copy_from_slice(&(take as u16).to_le_bytes());
au[start + 2..start + 4].copy_from_slice(&kind.to_le_bytes());
au.extend_from_slice(&bytes[off..off + take]);
au.resize(start + chunk, 0);
off += take;
}
}
}
close(&mut au, &mut open, chunk);
au
} else {
let p = &packets[0];
self.bitstream[p.offset..p.offset + p.size].to_vec()
};
// Correct pyrowave's zeroed sequence-header VUI: it signals ycbcr_range=FULL, but our CSC
// emits BT.709 LIMITED — patch the bits HONEST so VUI-honoring clients don't wash out
// blacks. (Linux capture has no HDR path, so this side never stamps BT.2020/PQ.)
if let Some(p) = packets.first() {
crate::pyrowave_wire::stamp_color_bits(&mut self.bitstream, p.offset, false);
}
// Frame into the wire AU via the shared helper (byte-identical on Linux + Windows): the dense
// single packet, or the datagram-aligned windowed AU (§4.4).
let pkts: Vec<(usize, usize)> = packets.iter().map(|p| (p.offset, p.size)).collect();
let au = crate::pyrowave_wire::build_au(&pkts, &self.bitstream, self.wire_chunk);
self.frame_count += 1;
self.pending.push_back(EncodedFrame {
data: au,
@@ -1184,9 +1173,14 @@ impl Encoder for PyroWaveEncoder {
}
fn caps(&self) -> EncoderCaps {
// All defaults: no RFI (meaningless — every frame is intra), no HDR (8-bit SDR codec),
// no intra-refresh wave (ditto). 4:2:0 only until the 4:4:4 ride-along (plan §6).
EncoderCaps::default()
// No RFI / no intra-refresh wave (every frame is intra). Report the real opened chroma so
// the session glue's post-open cross-check stays quiet on a genuine 4:4:4 session — a
// hardcoded `default()` here mis-reports a 4:4:4 open as 4:2:0 and fires a spurious
// "chroma disagrees with the negotiated Welcome" warn.
EncoderCaps {
chroma_444: self.chroma444,
..EncoderCaps::default()
}
}
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
@@ -1205,7 +1199,11 @@ impl Encoder for PyroWaveEncoder {
device: self.pw_dev,
width: self.width as i32,
height: self.height as i32,
chroma: pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420,
chroma: if self.chroma444 {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_444
} else {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420
},
};
let mut enc: pw::pyrowave_encoder = std::ptr::null_mut();
let r = pw::pyrowave_encoder_create(&einfo, &mut enc);
@@ -1327,10 +1325,16 @@ mod tests {
)
}
/// Decode an AU with a standalone pyrowave decoder and return the full YUV420P planes.
/// This is the golden oracle for both the Phase-1 smoke check (plane means) and the Apple
/// Metal port's committed PSNR fixtures (`pyrowave_dump_golden`).
unsafe fn decode_planes(w: u32, h: u32, au: &[u8]) -> (Vec<u8>, Vec<u8>, Vec<u8>) {
/// Decode an AU with a standalone pyrowave decoder and return the full planar YUV
/// (half-res chroma for 4:2:0, full-res for 4:4:4). This is the golden oracle for the
/// smoke checks (plane means) and the Apple Metal port's committed PSNR fixtures
/// (`pyrowave_dump_golden`).
unsafe fn decode_planes_chroma(
w: u32,
h: u32,
au: &[u8],
chroma444: bool,
) -> (Vec<u8>, Vec<u8>, Vec<u8>) {
let mut dev: pw::pyrowave_device = std::ptr::null_mut();
assert_eq!(
pw::pyrowave_create_default_device(&mut dev),
@@ -1340,7 +1344,11 @@ mod tests {
device: dev,
width: w as i32,
height: h as i32,
chroma: pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420,
chroma: if chroma444 {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_444
} else {
pw::pyrowave_chroma_subsampling_PYROWAVE_CHROMA_SUBSAMPLING_420
},
fragment_path: false,
};
let mut dec: pw::pyrowave_decoder = std::ptr::null_mut();
@@ -1354,11 +1362,16 @@ mod tests {
);
assert!(pw::pyrowave_decoder_decode_is_ready(dec, false));
let (cw, ch) = if chroma444 { (w, h) } else { (w / 2, h / 2) };
let mut y = vec![0u8; (w * h) as usize];
let mut cb = vec![0u8; (w * h / 4) as usize];
let mut cr = vec![0u8; (w * h / 4) as usize];
let mut cb = vec![0u8; (cw * ch) as usize];
let mut cr = vec![0u8; (cw * ch) as usize];
let mut buf: pw::pyrowave_cpu_buffer = std::mem::zeroed();
buf.format = pw::pyrowave_cpu_buffer_format_PYROWAVE_CPU_BUFFER_FORMAT_YUV420P;
buf.format = if chroma444 {
pw::pyrowave_cpu_buffer_format_PYROWAVE_CPU_BUFFER_FORMAT_YUV444P
} else {
pw::pyrowave_cpu_buffer_format_PYROWAVE_CPU_BUFFER_FORMAT_YUV420P
};
buf.width = w as i32;
buf.height = h as i32;
buf.data = [
@@ -1366,7 +1379,7 @@ mod tests {
cb.as_mut_ptr() as *mut _,
cr.as_mut_ptr() as *mut _,
];
buf.row_stride_in_bytes = [w as usize, (w / 2) as usize, (w / 2) as usize];
buf.row_stride_in_bytes = [w as usize, cw as usize, cw as usize];
buf.plane_size_in_bytes = [y.len(), cb.len(), cr.len()];
assert_eq!(
pw::pyrowave_decoder_decode_cpu_buffer_synchronous(dec, &buf),
@@ -1377,10 +1390,15 @@ mod tests {
(y, cb, cr)
}
/// Plane means of an upstream-decoded AU — the Phase-1 smoke assertion.
unsafe fn decode_plane_means(w: u32, h: u32, au: &[u8]) -> (f64, f64, f64) {
unsafe fn decode_planes(w: u32, h: u32, au: &[u8]) -> (Vec<u8>, Vec<u8>, Vec<u8>) {
// SAFETY: forwarded — same contract as the caller.
let (y, cb, cr) = unsafe { decode_planes(w, h, au) };
unsafe { decode_planes_chroma(w, h, au, false) }
}
/// Plane means of an upstream-decoded AU — the smoke assertion.
unsafe fn decode_plane_means(w: u32, h: u32, au: &[u8], chroma444: bool) -> (f64, f64, f64) {
// SAFETY: forwarded — same contract as the caller.
let (y, cb, cr) = unsafe { decode_planes_chroma(w, h, au, chroma444) };
let mean = |v: &[u8]| v.iter().map(|&x| x as f64).sum::<f64>() / v.len() as f64;
(mean(&y), mean(&cb), mean(&cr))
}
@@ -1394,7 +1412,8 @@ mod tests {
#[ignore = "needs a real Vulkan 1.3 compute device (run on a GPU host, not the build box)"]
fn pyrowave_smoke() {
let (w, h) = (256u32, 256u32);
let mut enc = PyroWaveEncoder::open(w, h, 60, 40_000_000).expect("open");
let mut enc =
PyroWaveEncoder::open(w, h, 60, 40_000_000, crate::ChromaFormat::Yuv420).expect("open");
assert!(!enc.caps().supports_rfi);
let colors = [
@@ -1414,7 +1433,7 @@ mod tests {
"AU exceeds rate budget"
);
// SAFETY: test-only FFI into the vendored decoder with locally-owned buffers.
let (ym, cbm, crm) = unsafe { decode_plane_means(w, h, &au.data) };
let (ym, cbm, crm) = unsafe { decode_plane_means(w, h, &au.data, false) };
let (ye, cbe, cre) = bt709(*c);
assert!(
(ym - ye).abs() < 3.0 && (cbm - cbe).abs() < 3.0 && (crm - cre).abs() < 3.0,
@@ -1508,6 +1527,68 @@ mod tests {
assert!(enc.poll().expect("poll").is_some());
}
/// The 4:4:4 twin of `pyrowave_smoke`: per-pixel CSC into full-res RG8 chroma +
/// `Chroma444` pyrowave objects, verified by upstream's own 4:4:4 CPU decode. The
/// busy-card leg then drives the rate controller at the ~2.6 bpp operating point —
/// exactly the regime that overran upstream's 4:2:0-sized payload staging before
/// `patches/0001-payload-data-444-sizing.patch` (the Phase-0 finding): it must stay
/// within budget, decode, and be run-to-run deterministic (the overrun was not).
#[test]
#[ignore = "needs a real Vulkan 1.3 compute device (run on a GPU host, not the build box)"]
fn pyrowave_smoke_444() {
let (w, h) = (256u32, 256u32);
let mut enc =
PyroWaveEncoder::open(w, h, 60, 40_000_000, crate::ChromaFormat::Yuv444).expect("open");
let colors = [
[40u8, 40, 200, 255],
[40, 200, 40, 255],
[200, 40, 40, 255],
[128, 128, 128, 255],
];
for (i, c) in colors.iter().enumerate() {
enc.submit(&cpu_frame(w, h, i as u64 * 16_666_667, *c))
.expect("submit");
let au = enc.poll().expect("poll").expect("one AU per frame");
assert!(au.keyframe);
assert!(
au.data.len() <= enc.frame_budget + BS_SLACK,
"AU exceeds rate budget"
);
// SAFETY: test-only FFI into the vendored decoder with locally-owned buffers.
let (ym, cbm, crm) = unsafe { decode_plane_means(w, h, &au.data, true) };
let (ye, cbe, cre) = bt709(*c);
assert!(
(ym - ye).abs() < 3.0 && (cbm - cbe).abs() < 3.0 && (crm - cre).abs() < 3.0,
"frame {i}: decoded plane means (Y {ym:.1}, Cb {cbm:.1}, Cr {crm:.1}) vs \
expected (Y {ye:.1}, Cb {cbe:.1}, Cr {cre:.1})"
);
}
// Busy content at the 4:4:4 operating point (~2.6 bpp).
let budget_bps = w as u64 * h as u64 * 60 * 26 / 10;
let mut enc =
PyroWaveEncoder::open(w, h, 60, budget_bps, crate::ChromaFormat::Yuv444).expect("open");
let mut sizes = Vec::new();
for _ in 0..3 {
enc.submit(&test_card(w, h, 7)).expect("busy submit");
let au = enc.poll().expect("poll").expect("busy AU");
assert!(
au.data.len() <= enc.frame_budget + BS_SLACK,
"busy 4:4:4 AU exceeds rate budget ({} > {})",
au.data.len(),
enc.frame_budget + BS_SLACK
);
// Upstream's own decoder accepts it (a corrupt stream errors or garbles).
// SAFETY: test-only FFI with locally-owned buffers.
let _ = unsafe { decode_planes_chroma(w, h, &au.data, true) };
sizes.push(au.data.len());
}
assert!(
sizes.windows(2).all(|s| s[0] == s[1]),
"identical input produced varying AU sizes (the Phase-0 overrun signature): {sizes:?}"
);
}
/// A deterministic busy BGRA test card (gradients + checker + LCG noise) — flat fills
/// exercise almost none of the entropy decoder, this hits every subband.
fn test_card(w: u32, h: u32, seed: u32) -> CapturedFrame {
@@ -1558,7 +1639,8 @@ mod tests {
// Odd-block geometry on purpose: 256 aligns clean, 144 → aligned 160 exercises the
// block-grid overhang. ~1.6 bpp at 60 fps.
let (w, h) = (256u32, 144u32);
let mut enc = PyroWaveEncoder::open(w, h, 60, 4_000_000).expect("open");
let mut enc =
PyroWaveEncoder::open(w, h, 60, 4_000_000, crate::ChromaFormat::Yuv420).expect("open");
let dump = |name: &str, bytes: &[u8]| {
std::fs::write(dir.join(name), bytes).expect("write fixture");
@@ -1610,5 +1692,19 @@ mod tests {
dump("ref-chunked-y.bin", &y);
dump("ref-chunked-cb.bin", &cb);
dump("ref-chunked-cr.bin", &cr);
// 4:4:4 dense AU + its reference (full-res chroma planes) — the Apple 4:4:4 layout's
// golden (design/pyrowave-444-hdr.md Phase 4). Same odd-block geometry.
let mut enc =
PyroWaveEncoder::open(w, h, 60, 6_500_000, crate::ChromaFormat::Yuv444).expect("open");
enc.submit(&test_card(w, h, 13)).expect("444 submit");
let au = enc.poll().expect("poll").expect("444 AU");
assert!(!au.chunk_aligned);
dump("au-dense444.bin", &au.data);
// SAFETY: test-only FFI with locally-owned buffers.
let (y, cb, cr) = unsafe { decode_planes_chroma(w, h, &au.data, true) };
dump("ref-dense444-y.bin", &y);
dump("ref-dense444-cb.bin", &cb);
dump("ref-dense444-cr.bin", &cr);
}
}
@@ -0,0 +1,37 @@
#version 450
// RGB(A) -> full-res Y + FULL-res interleaved CbCr (BT.709 limited range): the 4:4:4 twin of
// rgb2yuv.comp — one invocation per pixel, no chroma box filter, no siting. Same coefficients
// byte-for-byte (the wavelet clients' planar CSC decodes both layouts identically), same
// cursor-as-metadata blend, same source-edge clamp for the 32-aligned coded extent.
layout(local_size_x = 8, local_size_y = 8) in;
layout(binding = 0) uniform sampler2D rgb; // packed RGB input (sampled; BGRA import ok)
layout(binding = 1, r8) uniform writeonly image2D yImg; // full-res Y
layout(binding = 2, rg8) uniform writeonly image2D uvImg; // full-res UV (interleaved)
layout(binding = 3) uniform sampler2D cursorTex; // straight-alpha RGBA cursor (top-left)
layout(push_constant) uniform Push {
ivec2 curOrigin; // top-left of the cursor in frame pixels (position - hotspot)
ivec2 curSize; // cursor w,h in pixels; x <= 0 => disabled
} pc;
float lumaY(vec3 c) { return 16.0/255.0 + 0.1826*c.r + 0.6142*c.g + 0.0620*c.b; }
vec3 withCursor(ivec2 p, vec3 col) {
if (pc.curSize.x <= 0) return col;
ivec2 cp = p - pc.curOrigin;
if (cp.x < 0 || cp.y < 0 || cp.x >= pc.curSize.x || cp.y >= pc.curSize.y) return col;
vec4 c = texelFetch(cursorTex, cp, 0);
return mix(col, c.rgb, c.a);
}
void main() {
ivec2 sz = imageSize(yImg);
ivec2 rmax = textureSize(rgb, 0) - 1;
ivec2 p = ivec2(gl_GlobalInvocationID.xy);
if (p.x >= sz.x || p.y >= sz.y) return;
vec3 c = withCursor(p, texelFetch(rgb, min(p, rmax), 0).rgb);
imageStore(yImg, p, vec4(lumaY(c), 0, 0, 1));
float U = 128.0/255.0 - 0.1006*c.r - 0.3386*c.g + 0.4392*c.b;
float V = 128.0/255.0 + 0.4392*c.r - 0.3989*c.g - 0.0403*c.b;
imageStore(uvImg, p, vec4(U, V, 0, 1));
}
Binary file not shown.
+179 -49
View File
@@ -61,6 +61,10 @@ fn vaapi_sws_src(format: PixelFormat) -> Result<Pixel> {
PixelFormat::Rgba => Pixel::RGBA,
PixelFormat::Rgb => Pixel::RGB24,
PixelFormat::Bgr => Pixel::BGR24,
// The GNOME 50+ HDR capture formats (PQ/BT.2020 packed 2:10:10:10) — the HDR CPU path's
// swscale source for the X2RGB10→P010 conversion.
PixelFormat::X2Rgb10 => Pixel::X2RGB10LE,
PixelFormat::X2Bgr10 => Pixel::X2BGR10LE,
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => {
bail!("VAAPI CPU-input path supports packed RGB/BGR only; got {format:?}")
}
@@ -101,6 +105,7 @@ fn low_power_override() -> Option<bool> {
/// default on those kernels). AMD keeps its first-try full-feature open byte-for-byte unchanged.
/// The resolved mode is cached per codec; `PUNKTFUNK_VAAPI_LOW_POWER` pins it.
/// Safety contract is [`open_vaapi_encoder_mode`]'s (borrowed `device_ref`/`frames_ref`).
#[allow(clippy::too_many_arguments)]
unsafe fn open_vaapi_encoder(
codec: Codec,
width: u32,
@@ -109,6 +114,7 @@ unsafe fn open_vaapi_encoder(
bitrate_bps: u64,
device_ref: *mut ffi::AVBufferRef,
frames_ref: *mut ffi::AVBufferRef,
ten_bit: bool,
) -> Result<encoder::video::Encoder> {
let idx = lp_idx(codec);
let modes: &[bool] = match low_power_override() {
@@ -130,6 +136,7 @@ unsafe fn open_vaapi_encoder(
bitrate_bps,
device_ref,
frames_ref,
ten_bit,
lp,
) {
Ok(enc) => {
@@ -158,8 +165,9 @@ unsafe fn open_vaapi_encoder(
}
/// Build the FFmpeg encoder context (shared by both inner paths): name, mode, low-latency RC,
/// infinite GOP, BT.709-limited VUI, `pix_fmt=VAAPI`, and the given hw device + frames contexts.
/// Returns the opened encoder. `device_ref`/`frames_ref` are borrowed (ref'd into the context).
/// infinite GOP, the VUI (BT.709 limited SDR, or BT.2020 PQ limited for `ten_bit` HDR),
/// `pix_fmt=VAAPI`, and the given hw device + frames contexts. Returns the opened encoder.
/// `device_ref`/`frames_ref` are borrowed (ref'd into the context).
#[allow(clippy::too_many_arguments)]
unsafe fn open_vaapi_encoder_mode(
codec: Codec,
@@ -169,6 +177,7 @@ unsafe fn open_vaapi_encoder_mode(
bitrate_bps: u64,
device_ref: *mut ffi::AVBufferRef,
frames_ref: *mut ffi::AVBufferRef,
ten_bit: bool,
low_power: bool,
) -> Result<encoder::video::Encoder> {
let name = codec.vaapi_name();
@@ -181,23 +190,39 @@ unsafe fn open_vaapi_encoder_mode(
.context("alloc video encoder")?;
video.set_width(width);
video.set_height(height);
video.set_format(Pixel::NV12); // sw view; pix_fmt overridden to VAAPI below
// Fixed rate, CBR, no B-frames, ~1-frame VBV — the shared low-latency RC contract.
// sw view (pix_fmt overridden to VAAPI below): NV12, or P010 for the 10-bit HDR session.
video.set_format(if ten_bit { Pixel::P010LE } else { Pixel::NV12 });
// Fixed rate, CBR, no B-frames, ~1-frame VBV — the shared low-latency RC contract.
apply_low_latency_rc(&mut video, fps, bitrate_bps);
let raw = video.as_mut_ptr();
(*raw).gop_size = i32::MAX; // no periodic IDR (forced-IDR via pict_type=I on RFI)
// We hand the encoder BT.709 *limited* NV12 (swscale CSC on the CPU path; scale_vaapi pinned
// to `out_color_matrix=bt709:out_range=limited` on the zero-copy path, with the full-range
// RGB input tagged), so signal that VUI — else the client decoder washes the picture out.
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT709;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT709;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709;
if ten_bit {
// HDR10: BT.2020 primaries + SMPTE-2084 (PQ) transfer, limited range — matches the P010
// the CSC produces (swscale BT.2020 on the CPU path; scale_vaapi pinned to bt2020 on the
// zero-copy path). The client decoder auto-detects PQ from the VUI.
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT2020;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084;
} else {
// We hand the encoder BT.709 *limited* NV12 (swscale CSC on the CPU path; scale_vaapi pinned
// to `out_color_matrix=bt709:out_range=limited` on the zero-copy path, with the full-range
// RGB input tagged), so signal that VUI — else the client decoder washes the picture out.
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT709;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT709;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709;
}
(*raw).pix_fmt = ffi::AVPixelFormat::AV_PIX_FMT_VAAPI;
(*raw).hw_device_ctx = ffi::av_buffer_ref(device_ref);
(*raw).hw_frames_ctx = ffi::av_buffer_ref(frames_ref);
let mut opts = Dictionary::new();
if ten_bit && codec == Codec::H265 {
// HEVC Main10. `hevc_vaapi` derives it from the P010 surfaces, but pin it explicitly so
// the depth is never silently dropped. (10-bit AV1 is input-driven — no profile knob.)
opts.set("profile", "main10");
}
// async_depth=1: `send_frame` blocks until THIS frame's ASIC encode completes — the lowest
// latency structure libavcodec's vaapi_encode offers. Measured on the 780M at 1440p60: depth 1
// = 8.3 ms end-to-end p50 vs depth 2 = 18 ms, because with depth ≥ 2 frame N's packet only
@@ -242,10 +267,59 @@ pub fn probe_can_encode(codec: Codec) -> bool {
let prev = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, 640, 480, 2) {
Ok(hw) => {
open_vaapi_encoder(codec, 640, 480, 30, 2_000_000, hw.device_ref, hw.frames_ref)
.is_ok()
}
Ok(hw) => open_vaapi_encoder(
codec,
640,
480,
30,
2_000_000,
hw.device_ref,
hw.frames_ref,
false,
)
.is_ok(),
Err(_) => false,
};
ffi::av_log_set_level(prev);
ok
}
}
/// Probe whether the active VAAPI GPU can encode **10-bit** (HEVC Main10 / 10-bit AV1) from P010
/// surfaces — the exact shape a live HDR session opens (P010 pool + Main10 profile + PQ VUI). The
/// driver rejects what the video engine can't do; the result is cached by the caller
/// ([`crate::can_encode_10bit`]), so a non-Main10 GPU resolves every session to 8-bit SDR before
/// the Welcome (honest downgrade).
pub fn probe_can_encode_10bit(codec: Codec) -> bool {
if !codec.supports_10bit() || codec == Codec::PyroWave {
return false;
}
if ffmpeg::init().is_err() {
return false;
}
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `av_log_{get,set}_level`
// only read/write libav's global integer log level (no pointer args). `VaapiHw::new` (an
// `unsafe fn`) builds a VAAPI device + P010 frames pool from the literal args and hands back a
// RAII handle; `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` —
// the two non-null refs `VaapiHw::new` just created, live locals for the whole match arm — and
// `av_buffer_ref`s them into the probe encoder. Both `hw` and the encoder drop (RAII) at arm end.
unsafe {
// A missing VA device / no Main10 entrypoint is an expected probe outcome — quiet ffmpeg's
// error for the probe, then restore the level.
let prev = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_P010LE, 640, 480, 2) {
Ok(hw) => open_vaapi_encoder(
codec,
640,
480,
30,
2_000_000,
hw.device_ref,
hw.frames_ref,
true,
)
.is_ok(),
Err(_) => false,
};
ffi::av_log_set_level(prev);
@@ -348,12 +422,21 @@ impl CpuInner {
bitrate_bps: u64,
) -> Result<Self> {
let src_pixel = vaapi_sws_src(format)?;
// A 10-bit HDR capture (X2RGB10/X2BGR10, PQ/BT.2020) uploads P010 and encodes Main10; the
// 8-bit paths keep NV12/BT.709 byte-for-byte unchanged.
let ten_bit = format.is_hdr_rgb10();
let staging_av = if ten_bit {
ffi::AVPixelFormat::AV_PIX_FMT_P010LE
} else {
ffi::AVPixelFormat::AV_PIX_FMT_NV12
};
const POOL: c_int = 16;
// SAFETY: `VaapiHw::new` (an `unsafe fn`) requires libav initialized — guaranteed because the
// only path here is `VaapiEncoder::open` → `ensure_inner` → `CpuInner::open`, and `open` ran
// `ffmpeg::init()`. The args are valid: NV12 sw_format, the validated positive `width`/`height`,
// pool=16. It returns a RAII `VaapiHw` that unrefs its two `AVBufferRef`s on drop.
let hw = unsafe { VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, width, height, POOL)? };
// `ffmpeg::init()`. The args are valid: an NV12/P010 sw_format, the validated positive
// `width`/`height`, pool=16. It returns a RAII `VaapiHw` that unrefs its two `AVBufferRef`s
// on drop.
let hw = unsafe { VaapiHw::new(staging_av, width, height, POOL)? };
// SAFETY: `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` — both
// non-null (`VaapiHw::new` guarantees it) and from the `hw` just built above, which is a live
// local that outlives this synchronous call. The fn `av_buffer_ref`s them into the encoder, so
@@ -368,16 +451,19 @@ impl CpuInner {
bitrate_bps,
hw.device_ref,
hw.frames_ref,
ten_bit,
)?
};
// swscale RGB→NV12, BT.709 limited (matches the VUI), no rescale.
// swscale RGB→NV12 (BT.709 limited) or X2RGB10→P010 (BT.2020 limited, HDR) — matches the
// VUI; no rescale.
let src_av = pixel_to_av(src_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dimensions and
// pixel formats. All four dims are the encoder's positive `width`/`height` cast to `c_int`;
// `src_av` is a valid `AVPixelFormat` (from `pixel_to_av` of the `vaapi_sws_src`-validated
// `src_pixel`), the dst is NV12. The three trailing pointers (srcFilter, dstFilter, param) are
// explicitly null = "use defaults", which the API documents as accepted. No Rust memory is
// borrowed — only by-value ints/enums — and the returned pointer is null-checked just below.
// `src_pixel`), the dst is NV12/P010. The three trailing pointers (srcFilter, dstFilter,
// param) are explicitly null = "use defaults", which the API documents as accepted. No Rust
// memory is borrowed — only by-value ints/enums — and the returned pointer is null-checked
// just below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
@@ -385,7 +471,7 @@ impl CpuInner {
src_av,
width as c_int,
height as c_int,
ffi::AVPixelFormat::AV_PIX_FMT_NV12,
staging_av,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
@@ -393,45 +479,54 @@ impl CpuInner {
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→NV12) failed");
bail!(
"sws_getContext(RGB→{})",
if ten_bit { "P010" } else { "NV12" }
);
}
// SAFETY: `sws` is the non-null `SwsContext` from `sws_getContext` above (the `is_null()`
// check immediately preceding returned false). `sws_getCoefficients(SWS_CS_ITU709)` returns a
// pointer into a libswscale static const coefficient table valid for the whole process, reused
// here for both the inverse (src) and forward (dst) matrices. `sws_setColorspaceDetails` only
// reads those tables and writes scalar CSC settings into `sws`; the table pointer outlives the
// synchronous call and no Rust memory is passed.
// check immediately preceding returned false). The coefficient table from
// `sws_getCoefficients` (ITU-709, or BT.2020 NCL for the HDR path — matching the VUI) is a
// libswscale static const valid for the whole process, reused here for both the inverse
// (src) and forward (dst) matrices. `sws_setColorspaceDetails` only reads those tables and
// writes scalar CSC settings into `sws`; the table pointer outlives the synchronous call and
// no Rust memory is passed.
unsafe {
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, 0, 0, 1 << 16, 1 << 16);
let cs = ffi::sws_getCoefficients(if ten_bit {
super::libav::SWS_CS_BT2020
} else {
SWS_CS_ITU709
});
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, 0, 0, 1 << 16, 1 << 16);
}
// SAFETY: `av_frame_alloc` returns a fresh, uniquely-owned heap `AVFrame` (null-checked — on
// null we free the already-built `sws` and bail). We then write the plain `format`/`width`/
// `height` fields through the non-null, properly-aligned `f` (sole owner, not yet shared).
// `av_frame_get_buffer(f, 0)` allocates backing storage for those dims/format; on failure we
// free `f` and `sws` (unwinding the half-built state) and bail. On success `f` is a fully-owned
// NV12 frame stored in `CpuInner.nv12` and freed once in `CpuInner::drop`. `f` is a unique
// fresh pointer, so none of these writes alias anything.
// NV12/P010 frame stored in `CpuInner.nv12` and freed once in `CpuInner::drop`. `f` is a
// unique fresh pointer, so none of these writes alias anything.
let nv12 = unsafe {
let f = ffi::av_frame_alloc();
if f.is_null() {
ffi::sws_freeContext(sws);
bail!("av_frame_alloc(NV12) failed");
bail!("av_frame_alloc(staging) failed");
}
(*f).format = ffi::AVPixelFormat::AV_PIX_FMT_NV12 as c_int;
(*f).format = staging_av as c_int;
(*f).width = width as c_int;
(*f).height = height as c_int;
if ffi::av_frame_get_buffer(f, 0) < 0 {
let mut f = f;
ffi::av_frame_free(&mut f);
ffi::sws_freeContext(sws);
bail!("av_frame_get_buffer(NV12) failed");
bail!("av_frame_get_buffer(staging) failed");
}
f
};
tracing::info!(
encoder = codec.vaapi_name(),
"VAAPI encode active ({width}x{height}@{fps}, CPU→NV12 upload path)"
"VAAPI encode active ({width}x{height}@{fps}, CPU→{} upload path)",
if ten_bit { "P010 (HDR10)" } else { "NV12" }
);
Ok(CpuInner {
enc,
@@ -563,6 +658,15 @@ impl DmabufInner {
) -> Result<Self> {
let drm_fourcc = pf_frame::drm_fourcc(format)
.ok_or_else(|| anyhow!("no DRM fourcc for {format:?} (VAAPI zero-copy)"))?;
// A 10-bit HDR capture (X2RGB10/X2BGR10 dmabufs, PQ/BT.2020) maps + CSCs to P010 and
// encodes Main10; the 8-bit paths keep the NV12/BT.709 graph byte-for-byte unchanged.
let ten_bit = format.is_hdr_rgb10();
let sw_format = match format {
PixelFormat::X2Rgb10 => ffi::AVPixelFormat::AV_PIX_FMT_X2RGB10LE,
PixelFormat::X2Bgr10 => ffi::AVPixelFormat::AV_PIX_FMT_X2BGR10LE,
// The 8-bit capture formats are all XR24-shaped packed RGB (the historical BGR0 view).
_ => ffi::AVPixelFormat::AV_PIX_FMT_BGR0,
};
let node = render_node();
// SAFETY: libav is initialized (`VaapiEncoder::open` ran `ffmpeg::init()` before
// `ensure_inner` → `DmabufInner::open`). Every raw pointer dereferenced below is either freshly
@@ -628,7 +732,7 @@ impl DmabufInner {
}
let fc = (*drm_frames).data as *mut ffi::AVHWFramesContext;
(*fc).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME;
(*fc).sw_format = ffi::AVPixelFormat::AV_PIX_FMT_BGR0; // packed XR24 RGB plane
(*fc).sw_format = sw_format; // packed XR24 RGB plane, or XR30/XB30 for HDR
(*fc).width = width as c_int;
(*fc).height = height as c_int;
if ffi::av_hwframe_ctx_init(drm_frames) < 0 {
@@ -715,14 +819,24 @@ impl DmabufInner {
}
init!(src, ptr::null(), "buffer");
init!(hwmap, c"mode=read".as_ptr(), "hwmap");
// Pin the VPP's output colour to what the encoder's VUI signals (BT.709 limited).
// Without the explicit options the conversion matrix is whatever the driver defaults
// to for an unspecified output (Mesa: BT.601) — a hue shift against the signaled VUI.
init!(
scale,
c"format=nv12:out_color_matrix=bt709:out_range=limited".as_ptr(),
"scale_vaapi"
);
// Pin the VPP's output colour to what the encoder's VUI signals (BT.709 limited SDR,
// or BT.2020 limited P010 for HDR — the PQ transfer is per-channel and rides through
// the matrix untouched). Without the explicit options the conversion matrix is
// whatever the driver defaults to for an unspecified output (Mesa: BT.601) — a hue
// shift against the signaled VUI.
if ten_bit {
init!(
scale,
c"format=p010:out_color_matrix=bt2020:out_range=limited".as_ptr(),
"scale_vaapi"
);
} else {
init!(
scale,
c"format=nv12:out_color_matrix=bt709:out_range=limited".as_ptr(),
"scale_vaapi"
);
}
init!(sink, ptr::null(), "buffersink");
let link = |a: *mut ffi::AVFilterContext, b: *mut ffi::AVFilterContext| -> c_int {
@@ -766,6 +880,7 @@ impl DmabufInner {
bitrate_bps,
vaapi_device,
nv12_ctx,
ten_bit,
) {
Ok(enc) => enc,
Err(e) => {
@@ -779,7 +894,8 @@ impl DmabufInner {
tracing::info!(
encoder = codec.vaapi_name(),
"VAAPI encode active ({width}x{height}@{fps}, zero-copy dmabuf → GPU NV12)"
"VAAPI encode active ({width}x{height}@{fps}, zero-copy dmabuf → GPU {})",
if ten_bit { "P010 (HDR10)" } else { "NV12" }
);
Ok(DmabufInner {
enc,
@@ -987,8 +1103,22 @@ impl VaapiEncoder {
bit_depth: u8,
chroma: super::ChromaFormat,
) -> Result<Self> {
if bit_depth != 8 {
tracing::warn!(bit_depth, "VAAPI 10-bit not yet wired — encoding 8-bit");
// 10-bit rides on the captured format: an HDR capture (X2RGB10/X2BGR10) opens the P010 /
// Main10 / PQ-VUI variant of whichever inner path the first frame selects. A 10-bit
// request whose capture stayed SDR honestly encodes 8-bit; the reverse (PQ frames on an
// 8-bit session) is refused so PQ content is never mislabeled BT.709.
if format.is_hdr_rgb10() && bit_depth != 10 {
bail!(
"captured 10-bit HDR frames ({format:?}) on an {bit_depth}-bit VAAPI session — \
refusing to mislabel PQ content"
);
}
if bit_depth == 10 && !format.is_hdr_rgb10() {
tracing::warn!(
bit_depth,
?format,
"10-bit requested but the capture stayed SDR — encoding 8-bit"
);
}
// VAAPI 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel hardware in the
// lab exposes a HEVC 4:4:4 encode entrypoint, and the probe returns false so the host never
+268
View File
@@ -0,0 +1,268 @@
//! Shared PyroWave AU wire-framing (design/pyrowave-codec-plan.md §4.4) — the single source of
//! truth for the on-wire access-unit shape, used by BOTH the Linux (dmabuf/CSC) and Windows (NV12
//! zero-copy) host encoders. It turns pyrowave's packetized bitstream into either the **dense**
//! single-packet AU or the **datagram-aligned** windowed AU. Pure (no GPU/FFI) so it is unit-tested
//! on any platform and both encoders emit byte-identical framing — the clients parse this exact
//! layout, so it must stay in ONE place.
//!
//! Datagram-aligned AU: each `chunk`-sized window opens with a 4-byte prefix (`u16` used-length +
//! `u16` kind) and carries either WHOLE self-delimiting codec packets (`WIN_PACKED` — several small
//! ones share a window) or one fragment of an oversized ATOMIC packet (a `FRAG` chain — pyrowave's
//! 32×32 blocks are atomic and can exceed a shard). A lost shard zeroes its window (`used = 0`) so
//! the receiver skips it and drops any fragment chain it interrupts. Padding after `used` is zeroed.
/// The 4-byte per-window framing prefix (`u16` used-length + `u16` kind).
pub(crate) const WINDOW_PREFIX: usize = 4;
/// Window kinds: whole packets / an oversized packet's fragments.
const WIN_PACKED: u16 = 0;
const WIN_FRAG_FIRST: u16 = 1;
const WIN_FRAG_CONT: u16 = 2;
const WIN_FRAG_LAST: u16 = 3;
/// The packetize boundary to request from pyrowave: for a `wire_chunk` shard it is the shard payload
/// minus the 4-byte window prefix (so a whole codec packet + its prefix fits one shard); for the
/// dense case it is the whole-bitstream cap (one packet per AU).
pub(crate) fn packet_boundary(wire_chunk: Option<usize>, dense_cap: usize) -> usize {
wire_chunk.map(|c| c - WINDOW_PREFIX).unwrap_or(dense_cap)
}
/// Patch the frame's `BitstreamSequenceHeader` to signal `ycbcr_range = LIMITED`. pyrowave's C API
/// fills the header with `= {}` (all VUI fields zeroed) and offers NO way to set colour/range, so it
/// signals `ycbcr_range = 0 = YCBCR_RANGE_FULL` — but BOTH host CSCs (`rgb2yuv.comp` on Linux, the
/// D3D11 `BgraToYuvPlanes` on Windows) always emit BT.709 **LIMITED** YCbCr (black = Y16). A client
/// that honours the VUI (the Apple wavelet decoder reads `(word1 >> 30) & 1`) then skips the
/// limited→full expansion and shows washed-out, raised blacks. Patching the bit makes the bitstream
/// HONEST for every client — clients that hardcode limited (the Vulkan `video_pyrowave` path) are
/// unaffected, and pyrowave's own decode ignores the flag (it reconstructs raw YCbCr). The other
/// zeroed VUI fields (BT.709 primaries / transform / transfer) are already correct.
///
/// `seq_offset` is the byte offset of the frame's 8-byte `BitstreamSequenceHeader` in `bitstream` —
/// the SOF packet's offset. The colour bits live in the little-endian second word's top byte
/// (`seq_offset + 7`): `color_primaries` bit 27 (`0x08`), `transfer_function` bit 28 (`0x10`),
/// `ycbcr_transform` bit 29 (`0x20`), `ycbcr_range` bit 30 (`0x40`); `chroma_siting` bit 31 stays 0
/// (CENTER — the pyrowave CSCs use the centre-sited 2×2 box, unlike the left-cosited P010 path).
/// Range is ALWAYS stamped LIMITED (both CSCs emit studio range); `bt2020_pq` additionally stamps
/// BT.2020 primaries + PQ transfer + BT.2020 matrix — upstream's own enum semantics
/// (`pyrowave_common.hpp`), matching the session's negotiated `ColorInfo`.
pub(crate) fn stamp_color_bits(bitstream: &mut [u8], seq_offset: usize, bt2020_pq: bool) {
if let Some(b) = bitstream.get_mut(seq_offset + 7) {
*b |= 0x40;
if bt2020_pq {
*b |= 0x08 | 0x10 | 0x20;
}
}
}
/// The wavelet block space's total 32x32-block count for a mode — the exact counting walk of
/// upstream `WaveletBuffers::init_block_meta` (also ported to the Apple `WaveletLayout`, whose
/// golden tests pin it against real host AUs). Needed because the vendored RDO pass packs the
/// block index into 16 bits (`RDOperation.block_offset_saving` — see
/// `patches/0002-rdo-saving-clamp.patch`): a mode whose count exceeds `u16::MAX` would wrap
/// inside the rate controller, so the host guards such modes out (≈8K 4:4:4 territory).
pub(crate) fn block_count_32x32(width: u32, height: u32, chroma444: bool) -> u32 {
const LEVELS: u32 = 5;
let align = |v: u32| ((v + 31) & !31).max(128);
let (aw, ah) = (align(width), align(height));
let mut count = 0u32;
for level in (0..LEVELS).rev() {
let lw = (aw / 2) >> level;
let lh = (ah / 2) >> level;
let blocks_x8 = lw.div_ceil(8);
let blocks_y8 = lh.div_ceil(8);
let per_band = blocks_x8.div_ceil(4) * blocks_y8.div_ceil(4);
let bands = if level == LEVELS - 1 { 4 } else { 3 };
for component in 0..3u32 {
if level == 0 && component != 0 && !chroma444 {
continue;
}
count += per_band * bands;
}
}
count
}
/// Frame pyrowave's `packets` (each an `(offset, size)` into `bitstream`) into the wire AU.
/// `wire_chunk = None` copies the single dense packet; `Some(chunk)` produces the windowed
/// datagram-aligned AU (a whole number of `chunk`-sized windows).
pub(crate) fn build_au(
packets: &[(usize, usize)],
bitstream: &[u8],
wire_chunk: Option<usize>,
) -> Vec<u8> {
let Some(chunk) = wire_chunk else {
// Dense (default): boundary == whole buffer → the AU is exactly one pyrowave packet.
let (off, size) = packets[0];
return bitstream[off..off + size].to_vec();
};
let payload_max = chunk - WINDOW_PREFIX;
let mut au: Vec<u8> = Vec::with_capacity((packets.len() + 1) * chunk);
// The currently-open PACKED window: (start offset of its prefix, bytes used).
let mut open: Option<(usize, usize)> = None;
let close = |au: &mut Vec<u8>, open: &mut Option<(usize, usize)>, chunk: usize| {
if let Some((start, used)) = open.take() {
au[start..start + 2].copy_from_slice(&(used as u16).to_le_bytes());
au[start + 2..start + 4].copy_from_slice(&WIN_PACKED.to_le_bytes());
au.resize(start + chunk, 0);
}
};
for &(off, size) in packets {
let bytes = &bitstream[off..off + size];
if size <= payload_max {
let fits = open.is_some_and(|(_, used)| used + size <= payload_max);
if !fits {
close(&mut au, &mut open, chunk);
let start = au.len();
au.resize(start + WINDOW_PREFIX, 0);
open = Some((start, 0));
}
au.extend_from_slice(bytes);
if let Some((_, used)) = open.as_mut() {
*used += size;
}
} else {
// Oversized packet: its own FRAG chain of full windows.
close(&mut au, &mut open, chunk);
let mut o = 0usize;
while o < size {
let take = (size - o).min(payload_max);
let kind = if o == 0 {
WIN_FRAG_FIRST
} else if o + take == size {
WIN_FRAG_LAST
} else {
WIN_FRAG_CONT
};
let start = au.len();
au.resize(start + WINDOW_PREFIX, 0);
au[start..start + 2].copy_from_slice(&(take as u16).to_le_bytes());
au[start + 2..start + 4].copy_from_slice(&kind.to_le_bytes());
au.extend_from_slice(&bytes[o..o + take]);
au.resize(start + chunk, 0);
o += take;
}
}
}
close(&mut au, &mut open, chunk);
au
}
#[cfg(test)]
mod tests {
use super::*;
/// Walk a windowed AU back into the flat codec-packet stream (the client's parse), asserting the
/// framing invariants the encoder promises: whole windows, in-bounds `used`, zeroed padding.
fn walk(au: &[u8], chunk: usize) -> Vec<u8> {
assert_eq!(au.len() % chunk, 0, "AU is a whole number of windows");
let mut out = Vec::new();
let mut frag: Vec<u8> = Vec::new();
for win in au.chunks(chunk) {
let used = u16::from_le_bytes([win[0], win[1]]) as usize;
let kind = u16::from_le_bytes([win[2], win[3]]);
assert!(WINDOW_PREFIX + used <= win.len(), "window overrun");
assert!(
win[WINDOW_PREFIX + used..].iter().all(|&b| b == 0),
"non-zero padding after used"
);
let body = &win[WINDOW_PREFIX..WINDOW_PREFIX + used];
match kind {
0 => out.extend_from_slice(body),
1 => frag = body.to_vec(),
2 => frag.extend_from_slice(body),
3 => {
frag.extend_from_slice(body);
out.extend_from_slice(&frag);
frag.clear();
}
k => panic!("unknown window kind {k}"),
}
}
out
}
#[test]
fn dense_is_the_single_packet() {
let bs = (0u8..=200).collect::<Vec<u8>>();
let au = build_au(&[(10, 50)], &bs, None);
assert_eq!(au, bs[10..60]);
}
#[test]
fn packed_windows_pack_small_packets_and_reconstruct() {
// Three small packets that share windows; walking must reproduce them concatenated in order.
let bs: Vec<u8> = (0..255u32).map(|i| i as u8).collect();
let packets = [(0, 20), (20, 20), (40, 100)];
let chunk = 64; // payload_max = 60
let au = build_au(&packets, &bs, Some(chunk));
let flat = walk(&au, chunk);
let mut expect = Vec::new();
for &(o, s) in &packets {
expect.extend_from_slice(&bs[o..o + s]);
}
assert_eq!(flat, expect);
}
#[test]
fn oversized_packet_fragments_and_reassembles() {
// One atomic packet larger than a window → a FRAG chain the walk reassembles exactly.
let bs: Vec<u8> = (0..1000u32).map(|i| i as u8).collect();
let chunk = 64; // payload_max = 60
let au = build_au(&[(0, 500)], &bs, Some(chunk));
assert_eq!(walk(&au, chunk), bs[0..500]);
}
#[test]
fn boundary_reserves_the_window_prefix() {
assert_eq!(packet_boundary(Some(1408), 999_999), 1404);
assert_eq!(packet_boundary(None, 777), 777);
}
#[test]
fn block_count_matches_the_apple_layout_invariant() {
// 256x144 (the golden-fixture geometry, aligned 256x160): recompute via the same walk
// the validated Apple WaveletLayout uses and pin a few mode-level facts.
let manual = |w: u32, h: u32, c444: bool| {
let align = |v: u32| ((v + 31) & !31).max(128);
let (aw, ah) = (align(w), align(h));
let mut n = 0u32;
for level in (0..5u32).rev() {
let per = (((aw / 2) >> level).div_ceil(8).div_ceil(4))
* (((ah / 2) >> level).div_ceil(8).div_ceil(4));
let bands = if level == 4 { 4 } else { 3 };
for c in 0..3 {
if level == 0 && c != 0 && !c444 {
continue;
}
n += per * bands;
}
}
n
};
for (w, h) in [(256, 144), (1920, 1080), (3840, 2160), (7680, 4320)] {
assert_eq!(block_count_32x32(w, h, false), manual(w, h, false));
assert_eq!(block_count_32x32(w, h, true), manual(w, h, true));
}
// 4:4:4 fits comfortably at 4K; the 16-bit RDO block index wraps around 8K 4:4:4.
assert!(block_count_32x32(3840, 2160, true) <= u16::MAX as u32);
assert!(block_count_32x32(7680, 4320, true) > u16::MAX as u32);
assert!(block_count_32x32(7680, 4320, false) <= u16::MAX as u32);
}
#[test]
fn stamp_color_bits_sets_range_and_hdr_bits() {
let mut bs = vec![0u8; 16];
stamp_color_bits(&mut bs, 0, false);
// ycbcr_range = bit 30 of the LE second word = bit 6 of byte 7 (0x40); nothing else touched.
assert_eq!(bs[7], 0x40);
assert!(bs[..7].iter().all(|&b| b == 0));
assert!(bs[8..].iter().all(|&b| b == 0));
// Idempotent; an out-of-range offset is a silent no-op (never panics).
stamp_color_bits(&mut bs, 0, false);
assert_eq!(bs[7], 0x40);
stamp_color_bits(&mut bs, 100, false);
// HDR adds BT.2020 primaries (0x08) + PQ transfer (0x10) + BT.2020 matrix (0x20);
// chroma_siting (0x80) stays CENTER.
stamp_color_bits(&mut bs, 0, true);
assert_eq!(bs[7], 0x78);
}
}
+7 -4
View File
@@ -181,10 +181,13 @@ impl Encoder for OpenH264Encoder {
PixelFormat::Bgr => (3, 2, 1, 0),
PixelFormat::Rgba | PixelFormat::Rgbx => (4, 0, 1, 2),
PixelFormat::Bgra | PixelFormat::Bgrx => (4, 2, 1, 0),
// 10-bit HDR comes only from the GPU NVENC path; the software 8-bit H.264 encoder
// can't represent it (and never receives it — the capturer pairs Rgb10a2 with NVENC).
PixelFormat::Rgb10a2 => {
anyhow::bail!("software H.264 encoder cannot encode 10-bit HDR (Rgb10a2)")
// 10-bit HDR comes only from the GPU paths; the software 8-bit H.264 encoder can't
// represent it (and never receives it — HDR is never negotiated on a software host).
PixelFormat::Rgb10a2 | PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => {
anyhow::bail!(
"software H.264 encoder cannot encode 10-bit HDR ({:?})",
self.src_format
)
}
// NV12/P010 are GPU-resident video-processor outputs for the NVENC path; the software
// encoder never receives them (it only gets CPU RGB frames).
+4
View File
@@ -2788,6 +2788,7 @@ mod tests {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
@@ -2973,6 +2974,7 @@ mod tests {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
@@ -3114,6 +3116,7 @@ mod tests {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
@@ -3261,6 +3264,7 @@ mod tests {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
@@ -136,7 +136,15 @@ fn sws_src(format: PixelFormat) -> Result<Pixel> {
PixelFormat::Rgba => Pixel::RGBA,
PixelFormat::Rgb => Pixel::RGB24,
PixelFormat::Bgr => Pixel::BGR24,
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => {
// X2Rgb10/X2Bgr10 are the Linux GNOME 50 HDR screencast formats — the Windows HDR path
// stays Rgb10a2/P010, so they can't reach this capture-side conversion. Listed explicitly
// (not via `_`) so the next PixelFormat addition breaks this match again on purpose.
PixelFormat::Nv12
| PixelFormat::P010
| PixelFormat::Rgb10a2
| PixelFormat::Yuv444
| PixelFormat::X2Rgb10
| PixelFormat::X2Bgr10 => {
bail!("ffmpeg_win swscale path supports packed RGB/BGR only; got {format:?}")
}
})
@@ -1811,6 +1811,7 @@ mod tests {
payload: FramePayload::D3d11(D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
@@ -1913,6 +1914,7 @@ mod tests {
payload: FramePayload::D3d11(D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
File diff suppressed because it is too large Load Diff
+1
View File
@@ -1746,6 +1746,7 @@ mod tests {
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
+111 -19
View File
@@ -48,7 +48,19 @@ impl Codec {
} else {
0u8
};
#[cfg(not(all(target_os = "linux", feature = "pyrowave")))]
// Windows: the wavelet encoder rides on top of whatever GPU backend the box has (NVENC/AMF/
// QSV) — it opens its OWN Vulkan device by the render GPU's vendor/device-id and
// zero-copy-imports the capturer's NV12 D3D11 texture, so the H.26x backend is irrelevant to
// it. Only a software/GPU-less host keeps the bit off (no Vulkan GPU to open). Whether the
// Session-0 external-memory import actually works is confirmed at encoder open
// (`pyrowave_device_confirm_interop_support`); a failed open renegotiates to HEVC.
#[cfg(all(target_os = "windows", feature = "pyrowave"))]
let pyro = if windows_resolved_backend() != WindowsBackend::Software {
punktfunk_core::quic::CODEC_PYROWAVE
} else {
0u8
};
#[cfg(not(all(any(target_os = "linux", target_os = "windows"), feature = "pyrowave")))]
let pyro = 0u8;
let base = (|| {
/// The static GPU superset (H.264 | HEVC | AV1) — mirrors the GameStream
@@ -236,10 +248,11 @@ fn open_video_backend(
if fps == 0 || fps > 1000 {
anyhow::bail!("invalid refresh/fps {fps}: must be 1..=1000 Hz");
}
// 4:4:4 is HEVC-only. The negotiator should never pass `Yuv444` for another codec (it gates on
// `codec == H265`), but defend the contract here so a future caller can't silently emit a stream
// no decoder expects: a non-HEVC 4:4:4 request degrades to 4:2:0 with a warning.
let chroma = if chroma.is_444() && codec != Codec::H265 {
// 4:4:4 is HEVC- and PyroWave-only. The negotiator should never pass `Yuv444` for another
// codec (it gates on the codec + `can_encode_444`), but defend the contract here so a future
// caller can't silently emit a stream no decoder expects: an unsupported 4:4:4 request
// degrades to 4:2:0 with a warning.
let chroma = if chroma.is_444() && codec != Codec::H265 && codec != Codec::PyroWave {
tracing::warn!(
?codec,
"4:4:4 requested for a non-HEVC codec — encoding 4:2:0"
@@ -255,7 +268,7 @@ fn open_video_backend(
if codec == Codec::PyroWave {
#[cfg(feature = "pyrowave")]
{
return pyrowave::PyroWaveEncoder::open(width, height, fps, bitrate_bps)
return pyrowave::PyroWaveEncoder::open(width, height, fps, bitrate_bps, chroma)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "pyrowave"));
}
#[cfg(not(feature = "pyrowave"))]
@@ -275,8 +288,14 @@ fn open_video_backend(
// stream never dies over the new path. `format`/`bit_depth`/`chroma` only matter to VAAPI —
// the Vulkan backend imports the dmabuf and does its own 8-bit 4:2:0 CSC.
let open_amd_intel = || -> Result<(Box<dyn Encoder>, &'static str)> {
// An HDR session (10-bit + a PQ/BT.2020 capture format) must skip the Vulkan Video
// backend — it hardcodes an 8-bit 4:2:0 BT.709 CSC — and take the libav VAAPI path,
// which has the P010/Main10/PQ wiring. SDR sessions keep the Vulkan default.
#[cfg(feature = "vulkan-encode")]
if matches!(codec, Codec::H265 | Codec::Av1) && vulkan_encode_enabled() {
if matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
&& !(bit_depth == 10 && format.is_hdr_rgb10())
{
match vulkan_video::VulkanVideoEncoder::open(codec, width, height, fps, bitrate_bps)
{
Ok(e) => {
@@ -357,8 +376,17 @@ fn open_video_backend(
that ALSO preferred CODEC_PYROWAVE can display it (lab override; \
normal sessions negotiate it instead)"
);
pyrowave::PyroWaveEncoder::open(width, height, fps, bitrate_bps)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "pyrowave"))
// The lab override forces the wavelet stream onto a session negotiated for
// another codec — that session's chroma may be HEVC-4:4:4, which the
// pyrowave encoder doesn't do yet, so pin the override to 4:2:0.
pyrowave::PyroWaveEncoder::open(
width,
height,
fps,
bitrate_bps,
ChromaFormat::Yuv420,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "pyrowave"))
}
#[cfg(not(feature = "pyrowave"))]
{
@@ -399,10 +427,29 @@ fn open_video_backend(
}
#[cfg(target_os = "windows")]
{
// The Windows host leg is blocked on the .173 D3D11-interop debt (plan Phase 0 §3);
// host_wire_caps never advertises the bit here, so this only guards a forged preference.
// A NEGOTIATED PyroWave session (client advertised + preferred it) routes straight to the
// NV12 zero-copy wavelet backend (design/pyrowave-windows-host-zerocopy.md) — placed FIRST,
// like the Linux branch. It opens its own Vulkan device by the render GPU's vendor/device-id
// and imports the capturer's shared NV12 texture; the H.26x backend selection below is moot.
if codec == Codec::PyroWave {
anyhow::bail!("PyroWave host encode is not available on Windows yet");
#[cfg(feature = "pyrowave")]
{
let _ = (format, cuda);
return pyrowave::PyroWaveEncoder::open(
width,
height,
fps,
bitrate_bps,
chroma,
bit_depth,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "pyrowave"));
}
#[cfg(not(feature = "pyrowave"))]
anyhow::bail!(
"session negotiated PyroWave but this host was built without --features \
punktfunk-host/pyrowave (the advertisement bit should not have been set)"
);
}
let _ = cuda; // always false on Windows (no Cuda payload)
// NVIDIA → NVENC (direct SDK), AMD → AMF, Intel → QSV (both libavcodec), else → software
@@ -835,6 +882,13 @@ pub fn vaapi_codec_support() -> CodecSupport {
pub fn can_encode_444(codec: Codec) -> bool {
use std::collections::HashMap;
use std::sync::{Mutex, OnceLock};
if codec == Codec::PyroWave {
// PyroWave does its own RGB→YCbCr CSC (capture always hands it a full-chroma source),
// so 4:4:4 needs no GPU encode probe — only the full-res-chroma CSC variant:
// `rgb2yuv444.comp` on Linux (Phase 2) and the mode-aware `BgraToYuvPlanes` on
// Windows (Phase 3) — both landed (design/pyrowave-444-hdr.md).
return true;
}
if codec != Codec::H265 {
return false;
}
@@ -908,11 +962,10 @@ pub fn can_encode_444(_codec: Codec) -> bool {
/// Backend truth: Windows **NVENC** queries the per-codec `NV_ENC_CAPS_SUPPORT_10BIT_ENCODE` cap;
/// native **AMF** `Init`s a tiny P010 encoder with the 10-bit profile props (the driver rejects
/// what the VCN can't do). **QSV** stays `false` until validated on Intel glass — the libavcodec
/// Main10 incantation can silently encode 8-bit, the same stance as its 4:4:4 probe. Every
/// **Linux** backend is `false` today: direct-NVENC/CUDA pins 8-bit until a P010 capture path
/// exists (Phase 5.1), libav `hevc_nvenc` needs a 10-bit input format the capturer never feeds,
/// VAAPI 10-bit isn't wired, and Vulkan-video hardcodes 8-bit — so Linux hosts honestly negotiate
/// 8-bit SDR.
/// Main10 incantation can silently encode 8-bit, the same stance as its 4:4:4 probe. **Linux**
/// probes a tiny real Main10 open on the auto-resolved backend — libav NVENC (the HDR X2RGB10→
/// P010 swscale path) or VAAPI (P010 pool + Main10) — for the GNOME 50+ HDR portal capture;
/// the direct-SDK CUDA path and Vulkan-video stay 8-bit and a 10-bit session routes around them.
#[cfg(any(target_os = "linux", target_os = "windows"))]
pub fn can_encode_10bit(codec: Codec) -> bool {
use std::collections::HashMap;
@@ -920,6 +973,12 @@ pub fn can_encode_10bit(codec: Codec) -> bool {
if !codec.supports_10bit() {
return false;
}
if codec == Codec::PyroWave {
// PyroWave needs no GPU encode probe (the wavelet is depth-agnostic) — only the HDR
// capture CSC (scRGB FP16 → 16-bit studio-code planes), which exists on the Windows
// IDD-push path only (design/pyrowave-444-hdr.md Phase 3; Linux capture has no HDR).
return cfg!(target_os = "windows");
}
// Cached per (selected GPU, codec) — a web-console preference change re-probes on the newly
// selected adapter before the next Welcome, mirroring `can_encode_444`.
static CACHE: OnceLock<Mutex<HashMap<(String, &'static str), bool>>> = OnceLock::new();
@@ -931,8 +990,18 @@ pub fn can_encode_10bit(codec: Codec) -> bool {
let supported = {
#[cfg(target_os = "linux")]
{
// No Linux backend encodes 10-bit yet (see the fn doc) — never negotiate it.
false
// NVENC (libav, the HDR P010 swscale path) or VAAPI (P010 upload / dmabuf graph),
// probed by opening a tiny real Main10 encoder — the same honesty contract as
// `can_encode_444`. Vulkan-video and the direct-SDK CUDA path stay 8-bit; a 10-bit
// session routes around them (see `open_video_backend`). NOTE: encode capability is
// only half the Linux gate — the capture side (GNOME 50+ portal monitor in HDR mode)
// is resolved separately by the host (`capturer_supports_hdr` / the GameStream RTSP
// honor), since this probe can't know what the compositor will negotiate.
if linux_auto_is_vaapi() {
vaapi::probe_can_encode_10bit(codec)
} else {
linux::probe_can_encode_10bit(codec)
}
}
#[cfg(target_os = "windows")]
{
@@ -1260,6 +1329,29 @@ mod vk_util;
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[path = "enc/linux/pyrowave.rs"]
mod pyrowave;
// The Windows PyroWave encoder — NV12 zero-copy D3D11→Vulkan via pyrowave's own compat device
// (design/pyrowave-windows-host-zerocopy.md). Same module name as the Linux one (per-platform
// `#[path]`, mutually-exclusive cfg) so `crate::pyrowave::*` is flat on both.
#[cfg(all(target_os = "windows", feature = "pyrowave"))]
#[path = "enc/windows/pyrowave.rs"]
mod pyrowave;
// Shared PyroWave AU wire-framing (§4.4) — the single source of truth both platform backends emit,
// so the on-wire access-unit layout the clients parse can never drift between Linux and Windows.
#[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "pyrowave"))]
#[path = "enc/pyrowave_wire.rs"]
mod pyrowave_wire;
/// Whether a PyroWave mode fits the vendored rate controller's packed 16-bit block index
/// (`patches/0002-rdo-saving-clamp.patch` note): false ≈ 8K-class 4:4:4. The negotiator
/// downgrades such a session to 4:2:0 before the Welcome; the encoders also refuse outright.
#[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "pyrowave"))]
pub fn pyrowave_mode_fits_rdo(width: u32, height: u32, chroma444: bool) -> bool {
pyrowave_wire::block_count_32x32(width, height, chroma444) <= u16::MAX as u32
}
#[cfg(not(all(any(target_os = "linux", target_os = "windows"), feature = "pyrowave")))]
pub fn pyrowave_mode_fits_rdo(_width: u32, _height: u32, _chroma444: bool) -> bool {
false
}
#[cfg(test)]
mod tests {
+26 -1
View File
@@ -34,10 +34,35 @@ pub struct WinCaptureTarget {
pub wudf_pid: u32,
}
/// A GPU-resident captured texture (future NVENC-D3D11 zero-copy path).
/// The PyroWave (Windows) zero-copy sharing payload attached to a captured frame: the SECOND plane
/// texture + the cross-device fence the wavelet encoder needs (design/pyrowave-windows-host-
/// zerocopy.md). The wavelet encoder ingests **two SEPARATE** shareable plane textures — the full-res
/// `R8_UNORM` **Y** rides [`D3d11Frame::texture`], and the half-res `R8G8_UNORM` **CbCr** rides
/// [`cbcr`](Self::cbcr) — because importing a single *planar* NV12 texture into Vulkan is unreliable
/// on NVIDIA at arbitrary sizes; separate single/two-component textures import reliably. `None` on
/// every non-PyroWave frame (NVENC/AMF/QSV encode the in-place NV12/BGRA and need no cross-device
/// fence). The encoder makes each texture's shared handle on demand.
pub struct PyroFrameShare {
/// The half-res `R8G8_UNORM` interleaved CbCr plane (created `SHARED | SHARED_NTHANDLE`). The
/// full-res Y plane is [`D3d11Frame::texture`].
pub cbcr: ID3D11Texture2D,
/// The shared D3D11/D3D12 **fence** NT handle (raw), passed on EVERY frame; the encoder imports
/// it (duplicating) whenever it has no timeline yet (first frame or after an encoder rebuild).
pub fence_handle: Option<isize>,
/// The fence value the capturer signalled after THIS frame's convert. The encoder's Vulkan
/// acquire waits on it, so the wavelet read is ordered after the D3D11 CSC.
pub fence_value: u64,
}
/// A GPU-resident captured texture (the Windows zero-copy path: NVENC/AMF/QSV encode it in place;
/// the PyroWave backend imports it — plus the second plane in [`pyro`](Self::pyro) — into its own
/// Vulkan device). For a PyroWave frame, `texture` is the full-res `R8_UNORM` Y plane.
pub struct D3d11Frame {
pub texture: ID3D11Texture2D,
pub device: ID3D11Device,
/// PyroWave zero-copy sharing info (the CbCr plane + fence); `None` unless this is a PyroWave
/// session. See [`PyroFrameShare`].
pub pyro: Option<PyroFrameShare>,
}
// SAFETY: `D3d11Frame` owns an `ID3D11Texture2D` + `ID3D11Device`, which are COM interface pointers.
// D3D11 devices/resources use thread-safe (interlocked) COM reference counting, and the device is
+31
View File
@@ -56,6 +56,19 @@ pub enum PixelFormat {
/// `DeviceBuffer::yuv444` — three full-res planes stacked in one allocation); NVENC encodes
/// it natively under the Range-Extensions profile. Never a CPU payload.
Yuv444,
/// 10-bit RGB packed `x:R:G:B 2:10:10:10` little-endian (SPA `xRGB_210LE`, DRM `XRGB2101010` /
/// `XR30`, ffmpeg `x2rgb10le`, NVENC `ARGB10`) — as an LE u32: B in bits 0-9, G 10-19, R 20-29.
/// The Linux GNOME 50+ HDR screencast source format: Mutter advertises it (with BT.2020
/// primaries + SMPTE ST.2084 PQ transfer) for a monitor in HDR mode, so the samples are
/// PQ-encoded BT.2020 RGB. Linux-only; the Windows HDR path stays `Rgb10a2`/`P010`.
X2Rgb10,
/// 10-bit RGB packed `x:B:G:R 2:10:10:10` little-endian (SPA `xBGR_210LE`, DRM `XBGR2101010` /
/// `XB30`, ffmpeg `x2bgr10le`, NVENC `ABGR10`) — as an LE u32: R in bits 0-9, G 10-19, B 20-29;
/// the same memory layout as the Windows [`Rgb10a2`](Self::Rgb10a2) (DXGI `R10G10B10A2`). The
/// second GNOME 50+ HDR screencast format (same PQ/BT.2020 colorimetry as
/// [`X2Rgb10`](Self::X2Rgb10)); kept separate from `Rgb10a2` so the Linux and Windows HDR
/// paths stay independently greppable.
X2Bgr10,
}
impl PixelFormat {
@@ -67,6 +80,12 @@ impl PixelFormat {
_ => 4,
}
}
/// True for the packed 10-bit RGB layouts a Linux HDR (BT.2020 PQ) capture negotiates —
/// the formats that make a session's encode bit depth 10 (HEVC Main10 / 10-bit AV1).
pub fn is_hdr_rgb10(self) -> bool {
matches!(self, PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10)
}
}
/// DRM FourCC for a packed 32-bit format name (little-endian, e.g. `b"XR24"`).
@@ -86,6 +105,9 @@ pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020).
X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010
X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format.
@@ -115,6 +137,13 @@ pub struct OutputFormat {
/// Linux it forces the CPU RGB path the encoder swscales to `YUV444P`. `false` on every
/// 4:2:0 session.
pub chroma_444: bool,
/// A PyroWave (wavelet) session on Windows: the IDD-push capturer must make its NV12 out-ring
/// **shareable** (`SHARED | SHARED_NTHANDLE`) and signal a **shared fence** after each convert,
/// so the pyrowave encoder can zero-copy-import the texture into its own Vulkan device
/// (design/pyrowave-windows-host-zerocopy.md). Also forces the NV12 4:2:0 SDR convert branch
/// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the
/// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture).
pub pyrowave: bool,
}
impl OutputFormat {
@@ -130,6 +159,8 @@ impl OutputFormat {
hdr,
// The GameStream + spike paths are always 4:2:0 (4:4:4 is punktfunk/1-native only).
chroma_444: false,
// GameStream never negotiates PyroWave (native punktfunk/1 only).
pyrowave: false,
}
}
}
+15 -3
View File
@@ -76,10 +76,16 @@ pub struct HostConfig {
/// backend (the legacy SudoVDA backend was removed), so this is currently informational — kept for the
/// shipped `host.env` and as a forward seam if a second backend is ever added.
pub vdisplay: Option<String>,
/// `PUNKTFUNK_GAMESCOPE_STEAM` — opt the bare headless gamescope spawn into its Steam
/// integration mode (`--steam`). Managed gamescope-session-plus/SteamOS sessions own their
/// own flags and do not consult this.
/// `PUNKTFUNK_GAMESCOPE_STEAM` — force the bare headless gamescope spawn into its Steam
/// integration mode (`--steam`) for EVERY launch. A Steam title auto-enables `--steam` on its
/// own regardless of this knob; it exists to force it on for non-Steam launches too. Managed
/// gamescope-session-plus/SteamOS sessions own their own flags and do not consult this.
pub gamescope_steam: bool,
/// `PUNKTFUNK_GAMESCOPE_GRAB_CURSOR` — add `--force-grab-cursor` to the bare headless gamescope
/// spawn for an actual game launch, forcing relative-mouse capture so FPS mouselook works over the
/// injected pointer. Default OFF: it forces relative mode, which breaks absolute-pointer titles
/// and menus, so it's opt-in per host until validated on-glass.
pub gamescope_grab_cursor: bool,
/// `PUNKTFUNK_RECOVER_SESSION_CMD` — operator hook fired (debounced) when a client connects while NO
/// graphical session is live for this uid: the state a compositor crash leaves behind (gnome-shell
/// SIGSEGV → GDM greeter, whose auto-login is once-per-boot, so the box would otherwise need a walk-up
@@ -152,6 +158,12 @@ impl HostConfig {
"1" | "true" | "yes" | "on"
)
}),
gamescope_grab_cursor: val("PUNKTFUNK_GAMESCOPE_GRAB_CURSOR").is_some_and(|s| {
matches!(
s.trim().to_ascii_lowercase().as_str(),
"1" | "true" | "yes" | "on"
)
}),
recover_session_cmd: val("PUNKTFUNK_RECOVER_SESSION_CMD")
.filter(|s| !s.trim().is_empty()),
on_connect_cmd: val("PUNKTFUNK_ON_CONNECT_CMD").filter(|s| !s.trim().is_empty()),
+1 -1
View File
@@ -6,7 +6,7 @@
# gamescope-EI socket path is the shared pf-paths contract, not a vdisplay reach-in).
[package]
name = "pf-inject"
version = "0.12.0"
version.workspace = true
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
+4 -1
View File
@@ -1009,7 +1009,10 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// else decodes the codec); only device loss ends the session.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
DecodedImage::PyroWave(f) => {
st.hdr = false; // 8-bit SDR codec
// The wavelet stream carries the negotiated ColorInfo (no VUI): an
// HDR (PQ) pyrowave session presents through the HDR10 path exactly
// like the H.26x codecs (design/pyrowave-444-hdr.md Phase 3).
st.hdr = f.color.is_pq();
match presenter.present(
&window,
FrameInput::PyroWave(f),
+24 -4
View File
@@ -39,7 +39,7 @@ impl Presenter {
#[cfg(windows)]
FrameInput::D3d11(d) => Some(d.color.is_pq()),
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
FrameInput::PyroWave(f) => Some(f.color.is_pq()), // always SDR today
FrameInput::PyroWave(f) => Some(f.color.is_pq()),
};
if let Some(pq) = frame_pq {
// A PQ stream we can only tone-map (no HDR10 surface) is the silent failure behind
@@ -750,11 +750,31 @@ impl Presenter {
&[planar.desc_set],
&[],
);
let rows = csc_rows(color, 8, false);
// An HDR (PQ) pyrowave session carries P010-style 10-bit studio codes MSB-packed
// into 16-bit planes (design/pyrowave-444-hdr.md §2.2) — same sampling scale as
// the P010 path; SDR sessions are plain 8-bit BT.709 limited. Depth follows the
// colour contract (negotiation couples 10-bit ⟺ PQ for this codec).
let (depth, msb_packed) = if color.is_pq() {
(10, true)
} else {
(8, false)
};
let rows = csc_rows(color, depth, msb_packed);
// Mode 1 = PQ→SDR tonemap (PQ stream without an HDR10 surface); mode 0 passes
// the transfer through — identical to the NV12 arm above.
let mode = if color.is_pq() && !self.hdr_active {
1.0f32
} else {
0.0
};
let peak = std::env::var("PUNKTFUNK_TONEMAP_PEAK")
.ok()
.and_then(|v| v.parse::<f32>().ok())
.unwrap_or(4.9); // ≈1000 nits over the 203-nit reference
let mut pc = [0f32; 16];
pc[..12].copy_from_slice(bytemuck_rows(&rows));
pc[12] = 0.0; // SDR passthrough — PyroWave has no PQ path
pc[13] = 0.0;
pc[12] = mode;
pc[13] = peak;
let bytes = std::slice::from_raw_parts(pc.as_ptr().cast::<u8>(), 64);
self.device.cmd_push_constants(
self.cmd_buf,
+8 -4
View File
@@ -203,11 +203,15 @@ impl Presenter {
vk::Format::R8G8B8A8_UNORM
};
self.csc.destroy(&self.device); // fence-safe: only our cmd bufs reference it
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
if let Some(p) = &self.csc_planar {
p.destroy(&self.device);
}
self.csc = CscPass::new(&self.device, self.video_format)?;
// The planar (PyroWave) pass renders to the same intermediate — rebuild it at the
// new format too (an HDR pyrowave session needs the 10-bit intermediate exactly
// like the H.26x path; 8-bit PQ bands visibly).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
if let Some(p) = self.csc_planar.take() {
p.destroy(&self.device);
self.csc_planar = Some(CscPass::new_planar(&self.device, self.video_format)?);
}
if let Some(v) = self.video.take() {
unsafe {
self.device.destroy_framebuffer(v.framebuffer, None);
+2 -1
View File
@@ -315,7 +315,8 @@ impl Presenter {
ext_mem_win32: ash::khr::external_memory_win32::Device::new(&instance, &device),
});
let csc = CscPass::new(&device, vk::Format::R8G8B8A8_UNORM)?;
// PyroWave is 8-bit SDR only, so the planar pass never needs the HDR10 rebuild.
// Starts SDR like `csc`; an HDR (PQ) pyrowave session rebuilds it at the 10-bit
// intermediate via `set_hdr_mode`, exactly like the H.26x pass.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
let csc_planar = if pyrowave_ok {
Some(CscPass::new_planar(&device, vk::Format::R8G8B8A8_UNORM)?)
+1 -1
View File
@@ -7,7 +7,7 @@
# lifecycle events invert to a host-registered sink).
[package]
name = "pf-vdisplay"
version = "0.12.0"
version.workspace = true
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -78,7 +78,7 @@ pub use routing::{
#[cfg(target_os = "linux")]
pub use routing::{
cancel_pending_tv_restore, dedicated_game_exited, launch_into_gamescope_session,
launch_is_nested,
launch_is_nested, steam_appid_from_launch, watch_steam_game_exit,
};
/// Compositors punktfunk knows how to drive (plan §6).
@@ -25,7 +25,10 @@ use discovery::{
check_gamescope_version, find_gamescope_eis_socket, find_gamescope_node,
gamescope_node_present, poll_managed_node, wait_for_node,
};
pub(crate) use discovery::{game_session_exited, is_available};
pub(crate) use discovery::{
game_session_exited, is_available, steam_appid_from_launch, wait_for_steam_game_exit,
SteamGameWatch,
};
/// The gamescope virtual-display driver. Three modes by env, in precedence order:
/// * `PUNKTFUNK_GAMESCOPE_SESSION=<client>` — host-MANAGE a `gamescope-session-plus` session
@@ -1369,7 +1372,14 @@ fn shape_dedicated_command(app: &str) -> String {
/// Add the compositor-side arguments shared by every bare gamescope spawn. `steam_mode` belongs
/// before the `--` terminator; [`PUNKTFUNK_GAMESCOPE_APP`](spawn) configures the nested command
/// after it and therefore cannot enable gamescope's Steam integration itself.
fn add_bare_gamescope_args(command: &mut Command, w: u32, h: u32, hz: u32, steam_mode: bool) {
fn add_bare_gamescope_args(
command: &mut Command,
w: u32,
h: u32,
hz: u32,
steam_mode: bool,
grab_cursor: bool,
) {
command
.args(["--backend", "headless"])
.args(["-W", &w.to_string()])
@@ -1378,6 +1388,9 @@ fn add_bare_gamescope_args(command: &mut Command, w: u32, h: u32, hz: u32, steam
if steam_mode {
command.arg("--steam");
}
if grab_cursor {
command.arg("--force-grab-cursor");
}
command.args(["--xwayland-count", "1", "--"]);
}
@@ -1398,16 +1411,27 @@ fn spawn(w: u32, h: u32, hz: u32, cmd: Option<&str>, log: &std::path::Path) -> R
// Read the env fallback under the shared env lock so it can't race a concurrent session's
// `set_var` of the same key (security-review 2026-06-28 #7).
.or_else(|| crate::with_env_lock(|| std::env::var("PUNKTFUNK_GAMESCOPE_APP").ok()))
.filter(|s| !s.trim().is_empty())
.unwrap_or_else(|| "sleep infinity".to_string());
.filter(|s| !s.trim().is_empty());
// A real app was requested (vs. the `sleep infinity` keep-alive) — used to scope the game-only
// cursor-grab flag below.
let game_launch = app.is_some();
let app = app.unwrap_or_else(|| "sleep infinity".to_string());
// Dedicated-launch command shaping (Part B): a Steam URI runs with `-silent` so the game is the
// gamescope focus with no Steam client window to navigate.
let app = shape_dedicated_command(&app);
let relay = ei_socket_file();
let _ = std::fs::remove_file(&relay); // stale socket path from a previous session
let steam_mode = pf_host_config::config().gamescope_steam;
// Enable gamescope's Steam integration (`--steam`: in-game overlay, Steam+X shortcuts, gamepad-UI
// navigation) whenever we're launching Steam — the operator no longer has to set the global
// PUNKTFUNK_GAMESCOPE_STEAM knob for a Steam title. The knob still forces it on for every spawn.
let steam_mode = pf_host_config::config().gamescope_steam || is_steam_launch(&app);
// Opt-in relative-mouse capture for a nested game (`PUNKTFUNK_GAMESCOPE_GRAB_CURSOR`): the client
// already sends relative motion, but gamescope only enters relative mode when the app hides the
// cursor, which some FPS titles never signal over the injected pointer — grabbing fixes mouselook.
// Default OFF (it forces relative mode, which would break absolute-pointer games/menus).
let grab_cursor = game_launch && pf_host_config::config().gamescope_grab_cursor;
let mut cmd = Command::new("gamescope");
add_bare_gamescope_args(&mut cmd, w, h, hz, steam_mode);
add_bare_gamescope_args(&mut cmd, w, h, hz, steam_mode, grab_cursor);
cmd.args([
"sh",
"-c",
@@ -29,6 +29,122 @@ pub(crate) fn game_session_exited(node_id: u32) -> bool {
}
}
/// Outcome of watching a dedicated Steam launch's game lifetime ([`wait_for_steam_game_exit`]).
#[derive(Debug, PartialEq, Eq)]
pub(crate) enum SteamGameWatch {
/// The game was seen running and has now exited — the session should end (APP_EXITED).
Exited,
/// The watch was cancelled (the session ended for another reason) or the game never started
/// within the startup grace — leave the session as-is.
Cancelled,
}
/// Parse the Steam appid a dedicated launch targets from its resolved command
/// (`steam [-silent] steam://rungameid/<appid>`). `None` unless the first token is `steam` and a
/// `steam://rungameid/<digits>` URI is present — the trailing digits are the appid, which is exactly
/// what Steam's launch reaper carries as `AppId=<appid>` (gameid == appid for a plain library title,
/// the only kind the host ever resolves to this shape).
pub(crate) fn steam_appid_from_launch(cmd: &str) -> Option<u32> {
if cmd.split_whitespace().next() != Some("steam") {
return None;
}
const MARKER: &str = "steam://rungameid/";
let tail = &cmd[cmd.find(MARKER)? + MARKER.len()..];
let digits: String = tail
.chars()
.take_while(|c: &char| c.is_ascii_digit())
.collect();
digits.parse().ok()
}
/// Block until the dedicated Steam game `appid` has started and then exited, `cancel` is set, or the
/// game never appears within the startup grace. Same-uid `/proc` scan keyed on Steam's launch reaper
/// (`SteamLaunch AppId=<appid>`), whose lifetime is exactly the game's. Returns
/// [`SteamGameWatch::Exited`] only after the game was actually seen running and then stayed gone
/// across a short confirmation window — so a cold Steam boot / shader precompile (game not up yet) or
/// a transient scan miss can't end the stream early. Runs on the host's per-session watch thread.
pub(crate) fn wait_for_steam_game_exit(
appid: u32,
cancel: &std::sync::atomic::AtomicBool,
) -> SteamGameWatch {
use std::sync::atomic::Ordering;
// Cold Steam boot + first-launch shader precompile can delay the game window by minutes; give it a
// generous window to appear. A game that never starts leaves the session up (the Steam client is
// still streamed, and the node-death path still covers the Steam client itself dying).
const START_GRACE: Duration = Duration::from_secs(300);
const POLL: Duration = Duration::from_secs(1);
// Require the reaper gone across this window (a few polls) so a brief process swap can't fire early.
const EXIT_CONFIRM: Duration = Duration::from_secs(3);
let start_deadline = Instant::now() + START_GRACE;
// Phase 1: wait for the game's reaper to appear.
while !steam_game_running(appid) {
if cancel.load(Ordering::Relaxed) || Instant::now() >= start_deadline {
return SteamGameWatch::Cancelled;
}
std::thread::sleep(POLL);
}
// Phase 2: the game is up — wait for its reaper to disappear (confirmed across the window).
let mut gone_since: Option<Instant> = None;
loop {
if cancel.load(Ordering::Relaxed) {
return SteamGameWatch::Cancelled;
}
if steam_game_running(appid) {
gone_since = None;
} else if gone_since.get_or_insert_with(Instant::now).elapsed() >= EXIT_CONFIRM {
return SteamGameWatch::Exited;
}
std::thread::sleep(POLL);
}
}
/// Is Steam's launch reaper for appid `appid` alive right now (same uid as the host)? Steam wraps
/// every game launch — native or Proton — in `…/reaper SteamLaunch AppId=<appid> -- <game>`, and the
/// reaper lives for the game's whole lifetime, so its presence is a precise "the game is running"
/// signal. Matched on the `SteamLaunch` + `AppId=<appid>` argv tokens together (exact-match, so
/// `AppId=57` never matches appid 570) — specific to the game reaper, so Steam's own shader-precompile
/// step (not reaper-wrapped) can't be mistaken for the game.
fn steam_game_running(appid: u32) -> bool {
// SAFETY: `getuid()` is a parameterless POSIX call that always succeeds and touches no memory.
let uid = unsafe { libc::getuid() };
let appid_tok = format!("AppId={appid}");
let Ok(entries) = std::fs::read_dir("/proc") else {
return false;
};
for e in entries.flatten() {
let name = e.file_name();
let Some(pid_str) = name.to_str() else {
continue;
};
if !pid_str.bytes().all(|b| b.is_ascii_digit()) {
continue;
}
let Ok(md) = std::fs::metadata(e.path()) else {
continue;
};
use std::os::unix::fs::MetadataExt;
if md.uid() != uid {
continue;
}
let Ok(cmdline) = std::fs::read(e.path().join("cmdline")) else {
continue;
};
let (mut launch, mut appid_match) = (false, false);
for arg in cmdline.split(|&b| b == 0) {
if arg == b"SteamLaunch" {
launch = true;
} else if arg == appid_tok.as_bytes() {
appid_match = true;
}
}
if launch && appid_match {
return true;
}
}
false
}
/// Poll [`find_gamescope_node`] (unscoped) up to `timeout` — for the managed / SteamOS session, which
/// logs to journald (no per-spawn file) and is single-session (no scoping needed).
pub(super) fn poll_managed_node(timeout: Duration) -> Option<u32> {
@@ -263,7 +379,29 @@ fn parse_version(text: &str) -> Option<(u32, u32, u32)> {
#[cfg(test)]
mod tests {
use super::{parse_version, MIN_GAMESCOPE};
use super::{parse_version, steam_appid_from_launch, MIN_GAMESCOPE};
#[test]
fn parses_steam_appid_from_launch() {
// The resolved dedicated-launch command (pre- or post-`-silent` shaping) → the appid.
assert_eq!(
steam_appid_from_launch("steam steam://rungameid/570"),
Some(570)
);
assert_eq!(
steam_appid_from_launch("steam -silent steam://rungameid/1091500"),
Some(1091500)
);
// Non-Steam launches / bare Steam with no rungameid URI → no appid (no game-exit watch).
assert_eq!(steam_appid_from_launch("lutris lutris:rungameid/42"), None);
assert_eq!(steam_appid_from_launch("steam -gamepadui"), None);
assert_eq!(steam_appid_from_launch("vkcube"), None);
// A steam:// URI that isn't the first `steam` token (a custom command) is not treated as one.
assert_eq!(
steam_appid_from_launch("firefox steam://rungameid/570"),
None
);
}
#[test]
fn parses_version_banner() {
@@ -176,6 +176,28 @@ pub fn dedicated_game_exited(_node_id: u32) -> bool {
false
}
/// The Steam appid a dedicated launch targets (`steam … steam://rungameid/<appid>`), for the
/// game-exit watcher. `None` for a non-Steam launch (those are covered by the node-death path
/// [`dedicated_game_exited`] — gamescope's nested child IS the game). See
/// [`gamescope::steam_appid_from_launch`].
#[cfg(target_os = "linux")]
pub fn steam_appid_from_launch(cmd: &str) -> Option<u32> {
gamescope::steam_appid_from_launch(cmd)
}
/// Block until the dedicated Steam game `appid` has started and then exited — returns `true` when the
/// session should end cleanly (APP_EXITED). Returns `false` if `cancel` is set (the session ended for
/// another reason) or the game never started within the startup grace (leave the session up). Runs on
/// the host's per-session watch thread; `cancel` is the session's stop flag. See
/// [`gamescope::wait_for_steam_game_exit`].
#[cfg(target_os = "linux")]
pub fn watch_steam_game_exit(appid: u32, cancel: &std::sync::atomic::AtomicBool) -> bool {
matches!(
gamescope::wait_for_steam_game_exit(appid, cancel),
gamescope::SteamGameWatch::Exited
)
}
/// Cancel any pending TV-session restore because a client (re)connected (review #3). No-op off Linux.
#[cfg(target_os = "linux")]
pub fn cancel_pending_tv_restore() {
+70 -2
View File
@@ -45,12 +45,20 @@ const RECV_BUF: usize = MAX_DATAGRAM_BYTES + 1;
/// died at full rate over WiFi). Same lossy-drop contract as `WouldBlock`; FEC + the next frame
/// recover. Asynchronous network-path blips (`ENETUNREACH`/`EHOSTUNREACH`/`ENETDOWN`/`EHOSTDOWN`)
/// are droppable for the same reason a stale ICMP is.
/// - Windows `WSAENOBUFS` (10055): the exact analogue of unix `ENOBUFS` — a high-bitrate keyframe
/// burst (one `WSASendMsg` USO super-buffer is up to ~512 segments ≈ 700 KB) momentarily exhausts
/// the socket send buffer / AFD non-paged pool, and Winsock reports `WSAENOBUFS`, which Rust maps
/// to `ErrorKind::Uncategorized` (so the `WouldBlock` arm misses it, exactly like unix `ENOBUFS`).
/// Without treating it as transient a Windows host tears the whole session down under load
/// (observed live: `native::stream` "send failed — stopping stream" on a paced video burst). Same
/// lossy-drop contract; FEC + the next frame recover. The `WSAENET*`/`WSAEHOST*` family is the
/// Windows counterpart of the droppable unix network-path blips above.
fn is_transient_io(e: &std::io::Error) -> bool {
use std::io::ErrorKind::{ConnectionRefused, ConnectionReset, WouldBlock};
if matches!(e.kind(), WouldBlock | ConnectionRefused | ConnectionReset) {
return true;
}
// `ENOBUFS` & friends have no stable `ErrorKind`, so match the raw errno (unix only).
// `ENOBUFS` & friends have no stable `ErrorKind`, so match the raw errno.
#[cfg(unix)]
{
matches!(
@@ -62,7 +70,20 @@ fn is_transient_io(e: &std::io::Error) -> bool {
| Some(libc::EHOSTDOWN)
)
}
#[cfg(not(unix))]
// Windows Winsock codes (WSAE*), raw like the sibling `uso_unsupported`. WSAEWOULDBLOCK (10035)
// already maps to `ErrorKind::WouldBlock` above, so it isn't repeated here.
#[cfg(windows)]
{
matches!(
e.raw_os_error(),
Some(10055) // WSAENOBUFS — tx queue / send buffer full (the dominant high-bitrate drop)
| Some(10051) // WSAENETUNREACH
| Some(10065) // WSAEHOSTUNREACH
| Some(10050) // WSAENETDOWN
| Some(10064) // WSAEHOSTDOWN
)
}
#[cfg(not(any(unix, windows)))]
{
false
}
@@ -324,6 +345,53 @@ mod tests {
}
}
/// The raw-errno tx-queue-full / network-blip codes have no stable `ErrorKind` (they surface as
/// `Uncategorized`), so they only get caught by the platform `raw_os_error()` arms. A burst that
/// momentarily exhausts the send buffer must stay a lossy drop, never a teardown — this is the
/// regression guard for the Windows `WSAENOBUFS` (10055) session crash and the unix `ENOBUFS`
/// wlan-driver case. Gated per platform because a code is only classified on its own OS.
#[test]
fn transient_io_covers_raw_tx_queue_and_path_codes() {
use std::io::Error;
#[cfg(unix)]
{
for code in [
libc::ENOBUFS,
libc::ENETUNREACH,
libc::EHOSTUNREACH,
libc::ENETDOWN,
libc::EHOSTDOWN,
] {
assert!(
is_transient_io(&Error::from_raw_os_error(code)),
"unix errno {code} should be transient"
);
}
// A genuine failure with no stable ErrorKind must still tear down.
assert!(
!is_transient_io(&Error::from_raw_os_error(libc::EACCES)),
"EACCES must stay fatal"
);
}
#[cfg(windows)]
{
// WSAENOBUFS / WSAENETUNREACH / WSAEHOSTUNREACH / WSAENETDOWN / WSAEHOSTDOWN.
for code in [10055, 10051, 10065, 10050, 10064] {
assert!(
is_transient_io(&Error::from_raw_os_error(code)),
"WSA code {code} should be transient"
);
}
// WSAEACCES (10013) — a real failure that must stay fatal.
assert!(
!is_transient_io(&Error::from_raw_os_error(10013)),
"WSAEACCES must stay fatal"
);
}
}
/// `send_batch` delivers a whole frame's worth of packets over real loopback UDP — exercising
/// the `sendmmsg` path on Linux (the scalar-loop default elsewhere). 100 × 200 B = 20 KB fits
/// the socket buffer, so loopback is lossless and every packet must arrive intact + in order.
+27 -12
View File
@@ -12,7 +12,10 @@ use anyhow::Result;
// `crate::capture::*` (the capture mechanics that used the rest moved into pf-capture).
pub use pf_frame::{CapturedFrame, OutputFormat, PixelFormat};
// The capturer types + trait + synthetics live in `pf-capture`; re-export them at the old paths.
pub use pf_capture::{capturer_supports_444, Capturer, FastSyntheticCapturer, SyntheticCapturer};
pub use pf_capture::{
capturer_supports_444, capturer_supports_hdr, Capturer, FastSyntheticCapturer,
SyntheticCapturer,
};
// `crate::capture::dxgi::{install_gpu_pref_hook, hdr_p010_selftest}` (main.rs subcommands) and
// `crate::capture::synthetic_nv12` resolve through pf-capture's Windows modules.
#[cfg(target_os = "windows")]
@@ -45,18 +48,20 @@ fn zero_copy_policy() -> pf_capture::ZeroCopyPolicy {
}
}
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal.
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal. `want_hdr`
/// offers the GNOME 50+ 10-bit PQ/BT.2020 formats (pass it only when the session negotiated HDR
/// AND the mirrored monitor is in HDR mode — see [`pf_capture::gnome_hdr_monitor_active`]).
#[cfg(target_os = "linux")]
pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
pub fn open_portal_monitor(want_hdr: bool) -> Result<Box<dyn Capturer>> {
// On RemoteDesktop-capable desktops (KWin/GNOME) anchor ScreenCast to a RemoteDesktop
// session so it inherits that grant headlessly; wlroots/Sway has no RemoteDesktop portal,
// so use a plain ScreenCast session there.
let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei;
pf_capture::open_portal_monitor(anchored, zero_copy_policy())
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy())
}
#[cfg(not(target_os = "linux"))]
pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
pub fn open_portal_monitor(_want_hdr: bool) -> Result<Box<dyn Capturer>> {
anyhow::bail!("portal capture requires Linux (xdg-desktop-portal + PipeWire)")
}
@@ -69,11 +74,13 @@ pub fn capture_virtual_output(
want: OutputFormat,
_capture: crate::session_plan::CaptureBackend,
) -> Result<Box<dyn Capturer>> {
// The Linux host stays 8-bit (HDR is blocked upstream) and the portal negotiates its own pixel
// format, so `want.gpu` gates GPU zero-copy capture (the capture backend is always the portal
// the `CaptureBackend` arg is a Windows-only dispatch) and `want.chroma_444` selects the
// worker's planar-YUV444 GPU convert. `gpu = false` (4:4:4 without zero-copy) forces the CPU
// mmap path so the encoder gets CPU-resident RGB to swscale into YUV444P.
// The Linux NATIVE plane stays 8-bit (Mutter's virtual-monitor streams are SDR-only upstream;
// the GNOME 50+ HDR path is monitor-mirror only — `open_portal_monitor`) and the portal
// negotiates its own pixel format, so `want.gpu` gates GPU zero-copy capture (the capture
// backend is always the portal — the `CaptureBackend` arg is a Windows-only dispatch) and
// `want.chroma_444` selects the worker's planar-YUV444 GPU convert. `gpu = false` (4:4:4
// without zero-copy) forces the CPU mmap path so the encoder gets CPU-resident RGB to swscale
// into YUV444P.
pf_capture::open_virtual_output(
vout.remote_fd,
vout.node_id,
@@ -131,8 +138,16 @@ pub fn capture_virtual_output(
// proactively enables advanced color and selects the per-frame conversion. There is NO fallback:
// if it can't open or the driver doesn't attach, the session fails cleanly and the client
// reconnects.
pf_capture::open_idd_push(target, pref, want.hdr, want.chroma_444, keep, sender)
.map_err(|(e, _keep)| e.context("IDD-push capture open (no fallback)"))
pf_capture::open_idd_push(
target,
pref,
want.hdr,
want.chroma_444,
want.pyrowave,
keep,
sender,
)
.map_err(|(e, _keep)| e.context("IDD-push capture open (no fallback)"))
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
+20
View File
@@ -164,6 +164,12 @@ pub enum EventKind {
/// API (RFC §8) lands.
source: String,
},
#[serde(rename = "plugins.changed")]
PluginsChanged {
/// The plugin whose registration changed (registered, restarted, deregistered, or
/// lease-expired). A consumer re-reads `GET /api/v1/plugins` for the new set.
id: String,
},
#[serde(rename = "host.started")]
HostStarted {
version: String,
@@ -190,6 +196,7 @@ impl EventKind {
EventKind::DisplayCreated { .. } => "display.created",
EventKind::DisplayReleased { .. } => "display.released",
EventKind::LibraryChanged { .. } => "library.changed",
EventKind::PluginsChanged { .. } => "plugins.changed",
EventKind::HostStarted { .. } => "host.started",
EventKind::HostStopping => "host.stopping",
}
@@ -495,6 +502,19 @@ mod tests {
serde_json::to_string(&ev).unwrap(),
r#"{"seq":2,"ts_ms":1700000000000,"schema":1,"kind":"host.stopping"}"#
);
let ev = HostEvent {
seq: 3,
ts_ms: 1_700_000_000_000,
schema: 1,
kind: EventKind::PluginsChanged {
id: "rom-manager".into(),
},
};
assert_eq!(
serde_json::to_string(&ev).unwrap(),
r#"{"seq":3,"ts_ms":1700000000000,"schema":1,"kind":"plugins.changed","id":"rom-manager"}"#
);
}
#[test]
+16 -3
View File
@@ -5,7 +5,7 @@
use anyhow::{anyhow, Context, Result};
use pf_paths::config_dir;
use rsa::pkcs1v15::SigningKey;
use rsa::pkcs8::DecodePrivateKey;
use rsa::pkcs8::{DecodePrivateKey, EncodePrivateKey, LineEnding};
use rsa::RsaPrivateKey;
use sha2::Sha256;
use std::fs;
@@ -70,7 +70,20 @@ impl ServerIdentity {
}
fn generate() -> Result<(String, String)> {
let key = rcgen::KeyPair::generate_for(&rcgen::PKCS_RSA_SHA256).context("rcgen RSA keygen")?;
// The workspace is ring-only (aws-lc-sys breaks Windows CI — see the rustls/rcgen pins), and
// `ring` can *sign* with an existing RSA key but cannot *generate* one: rcgen's ring backend
// returns `KeyGenerationUnavailable` for `generate_for(&PKCS_RSA_SHA256)`. Moonlight requires an
// RSA-2048 identity, so generate the key with the pure-Rust `rsa` crate (already a dep for the
// pairing signer) and hand the PKCS#8 PEM to rcgen, whose ring backend *can* load + self-sign
// with it. Returning that same PEM keeps it byte-identical to what `from_pems` re-parses.
let mut rng = rand::thread_rng();
let priv_key = RsaPrivateKey::new(&mut rng, 2048).context("generate RSA-2048 host key")?;
let key_pem = priv_key
.to_pkcs8_pem(LineEnding::LF)
.context("encode host key as PKCS#8 PEM")?
.to_string();
let key = rcgen::KeyPair::from_pkcs8_pem_and_sign_algo(&key_pem, &rcgen::PKCS_RSA_SHA256)
.context("load RSA host key into rcgen")?;
let mut params = rcgen::CertificateParams::new(Vec::<String>::new()).context("cert params")?;
params
.distinguished_name
@@ -78,7 +91,7 @@ fn generate() -> Result<(String, String)> {
params.not_before = rcgen::date_time_ymd(2020, 1, 1);
params.not_after = rcgen::date_time_ymd(2040, 1, 1);
let cert = params.self_signed(&key).context("self-sign cert")?;
Ok((cert.pem(), key.serialize_pem()))
Ok((cert.pem(), key_pem))
}
/// Extract the X.509 `signatureValue` bytes from a cert PEM.
+38 -11
View File
@@ -50,21 +50,45 @@ pub const SCM_AV1_MAIN10: u32 = 0x0002_0000;
/// The **SDR baseline** codec mask: H.264, HEVC Main, AV1 Main 8-bit (= 65793). HEVC Main10 (HDR) is
/// layered on top of this at runtime by `serverinfo::codec_mode_support` when — and only when — the
/// host can actually deliver it ([`host_hdr_capable`]); it is never a static claim, because a non-HDR
/// host (Linux, or a Windows host without the `PUNKTFUNK_10BIT` opt-in) must not invite a client into
/// an HDR mode it can't produce. (The previous placeholder 3843 = 0xF03 wrongly claimed HEVC Main10 +
/// host (a host without the `PUNKTFUNK_10BIT` opt-in, or a Linux host whose video source / encoder
/// can't do Main10) must not invite a client into an HDR mode it can't produce. (The previous placeholder 3843 = 0xF03 wrongly claimed HEVC Main10 +
/// 4:4:4 and *no* AV1.) 4:4:4 stays off entirely on GameStream: stock Moonlight is 4:2:0 —
/// full-chroma is a punktfunk/1-native negotiation only (`crate::capture::capturer_supports_444`).
pub const SERVER_CODEC_MODE_SUPPORT: u32 = SCM_H264 | SCM_HEVC | SCM_AV1_MAIN8;
/// Whether this host can deliver an **HDR** (HEVC Main10 / BT.2020 PQ) GameStream — the single gate
/// for advertising [`SCM_HEVC_MAIN10`] in serverinfo and `IsHdrSupported` per app, and for honoring a
/// client's `dynamicRangeMode` request. HDR capture+encode is **Windows-only** (the Linux host is
/// 8-bit, blocked upstream) and behind the operator's `PUNKTFUNK_10BIT` opt-in — the same policy gate
/// the native punktfunk/1 plane honors. When this is true the IDD-push capturer streams HEVC Main10 PQ
/// whenever the desktop is HDR, and a client HDR request makes the GameStream video path proactively
/// enable advanced color on the per-session virtual display so PQ flows even from an SDR desktop.
/// for advertising [`SCM_HEVC_MAIN10`] in serverinfo and `IsHdrSupported` per app, and (together
/// with the live capture-side check at RTSP time) for honoring a client's `dynamicRangeMode`
/// request. Behind the operator's `PUNKTFUNK_10BIT` opt-in — the same policy gate the native
/// punktfunk/1 plane honors — on both OSes.
///
/// **Windows**: the IDD-push capturer streams HEVC Main10 PQ whenever the desktop is HDR, and a
/// client HDR request proactively enables advanced color on the per-session virtual display so PQ
/// flows even from an SDR desktop.
///
/// **Linux**: the GNOME 50+ portal **monitor mirror** (`video_source=portal`) can negotiate the
/// 10-bit PQ formats while the mirrored monitor is in HDR mode, and the NVENC/VAAPI encoders have
/// a probed Main10 path ([`crate::encode::can_encode_10bit`]). The virtual-output source stays SDR
/// (Mutter's RecordVirtual streams are 8-bit-only upstream), so this is `false` for it. Whether
/// the monitor is ACTUALLY in HDR mode right now is checked live at RTSP honor time
/// ([`pf_capture::gnome_hdr_monitor_active`]) — this fn is the static serverinfo capability.
pub fn host_hdr_capable() -> bool {
cfg!(target_os = "windows") && pf_host_config::config().ten_bit
if !pf_host_config::config().ten_bit {
return false;
}
#[cfg(target_os = "windows")]
{
true
}
#[cfg(target_os = "linux")]
{
pf_host_config::config().video_source.as_deref() == Some("portal")
&& crate::encode::can_encode_10bit(crate::encode::Codec::H265)
}
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
{
false
}
}
/// Stable host identity + advertised capabilities, shared across control-plane handlers.
@@ -141,8 +165,11 @@ pub struct AppState {
pub rfi_range: std::sync::Arc<std::sync::Mutex<Option<(i64, i64)>>>,
/// Persistent screen capturer, reused across streams so reconnects don't spawn a second
/// (conflicting) screencast session. The video thread borrows it for the stream's duration
/// and returns it; `set_active` gates its cost while idle.
pub video_cap: std::sync::Arc<std::sync::Mutex<Option<Box<dyn crate::capture::Capturer>>>>,
/// and returns it; `set_active` gates its cost while idle. The slot's `bool` records whether
/// it was opened with the HDR (10-bit PQ) offer — a stream whose negotiated `hdr` differs
/// drops the pooled capturer and opens a fresh screencast session at the right depth
/// (mirroring the audio capturer's channel-count reuse gate).
pub video_cap: stream::CapturerSlot,
/// Persistent audio capturer, reused across streams when the channel count still matches
/// (avoids a PipeWire stream setup per reconnect); drained on reuse so no stale audio is
/// sent, dropped + reopened when a session negotiates a different channel count.
+21 -6
View File
@@ -396,18 +396,33 @@ fn stream_config(map: &HashMap<String, String>) -> Option<StreamConfig> {
_ => Codec::H264,
};
// 10-bit/HDR request (Moonlight sets `dynamicRangeMode != 0` only when it both saw our Main10 SCM
// bit AND the user enabled HDR). Honor it only when the host can actually deliver Main10 (Windows +
// PUNKTFUNK_10BIT, `host_hdr_capable`); when honored, the video path proactively enables advanced
// color on the virtual display so a PQ stream flows even from an SDR desktop. A request we can't
// honor degrades to 8-bit SDR (and a desktop that is ALREADY HDR still streams PQ regardless, since
// the IDD-push capturer follows the display).
// bit AND the user enabled HDR). Honor it only when the host can actually deliver Main10
// (`host_hdr_capable` — Windows IDD-push, or the Linux GNOME 50+ portal mirror). On Windows,
// when honored, the video path proactively enables advanced color on the virtual display so a
// PQ stream flows even from an SDR desktop. On Linux the portal can only deliver PQ while the
// MIRRORED monitor is in HDR mode, so additionally probe the live colour mode here (one D-Bus
// round-trip, sync RTSP thread) — an SDR desktop honestly degrades to 8-bit SDR up front
// instead of running the capture negotiation into its timeout. A request we can't honor
// degrades to 8-bit SDR (and a Windows desktop that is ALREADY HDR still streams PQ
// regardless, since the IDD-push capturer follows the display).
let hdr_requested = parse_u("x-nv-video[0].dynamicRangeMode").unwrap_or(0) != 0;
let hdr = hdr_requested && crate::gamestream::host_hdr_capable();
// `mut` is load-bearing on Linux only — the GNOME colour-mode probe below clears it. Scope the
// allow to non-Linux so `unused_mut` still fires here if that probe ever goes away.
#[cfg_attr(not(target_os = "linux"), allow(unused_mut))]
let mut hdr = hdr_requested && crate::gamestream::host_hdr_capable();
if hdr_requested && !hdr {
tracing::warn!(
"client requested HDR (dynamicRangeMode != 0) but host is not HDR-capable — streaming 8-bit SDR"
);
}
#[cfg(target_os = "linux")]
if hdr && !pf_capture::gnome_hdr_monitor_active() {
tracing::warn!(
"client requested HDR but no monitor is in BT.2100 (HDR) colour mode — enable HDR in \
GNOME Settings Displays (GNOME 50+) to stream it; streaming 8-bit SDR"
);
hdr = false;
}
// The client's requested CSC (moonlight-common-c SdpGenerator.c: `encoderCscMode =
// (colorspace << 1) | fullRange` — colorspace 0=Rec601, 1=Rec709, 2=Rec2020). Moonlight
// renderers configure their YUV→RGB from this REQUESTED value (not the bitstream VUI), so a
+32 -10
View File
@@ -34,8 +34,9 @@ pub struct StreamConfig {
}
/// Slot for the persistent screen capturer, shared with the control plane and reused across
/// streams so a reconnect doesn't open a second (conflicting) screencast session.
pub type CapturerSlot = Arc<std::sync::Mutex<Option<Box<dyn Capturer>>>>;
/// streams so a reconnect doesn't open a second (conflicting) screencast session. The `bool` is
/// the pooled capturer's HDR-ness (see `AppState::video_cap`).
pub type CapturerSlot = Arc<std::sync::Mutex<Option<(Box<dyn Capturer>, bool)>>>;
/// A pending client reference-frame-invalidation range (lost `firstFrame..=lastFrame`), set by the
/// control plane and drained by the video thread (see [`AppState::rfi_range`](super::AppState)).
@@ -120,7 +121,7 @@ fn run(
running: &Arc<AtomicBool>,
force_idr: &AtomicBool,
rfi_range: &std::sync::Mutex<Option<(i64, i64)>>,
video_cap: &std::sync::Mutex<Option<Box<dyn Capturer>>>,
video_cap: &std::sync::Mutex<Option<(Box<dyn Capturer>, bool)>>,
// Shared stats recorder for the web-console capture/graph. Threaded into `stream_body` (the
// encode loop); per-frame sample emission is wired by a later pass.
stats: &Arc<crate::stats_recorder::StatsRecorder>,
@@ -243,15 +244,31 @@ fn run(
}
// Reuse the persistent capturer (one screencast session → clean reconnect); create it on
// the first stream. Borrow it for this stream and return it on exit.
let mut capturer: Box<dyn Capturer> = match video_cap.lock().unwrap().take() {
// the first stream. Borrow it for this stream and return it on exit. Reuse is gated on the
// pooled capturer's HDR-ness matching this stream's negotiated `cfg.hdr` — the depth is a
// PipeWire-negotiation-time property of the screencast session, so an HDR↔SDR change needs a
// fresh session (same pattern as the audio capturer's channel-count gate).
let pooled = match video_cap.lock().unwrap().take() {
Some((c, was_hdr)) if was_hdr == cfg.hdr => Some(c),
Some((c, was_hdr)) => {
tracing::info!(
was_hdr,
want_hdr = cfg.hdr,
"video source: pooled capturer depth mismatch — opening a fresh screencast session"
);
drop(c);
None
}
None => None,
};
let mut capturer: Box<dyn Capturer> = match pooled {
Some(c) => {
tracing::info!("video source: reusing capturer");
c
}
None if pf_host_config::config().video_source.as_deref() == Some("portal") => {
tracing::info!("video source: portal desktop capture");
capture::open_portal_monitor().context("open portal capturer")?
tracing::info!(hdr = cfg.hdr, "video source: portal desktop capture");
capture::open_portal_monitor(cfg.hdr).context("open portal capturer")?
}
None => {
tracing::info!("video source: synthetic test pattern");
@@ -272,7 +289,7 @@ fn run(
&client_label,
);
capturer.set_active(false);
*video_cap.lock().unwrap() = Some(capturer);
*video_cap.lock().unwrap() = Some((capturer, cfg.hdr));
result
}
@@ -380,7 +397,9 @@ fn open_gs_virtual_source(
// HDR: pass the negotiated `cfg.hdr` (client asked for HDR AND the host can deliver it). On the
// Windows IDD-push path this proactively enables advanced color on the virtual display so a Main10
// PQ stream flows even from an SDR desktop; an already-HDR desktop streams PQ regardless (the
// capturer follows the display). No-op on Linux (8-bit, and `cfg.hdr` is always false there).
// capturer follows the display). No-op on Linux: virtual-output capture is SDR-only upstream
// (Mutter RecordVirtual), and `host_hdr_capable` therefore keeps `cfg.hdr` false for this
// source — the Linux HDR path is the portal monitor mirror (`video_source=portal`).
let capturer = capture::capture_virtual_output(
vout,
capture::OutputFormat::resolve(cfg.hdr, crate::encode::resolved_backend_is_gpu()),
@@ -399,7 +418,10 @@ fn open_gs_virtual_source(
fn gs_bit_depth(format: crate::capture::PixelFormat) -> u8 {
use crate::capture::PixelFormat;
match format {
PixelFormat::P010 | PixelFormat::Rgb10a2 => 10,
// Windows IDD-push HDR formats, and the Linux GNOME 50+ portal HDR formats.
PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => {
10
}
_ => 8,
}
}
+88 -9
View File
@@ -176,16 +176,48 @@ impl HooksConfig {
/// The persisted hooks store — the [`crate::vdisplay::policy::DisplayPolicyStore`] recipe:
/// private dir, temp-write + atomic rename, in-memory value changes only if the write succeeds.
///
/// A hand-edited `hooks.json` is honored WITHOUT a restart (the documented contract): [`get`]
/// re-stats the file and reloads when its identity (mtime + length) moved. The stat rides the
/// per-event dispatch, so the check costs one `metadata()` call per event, and a full re-read
/// happens only when the file actually changed.
///
/// [`get`]: HooksStore::get
pub struct HooksStore {
path: PathBuf,
cur: Mutex<Option<HooksConfig>>,
cur: Mutex<StoreState>,
}
struct StoreState {
cfg: Option<HooksConfig>,
/// Identity of the file revision `cfg` was parsed from (mtime + length); `None` = the file
/// did not exist. `get` compares against a fresh stat to detect hand edits.
file_id: Option<(std::time::SystemTime, u64)>,
}
impl HooksStore {
/// Load from `path`. Missing file ⇒ no hooks; corrupt file ⇒ no hooks with a warning
/// (never fail host startup over a settings file).
pub fn load_from(path: PathBuf) -> Self {
let cur = match std::fs::read(&path) {
let (cfg, file_id) = Self::read_disk(&path);
HooksStore {
path,
cur: Mutex::new(StoreState { cfg, file_id }),
}
}
/// The file's on-disk identity, `None` when it does not exist (or cannot be stat'd —
/// indistinguishable on purpose: both mean "no usable hooks file").
fn file_identity(path: &PathBuf) -> Option<(std::time::SystemTime, u64)> {
let meta = std::fs::metadata(path).ok()?;
Some((meta.modified().ok()?, meta.len()))
}
/// Read + validate the file. Same lenient contract as startup: missing ⇒ no hooks;
/// invalid/unreadable ⇒ no hooks with a warning naming the problem.
fn read_disk(path: &PathBuf) -> (Option<HooksConfig>, Option<(std::time::SystemTime, u64)>) {
let file_id = Self::file_identity(path);
let cfg = match std::fs::read(path) {
Ok(bytes) => match serde_json::from_slice::<HooksConfig>(&bytes) {
Ok(c) => {
if let Err(e) = c.validate() {
@@ -204,15 +236,23 @@ impl HooksStore {
},
Err(_) => None,
};
HooksStore {
path,
cur: Mutex::new(cur),
}
(cfg, file_id)
}
/// The stored configuration (empty when unconfigured) — the mgmt GET and the dispatcher.
/// Re-reads `hooks.json` first if it changed on disk since last load, so hand edits apply
/// on the next event, no restart ("changes apply immediately" — docs/automation.md).
pub fn get(&self) -> HooksConfig {
self.cur.lock().unwrap().clone().unwrap_or_default()
let mut st = self.cur.lock().unwrap();
let now_id = Self::file_identity(&self.path);
if now_id != st.file_id {
let (cfg, file_id) = Self::read_disk(&self.path);
tracing::info!(path = %self.path.display(), hooks = cfg.as_ref().map_or(0, |c| c.hooks.len()),
"hooks.json changed on disk — reloaded");
st.cfg = cfg;
st.file_id = file_id;
}
st.cfg.clone().unwrap_or_default()
}
/// Persist + adopt a new configuration (caller validates first). The in-memory value
@@ -224,12 +264,15 @@ impl HooksStore {
let tmp = self.path.with_extension("json.tmp");
pf_paths::write_secret_file(&tmp, &serde_json::to_vec_pretty(&cfg)?)?;
std::fs::rename(&tmp, &self.path)?;
*self.cur.lock().unwrap() = Some(cfg);
let mut st = self.cur.lock().unwrap();
st.file_id = Self::file_identity(&self.path);
st.cfg = Some(cfg);
Ok(())
}
}
/// The process-wide hooks store (`<config_dir>/hooks.json`), loaded once on first access.
/// The process-wide hooks store (`<config_dir>/hooks.json`), loaded on first access and
/// re-loaded whenever the file changes on disk (see [`HooksStore::get`]).
pub fn store() -> &'static HooksStore {
static STORE: OnceLock<HooksStore> = OnceLock::new();
STORE.get_or_init(|| HooksStore::load_from(pf_paths::config_dir().join("hooks.json")))
@@ -860,6 +903,42 @@ mod tests {
let _ = std::fs::remove_file(&path);
}
#[test]
fn hand_edited_file_reloads_without_restart() {
let path = std::env::temp_dir().join(format!(
"pf-hooks-reload-test-{}-{:p}.json",
std::process::id(),
&0u8 as *const u8
));
let _ = std::fs::remove_file(&path);
let store = HooksStore::load_from(path.clone());
assert!(store.get().hooks.is_empty());
// The documented flow: the operator writes hooks.json by hand and the SAME running
// store honors it on the next event — no restart, no PUT.
std::fs::write(
&path,
br#"{"hooks":[{"on":"stream.started","run":"true"}]}"#,
)
.unwrap();
assert_eq!(store.get().hooks.len(), 1, "hand edit applies on next read");
assert_eq!(store.get().hooks[0].on, "stream.started");
// A second edit applies too (length differs, so same-second mtime granularity can't
// mask it).
std::fs::write(
&path,
br#"{"hooks":[{"on":"stream.started","run":"true"},{"on":"client.*","run":"true"}]}"#,
)
.unwrap();
assert_eq!(store.get().hooks.len(), 2, "second hand edit applies too");
// Deleting the file removes the hooks.
std::fs::remove_file(&path).unwrap();
assert!(store.get().hooks.is_empty(), "deleted file = no hooks");
}
#[test]
fn filters_constrain_and_missing_fields_never_match() {
let ev = sample_event();
+120 -1
View File
@@ -145,6 +145,76 @@ pub(crate) fn fetch_image(url: &str) -> Option<(Vec<u8>, String)> {
(!bytes.is_empty()).then_some((bytes, ctype))
}
/// A stored [`Artwork`] value that is a **local filesystem path** to an image on the host — as
/// opposed to an `http(s)`/`data:` URL or an already-relative host proxy path. Provider plugins that
/// run on the host (e.g. the Playnite sync plugin) set these: the reconcile payload stays tiny
/// (paths, not inlined bytes, so it scales to thousands of titles) and the host serves the bytes
/// through the art proxy, exactly like Steam's cache art. Windows-shaped only (`C:\…`, `C:/…`, or a
/// `\\server\share` UNC) — Playnite, the only local-art provider, is Windows-only, and this keeps the
/// check from ever mistaking the `/api/…` proxy path (or a POSIX abs path) for a local file.
pub fn is_local_art_path(v: &str) -> bool {
if v.starts_with("http://") || v.starts_with("https://") || v.starts_with("data:") {
return false;
}
let b = v.as_bytes();
(b.len() >= 3 && b[1] == b':' && (b[2] == b'\\' || b[2] == b'/')) || v.starts_with("\\\\")
}
/// Read a local image file into `(bytes, content-type)` for the art proxy. `None` if it isn't an
/// existing regular file, is empty, or exceeds 16 MiB (a cover never approaches that; the cap bounds
/// host memory). Content-type is guessed from the extension.
pub fn local_art_bytes(path: &str) -> Option<(Vec<u8>, String)> {
let p = std::path::Path::new(path);
let meta = std::fs::metadata(p).ok()?;
if !meta.is_file() || meta.len() == 0 || meta.len() > 16 * 1024 * 1024 {
return None;
}
let ctype = match p
.extension()
.and_then(|e| e.to_str())
.map(|e| e.to_ascii_lowercase())
.as_deref()
{
Some("jpg" | "jpeg") => "image/jpeg",
Some("png") => "image/png",
Some("webp") => "image/webp",
Some("gif") => "image/gif",
Some("bmp") => "image/bmp",
Some("ico") => "image/x-icon",
Some("tga") => "image/x-tga",
_ => "application/octet-stream",
}
.to_string();
Some((std::fs::read(p).ok()?, ctype))
}
/// Resolve one art value to bytes for the Moonlight `/appasset` proxy: a local host file
/// ([`is_local_art_path`]) is read directly, anything else is a URL fetched by [`fetch_image`].
fn resolve_art_bytes(v: &str) -> Option<(Vec<u8>, String)> {
if is_local_art_path(v) {
local_art_bytes(v)
} else {
fetch_image(v)
}
}
/// Rewrite any **local-file** art paths on an entry into host art-proxy URLs
/// (`/api/v1/library/art/<id>/<kind>`, the same relative-proxy shape Steam art uses, resolved by the
/// client against the host). `http(s)`/`data:` URLs and already-relative proxy paths are left as-is.
/// Applied to the `GET /library` response so a client fetches a provider's local covers from the host
/// instead of receiving an unreachable `C:\…` path.
pub fn proxy_local_art(id: &str, art: &mut Artwork) {
let rw = |field: &mut Option<String>, kind: &str| {
if field.as_deref().is_some_and(is_local_art_path) {
*field = Some(format!("/api/v1/library/art/{id}/{kind}"));
}
};
rw(&mut art.portrait, "portrait");
rw(&mut art.hero, "hero");
rw(&mut art.logo, "logo");
rw(&mut art.header, "header");
}
/// Resolve + fetch the best box-art cover for a library id (the GameStream `/appasset` proxy — Moonlight
/// fetches per-app covers from the HOST, not the CDN, so we proxy the bytes). Tries the portrait (tall
/// capsule Moonlight wants) → header → hero → logo, returning the first that fetches as
@@ -171,7 +241,7 @@ pub fn fetch_box_art(id: &str) -> Option<(Vec<u8>, String)> {
[g.art.portrait, g.art.header, g.art.hero, g.art.logo]
.into_iter()
.flatten()
.find_map(|url| fetch_image(&url))
.find_map(|url| resolve_art_bytes(&url))
}
/// Make a protocol-relative URL (`//host/...`, common in GOG + MS catalog responses) absolute https.
@@ -264,4 +334,53 @@ mod tests {
// Empty payload → None (never serve a 0-byte cover).
assert!(fetch_image("data:image/png;base64,").is_none());
}
#[test]
fn local_art_path_detection() {
// Windows-shaped local paths a provider (Playnite) would store.
assert!(is_local_art_path(r"C:\Users\me\cover.jpg"));
assert!(is_local_art_path("C:/Users/me/cover.png"));
assert!(is_local_art_path(r"\\nas\share\art.jpg"));
// URLs and the host proxy path are NOT local files.
assert!(!is_local_art_path("https://cdn/x.jpg"));
assert!(!is_local_art_path("http://host/x.jpg"));
assert!(!is_local_art_path("data:image/png;base64,AAAA"));
assert!(!is_local_art_path(
"/api/v1/library/art/custom:abc/portrait"
));
}
#[test]
fn proxy_local_art_rewrites_only_local_paths() {
let mut art = Artwork {
portrait: Some(r"C:\art\p.jpg".into()),
hero: Some("https://cdn/h.jpg".into()),
logo: None,
header: Some("/api/v1/library/art/custom:x/header".into()),
};
proxy_local_art("custom:abc", &mut art);
// The local path becomes a host proxy URL; the remote URL and an already-proxied path stay.
assert_eq!(
art.portrait.as_deref(),
Some("/api/v1/library/art/custom:abc/portrait")
);
assert_eq!(art.hero.as_deref(), Some("https://cdn/h.jpg"));
assert_eq!(
art.header.as_deref(),
Some("/api/v1/library/art/custom:x/header")
);
}
#[test]
fn local_art_bytes_reads_a_real_file() {
let dir = std::env::temp_dir().join(format!("pf-art-test-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let f = dir.join("cover.png");
std::fs::write(&f, [1u8, 2, 3, 4]).unwrap();
let (bytes, ctype) = local_art_bytes(f.to_str().unwrap()).expect("reads file");
assert_eq!(bytes, vec![1, 2, 3, 4]);
assert_eq!(ctype, "image/png");
assert!(local_art_bytes(dir.join("nope.png").to_str().unwrap()).is_none());
let _ = std::fs::remove_dir_all(&dir);
}
}
@@ -92,6 +92,24 @@ pub fn load_custom() -> Vec<CustomEntry> {
}
}
/// Serve a custom/provider entry's stored **local** art file for one [`ArtKind`] — the non-Steam
/// branch of the art proxy (`GET /library/art/custom:<id>/<kind>`). `id` is the bare custom id (the
/// `custom:` prefix already stripped by the handler). `None` if the entry is unknown, has no art of
/// that kind, or that art value isn't a servable local file (e.g. an `http` URL the client fetches
/// itself). Blocking IO — call off the async runtime.
pub fn custom_local_art_bytes(id: &str, kind: ArtKind) -> Option<(Vec<u8>, String)> {
let entry = load_custom().into_iter().find(|e| e.id == id)?;
let field = match kind {
ArtKind::Portrait => entry.art.portrait,
ArtKind::Hero => entry.art.hero,
ArtKind::Logo => entry.art.logo,
ArtKind::Header => entry.art.header,
}?;
is_local_art_path(&field)
.then(|| local_art_bytes(&field))
.flatten()
}
fn save_custom(entries: &[CustomEntry]) -> Result<()> {
let dir = pf_paths::config_dir();
// Owner-private dir (0700 / SYSTEM+Admins DACL) so a non-privileged local user can't plant a
+18
View File
@@ -283,6 +283,24 @@ fn real_main() -> Result<()> {
// PASS/FAIL + max Y/Cb/Cr error.
#[cfg(target_os = "windows")]
Some("hdr-p010-selftest") => crate::capture::dxgi::hdr_p010_selftest(),
// Linux HDR readiness probe (GNOME 50+ portal path): prints whether a monitor is currently
// in BT.2100 (HDR) colour mode, whether the NVENC/VAAPI backend probes Main10 for
// HEVC/AV1, and the GameStream HDR capability the two combine into — the "why isn't my
// stream HDR?" diagnostic (no display/session needed for the encoder half).
#[cfg(target_os = "linux")]
Some("hdr-probe") => {
let monitor_hdr = pf_capture::gnome_hdr_monitor_active();
let hevc10 = encode::can_encode_10bit(encode::Codec::H265);
let av110 = encode::can_encode_10bit(encode::Codec::Av1);
println!("monitor in BT.2100 (HDR) colour mode: {monitor_hdr}");
println!("encoder Main10 (HEVC): {hevc10}");
println!("encoder 10-bit (AV1): {av110}");
println!(
"GameStream HDR capable (PUNKTFUNK_10BIT + video_source=portal + encoder): {}",
gamestream::host_hdr_capable()
);
Ok(())
}
// Compositor readiness probe: exit 0 iff the (detected or PUNKTFUNK_COMPOSITOR-forced)
// compositor is up and able to create a virtual output *now*. A session-bringup
// script polls this to gate on real readiness instead of a blind `sleep`.
+6 -1
View File
@@ -38,6 +38,7 @@ mod hooks;
mod host;
mod library;
mod native;
mod plugins;
mod session;
mod shared;
mod stats;
@@ -223,7 +224,10 @@ fn api_router_parts() -> (Router<Arc<MgmtState>>, utoipa::openapi::OpenApi) {
))
.routes(routes!(stats::logs_get))
.routes(routes!(events::stream_events))
.routes(routes!(hooks::get_hooks, hooks::set_hooks)),
.routes(routes!(hooks::get_hooks, hooks::set_hooks))
.routes(routes!(plugins::list_plugins))
.routes(routes!(plugins::register_plugin, plugins::delete_plugin))
.routes(routes!(plugins::get_ui_credential)),
)
.split_for_parts()
}
@@ -261,6 +265,7 @@ pub fn openapi_json() -> String {
(name = "logs", description = "Host log stream: the newest in-memory log entries, cursor-paged for live following"),
(name = "events", description = "Host lifecycle events: an SSE stream (client/session/stream lifecycle, pairing, displays, library, host) with Last-Event-ID resume and server-side kind filters"),
(name = "hooks", description = "Operator hooks: commands and webhooks fired on lifecycle events (fire-and-forget — hooks observe, never veto)"),
(name = "plugins", description = "Plugin directory: running `punktfunk-plugin-*` processes register a lease and, optionally, a loopback UI the web console proxies and adds to its nav"),
)
)]
struct ApiDoc;
+30 -7
View File
@@ -36,6 +36,12 @@ pub(crate) async fn get_library(
if let Some(provider) = q.provider.filter(|p| !p.is_empty()) {
games.retain(|g| g.provider.as_deref() == Some(provider.as_str()));
}
// Rewrite provider entries' local-file art into host art-proxy URLs so a client fetches covers
// from the host (a provider like Playnite stores on-host paths; the payload stays tiny at any
// library size, and the client never sees an unreachable `C:\…`).
for g in &mut games {
crate::library::proxy_local_art(&g.id, &mut g.art);
}
Json(games)
}
@@ -246,14 +252,31 @@ pub(crate) async fn get_library_art(Path((id, kind)): Path<(String, String)>) ->
let Some(kind) = crate::library::ArtKind::parse(&kind) else {
return api_error(StatusCode::NOT_FOUND, "unknown art kind");
};
let Some(appid) = id
// Steam: CDN / local-cache proxy (id `steam:<appid>`).
if let Some(appid) = id
.strip_prefix("steam:")
.and_then(|s| s.parse::<u32>().ok())
else {
return api_error(StatusCode::NOT_FOUND, "no art proxy for this store");
};
match tokio::task::spawn_blocking(move || crate::library::steam_art_bytes(appid, kind)).await {
Ok(Some((bytes, ctype))) => ([(header::CONTENT_TYPE, ctype)], bytes).into_response(),
_ => api_error(StatusCode::NOT_FOUND, "no art of that kind for this title"),
{
return match tokio::task::spawn_blocking(move || {
crate::library::steam_art_bytes(appid, kind)
})
.await
{
Ok(Some((bytes, ctype))) => ([(header::CONTENT_TYPE, ctype)], bytes).into_response(),
_ => api_error(StatusCode::NOT_FOUND, "no art of that kind for this title"),
};
}
// Custom/provider entry (id `custom:<id>`): serve its stored LOCAL art file — e.g. the Playnite
// plugin's covers, reconciled as on-host paths rather than inlined bytes.
if let Some(cid) = id.strip_prefix("custom:").map(str::to_owned) {
return match tokio::task::spawn_blocking(move || {
crate::library::custom_local_art_bytes(&cid, kind)
})
.await
{
Ok(Some((bytes, ctype))) => ([(header::CONTENT_TYPE, ctype)], bytes).into_response(),
_ => api_error(StatusCode::NOT_FOUND, "no art of that kind for this title"),
};
}
api_error(StatusCode::NOT_FOUND, "no art proxy for this store")
}
+554
View File
@@ -0,0 +1,554 @@
//! Plugin registry (plugin-ui-surface design): an in-memory, lease-based directory of running
//! `punktfunk-plugin-*` processes and the loopback UI each one serves.
//!
//! A plugin (an out-of-process script under the scripting runner, RFC §8) that wants a UI serves
//! it on a **loopback** port behind a per-boot secret, then **registers** here — `{title, ui:{port,
//! secret, icon}}` — over the admin/loopback lane it already holds via the SDK. The web console
//! reads [`list_plugins`] to grow a nav entry and reverse-proxies to the port (fetching the secret
//! from [`get_ui_credential`] server-side, never exposing it to the browser). The host itself never
//! dials the plugin, never health-checks it, and never persists any of this: it is a phone book with
//! expiry.
//!
//! Lease model (design §3, D8): a registration lives for [`LEASE_TTL`]; the plugin renews with the
//! same idempotent `PUT` every 30 s. Expiry is **lazy** — a crashed plugin's entry simply stops
//! listing once stale; there is no reaper task and nothing to persist across a host restart (the
//! supervised plugin re-registers on its next tick). Every consumer dials `127.0.0.1:<port>` only —
//! a registration stores a *port*, never an address, so it can never point the proxy elsewhere (D5).
//!
//! Auth: these routes carry no special handling — they are outside the [`super::auth::cert_may_access`]
//! read-only allowlist, so the middleware confines them to a **bearer + loopback** peer like every
//! other mutation. LAN clients have no business here.
use super::shared::*;
use crate::events::{emit, EventKind};
use std::collections::HashMap;
use std::sync::{OnceLock, RwLock};
use std::time::{Duration, Instant};
/// How long a registration stays live after its last renewal. The SDK helper renews every 30 s, so
/// this tolerates two missed ticks before a plugin drops out of the listing.
const LEASE_TTL: Duration = Duration::from_secs(90);
// ---------------------------------------------------------------- wire shapes
/// A plugin's UI surface as it registers it. Carries the secret — this shape is only ever a request
/// body, never a response ([`PluginUiPublic`] is the secret-free view).
#[derive(Deserialize, ToSchema)]
pub(crate) struct PluginUi {
/// The **loopback** port the plugin serves its UI on. The host and console only ever dial
/// `127.0.0.1:<port>`; a registration can never carry a hostname.
pub port: u16,
/// Per-boot shared secret the console proxy must present (as `Authorization: Bearer`) on every
/// request to the plugin's UI server. Rotated whenever the plugin restarts.
pub secret: String,
/// Optional lucide icon name for the console nav entry (`^[a-z0-9-]{1,48}$`).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub icon: Option<String>,
}
/// Register/renew body for `PUT /plugins/{id}`.
#[derive(Deserialize, ToSchema)]
pub(crate) struct PluginRegistration {
/// Human-readable title for the console nav entry (164 chars; control chars stripped).
pub title: String,
/// Optional plugin version, purely informational (≤32 chars).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub version: Option<String>,
/// Present iff the plugin serves a UI surface. A registration with no `ui` is a liveness/phone-book
/// entry only (e.g. a future runner-management listing) and grows no nav entry.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub ui: Option<PluginUi>,
}
/// The secret-free view of a plugin's UI surface — what [`list_plugins`] returns to the browser.
#[derive(Serialize, ToSchema)]
pub(crate) struct PluginUiPublic {
pub port: u16,
#[serde(skip_serializing_if = "Option::is_none")]
pub icon: Option<String>,
}
/// One entry in `GET /plugins`. **Never carries the secret** — the browser learns a plugin exists
/// and has a UI, nothing that lets it reach the plugin directly (it goes through the console proxy).
#[derive(Serialize, ToSchema)]
pub(crate) struct PluginSummary {
pub id: String,
pub title: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub version: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub ui: Option<PluginUiPublic>,
}
/// `GET /plugins/{id}/ui-credential` — the console proxy's server-side lookup (bearer + loopback).
/// This is the only endpoint that returns a secret; the console BFF denylists it from the browser.
#[derive(Serialize, ToSchema)]
pub(crate) struct UiCredential {
pub port: u16,
pub secret: String,
}
// ---------------------------------------------------------------- registry core
/// The stored UI surface (internal — parsed + validated, no wire derives).
#[derive(Clone, PartialEq)]
struct StoredUi {
port: u16,
secret: String,
icon: Option<String>,
}
/// One live registration. `expires_at` is a **monotonic** [`Instant`] (immune to wall-clock jumps).
struct Stored {
title: String,
version: Option<String>,
ui: Option<StoredUi>,
expires_at: Instant,
}
impl Stored {
/// Do the operator-visible fields match (ignoring the lease clock)? A pure lease renewal leaves
/// these unchanged and emits no event; a restart (new secret) or a re-scan (new title/icon) does.
fn public_eq(&self, title: &str, version: &Option<String>, ui: &Option<StoredUi>) -> bool {
self.title == title && self.version == *version && self.ui == *ui
}
}
/// The process-wide plugin registry.
pub(crate) struct PluginRegistry {
inner: RwLock<HashMap<String, Stored>>,
}
/// A validated registration ready to store (title trimmed, ui fields checked).
struct Valid {
title: String,
version: Option<String>,
ui: Option<StoredUi>,
}
impl PluginRegistry {
fn new() -> Self {
Self {
inner: RwLock::new(HashMap::new()),
}
}
/// Insert or renew `id`. Returns `true` when an operator-visible field changed (new plugin,
/// restart, or re-scan) — the signal the caller emits `plugins.changed` on. A pure renewal
/// returns `false`.
fn upsert(&self, id: &str, v: Valid) -> bool {
let expires_at = Instant::now() + LEASE_TTL;
let mut map = self.inner.write().unwrap_or_else(|e| e.into_inner());
let changed = match map.get(id) {
// An *expired* prior entry counts as a change (it had stopped listing).
Some(prev) => !prev.is_live() || !prev.public_eq(&v.title, &v.version, &v.ui),
None => true,
};
map.insert(
id.to_string(),
Stored {
title: v.title,
version: v.version,
ui: v.ui,
expires_at,
},
);
changed
}
/// The current listing (sorted by title, then id), plus the ids of entries that had expired and
/// were pruned by this call — the caller emits `plugins.changed` for those. Prunes under the
/// write lock so a stale entry is reaped exactly once.
fn snapshot(&self) -> (Vec<PluginSummary>, Vec<String>) {
let now = Instant::now();
let mut map = self.inner.write().unwrap_or_else(|e| e.into_inner());
let mut expired: Vec<String> = map
.iter()
.filter(|(_, s)| now >= s.expires_at)
.map(|(id, _)| id.clone())
.collect();
for id in &expired {
map.remove(id);
}
expired.sort();
let mut live: Vec<PluginSummary> = map
.iter()
.map(|(id, s)| PluginSummary {
id: id.clone(),
title: s.title.clone(),
version: s.version.clone(),
ui: s.ui.as_ref().map(|u| PluginUiPublic {
port: u.port,
icon: u.icon.clone(),
}),
})
.collect();
live.sort_by(|a, b| a.title.cmp(&b.title).then_with(|| a.id.cmp(&b.id)));
(live, expired)
}
/// The `{port, secret}` for a live plugin's UI, or `None` if unknown/expired/UI-less. Does not
/// prune (a read path) — a stale entry is reaped by the next [`snapshot`](Self::snapshot).
fn credential(&self, id: &str) -> Option<UiCredential> {
let map = self.inner.read().unwrap_or_else(|e| e.into_inner());
let s = map.get(id)?;
if !s.is_live() {
return None;
}
let ui = s.ui.as_ref()?;
Some(UiCredential {
port: ui.port,
secret: ui.secret.clone(),
})
}
/// Remove `id`. Returns `true` if a **live** entry existed (a clean deregister); removing an
/// already-expired or unknown id returns `false` (nothing to announce).
fn remove(&self, id: &str) -> bool {
let mut map = self.inner.write().unwrap_or_else(|e| e.into_inner());
map.remove(id).is_some_and(|s| s.is_live())
}
}
impl Stored {
fn is_live(&self) -> bool {
Instant::now() < self.expires_at
}
}
/// The process-wide registry singleton (the [`crate::events::bus`] shape).
pub(crate) fn registry() -> &'static PluginRegistry {
static REG: OnceLock<PluginRegistry> = OnceLock::new();
REG.get_or_init(PluginRegistry::new)
}
// ---------------------------------------------------------------- validation
/// A plugin id: `definePlugin`'s kebab-case name (`^[a-z][a-z0-9-]*$`, ≤64) — the same regex the SDK
/// enforces, so a plugin's registration id always matches its package name.
fn valid_plugin_id(id: &str) -> bool {
!id.is_empty()
&& id.len() <= 64
&& id.as_bytes()[0].is_ascii_lowercase()
&& id
.bytes()
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-')
}
/// Strip control characters (defense against a title/version smuggling terminal escapes or newlines
/// into a log line or the console nav), then trim.
fn sanitize(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control())
.collect::<String>()
.trim()
.to_string()
}
/// Validate a registration body into the internal [`Valid`] form, or a human-readable reason.
fn validate(reg: PluginRegistration) -> Result<Valid, String> {
let title = sanitize(&reg.title);
if title.is_empty() {
return Err("title must not be empty".into());
}
if title.chars().count() > 64 {
return Err("title must be at most 64 characters".into());
}
let version = match reg.version {
Some(v) => {
let v = sanitize(&v);
if v.chars().count() > 32 {
return Err("version must be at most 32 characters".into());
}
(!v.is_empty()).then_some(v)
}
None => None,
};
let ui = match reg.ui {
Some(u) => Some(validate_ui(u)?),
None => None,
};
Ok(Valid { title, version, ui })
}
fn validate_ui(u: PluginUi) -> Result<StoredUi, String> {
if u.port < 1024 {
return Err("ui.port must be a non-privileged port (>= 1024)".into());
}
let n = u.secret.len();
if !(16..=128).contains(&n) {
return Err("ui.secret must be 16128 characters".into());
}
if !u
.secret
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'_' || b == b'-')
{
return Err("ui.secret must be [A-Za-z0-9_-]".into());
}
let icon = match u.icon {
Some(icon) => {
let ok = (1..=48).contains(&icon.len())
&& icon
.bytes()
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-');
if !ok {
return Err("ui.icon must be a lucide name ([a-z0-9-], 148 chars)".into());
}
Some(icon)
}
None => None,
};
Ok(StoredUi {
port: u.port,
secret: u.secret,
icon,
})
}
// ---------------------------------------------------------------- handlers
/// Register or renew a plugin
///
/// Upserts the plugin's directory entry and renews its lease (TTL 90 s). Idempotent: a plugin PUTs
/// this every ~30 s while it runs. The optional `ui` block declares a loopback UI surface the console
/// will proxy and add to its nav. Emits `plugins.changed` when an operator-visible field changed
/// (first registration, restart, or re-scan) — a pure renewal is silent.
#[utoipa::path(
put,
path = "/plugins/{id}",
tag = "plugins",
operation_id = "registerPlugin",
params(("id" = String, Path, description = "The plugin id (its `definePlugin` name: `[a-z][a-z0-9-]*`)")),
request_body = PluginRegistration,
responses(
(status = NO_CONTENT, description = "Registered / renewed"),
(status = BAD_REQUEST, description = "Invalid id or registration", body = ApiError),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
pub(crate) async fn register_plugin(
Path(id): Path<String>,
ApiJson(reg): ApiJson<PluginRegistration>,
) -> Response {
if !valid_plugin_id(&id) {
return api_error(
StatusCode::BAD_REQUEST,
"invalid plugin id (expected kebab-case `[a-z][a-z0-9-]*`, ≤64)",
);
}
let valid = match validate(reg) {
Ok(v) => v,
Err(e) => return api_error(StatusCode::BAD_REQUEST, &e),
};
if registry().upsert(&id, valid) {
tracing::info!(plugin = %id, "plugin registered");
emit(EventKind::PluginsChanged { id });
}
StatusCode::NO_CONTENT.into_response()
}
/// List registered plugins
///
/// The live plugin directory (lease not expired), sorted by title. **Secret-free**: each entry
/// reports its id, title, optional version, and — for plugins that serve one — a UI descriptor
/// (loopback port + icon). The console renders these as nav entries and proxies to the port; it
/// fetches the secret separately, server-side.
#[utoipa::path(
get,
path = "/plugins",
tag = "plugins",
operation_id = "listPlugins",
responses(
(status = OK, description = "Live plugin registrations", body = [PluginSummary]),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
pub(crate) async fn list_plugins() -> Json<Vec<PluginSummary>> {
let (plugins, expired) = registry().snapshot();
// Lazy expiry: an entry that aged out is reaped here (exactly once) and announced, so a consumer
// watching the event stream sees the departure even though nothing actively deregistered it.
for id in expired {
tracing::info!(plugin = %id, "plugin lease expired");
emit(EventKind::PluginsChanged { id });
}
Json(plugins)
}
/// Fetch a plugin UI's proxy credential
///
/// Returns `{port, secret}` for a live plugin's loopback UI — the console proxy's server-side lookup.
/// Bearer + loopback only (like every mutation), and additionally excluded from the console's browser
/// passthrough: the secret never reaches a browser.
#[utoipa::path(
get,
path = "/plugins/{id}/ui-credential",
tag = "plugins",
operation_id = "getPluginUiCredential",
params(("id" = String, Path, description = "The plugin id")),
responses(
(status = OK, description = "The proxy credential", body = UiCredential),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
(status = NOT_FOUND, description = "No live plugin with that id, or it serves no UI", body = ApiError),
)
)]
pub(crate) async fn get_ui_credential(Path(id): Path<String>) -> Response {
match registry().credential(&id) {
Some(cred) => Json(cred).into_response(),
None => api_error(StatusCode::NOT_FOUND, "no live plugin UI with that id"),
}
}
/// Deregister a plugin
///
/// The clean-shutdown path: removes the plugin's directory entry immediately (the SDK helper calls
/// this from its scope finalizer on `SIGTERM`). Emits `plugins.changed` when a live entry was
/// removed. Idempotent — deleting an unknown/expired id is a no-op `204`.
#[utoipa::path(
delete,
path = "/plugins/{id}",
tag = "plugins",
operation_id = "deregisterPlugin",
params(("id" = String, Path, description = "The plugin id")),
responses(
(status = NO_CONTENT, description = "Deregistered (or already absent)"),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
pub(crate) async fn delete_plugin(Path(id): Path<String>) -> Response {
if registry().remove(&id) {
tracing::info!(plugin = %id, "plugin deregistered");
emit(EventKind::PluginsChanged { id });
}
StatusCode::NO_CONTENT.into_response()
}
#[cfg(test)]
mod tests {
use super::*;
fn reg(title: &str, port: u16, secret: &str) -> PluginRegistration {
PluginRegistration {
title: title.into(),
version: None,
ui: Some(PluginUi {
port,
secret: secret.into(),
icon: Some("gamepad-2".into()),
}),
}
}
const SECRET: &str = "abcdefghijklmnop0123"; // 20 chars, valid alphabet
#[test]
fn id_validation() {
assert!(valid_plugin_id("rom-manager"));
assert!(valid_plugin_id("a"));
assert!(valid_plugin_id("x9"));
assert!(!valid_plugin_id("")); // empty
assert!(!valid_plugin_id("9lives")); // must start with a letter
assert!(!valid_plugin_id("-lead")); // must start with a letter
assert!(!valid_plugin_id("Rom")); // no uppercase
assert!(!valid_plugin_id("rom_manager")); // no underscore
assert!(!valid_plugin_id(&"a".repeat(65))); // too long
}
#[test]
fn registration_validation() {
assert!(validate(reg("ROM Manager", 49321, SECRET)).is_ok());
// control chars stripped from the title
let v = validate(PluginRegistration {
title: "Ro\u{7}m\n".into(),
version: None,
ui: None,
})
.unwrap();
assert_eq!(v.title, "Rom");
// privileged port rejected
assert!(validate(reg("x", 80, SECRET)).is_err());
// short secret rejected
assert!(validate(reg("x", 49321, "tooshort")).is_err());
// bad secret alphabet rejected
assert!(validate(reg("x", 49321, "bad secret with spaces!!")).is_err());
// empty title rejected
assert!(validate(reg(" ", 49321, SECRET)).is_err());
}
#[test]
fn upsert_reports_change_but_not_renewal() {
let r = PluginRegistry::new();
// first registration is a change
assert!(r.upsert("p", validate(reg("Title", 49321, SECRET)).unwrap()));
// identical renewal is not
assert!(!r.upsert("p", validate(reg("Title", 49321, SECRET)).unwrap()));
// a new secret (restart) is a change
assert!(r.upsert(
"p",
validate(reg("Title", 49321, "ZZZZZZZZZZZZZZZZ")).unwrap()
));
// a new title (re-scan) is a change
assert!(r.upsert(
"p",
validate(reg("New Title", 49321, "ZZZZZZZZZZZZZZZZ")).unwrap()
));
}
#[test]
fn snapshot_lists_secret_free_sorted() {
let r = PluginRegistry::new();
r.upsert("zeta", validate(reg("Zeta", 50000, SECRET)).unwrap());
r.upsert("alpha", validate(reg("Alpha", 50001, SECRET)).unwrap());
let (plugins, expired) = r.snapshot();
assert!(expired.is_empty());
assert_eq!(
plugins.iter().map(|p| p.id.as_str()).collect::<Vec<_>>(),
vec!["alpha", "zeta"] // sorted by title
);
// the summary type has no secret field at all — check the credential path carries it
assert_eq!(r.credential("alpha").unwrap().secret, SECRET);
assert_eq!(r.credential("alpha").unwrap().port, 50001);
}
#[test]
fn credential_absent_for_ui_less_and_unknown() {
let r = PluginRegistry::new();
r.upsert(
"headless",
validate(PluginRegistration {
title: "Headless".into(),
version: None,
ui: None,
})
.unwrap(),
);
assert!(r.credential("headless").is_none()); // registered but no UI
assert!(r.credential("nope").is_none()); // unknown
}
#[test]
fn expired_entries_drop_from_listing_and_credential() {
let r = PluginRegistry::new();
r.upsert("p", validate(reg("P", 49321, SECRET)).unwrap());
// Force the lease into the past.
{
let mut map = r.inner.write().unwrap();
map.get_mut("p").unwrap().expires_at =
Instant::now().checked_sub(Duration::from_secs(1)).unwrap();
}
assert!(r.credential("p").is_none()); // expired → no credential
let (plugins, expired) = r.snapshot();
assert!(plugins.is_empty());
assert_eq!(expired, vec!["p".to_string()]); // reaped + announced once
// second snapshot no longer reports it as freshly-expired
assert!(r.snapshot().1.is_empty());
}
#[test]
fn remove_reports_live_only() {
let r = PluginRegistry::new();
r.upsert("p", validate(reg("P", 49321, SECRET)).unwrap());
assert!(r.remove("p")); // live → announced
assert!(!r.remove("p")); // already gone → silent
}
}
+95
View File
@@ -138,6 +138,18 @@ async fn cert_auth_is_a_read_only_allowlist() {
"the client roster {p} must require the bearer token, not just a paired cert"
);
}
// The plugin directory is admin-only — a paired streaming cert has no business enumerating the
// host's running plugins or reaching a plugin UI's proxy credential (plugin-ui-surface §3).
for p in [
"/api/v1/plugins",
"/api/v1/plugins/rom-manager/ui-credential",
] {
assert_eq!(
send_cert(&app, get_req(p), fp).await,
StatusCode::UNAUTHORIZED,
"the plugin directory {p} must require the bearer token, not just a paired cert"
);
}
// PIN-exposing GET + state-changing routes → token-only (cert rejected without a bearer).
assert_eq!(
send_cert(&app, get_req("/api/v1/native/pair"), fp).await,
@@ -574,6 +586,89 @@ async fn idr_requires_an_active_stream() {
assert!(state.force_idr.load(Ordering::SeqCst));
}
/// The plugin registry round-trips through the router: register → list (secret-free) → credential
/// (secret present) → deregister. Guards the wiring, auth, and — the security-critical bit — that
/// the UI secret never appears in the browser-visible listing (plugin-ui-surface §7, D6).
#[tokio::test]
async fn plugin_registry_roundtrip() {
let app = test_app(test_state(), None);
let id = "test-plugin-roundtrip";
let secret = "s3cr3t-abcdefghijkl"; // 19 chars, valid [A-Za-z0-9_-]
// Register with a UI surface → 204.
let (status, _) = send(
&app,
put_json(
&format!("/api/v1/plugins/{id}"),
serde_json::json!({
"title": "Test Plugin",
"ui": { "port": 49321, "secret": secret, "icon": "gamepad-2" }
}),
),
)
.await;
assert_eq!(status, StatusCode::NO_CONTENT);
// It lists — and the secret appears NOWHERE in the listing body.
let (status, body) = send(&app, get_req("/api/v1/plugins")).await;
assert_eq!(status, StatusCode::OK);
let mine = body
.as_array()
.unwrap()
.iter()
.find(|p| p["id"] == id)
.expect("registered plugin is listed");
assert_eq!(mine["title"], "Test Plugin");
assert_eq!(mine["ui"]["port"], 49321);
assert_eq!(mine["ui"]["icon"], "gamepad-2");
assert!(
!body.to_string().contains(secret),
"the listing must never carry the UI secret"
);
// The credential endpoint (server-side proxy lookup) DOES carry it.
let (status, body) = send(
&app,
get_req(&format!("/api/v1/plugins/{id}/ui-credential")),
)
.await;
assert_eq!(status, StatusCode::OK);
assert_eq!(body["secret"], secret);
assert_eq!(body["port"], 49321);
// Deregister → gone from the listing, credential 404s.
let (status, _) = send(
&app,
axum::http::Request::delete(format!("/api/v1/plugins/{id}"))
.body(Body::empty())
.unwrap(),
)
.await;
assert_eq!(status, StatusCode::NO_CONTENT);
let (_, body) = send(&app, get_req("/api/v1/plugins")).await;
assert!(
body.as_array().unwrap().iter().all(|p| p["id"] != id),
"deregistered plugin must not list"
);
let (status, _) = send(
&app,
get_req(&format!("/api/v1/plugins/{id}/ui-credential")),
)
.await;
assert_eq!(status, StatusCode::NOT_FOUND);
// A structurally invalid registration is a 400 (privileged port).
let (status, _) = send(
&app,
put_json(
&format!("/api/v1/plugins/{id}"),
serde_json::json!({ "title": "x", "ui": { "port": 80, "secret": secret } }),
),
)
.await;
assert_eq!(status, StatusCode::BAD_REQUEST);
}
/// The OpenAPI document lists every route with a unique operationId (codegen relies
/// on both), and the checked-in copy is current.
#[test]
+124 -6
View File
@@ -521,17 +521,55 @@ fn resolve_bitrate_kbps_for(
codec: crate::encode::Codec,
requested: u32,
mode: &punktfunk_core::config::Mode,
chroma: crate::encode::ChromaFormat,
bit_depth: u8,
) -> u32 {
if requested == 0 && codec == crate::encode::Codec::PyroWave {
let bps =
mode.width as u64 * mode.height as u64 * u64::from(mode.refresh_hz.max(1)) * 16 / 10;
return u32::try_from(bps / 1000)
// ~1.6 bpp for 4:2:0. 4:4:4 doubles the samples per pixel (3 vs 1.5) but chroma
// compresses better than luma → ×1.625 ≈ 2.6 bpp; 16-bit planes add ~15 % (both
// factors measured against the Phase-0 fixture matrix, design/pyrowave-444-hdr.md).
let bpp_x10: u64 = if chroma.is_444() { 26 } else { 16 };
let mut bps =
mode.width as u64 * mode.height as u64 * u64::from(mode.refresh_hz.max(1)) * bpp_x10
/ 10;
if bit_depth >= 10 {
bps = bps * 115 / 100;
}
let pin = u32::try_from(bps / 1000)
.unwrap_or(MAX_BITRATE_KBPS)
.clamp(MIN_BITRATE_KBPS, MAX_BITRATE_KBPS);
// Operator link ceiling. PyroWave's Automatic pin is open-loop (all-intra, so ABR and the
// capacity probe are off) — at a high pixel rate it can outrun the physical link (e.g.
// 4:4:4 + HDR at 5120x1440@240 pins ~5.3 Gbps, over a 5 GbE link), and the overshoot just
// becomes packet loss / partial frames. `PUNKTFUNK_PYROWAVE_MAX_MBPS` lets a host on a
// constrained link cap the pin to what the fabric carries; unset ⇒ no cap (unchanged).
if let Some(ceiling) = pyrowave_auto_pin_ceiling_kbps() {
if pin > ceiling {
tracing::warn!(
pin_kbps = pin,
ceiling_kbps = ceiling,
"PyroWave Automatic bitrate pin exceeds PUNKTFUNK_PYROWAVE_MAX_MBPS — capping \
to the link ceiling (set an explicit client bitrate to choose your own)"
);
return ceiling.max(MIN_BITRATE_KBPS);
}
}
return pin;
}
resolve_bitrate_kbps(requested)
}
/// Operator ceiling for PyroWave's open-loop Automatic bitrate pin: `PUNKTFUNK_PYROWAVE_MAX_MBPS`
/// (megabits/s) → kbps, or `None` when unset/zero/invalid (no cap — the raw bpp pin stands).
/// Only consulted for `requested == 0` PyroWave sessions; an explicit client bitrate bypasses it.
fn pyrowave_auto_pin_ceiling_kbps() -> Option<u32> {
std::env::var("PUNKTFUNK_PYROWAVE_MAX_MBPS")
.ok()
.and_then(|s| s.trim().parse::<u32>().ok())
.filter(|&m| m > 0)
.map(|m| m.saturating_mul(1000))
}
/// Resolve the audio channel count the session will capture + encode from the client's request.
/// Normalizes to one of 2 (stereo) / 6 (5.1) / 8 (7.1); anything else (older client, garbage)
/// becomes stereo. Both backends can produce the requested count (PipeWire pads/upmixes positions,
@@ -1458,22 +1496,102 @@ mod tests {
height: 1080,
refresh_hz: 60,
};
use crate::encode::ChromaFormat;
// Automatic (0) on PyroWave → the ~1.6 bpp operating point, not the 20 Mbps H.26x
// default (which would turn wavelets to mush — plan §4.6).
let kbps = resolve_bitrate_kbps_for(crate::encode::Codec::PyroWave, 0, &mode);
let kbps = resolve_bitrate_kbps_for(
crate::encode::Codec::PyroWave,
0,
&mode,
ChromaFormat::Yuv420,
8,
);
assert_eq!(kbps, 1920 * 1080 * 60 * 16 / 10 / 1000);
// 4:4:4 scales the pin to ~2.6 bpp, 10-bit adds 15 % (design/pyrowave-444-hdr.md §2.5).
assert_eq!(
resolve_bitrate_kbps_for(
crate::encode::Codec::PyroWave,
0,
&mode,
ChromaFormat::Yuv444,
8
),
1920 * 1080 * 60 * 26 / 10 / 1000
);
assert_eq!(
resolve_bitrate_kbps_for(
crate::encode::Codec::PyroWave,
0,
&mode,
ChromaFormat::Yuv444,
10
),
(1920u64 * 1080 * 60 * 26 / 10 * 115 / 100 / 1000) as u32
);
// An explicit client rate is honored (clamped like any other codec)...
assert_eq!(
resolve_bitrate_kbps_for(crate::encode::Codec::PyroWave, 130_000, &mode),
resolve_bitrate_kbps_for(
crate::encode::Codec::PyroWave,
130_000,
&mode,
ChromaFormat::Yuv420,
8
),
130_000
);
// ...and the H.26x codecs keep the legacy default.
assert_eq!(
resolve_bitrate_kbps_for(crate::encode::Codec::H265, 0, &mode),
resolve_bitrate_kbps_for(
crate::encode::Codec::H265,
0,
&mode,
ChromaFormat::Yuv420,
8
),
DEFAULT_BITRATE_KBPS
);
}
#[test]
fn pyrowave_auto_pin_respects_operator_ceiling() {
use crate::encode::{ChromaFormat, Codec};
use punktfunk_core::config::Mode;
// 5120x1440@240 4:4:4 10-bit pins ~5.29 Gbps open-loop — above a 5 GbE link.
let mode = Mode {
width: 5120,
height: 1440,
refresh_hz: 240,
};
let uncapped =
resolve_bitrate_kbps_for(Codec::PyroWave, 0, &mode, ChromaFormat::Yuv444, 10);
assert!(
uncapped > 5_000_000,
"expected the open-loop pin, got {uncapped}"
);
// With the operator ceiling set, the Automatic pin is capped to the link rate...
std::env::set_var("PUNKTFUNK_PYROWAVE_MAX_MBPS", "4500");
assert_eq!(
resolve_bitrate_kbps_for(Codec::PyroWave, 0, &mode, ChromaFormat::Yuv444, 10),
4_500_000
);
// ...but a pin already under the ceiling is untouched (1080p60 4:2:0 ≈ 199 Mbps)...
let small = Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
};
assert_eq!(
resolve_bitrate_kbps_for(Codec::PyroWave, 0, &small, ChromaFormat::Yuv420, 8),
1920 * 1080 * 60 * 16 / 10 / 1000
);
// ...and an explicit client rate bypasses the ceiling entirely.
assert_eq!(
resolve_bitrate_kbps_for(Codec::PyroWave, 6_000_000, &mode, ChromaFormat::Yuv444, 10),
6_000_000
);
std::env::remove_var("PUNKTFUNK_PYROWAVE_MAX_MBPS");
}
#[test]
fn adapt_fec_maps_loss_to_recovery_band() {
// A perfectly clean window (0 loss) lands on the floor.
+53 -14
View File
@@ -187,14 +187,8 @@ pub(super) async fn negotiate(
// needed; the actual pads are created lazily by the input thread).
let gamepad = resolve_gamepad(hello.gamepad);
// Resolve the encoder bitrate (client request clamped to a sane range, or a
// codec-aware host default — PyroWave pins ~1.6 bpp for the mode).
let bitrate_kbps = resolve_bitrate_kbps_for(codec, hello.bitrate_kbps, &hello.mode);
tracing::info!(
requested_kbps = hello.bitrate_kbps,
resolved_kbps = bitrate_kbps,
"encoder bitrate"
);
// (The encoder bitrate is resolved below, AFTER bit depth + chroma: PyroWave's automatic
// ~bpp pin scales with both — design/pyrowave-444-hdr.md §2.5.)
// Resolve the audio channel count (client request → stereo / 5.1 / 7.1). The capturer opens
// at this count: PipeWire synthesizes the requested positions (padding with silence when the
@@ -218,10 +212,22 @@ pub(super) async fn negotiate(
// label that matches the stream.
let host_wants_10bit = pf_host_config::config().ten_bit;
let client_supports_10bit = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_10BIT != 0;
// The capture side must be able to deliver a 10-bit HDR source for the NATIVE plane's
// virtual-output capture — the honest-downgrade gate, mirroring `capturer_supports_444`.
// Windows IDD-push can (it proactively enables advanced colour); Linux cannot: Mutter's
// RecordVirtual virtual-monitor streams are 8-bit-only upstream (GNOME 50 added HDR for
// *monitor* streams only — the GameStream portal-mirror path uses that; see
// `gamestream::host_hdr_capable`), so a Linux native session honestly stays 8-bit SDR even
// though `can_encode_10bit` now probes true on a Main10-capable GPU.
let capture_supports_hdr = crate::capture::capturer_supports_hdr();
// The GPU probe may open a tiny encoder on first use, so run it off the reactor like the
// 4:4:4 probe below (blocking probes → spawn_blocking), short-circuited behind the cheap
// gates. The result is cached process-wide per (GPU, codec).
let gpu_can_10bit = if host_wants_10bit && client_supports_10bit && codec.supports_10bit() {
let gpu_can_10bit = if host_wants_10bit
&& client_supports_10bit
&& codec.supports_10bit()
&& capture_supports_hdr
{
tokio::task::spawn_blocking(move || crate::encode::can_encode_10bit(codec))
.await
.context("10-bit capability probe task")?
@@ -233,6 +239,7 @@ pub(super) async fn negotiate(
bit_depth,
host_wants_10bit,
client_supports_10bit,
capture_supports_hdr,
codec = ?codec,
gpu_can_10bit,
client_video_caps = hello.video_caps,
@@ -255,19 +262,24 @@ pub(super) async fn negotiate(
// today (full-chroma IDD-push capture is a follow-up), so it returns false there and the host
// negotiates 4:2:0. (Replaces the old `single_process` gate — single-process is now the only
// topology, and 4:4:4 routed to DDA, which was removed.)
let capture_supports_444 =
crate::capture::capturer_supports_444(crate::encode::resolved_backend_ingests_rgb_444());
// PyroWave does its own RGB→YCbCr CSC and its capture mode always delivers a full-chroma
// (RGB/BGRA) source on both OSes — the capturer gate is inherently satisfied; the real
// gate is `can_encode_444` (the full-res-chroma CSC variant existing on this OS).
let capture_supports_444 = codec == crate::encode::Codec::PyroWave
|| crate::capture::capturer_supports_444(crate::encode::resolved_backend_ingests_rgb_444());
// The GPU probe opens a real (tiny) encoder on first use, so run it off the reactor like the
// compositor probe above (blocking probes → spawn_blocking). Short-circuit so it only runs when
// the cheap gates already pass. The result is cached process-wide (a negative latches until
// restart — acceptable: a GPU either supports HEVC 4:4:4 or it doesn't, and a transient open
// failure here is rare since the session's own encoder isn't open yet).
let gpu_supports_444 = if codec == crate::encode::Codec::H265
&& host_wants_444
let gpu_supports_444 = if matches!(
codec,
crate::encode::Codec::H265 | crate::encode::Codec::PyroWave
) && host_wants_444
&& client_supports_444
&& capture_supports_444
{
tokio::task::spawn_blocking(|| crate::encode::can_encode_444(crate::encode::Codec::H265))
tokio::task::spawn_blocking(move || crate::encode::can_encode_444(codec))
.await
.context("4:4:4 capability probe task")?
} else {
@@ -278,6 +290,22 @@ pub(super) async fn negotiate(
} else {
crate::encode::ChromaFormat::Yuv420
};
// PyroWave-only mode-size gate: the vendored rate controller packs its block index into
// 16 bits (pyrowave-sys patches/0002 note), which ≈8K-class 4:4:4 overflows — downgrade
// to 4:2:0 BEFORE the Welcome (the honest-downgrade channel), like every gate above.
let chroma = if codec == crate::encode::Codec::PyroWave
&& chroma.is_444()
&& !crate::encode::pyrowave_mode_fits_rdo(hello.mode.width, hello.mode.height, true)
{
tracing::warn!(
mode = %format_args!("{}x{}", hello.mode.width, hello.mode.height),
"PyroWave 4:4:4 at this mode exceeds the rate controller's block-index range — \
negotiating 4:2:0"
);
crate::encode::ChromaFormat::Yuv420
} else {
chroma
};
tracing::info!(
chroma = ?chroma,
host_wants_444,
@@ -299,6 +327,17 @@ pub(super) async fn negotiate(
bit_depth
};
// Resolve the encoder bitrate (client request clamped to a sane range, or a codec-aware
// host default). Resolved AFTER depth + chroma: PyroWave's Automatic rate is a ~bpp pin
// for the negotiated mode that scales with both (design/pyrowave-444-hdr.md §2.5).
let bitrate_kbps =
resolve_bitrate_kbps_for(codec, hello.bitrate_kbps, &hello.mode, chroma, bit_depth);
tracing::info!(
requested_kbps = hello.bitrate_kbps,
resolved_kbps = bitrate_kbps,
"encoder bitrate"
);
// Reserve the data-plane UDP socket up front and HOLD it through streaming (no
// bind→read→drop→rebind window a concurrent session could race for a fixed port). A fixed
// `--data-port` yields `direct = true` (stream straight to the client's reported address,
+51 -2
View File
@@ -978,6 +978,45 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
let _ = &launch;
// Dedicated Steam launch: end the stream cleanly when the LAUNCHED GAME exits. The node-death
// check in the capture-loss branch below can't see this for Steam — the nested `steam` is the
// resident client and stays up after a game quits, so gamescope (and its node) never dies and the
// stream would sit on a hidden Steam session forever. Watch the game process directly (Steam's
// `SteamLaunch AppId=<id>` reaper, whose lifetime == the game's) and close with APP_EXITED when
// it's gone, so a launcher client returns to its library. Non-Steam nested launches keep the
// node-death path (gamescope's child IS the game). Cancelled via `stop` when the session ends for
// another reason first; the thread self-terminates, so we don't join it.
#[cfg(target_os = "linux")]
let _game_watch = launch
.as_deref()
.filter(|_| crate::vdisplay::launch_is_nested(compositor))
.and_then(crate::vdisplay::steam_appid_from_launch)
.map(|appid| {
let conn = conn.clone();
let stop = stop.clone();
let quit = quit.clone();
std::thread::Builder::new()
.name("pf1-gamewatch".into())
.spawn(move || {
if crate::vdisplay::watch_steam_game_exit(appid, &stop) {
tracing::info!(
appid,
"dedicated Steam game exited — ending the session cleanly (APP_EXITED)"
);
// Close FIRST so APP_EXITED is the winning close code (quinn keeps the first
// application close), then set the flags: `quit` skips the display lease's
// keep-alive linger and `stop` wakes the encode/send loops out.
conn.close(
punktfunk_core::quic::APP_EXITED_CLOSE_CODE.into(),
b"game exited",
);
quit.store(true, Ordering::SeqCst);
stop.store(true, Ordering::SeqCst);
}
})
.ok()
});
let perf = pf_host_config::config().perf;
// Microburst cap (applied in send_loop/paced_submit): a frame ≤ the cap bursts out
// immediately; only a bigger frame's overflow is spread. `None` = auto — max(128 KB, the
@@ -1262,7 +1301,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// so re-resolve it for the new mode. Explicit client rates stay put (the operator knows
// the link), and the H.26x codecs keep their mode-independent rate (ABR owns it).
let mode_bitrate = if bitrate_auto && plan.codec == crate::encode::Codec::PyroWave {
resolve_bitrate_kbps_for(plan.codec, 0, &new_mode)
resolve_bitrate_kbps_for(plan.codec, 0, &new_mode, plan.chroma, plan.bit_depth)
} else {
bitrate_kbps
};
@@ -2002,7 +2041,17 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// rate at ~1.11× target when the source runs faster (compositor Hz > session fps);
// the +0.5×interval keepalive keeps a static desktop re-encoding (bitrate shape,
// client liveness) at 1.5×interval cadence and bounds control-servicing latency.
let earliest = next - interval.mul_f32(0.1);
//
// Anchor the floor to THIS frame's arrival (`t_cap`), not to `next` — `next` is
// re-based to the instant *after* submit(), so a synchronous encoder folds its whole
// encode into the cadence: PyroWave's ~2 ms inline encode pushes the floor out by
// that much, the loop misses the next arrival and samples one interval late, and the
// period becomes interval + encode (≈158 fps off a 240 Hz source; 360 Hz → ~200).
// An async encoder (NVENC) returns from submit in ≈0, so t_cap ≈ post-submit and this
// is a no-op for it — which is why H.26x already holds full rate. Arrival-anchoring
// lets the synchronous encode overlap the interval; the ≥0.9×interval spacing from
// the last grab still caps the rate at ~1.11× target.
let earliest = t_cap + interval.mul_f32(0.9);
if let Some(d) = earliest.checked_duration_since(std::time::Instant::now()) {
std::thread::sleep(d);
}
+8 -1
View File
@@ -89,7 +89,9 @@ pub struct SessionPlan {
/// Handshake-negotiated encode bit depth (8, or 10 = HEVC Main10).
pub bit_depth: u8,
/// The IDD-push HDR hint (`bit_depth >= 10`) — the want-HDR flag handed to the capturer so it
/// proactively enables advanced color on the virtual display. Linux is 8-bit (HDR blocked upstream).
/// proactively enables advanced color on the virtual display. The Linux NATIVE plane is 8-bit
/// (Mutter's virtual-monitor streams are SDR-only upstream — GNOME 50 HDR is monitor-mirror
/// only, which the GameStream portal path uses; see `capture::capturer_supports_hdr`).
pub hdr: bool,
/// Handshake-negotiated chroma subsampling (4:2:0, or full-chroma 4:4:4 when the client + host +
/// GPU all support it). Resolved before the Welcome; `Yuv420` on every backend that declined it.
@@ -179,6 +181,11 @@ impl SessionPlan {
// 4:4:4 needs a full-chroma source: on Windows this keeps the capturer on RGB (not the
// default NV12/P010 video-engine output) so NVENC can CSC to 4:4:4.
chroma_444: self.chroma.is_444(),
// PyroWave (Windows): the IDD-push capturer makes its NV12 out-ring shareable + signals a
// shared fence so the wavelet encoder can zero-copy-import the texture into its own Vulkan
// device. Inert on Linux (the wavelet backend ingests dmabufs / CPU RGB there — handled
// by the `gpu` flips above, not this flag).
pyrowave: self.codec == crate::encode::Codec::PyroWave,
}
}
}
+8 -2
View File
@@ -87,8 +87,14 @@ pub fn run(opts: Options) -> Result<()> {
}
}
Source::Portal => {
tracing::info!("spike source: xdg ScreenCast portal (live monitor)");
capture::open_portal_monitor().context("open portal capturer")?
// PUNKTFUNK_SPIKE_HDR=1: run the GNOME 50+ HDR offer (10-bit PQ dmabufs) — the dev
// validation lever for the Linux HDR capture path without a full GameStream client.
let want_hdr = std::env::var("PUNKTFUNK_SPIKE_HDR").as_deref() == Ok("1");
tracing::info!(
want_hdr,
"spike source: xdg ScreenCast portal (live monitor)"
);
capture::open_portal_monitor(want_hdr).context("open portal capturer")?
}
Source::KwinVirtual => {
let compositor = crate::vdisplay::detect().unwrap_or(crate::vdisplay::Compositor::Kwin);
@@ -0,0 +1,20 @@
diff --git a/crates/pyrowave-sys/vendor/pyrowave/pyrowave_encoder.cpp b/crates/pyrowave-sys/vendor/pyrowave/pyrowave_encoder.cpp
index f5ac6dcc..ad4e9746 100644
--- a/crates/pyrowave-sys/vendor/pyrowave/pyrowave_encoder.cpp
+++ b/crates/pyrowave-sys/vendor/pyrowave/pyrowave_encoder.cpp
@@ -187,8 +187,13 @@ void Encoder::Impl::init_block_meta()
meta_buffer = device->create_buffer(info);
device->set_name(*meta_buffer, "meta-buffer");
- // Worst case estimate.
- info.size = aligned_width * aligned_height * 2;
+ // Worst case estimate. PUNKTFUNK PATCH (patches/0001-payload-data-444-sizing.patch):
+ // 4:4:4 carries 3 samples per pixel vs 4:2:0's 1.5 — the same per-sample headroom
+ // needs twice the bytes, or busy 4:4:4 content overruns this buffer on the GPU and
+ // corrupts the adjacent meta/bucket allocations (nondeterministic bad bitstreams and
+ // crashes at ANY target bitrate). Validated on RTX 5070 Ti @1080p/4K, 8/16-bit.
+ info.size = VkDeviceSize(aligned_width) * aligned_height *
+ (chroma == ChromaSubsampling::Chroma444 ? 4 : 2);
payload_data = device->create_buffer(info);
device->set_name(*payload_data, "payload-data");
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,16 @@
diff --git a/crates/pyrowave-sys/vendor/pyrowave/encode.cpp b/crates/pyrowave-sys/vendor/pyrowave/encode.cpp
index 95725551..2e78fada 100644
--- a/crates/pyrowave-sys/vendor/pyrowave/encode.cpp
+++ b/crates/pyrowave-sys/vendor/pyrowave/encode.cpp
@@ -201,7 +201,10 @@ static void run_encoder(Device &device, const char *out_path, const char *in_pat
for (auto &img : inputs.images)
{
auto *y = cmd->update_image(*img);
- if (!input.read(y, img->get_width() * img->get_height()))
+ // PUNKTFUNK PATCH 0003: size the plane read in BYTES, not texels — 16-bit y4m
+ // inputs otherwise read half a plane and desync the stream after frame 1.
+ if (!input.read(y, size_t(img->get_width()) * img->get_height() *
+ YUV4MPEGFile::format_to_bytes_per_component(input.get_format())))
{
LOGE("Failed to read plane.\n");
device.submit_discard(cmd);
@@ -0,0 +1,108 @@
Encoder scratch-buffer pool — PUNKTFUNK LOCAL PATCH.
Not upstream. pyrowave_encoder_encode_gpu_synchronous() allocated four Vulkan
buffers on EVERY frame (meta + bitstream, each Device-local and CachedHost).
At streaming rates (240 fps, 1-3 MB bitstreams) that per-frame allocator churn
did not just cost CPU — it stalled the encode: measured on an RTX 4090 the
per-frame submit+fence-wait at 5120x1440 was ~15 ms (≈64 fps ceiling) and
collapsed to ~1 ms (≈1025 fps) once the buffers are pooled. The four buffers are
sized from the fixed session resolution + pinned bitrate budget, so after the
first frame they are pure reuse; each is only re-created if a larger size is
requested. Safe because the encode is synchronous (compute_num_packets/packetize
wait the fence before the next encode reuses a buffer), and holding the handles
on the encoder keeps next_frame_context() from recycling them.
Correctness re-validated by the pyrowave_win_smoke round-trip (100/180/60 across
SDR/HDR x 420/444) after the change.
diff --git a/crates/pyrowave-sys/vendor/pyrowave/pyrowave_c.cpp b/crates/pyrowave-sys/vendor/pyrowave/pyrowave_c.cpp
index 114b0bfc..eb917e64 100644
--- a/crates/pyrowave-sys/vendor/pyrowave/pyrowave_c.cpp
+++ b/crates/pyrowave-sys/vendor/pyrowave/pyrowave_c.cpp
@@ -799,6 +799,10 @@ struct pyrowave_encoder_opaque
Fence queued_fence;
BufferHandle queued_meta;
BufferHandle queued_bitstream;
+ // PUNKTFUNK: the GPU-side twins of queued_meta/queued_bitstream, pooled on the encoder so
+ // the encode path reuses them across frames instead of allocating four buffers per frame.
+ BufferHandle queued_meta_gpu;
+ BufferHandle queued_bitstream_gpu;
ChromaSubsampling chroma = {};
int width = 0;
int height = 0;
@@ -936,41 +940,49 @@ pyrowave_encoder_encode_gpu_synchronous(pyrowave_encoder encoder,
device->next_frame_context();
- BufferCreateInfo bufinfo = {};
- bufinfo.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT |
- VK_BUFFER_USAGE_TRANSFER_DST_BIT |
- VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
-
- bufinfo.size = encoder->encoder.get_meta_required_size();
- bufinfo.domain = BufferDomain::CachedHost;
- encoder->queued_meta = device->create_buffer(bufinfo);
-
- if (!encoder->queued_meta)
- return PYROWAVE_ERROR_OUT_OF_HOST_MEMORY;
-
- bufinfo.domain = BufferDomain::Device;
- auto queued_meta_gpu = device->create_buffer(bufinfo);
-
- if (!queued_meta_gpu)
- return PYROWAVE_ERROR_OUT_OF_DEVICE_MEMORY;
-
auto target_bitstream_size = rate_control->maximum_bitstream_size & ~VkDeviceSize(3u);
// Check for bogus sizes.
if (target_bitstream_size > UINT32_MAX || target_bitstream_size == 0)
return PYROWAVE_ERROR_INVALID_ARGUMENT;
- bufinfo.size = target_bitstream_size + encoder->encoder.get_meta_required_size();
- bufinfo.domain = BufferDomain::CachedHost;
- encoder->queued_bitstream = device->create_buffer(bufinfo);
+ const VkDeviceSize meta_size = encoder->encoder.get_meta_required_size();
+ const VkDeviceSize bitstream_size = target_bitstream_size + meta_size;
+
+ // PUNKTFUNK: pool the four scratch buffers on the encoder and only (re)create one when a
+ // larger size is needed. Upstream allocated all four (meta + bitstream, each Device +
+ // CachedHost) on every call; at streaming rates (240 fps, MB-scale bitstreams) that
+ // allocator churn dominated the per-frame CPU cost. The sizes are effectively constant per
+ // session (fixed resolution + a pinned bitrate budget), so after the first frame these are
+ // pure reuse. The synchronous encode model (packetize()/compute_num_packets() wait the
+ // fence before the next encode reuses a buffer) guarantees no in-flight GPU access to a
+ // buffer we hand back, and holding the handles keeps next_frame_context() from recycling
+ // them.
+ BufferCreateInfo bufinfo = {};
+ bufinfo.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT |
+ VK_BUFFER_USAGE_TRANSFER_DST_BIT |
+ VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
- if (!encoder->queued_bitstream)
- return PYROWAVE_ERROR_OUT_OF_HOST_MEMORY;
+ auto ensure_buffer = [&](BufferHandle &handle, VkDeviceSize size, BufferDomain domain) -> bool {
+ if (!handle || handle->get_create_info().size < size)
+ {
+ bufinfo.size = size;
+ bufinfo.domain = domain;
+ handle = device->create_buffer(bufinfo);
+ }
+ return bool(handle);
+ };
- bufinfo.domain = BufferDomain::Device;
- auto queued_bitstream_gpu = device->create_buffer(bufinfo);
+ auto &queued_meta_gpu = encoder->queued_meta_gpu;
+ auto &queued_bitstream_gpu = encoder->queued_bitstream_gpu;
- if (!queued_bitstream_gpu)
+ if (!ensure_buffer(encoder->queued_meta, meta_size, BufferDomain::CachedHost))
+ return PYROWAVE_ERROR_OUT_OF_HOST_MEMORY;
+ if (!ensure_buffer(queued_meta_gpu, meta_size, BufferDomain::Device))
+ return PYROWAVE_ERROR_OUT_OF_DEVICE_MEMORY;
+ if (!ensure_buffer(encoder->queued_bitstream, bitstream_size, BufferDomain::CachedHost))
+ return PYROWAVE_ERROR_OUT_OF_HOST_MEMORY;
+ if (!ensure_buffer(queued_bitstream_gpu, bitstream_size, BufferDomain::Device))
return PYROWAVE_ERROR_OUT_OF_DEVICE_MEMORY;
Encoder::BitstreamBuffers bitstream_buffers = {};
@@ -0,0 +1,121 @@
Elevated global-priority encode queue — PUNKTFUNK LOCAL PATCH.
Not upstream. PyroWave's wavelet encode runs on the GPU's compute/shader cores, so a GPU-bound
game starves it: `pyrowave_encoder_encode_gpu_synchronous` spikes from ~2 ms to ~15 ms under a
95%+ game load and the stream framerate collapses (NVENC is immune — it uses the separate encoder
ASIC). A WDDM *process* scheduling-priority raise does not help (it orders packet submission, not
hardware preemption). This requests a global-priority Vulkan *queue* — the actual NVIDIA compute-
preemption lever — on the pyrowave encode device.
`Context::create_device` (the pyrowave path uses plain vkCreateDevice, not the vpCreateDevice
profile path): enable VK_KHR_global_priority (or the EXT alias), chain a
VkDeviceQueueGlobalPriorityCreateInfoKHR into every queue-create-info, and a create loop that tries
`PYROWAVE_QUEUE_PRIORITY` (off|high|realtime, default realtime) → HIGH → no-priority, downgrading on
VK_ERROR_NOT_PERMITTED_KHR so a refused class NEVER regresses the encoder. Gated on !inherit_info
(only the fresh encoder device). This Granite copy is vendored solely for the PyroWave codec.
NOTE: on an RTX 4090 / Windows / WDDM this did not reduce the spikes (the graphics-vs-compute
preemption granularity is the wall) — kept because it is correct, harmless (graceful fallback), and
may help other GPUs/drivers. Reduce the encode's GPU cost (4:2:0/8-bit) or use H.265 for a
GPU-saturated game.
diff --git a/crates/pyrowave-sys/vendor/pyrowave/Granite/vulkan/context.cpp b/crates/pyrowave-sys/vendor/pyrowave/Granite/vulkan/context.cpp
index 5257fc33..479eeded 100644
--- a/crates/pyrowave-sys/vendor/pyrowave/Granite/vulkan/context.cpp
+++ b/crates/pyrowave-sys/vendor/pyrowave/Granite/vulkan/context.cpp
@@ -2115,6 +2115,51 @@ bool Context::create_device(VkPhysicalDevice gpu_, VkSurfaceKHR surface,
vpGetProfileProperties(profile.profile, &props);
#endif
+ // PUNKTFUNK PATCH (patches/0005-global-priority-queue.patch, NOT upstream): request a high
+ // global-priority queue so PyroWave's compute-shader encode can PREEMPT a GPU-bound game on
+ // the shared shader cores. A WDDM process-priority raise does not help (it only orders packet
+ // submission, not hardware preemption); a global-priority queue is the actual NVIDIA compute-
+ // preemption lever. `PYROWAVE_QUEUE_PRIORITY` = off | high | realtime (default realtime). The
+ // device-create loop below downgrades on NOT_PERMITTED, so a refused class NEVER fails the
+ // encoder — it just runs at default priority. Only the fresh encoder device (no inherit_info)
+ // is affected; this Granite copy is vendored solely for the PyroWave codec.
+ Util::SmallVector<VkQueueGlobalPriorityKHR> pf_priority_candidates;
+ Util::SmallVector<VkDeviceQueueGlobalPriorityCreateInfoKHR> pf_global_priority_infos;
+ if (!inherit_info)
+ {
+ const char *pf_gp_ext = nullptr;
+ if (has_extension(VK_KHR_GLOBAL_PRIORITY_EXTENSION_NAME))
+ pf_gp_ext = VK_KHR_GLOBAL_PRIORITY_EXTENSION_NAME;
+ else if (has_extension(VK_EXT_GLOBAL_PRIORITY_EXTENSION_NAME))
+ pf_gp_ext = VK_EXT_GLOBAL_PRIORITY_EXTENSION_NAME;
+ std::string pf_prio;
+ if (!Util::get_environment("PYROWAVE_QUEUE_PRIORITY", pf_prio))
+ pf_prio = "realtime";
+ for (auto &pf_ch : pf_prio)
+ if (pf_ch >= 'A' && pf_ch <= 'Z')
+ pf_ch = char(pf_ch + 32);
+ if (pf_gp_ext && pf_prio != "off")
+ {
+ if (pf_prio == "high")
+ {
+ pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_HIGH_KHR);
+ }
+ else
+ {
+ pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_REALTIME_KHR);
+ pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_HIGH_KHR);
+ }
+ enabled_extensions.push_back(pf_gp_ext);
+ pf_global_priority_infos.resize(queue_infos.size());
+ for (size_t pf_i = 0; pf_i < queue_infos.size(); pf_i++)
+ {
+ pf_global_priority_infos[pf_i] = { VK_STRUCTURE_TYPE_DEVICE_QUEUE_GLOBAL_PRIORITY_CREATE_INFO_KHR };
+ pf_global_priority_infos[pf_i].globalPriority = pf_priority_candidates.front();
+ queue_infos[pf_i].pNext = &pf_global_priority_infos[pf_i];
+ }
+ }
+ }
+
if (inherit_info)
{
device_info.enabledExtensionCount = inherit_info->enabledExtensionCount;
@@ -2144,8 +2189,41 @@ bool Context::create_device(VkPhysicalDevice gpu_, VkSurfaceKHR surface,
if (device == VK_NULL_HANDLE)
return false;
}
- else if (vkCreateDevice(gpu, &device_info, nullptr, &device) != VK_SUCCESS)
- return false;
+ else
+ {
+ // PUNKTFUNK: try the requested global-priority class, downgrade through the
+ // candidate list on NOT_PERMITTED, then finally create with no global priority.
+ VkResult pf_res;
+ if (pf_priority_candidates.empty())
+ {
+ pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
+ }
+ else
+ {
+ pf_res = VK_ERROR_NOT_PERMITTED_KHR;
+ for (size_t pf_a = 0; pf_a < pf_priority_candidates.size(); pf_a++)
+ {
+ for (auto &pf_gp : pf_global_priority_infos)
+ pf_gp.globalPriority = pf_priority_candidates[pf_a];
+ pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
+ if (pf_res != VK_ERROR_NOT_PERMITTED_KHR)
+ break;
+ LOGW("PyroWave: global queue priority %u not permitted; downgrading.\n",
+ unsigned(pf_priority_candidates[pf_a]));
+ }
+ if (pf_res == VK_ERROR_NOT_PERMITTED_KHR)
+ {
+ for (auto &pf_qi : queue_infos)
+ pf_qi.pNext = nullptr;
+ pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
+ LOGW("PyroWave: all global queue priorities refused; default priority.\n");
+ }
+ else
+ LOGI("PyroWave: encode device created with an elevated global queue priority.\n");
+ }
+ if (pf_res != VK_SUCCESS)
+ return false;
+ }
}
if (inherit_info)
@@ -2115,6 +2115,51 @@ bool Context::create_device(VkPhysicalDevice gpu_, VkSurfaceKHR surface,
vpGetProfileProperties(profile.profile, &props);
#endif
// PUNKTFUNK PATCH (patches/0005-global-priority-queue.patch, NOT upstream): request a high
// global-priority queue so PyroWave's compute-shader encode can PREEMPT a GPU-bound game on
// the shared shader cores. A WDDM process-priority raise does not help (it only orders packet
// submission, not hardware preemption); a global-priority queue is the actual NVIDIA compute-
// preemption lever. `PYROWAVE_QUEUE_PRIORITY` = off | high | realtime (default realtime). The
// device-create loop below downgrades on NOT_PERMITTED, so a refused class NEVER fails the
// encoder — it just runs at default priority. Only the fresh encoder device (no inherit_info)
// is affected; this Granite copy is vendored solely for the PyroWave codec.
Util::SmallVector<VkQueueGlobalPriorityKHR> pf_priority_candidates;
Util::SmallVector<VkDeviceQueueGlobalPriorityCreateInfoKHR> pf_global_priority_infos;
if (!inherit_info)
{
const char *pf_gp_ext = nullptr;
if (has_extension(VK_KHR_GLOBAL_PRIORITY_EXTENSION_NAME))
pf_gp_ext = VK_KHR_GLOBAL_PRIORITY_EXTENSION_NAME;
else if (has_extension(VK_EXT_GLOBAL_PRIORITY_EXTENSION_NAME))
pf_gp_ext = VK_EXT_GLOBAL_PRIORITY_EXTENSION_NAME;
std::string pf_prio;
if (!Util::get_environment("PYROWAVE_QUEUE_PRIORITY", pf_prio))
pf_prio = "realtime";
for (auto &pf_ch : pf_prio)
if (pf_ch >= 'A' && pf_ch <= 'Z')
pf_ch = char(pf_ch + 32);
if (pf_gp_ext && pf_prio != "off")
{
if (pf_prio == "high")
{
pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_HIGH_KHR);
}
else
{
pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_REALTIME_KHR);
pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_HIGH_KHR);
}
enabled_extensions.push_back(pf_gp_ext);
pf_global_priority_infos.resize(queue_infos.size());
for (size_t pf_i = 0; pf_i < queue_infos.size(); pf_i++)
{
pf_global_priority_infos[pf_i] = { VK_STRUCTURE_TYPE_DEVICE_QUEUE_GLOBAL_PRIORITY_CREATE_INFO_KHR };
pf_global_priority_infos[pf_i].globalPriority = pf_priority_candidates.front();
queue_infos[pf_i].pNext = &pf_global_priority_infos[pf_i];
}
}
}
if (inherit_info)
{
device_info.enabledExtensionCount = inherit_info->enabledExtensionCount;
@@ -2144,8 +2189,41 @@ bool Context::create_device(VkPhysicalDevice gpu_, VkSurfaceKHR surface,
if (device == VK_NULL_HANDLE)
return false;
}
else if (vkCreateDevice(gpu, &device_info, nullptr, &device) != VK_SUCCESS)
return false;
else
{
// PUNKTFUNK: try the requested global-priority class, downgrade through the
// candidate list on NOT_PERMITTED, then finally create with no global priority.
VkResult pf_res;
if (pf_priority_candidates.empty())
{
pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
}
else
{
pf_res = VK_ERROR_NOT_PERMITTED_KHR;
for (size_t pf_a = 0; pf_a < pf_priority_candidates.size(); pf_a++)
{
for (auto &pf_gp : pf_global_priority_infos)
pf_gp.globalPriority = pf_priority_candidates[pf_a];
pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
if (pf_res != VK_ERROR_NOT_PERMITTED_KHR)
break;
LOGW("PyroWave: global queue priority %u not permitted; downgrading.\n",
unsigned(pf_priority_candidates[pf_a]));
}
if (pf_res == VK_ERROR_NOT_PERMITTED_KHR)
{
for (auto &pf_qi : queue_infos)
pf_qi.pNext = nullptr;
pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
LOGW("PyroWave: all global queue priorities refused; default priority.\n");
}
else
LOGI("PyroWave: encode device created with an elevated global queue priority.\n");
}
if (pf_res != VK_SUCCESS)
return false;
}
}
if (inherit_info)
@@ -8,3 +8,42 @@ vulkan-headers: 015e25c3c91b70eb1a754d36fb14c4ba6ad9b0b9
Tree is pruned to what the pyrowave-sys standalone build needs
(see the rm -rf list in the script). All parts are MIT-licensed
(pyrowave, Granite) or Apache-2.0/MIT (volk, Vulkan-Headers).
Local patches (crates/pyrowave-sys/patches/, re-applied on re-vendor).
These are OUR fixes — kept local by decision (we vendor anyway), not filed
upstream:
0001-payload-data-444-sizing.patch — encoder payload_data worst-case buffer
was sized for 4:2:0's 1.5 samples/px; busy 4:4:4 (3 samples/px) overran it
on the GPU → nondeterministic corrupt bitstreams/crashes at any bitrate.
Found + validated 2026-07-18 (RTX 5070 Ti, 1080p/4K, 8/16-bit).
0002-rdo-saving-clamp.patch — analyze_rate_control.comp accumulates the full
32-bit per-block rate saving into the RDO bucket totals but packs only 16
bits into each RDOperation; once a block's saving exceeds 65535 cost units
the resolve pass over-credits applied ops and the bitstream can overshoot
the hard rate target (the same overrun class as 0001). Clamped to the
packed width (conservative direction). Includes the regenerated
shaders/slangmosh.hpp (built with the pinned Granite slangmosh, -O --strip
per upstream's slangmosh.sh); behavior-neutral at our operating points
(byte-identical outputs, validated on the RTX 5070 Ti).
NOTE the related UNPATCHED limit: the packed 16-bit block_index wraps when
block_count_32x32 > 65535 (~8K 4:4:4) — guarded host-side instead
(pf-encode rejects such modes for PyroWave).
0003-devel-encode-16bit-read.patch — devel tool encode.cpp read y4m planes
with texel-count math (bytes for 8-bit): 16-bit inputs got half-plane
reads and a desynced stream after frame 1. Tool-only (our build never
compiles the devel tools; kept so the vendored source is honest).
0004-encoder-buffer-pool.patch — pyrowave_c.cpp allocated four Vulkan buffers
(meta + bitstream, Device + CachedHost) on EVERY encode. At 240 fps with
MB-scale bitstreams that per-frame churn stalled the encode itself: on an
RTX 4090 the 5120x1440 submit+fence-wait was ~15 ms (~64 fps ceiling) and
dropped to ~1 ms (~1025 fps) once the buffers are pooled on the encoder and
reused. Safe under the synchronous encode model; re-validated by
pyrowave_win_smoke. Perf fix, not a correctness fix.
0005-global-priority-queue.patch — Context::create_device requests a global-priority Vulkan
compute queue (VK_KHR_global_priority, PYROWAVE_QUEUE_PRIORITY=off|high|realtime, default
realtime) so the wavelet encode can preempt a GPU-bound game on the shared shader cores. A
create loop downgrades on NOT_PERMITTED so a refused class never regresses the encoder. Did
not overcome the graphics-vs-compute preemption wall on an RTX 4090 (kept: correct + harmless,
may help other HW/drivers).
+4 -1
View File
@@ -201,7 +201,10 @@ static void run_encoder(Device &device, const char *out_path, const char *in_pat
for (auto &img : inputs.images)
{
auto *y = cmd->update_image(*img);
if (!input.read(y, img->get_width() * img->get_height()))
// PUNKTFUNK PATCH 0003: size the plane read in BYTES, not texels — 16-bit y4m
// inputs otherwise read half a plane and desync the stream after frame 1.
if (!input.read(y, size_t(img->get_width()) * img->get_height() *
YUV4MPEGFile::format_to_bytes_per_component(input.get_format())))
{
LOGE("Failed to read plane.\n");
device.submit_discard(cmd);
+39 -27
View File
@@ -799,6 +799,10 @@ struct pyrowave_encoder_opaque
Fence queued_fence;
BufferHandle queued_meta;
BufferHandle queued_bitstream;
// PUNKTFUNK: the GPU-side twins of queued_meta/queued_bitstream, pooled on the encoder so
// the encode path reuses them across frames instead of allocating four buffers per frame.
BufferHandle queued_meta_gpu;
BufferHandle queued_bitstream_gpu;
ChromaSubsampling chroma = {};
int width = 0;
int height = 0;
@@ -936,41 +940,49 @@ pyrowave_encoder_encode_gpu_synchronous(pyrowave_encoder encoder,
device->next_frame_context();
BufferCreateInfo bufinfo = {};
bufinfo.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT |
VK_BUFFER_USAGE_TRANSFER_DST_BIT |
VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
bufinfo.size = encoder->encoder.get_meta_required_size();
bufinfo.domain = BufferDomain::CachedHost;
encoder->queued_meta = device->create_buffer(bufinfo);
if (!encoder->queued_meta)
return PYROWAVE_ERROR_OUT_OF_HOST_MEMORY;
bufinfo.domain = BufferDomain::Device;
auto queued_meta_gpu = device->create_buffer(bufinfo);
if (!queued_meta_gpu)
return PYROWAVE_ERROR_OUT_OF_DEVICE_MEMORY;
auto target_bitstream_size = rate_control->maximum_bitstream_size & ~VkDeviceSize(3u);
// Check for bogus sizes.
if (target_bitstream_size > UINT32_MAX || target_bitstream_size == 0)
return PYROWAVE_ERROR_INVALID_ARGUMENT;
bufinfo.size = target_bitstream_size + encoder->encoder.get_meta_required_size();
bufinfo.domain = BufferDomain::CachedHost;
encoder->queued_bitstream = device->create_buffer(bufinfo);
const VkDeviceSize meta_size = encoder->encoder.get_meta_required_size();
const VkDeviceSize bitstream_size = target_bitstream_size + meta_size;
if (!encoder->queued_bitstream)
// PUNKTFUNK: pool the four scratch buffers on the encoder and only (re)create one when a
// larger size is needed. Upstream allocated all four (meta + bitstream, each Device +
// CachedHost) on every call; at streaming rates (240 fps, MB-scale bitstreams) that
// allocator churn dominated the per-frame CPU cost. The sizes are effectively constant per
// session (fixed resolution + a pinned bitrate budget), so after the first frame these are
// pure reuse. The synchronous encode model (packetize()/compute_num_packets() wait the
// fence before the next encode reuses a buffer) guarantees no in-flight GPU access to a
// buffer we hand back, and holding the handles keeps next_frame_context() from recycling
// them.
BufferCreateInfo bufinfo = {};
bufinfo.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT |
VK_BUFFER_USAGE_TRANSFER_DST_BIT |
VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
auto ensure_buffer = [&](BufferHandle &handle, VkDeviceSize size, BufferDomain domain) -> bool {
if (!handle || handle->get_create_info().size < size)
{
bufinfo.size = size;
bufinfo.domain = domain;
handle = device->create_buffer(bufinfo);
}
return bool(handle);
};
auto &queued_meta_gpu = encoder->queued_meta_gpu;
auto &queued_bitstream_gpu = encoder->queued_bitstream_gpu;
if (!ensure_buffer(encoder->queued_meta, meta_size, BufferDomain::CachedHost))
return PYROWAVE_ERROR_OUT_OF_HOST_MEMORY;
bufinfo.domain = BufferDomain::Device;
auto queued_bitstream_gpu = device->create_buffer(bufinfo);
if (!queued_bitstream_gpu)
if (!ensure_buffer(queued_meta_gpu, meta_size, BufferDomain::Device))
return PYROWAVE_ERROR_OUT_OF_DEVICE_MEMORY;
if (!ensure_buffer(encoder->queued_bitstream, bitstream_size, BufferDomain::CachedHost))
return PYROWAVE_ERROR_OUT_OF_HOST_MEMORY;
if (!ensure_buffer(queued_bitstream_gpu, bitstream_size, BufferDomain::Device))
return PYROWAVE_ERROR_OUT_OF_DEVICE_MEMORY;
Encoder::BitstreamBuffers bitstream_buffers = {};
+7 -2
View File
@@ -187,8 +187,13 @@ void Encoder::Impl::init_block_meta()
meta_buffer = device->create_buffer(info);
device->set_name(*meta_buffer, "meta-buffer");
// Worst case estimate.
info.size = aligned_width * aligned_height * 2;
// Worst case estimate. PUNKTFUNK PATCH (patches/0001-payload-data-444-sizing.patch):
// 4:4:4 carries 3 samples per pixel vs 4:2:0's 1.5 — the same per-sample headroom
// needs twice the bytes, or busy 4:4:4 content overruns this buffer on the GPU and
// corrupts the adjacent meta/bucket allocations (nondeterministic bad bitstreams and
// crashes at ANY target bitrate). Validated on RTX 5070 Ti @1080p/4K, 8/16-bit.
info.size = VkDeviceSize(aligned_width) * aligned_height *
(chroma == ChromaSubsampling::Chroma444 ? 4 : 2);
payload_data = device->create_buffer(info);
device->set_name(*payload_data, "payload-data");
@@ -132,6 +132,10 @@ void emit_rdo_operations()
else if (gl_SubgroupInvocationID < 16)
{
uint saving = shared_rate_cost[gl_SubgroupInvocationID - 1] - shared_rate_cost[gl_SubgroupInvocationID];
// PUNKTFUNK PATCH 0002: RDOperation packs saving into 16 bits; clamp the bucket
// accounting to match, else resolve over-credits applied ops and the produced
// bitstream can overshoot the hard rate target (a buffer-overrun class).
saving = min(saving, 65535u);
if (saving != 0)
{
File diff suppressed because it is too large Load Diff
+4
View File
@@ -29,6 +29,7 @@ and nothing you configure here runs anywhere near the streaming path.
| `pairing.completed` / `pairing.denied` | a pairing is approved+stored / denied | device name, fingerprint, plane |
| `display.created` / `display.released` | a virtual display is minted / kept displays are released | backend + mode / count |
| `library.changed` | the game library is mutated | source: `manual`, or the provider id that reconciled (`PUT /api/v1/library/provider/{p}`) |
| `plugins.changed` | a plugin's registration changes (registered, restarted, deregistered, or its lease expired) | plugin id |
| `host.started` / `host.stopping` | the serve planes come up / wind down | version, whether GameStream is enabled |
Every event is a small JSON document with a monotonic `seq`, a `ts_ms` timestamp, a `schema`
@@ -147,6 +148,9 @@ directory of scripts and installed plugins as one service: crash-restarts with b
`systemctl stop` that interrupts plugins structurally so their cleanup runs. See the SDK README
for the five-line quickstart and unit templates.
For ready-made plugins — sync your ROM collection or your Playnite library into the game library,
with a console page to manage them — see [Plugins](/docs/plugins).
The canonical "decide, don't just observe" pattern — approve pairing from your phone: watch
`pairing.pending`, send yourself a notification, and call
`POST /api/v1/native/pending/{id}/approve` when you tap yes. The full API is documented at
+11 -11
View File
@@ -87,9 +87,9 @@ pick a host from the list, just like the other native apps.
A headless CLI path exists for scripting/measurement:
```sh
punktfunk-client # open the WinUI 3 window (host list / settings)
punktfunk-client --discover # list hosts on the network
punktfunk-client --headless --connect <host>:9777 # no window: connect, count frames, print stats
punktfunk-client # open the WinUI 3 window (host list / settings)
punktfunk-client --discover # list hosts on the network
punktfunk-client --headless --speed-test --connect <host>:9777 # no window: probe the link, print measured/recommended bitrate
```
Prefer the broadest compatibility, or no install? **Moonlight** also streams to Windows (see below).
@@ -121,13 +121,13 @@ punktfunk-probe --connect <host>:9777 --pin <fp> # connect to one
| You're streaming to… | Use |
|---|---|
| A Mac, iPhone, iPad, or Apple TV | The **Apple app** |
| A Linux desktop or laptop | **`punktfunk-client`** (GTK4) |
| A **Steam Deck** | The **[Decky plugin](/docs/steam-deck)** in Gaming Mode, or the GTK4 client in Desktop Mode |
| An Android phone or TV | The **Android app** |
| Windows | The native **`punktfunk-client`** (signed MSIX) or **Moonlight** |
| An **LG webOS TV** | The community **[`pf-webos`](https://github.com/dyptan-io/pf-webos)** client, or **Moonlight** |
| A browser, another smart TV, or any other device | **Moonlight** |
| Automated tests / latency measurement | **`punktfunk-probe`** (headless) |
| A Mac, iPhone, iPad, or Apple TV | The **[Apple app](#apple-app-mac-iphone-ipad-apple-tv)** |
| A Linux desktop or laptop | **[`punktfunk-client`](#linux-desktop-client-gtk4)** (GTK4) |
| A **Steam Deck** | The **[Decky plugin](/docs/steam-deck)** in Gaming Mode, or the [GTK4 client](#linux-desktop-client-gtk4) in Desktop Mode |
| An Android phone or TV | The **[Android app](#android-app-phone--android-tv)** |
| Windows | The native **[`punktfunk-client`](#windows-desktop-client)** (signed MSIX) or **[Moonlight](/docs/moonlight)** |
| An **LG webOS TV** | The community **[`pf-webos`](https://github.com/dyptan-io/pf-webos)** client, or **[Moonlight](/docs/moonlight)** |
| A browser, another smart TV, or any other device | **[Moonlight](/docs/moonlight)** |
| Automated tests / latency measurement | **[`punktfunk-probe`](#linux-reference-client-headless)** (headless) |
Whichever you choose, the first connection needs a one-time [pairing](/docs/pairing).
+4 -3
View File
@@ -88,8 +88,9 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
| Setting | Values | Meaning |
|---|---|---|
| `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. |
| `PUNKTFUNK_10BIT` | `1` | HEVC Main10 / HDR. Honored only when the client also advertises 10-bit. **Windows host only** (the Linux host stays 8-bit). |
| `PUNKTFUNK_444` | `1` | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. |
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. |
| `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. |
| `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. |
| `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. |
| `PUNKTFUNK_OH264_THREADS` / `PUNKTFUNK_OH264_GOP` | `N` | Software (openh264) encoder tuning: encode threads (default 2 — latency over throughput) and GOP length (default 0 = encoder-auto). Only relevant with `PUNKTFUNK_ENCODER=software`. |
@@ -142,7 +143,7 @@ notes for context.
|---|---|---|
| `PUNKTFUNK_GSO` | `1` · `0` | UDP Generic Segmentation Offload on the send path (coalesce a frame's packets into kernel super-buffers) — cuts send CPU ~30%, but its line-rate packet trains can cost delivered throughput on constrained links (measured on a 2.5GbE hop). Off by default until send pacing spaces the super-buffers; set `1` to opt in (auto-falls back to `sendmmsg` on kernels/paths without support). |
| `PUNKTFUNK_SPLIT_ENCODE` | `0`/`disable` · `1`/`auto` · `2` · `3` | NVENC N-way split-encode for very high pixel rates (5K@240). `auto` picks automatically above ~1 Gpix/s. |
| `PUNKTFUNK_GPU_PRIORITY_CLASS` | `off` · `normal` · `high` · `realtime` | **(Windows)** GPU scheduling priority for capture/encode under a GPU-saturating game. Default `high`; `realtime` is the strongest lever but can freeze NVENC on some setups. |
| `PUNKTFUNK_GPU_PRIORITY_CLASS` | `off` · `normal` · `high` · `realtime` · `auto` | **(Windows)** GPU scheduling priority for capture/encode under a GPU-saturating game. Default `auto` (starts `high`, upgrades to `realtime` when it's safe — e.g. HAGS off); `high` pins the static pre-gate behaviour; `realtime` is the strongest lever but can freeze NVENC on some setups. |
| `PUNKTFUNK_IDD_DEPTH` | `N` (default `2`) | **(Windows)** IDD-push pipeline depth. `1` cuts latency once GPU priority is raised; higher smooths a contended GPU. |
## Diagnostics
+4 -4
View File
@@ -43,15 +43,15 @@ GPU path** that keeps latency low even at high resolutions and frame rates.
punktfunk speaks two protocols over the same host:
- **GameStream** — the protocol Moonlight uses. Any [Moonlight](/docs/moonlight) client connects with
no special software. This is the most compatible way in.
- **GameStream** — the protocol Moonlight uses. Start the host with `--gamestream` and any
[Moonlight](/docs/moonlight) client connects with no special software. This is the most compatible way in.
- **punktfunk/1 (native)** — a purpose-built protocol with a QUIC control channel and a UDP data
channel hardened with forward error correction and encryption. It's lower-latency and more resilient
on imperfect networks, and it's what the [native clients](/docs/clients) (Apple, Linux, Windows,
Android) use.
Both run from a single host process, so you don't choose up front — Moonlight clients use GameStream,
the native clients use punktfunk/1.
The native `punktfunk/1` plane runs by default (the secure default); add `--gamestream` and both planes
serve from a single host process — Moonlight clients use GameStream, the native clients use punktfunk/1.
## Pairing and trust
+3 -3
View File
@@ -18,9 +18,9 @@ It's built for the things that make streaming feel native:
plug to deal with, even on the secure desktop.
- **Low latency, GPU end to end.** Frames go straight from the compositor to the GPU encoder
(NVENC) with zero CPU copies, and over a transport tuned for responsiveness rather than throughput.
- **Works with the apps you already have.** punktfunk speaks the GameStream protocol, so any
**Moonlight** client connects out of the box — and a faster **native protocol** with dedicated apps
for **macOS, iOS, tvOS, Linux, Windows, and Android**.
- **Works with the apps you already have.** punktfunk also speaks the GameStream protocol, so with
`--gamestream` any **Moonlight** client connects — alongside a faster **native protocol** with
dedicated apps for **macOS, iOS, tvOS, Linux, Windows, and Android**.
- **Secure by default.** Hosts require a one-time PIN pairing; after that, devices reconnect on a
pinned identity. No accounts, no cloud.
+5 -3
View File
@@ -83,15 +83,17 @@ The Windows client ships as a **signed MSIX** in the package registry. Builds us
certificate, so you import that certificate once before Windows will install the package.
1. Open the [packages page](https://git.unom.io/unom/-/packages) (generic group), find
**`punktfunk-client-windows`**, and download the newest **`.msix`** and its matching **`.cer`**.
**`punktfunk-client-windows`**, and download the newest **`.msix`** and its matching **`.cer`** for
your CPU — the artifacts are arch-suffixed (`…_x64.msix` / `…_arm64.msix`).
2. **Trust the publisher certificate**, then install. The MSIX won't install until the certificate is
trusted — but it's the **same certificate for every release**, so this is genuinely one-time and
later updates need nothing. In an **admin** PowerShell:
```powershell
Import-Certificate -FilePath .\punktfunk-client-windows.cer `
# use the _arm64 files instead on an Arm device
Import-Certificate -FilePath .\punktfunk-client-windows_x64.cer `
-CertStoreLocation Cert:\LocalMachine\TrustedPeople
Add-AppxPackage .\punktfunk-client-windows.msix
Add-AppxPackage .\punktfunk-client-windows_x64.msix
```
If Windows reports a missing dependency, install the
+2 -2
View File
@@ -37,13 +37,13 @@ punktfunk also runs as a native host on **Windows 11 22H2+ (x64)**, shipped as a
installer — see [Windows Host](/docs/windows-host) for what it includes and its limitations.
1. From the [packages page](https://git.unom.io/unom/-/packages) (generic group), download the newest
**`punktfunk-host-setup-<ver>.exe`** and its matching **`.cer`**.
**`punktfunk-host-setup-<ver>.exe`** and the matching **`punktfunk-host-windows_<ver>.cer`**.
2. **Trust the publisher certificate once.** The installer is signed with a self-signed certificate
whose public `.cer` is published next to it — the **same certificate for every release**, so this is
genuinely one-time and later updates need nothing. In an **admin** PowerShell:
```powershell
Import-Certificate -FilePath .\punktfunk-host-setup.cer `
Import-Certificate -FilePath .\punktfunk-host-windows_<ver>.cer `
-CertStoreLocation Cert:\LocalMachine\TrustedPublisher
```
+1
View File
@@ -27,6 +27,7 @@
"pyrowave",
"host-cli",
"automation",
"plugins",
"---Connecting---",
"clients",
"install-client",

Some files were not shown because too many files have changed in this diff Show More