Compare commits

..
Author SHA1 Message Date
enricobuehlerandClaude Opus 4.8 491a344f23 fix(client/windows): repair the v0.15.0 MSIX build — ARM64 feature leak + illegal XML comment
release / apple (push) Successful in 10m26s
android-screenshots / screenshots (push) Successful in 3m5s
decky / build-publish (push) Successful in 29s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
deb / build-publish (push) Successful in 12m14s
deb / build-publish-host (push) Successful in 12m25s
android / android (push) Successful in 13m56s
arch / build-publish (push) Successful in 13m30s
docker / deploy-docs (push) Successful in 17s
web-screenshots / screenshots (push) Successful in 4m11s
linux-client-screenshots / screenshots (push) Successful in 7m6s
flatpak / build-publish (push) Successful in 6m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m46s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m3s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m9s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m53s
windows-host / package (push) Failing after 14m35s
Both MSIX matrix jobs failed at the v0.15.0 tag, for two unrelated reasons, both
introduced by this release. 0.13.0 and 0.14.0 shipped both packages; 0.15.0 shipped
neither.

ARM64 failed to BUILD. The PyroWave Windows un-gating (1ef0229b) moved `pyrowave-sys`
in pf-client-core out of `cfg(target_os = "linux")` into a general optional dependency
that pf-client-core's own `default = ["pyrowave"]` turns on. The ARM64 leg builds
`--no-default-features` precisely to skip it, but that only disables defaults for the
two packages named with `-p` — every internal edge kept inheriting them:
clients/windows -> pf-client-core, clients/session -> pf-client-core, clients/session
-> pf-presenter, and pf-presenter's own `default = ["pyrowave"]` -> pf-client-core.
So the vendored Granite C++ compiled on ARM64 and stopped where it always does:
muglm.cpp reaching for _MM_TRANSPOSE4_PS/_mm_storeu_ps, then
`simd.hpp:103 #error "Implement me."`.

Those three internal edges now pass `default-features = false`; the feature is enabled
only through an explicit chain (clients/session's `pyrowave` -> pf-client-core/pyrowave
+ pf-presenter/pyrowave, and pf-presenter's `pyrowave` -> pf-client-core/pyrowave), so
`--no-default-features` finally means what it says. Verified by feature resolution
rather than assertion — `cargo tree -i pyrowave-sys`:

  aarch64-pc-windows-msvc, --no-default-features -> not in the graph (was: present)
  x86_64-pc-windows-msvc,  default features      -> present
  x86_64-unknown-linux-gnu, default features     -> present

PyroWave is NOT dropped from the Windows client. Decode has always run in the spawned
punktfunk-session binary, whose own default keeps the feature on, and unification turns
it back on for the shared pf-client-core wherever that binary is in the build (x64,
Linux). The shell only ever offered "pyrowave" as a codec preference string — it holds
no decoder and never called one, so linking the C++ into it was dead weight even on x64.

x64 built fine and failed at PACKAGING: `makepri new` rejected the manifest with
PRI191 "Appx manifest not found or is invalid" / malformed comment syntax. The console
tile added in 2508b720 documented its flag inside an XML comment, and XML forbids `--`
anywhere in a comment body, so the literal flag spelling made the whole manifest
unparseable. Reworded, with a note to keep the next person from reintroducing it.
Confirmed by parsing both revisions after template substitution: the tagged manifest
fails at line 63 col 9, this one parses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 23:18:57 +02:00
398 changed files with 9622 additions and 53382 deletions
+3 -6
View File
@@ -1,12 +1,9 @@
# The root build context is used by web/Dockerfile (which needs web/ and # Root build context is used only by web/Dockerfile, which needs web/ and
# api/openapi.json) and by ci/rust-ci-arm64cross.Dockerfile (which needs the toolchain # api/openapi.json. Allowlist those; keep everything else (target/, .git, crates)
# pin). Allowlist those; keep everything else (target/, .git, crates) out of the # out of the context upload.
# context upload.
* *
!web !web
!api/openapi.json !api/openapi.json
!rust-toolchain.toml
!ci/pf-host-cc
web/node_modules web/node_modules
web/.output web/.output
web/dist web/dist
+1 -11
View File
@@ -46,23 +46,13 @@ jobs:
# SHA-pinned: this workflow's release job carries the signing keystore + Play service-account # SHA-pinned: this workflow's release job carries the signing keystore + Play service-account
# secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e. # secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e.
uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3 uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3
with:
# Only platform-tools — NOT the action's default legacy `tools`, whose dependency chain
# drags in the ~250 MB emulator nobody here runs (instrumentation tests are deferred).
# That download was the single flakiest piece of this job: the shared runner fleet drops
# packets under parallel-job load and sdkmanager's streamed unzip turns a truncated
# stream into "Error on ZipFile unknown archive" (observed 2026-07-22, twice).
packages: platform-tools
- name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build) - name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build)
# cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path # cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path
# kit/build.gradle.kts prepends to PATH for cargo-ndk's audiopus_sys (libopus) CMake build. # kit/build.gradle.kts prepends to PATH for cargo-ndk's audiopus_sys (libopus) CMake build.
# Note: platforms;android-37 is sometimes missing from standard channels; AGP will # Note: platforms;android-37 is sometimes missing from standard channels; AGP will
# auto-download it if needed during the build. # auto-download it if needed during the build.
# retry.sh: sdkmanager is a single-shot multi-hundred-MB fetch, exactly the class the run: sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0" "ndk;30.0.14904198" "cmake;3.22.1"
# helper exists for (fleet-load packet drops truncate the stream mid-unzip); a failed
# attempt leaves no partial package behind, so a plain re-invoke is safe.
run: bash scripts/ci/retry.sh 4 sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0" "ndk;30.0.14904198" "cmake;3.22.1"
- name: Caches (cargo + gradle) - name: Caches (cargo + gradle)
uses: actions/cache@v4 uses: actions/cache@v4
-39
View File
@@ -1,39 +0,0 @@
# Announce a stable release to the Discord #releases channel.
#
# This is the deliberate "go" step for a release. Release notes live in the repo at
# docs/releases/<tag>.md and are seeded into the Gitea release body at creation by the build
# workflows (scripts/ci/gitea-release.sh), so the release is never noteless. Once every
# platform's CI is green for a tag, dispatch this workflow with that tag: it re-asserts the notes
# file over the live release and posts a formatted embed to #releases.
#
# Manual on purpose — pressing "go" is the quality gate that says "all platforms built, notes are
# final, tell the community." It is NOT wired to the tag push, so a half-built or failed release
# is never announced. Stable-only: a -rc/pre-release tag is refused unless allow_prerelease=true.
#
# Requires the repo secret DISCORD_RELEASE_WEBHOOK (the #releases channel webhook URL); GITEA auth
# reuses REGISTRY_TOKEN like the other release workflows.
name: announce
on:
workflow_dispatch:
inputs:
tag:
description: "Release tag to announce (e.g. v0.18.0)"
required: true
allow_prerelease:
description: "Announce even if the tag is a pre-release (-rc)"
required: false
default: "false"
jobs:
announce:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Post release announcement to Discord
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
DISCORD_RELEASE_WEBHOOK: ${{ secrets.DISCORD_RELEASE_WEBHOOK }}
ALLOW_PRERELEASE: ${{ inputs.allow_prerelease }}
run: bash scripts/ci/discord-announce.sh "${{ inputs.tag }}"
-84
View File
@@ -61,43 +61,6 @@ jobs:
- name: Test (unit + loopback + proptest + C ABI harness) - name: Test (unit + loopback + proptest + C ABI harness)
run: cargo test --workspace --locked run: cargo test --workspace --locked
# The GPU encode backends are OFF by default, so every step above compiles ~none of them:
# `nvenc` gates enc/linux/nvenc_cuda.rs (+ nvenc_core/nvenc_status) and `vulkan-encode` gates
# enc/linux/vulkan_video.rs (+ the vendored vk_av1_encode/vk_valve_rgb bindings) — ~8,150
# lines carrying ~70 `unsafe` blocks. Their ONLY prior CI coverage was deb.yml's
# `cargo build`, where warnings are not errors, so pf-encode's own
# `#![deny(clippy::undocumented_unsafe_blocks)]` — the crate's stated unsafe-proof gate —
# was never actually enforced on them. (`pyrowave` needs no extra step: punktfunk-host has
# `default = ["pyrowave"]`, so the steps above already cover it.)
#
# `--all-targets` is load-bearing, not decoration: without it the feature-gated
# `#[cfg(test)]` modules are never compiled, which is exactly how all ten
# `NvencCudaEncoder::open` call sites in nvenc_cuda.rs's tests drifted to the wrong arity
# (E0061 x10) without any job noticing.
#
# GPU-free: every test needing real hardware is `#[ignore]`d, and NVENC/CUDA resolve their
# entry points at RUNTIME (dlopen), so the test binary links without a driver present.
# (On MSVC the same crate link-imports those symbols instead, which is why windows-host.yml
# can only type-check these tests via clippy — see the note there.)
#
# Scoped to `-p pf-encode` with ITS OWN feature names: punktfunk-host has no code gated on
# `nvenc`/`vulkan-encode` (its only `cfg(feature)` sites are the two `pyrowave` ones in
# capture.rs, and pyrowave is default-on, so the steps above already cover them). Going
# through `--features punktfunk-host/...` would force punktfunk-host into the selection and
# re-run its entire test suite a second time for no extra coverage.
#
# `pyrowave` is listed explicitly even though it is punktfunk-host's default: selecting only
# `-p pf-encode` takes the host out of the resolution, and pf-encode's own default is empty.
# Naming it keeps this the SHIPPED Linux feature set — deb.yml builds
# `--features punktfunk-host/nvenc,punktfunk-host/vulkan-encode` WITHOUT
# `--no-default-features`, so the .deb carries nvenc + vulkan-encode + pyrowave together, and
# that combination is what deserves the lint.
- name: Clippy + test the feature-gated Linux encode backends
run: |
cargo clippy -p pf-encode --all-targets --locked \
--features nvenc,vulkan-encode,pyrowave -- -D warnings
cargo test -p pf-encode --locked --features nvenc,vulkan-encode,pyrowave
- name: C ABI harness (standalone link proof) - name: C ABI harness (standalone link proof)
run: bash crates/punktfunk-core/tests/c/run.sh run: bash crates/punktfunk-core/tests/c/run.sh
@@ -108,53 +71,6 @@ jobs:
git diff --exit-code include/punktfunk_core.h \ git diff --exit-code include/punktfunk_core.h \
|| (echo "include/punktfunk_core.h is stale — commit the regenerated header" && exit 1) || (echo "include/punktfunk_core.h is stale — commit the regenerated header" && exit 1)
# The client stack cross-checked for aarch64. NOT an artifact job — deb.yml ships those —
# this exists so a portability defect fails here instead of surfacing in a release build or
# on a user's board. It earns its runtime: the bug that motivated it (a Vulkan extension
# array typed `*const i8`, where `c_char` is signed on x86_64 and UNSIGNED on aarch64)
# compiled cleanly on every target CI built at the time.
#
# Client crates only, listed explicitly: the host's encode stack is x86 (NVENC/QSV/AMF) and
# `--workspace` would drag it in. Runs in the cross image (amd64 toolchain + arm64 sysroot,
# ci/rust-ci-arm64cross.Dockerfile) on the ordinary runner — no arm64 runner involved.
rust-arm64:
runs-on: ubuntu-24.04
container:
image: git.unom.io/unom/punktfunk-rust-ci-arm64cross:latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- name: Cache keys
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
- uses: actions/cache@v4
with:
path: |
/usr/local/cargo/registry
/usr/local/cargo/git
key: cargo-home-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-home-
- uses: actions/cache@v4
with:
path: target
# Its OWN prefix: aarch64 artifacts must never share the amd64 jobs' target cache.
key: cargo-target-arm64-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-target-arm64-v1-${{ env.rustc }}-
- name: Clippy for aarch64 (deny warnings)
run: |
cargo clippy --target aarch64-unknown-linux-gnu --all-targets --locked \
-p punktfunk-core -p pf-client-core -p pf-presenter -p pf-console-ui \
-p punktfunk-client-session -p punktfunk-client-linux \
-- -D warnings
# The minimal embedded build — no Skia, no PyroWave — is what a small image installs, so
# it has to keep compiling on its own, not just as a subset of the default features.
- name: Build the session binary, minimal features
run: |
cargo build --release --target aarch64-unknown-linux-gnu --locked \
-p punktfunk-client-session --no-default-features
web: web:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
container: container:
+1 -96
View File
@@ -1,13 +1,9 @@
# Build the punktfunk .debs and publish them to Gitea's Debian package registry, so Ubuntu # Build the punktfunk .debs and publish them to Gitea's Debian package registry, so Ubuntu
# boxes get new builds via `apt update && apt upgrade`. Three jobs, all publishing to the same # boxes get new builds via `apt update && apt upgrade`. Two jobs, both publishing to the same
# apt distribution/component: # apt distribution/component:
# #
# build-publish — client + web + scripting, on the Ubuntu 26.04 rust-ci image (the client # build-publish — client + web + scripting, on the Ubuntu 26.04 rust-ci image (the client
# needs 24.04-absent libs: SDL3, GTK4 ≥ 4.20). # needs 24.04-absent libs: SDL3, GTK4 ≥ 4.20).
# build-publish-client-arm64
# — the same client package for arm64, CROSS-compiled on the same amd64
# runner in the rust-ci-arm64cross image. No host counterpart: the Linux
# host's encode stack is x86 (NVENC/QSV/AMF).
# build-publish-host — the HOST, on the Ubuntu 24.04 rust-ci-noble image with a from-source # build-publish-host — the HOST, on the Ubuntu 24.04 rust-ci-noble image with a from-source
# FFmpeg 8 BUNDLED into the .deb. This lowers the host's glibc floor to 2.39 # FFmpeg 8 BUNDLED into the .deb. This lowers the host's glibc floor to 2.39
# and removes the hard `Depends: libavcodec62`, so the ONE host .deb installs # and removes the hard `Depends: libavcodec62`, so the ONE host .deb installs
@@ -273,94 +269,3 @@ jobs:
for DEB in dist/*.deb; do for DEB in dist/*.deb; do
upsert_asset "$RID" "$DEB" upsert_asset "$RID" "$DEB"
done done
# ---------------------------------------------------------------------------------------------
# The aarch64 CLIENT .deb. Cross-compiled on the ordinary amd64 runner in the
# punktfunk-rust-ci-arm64cross image (the rust-ci toolchain + an arm64 multiarch sysroot — see
# ci/rust-ci-arm64cross.Dockerfile); there is no arm64 runner in the fleet and none is needed.
# Client only, by decision: the Linux host encodes with NVENC/QSV/AMF, all x86.
# Publishes to the same distribution/component as the amd64 jobs — the apt registry keys pool
# entries by arch, so `apt` on an arm64 box picks this one up with no client-side configuration.
build-publish-client-arm64:
runs-on: ubuntu-24.04
container:
image: git.unom.io/unom/punktfunk-rust-ci-arm64cross:latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
# Byte-identical to build-publish's version step (pf-version.sh is deterministic per
# commit), so the arm64 package always shares the amd64 version line.
- name: Version + channel
run: |
eval "$(bash scripts/ci/pf-version.sh)"
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; DIST=stable ;;
*) V="${PF_BASE}~ci${GITHUB_RUN_NUMBER}.g${SHORT}"; DIST=canary ;;
esac
echo "VERSION=$V" >> "$GITHUB_ENV"
echo "DISTRIBUTION=$DIST" >> "$GITHUB_ENV"
echo "package version $V -> apt distribution '$DIST' (arm64)"
# dpkg-shlibdeps + dpkg-deb. The arm64 link deps themselves are the cross image's whole
# point and are already baked in; python3 is for scripts/ci/gitea-release.sh.
- name: dpkg-dev
run: |
apt-get update
apt-get install -y --no-install-recommends dpkg-dev python3
- name: Cache keys
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
- uses: actions/cache@v4
with:
path: |
/usr/local/cargo/registry
/usr/local/cargo/git
key: cargo-home-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-home-
- uses: actions/cache@v4
with:
path: target
# Its OWN key — these are aarch64 artifacts under target/aarch64-unknown-linux-gnu/
# and must never share the amd64 jobs' target cache.
key: cargo-target-arm64-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-target-arm64-v1-${{ env.rustc }}-
- name: Build the arm64 client .deb
env:
PUNKTFUNK_BUILD_VERSION: ${{ env.VERSION }} # stamped into the binaries (build.rs)
run: |
git config --global --add safe.directory "$PWD"
ARCH=arm64 TARGET=aarch64-unknown-linux-gnu \
bash packaging/debian/build-client-deb.sh
# Fail here rather than shipping an amd64 binary under an arm64 package name.
readelf -h target/aarch64-unknown-linux-gnu/release/punktfunk-session \
| grep -q AArch64 || { echo "ERROR: session binary is not AArch64"; exit 1; }
- name: Publish to the Gitea apt registry
env:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
for DEB in dist/*.deb; do
echo "uploading $DEB"
NAME=$(dpkg-deb -f "$DEB" Package)
VER=$(dpkg-deb -f "$DEB" Version)
ARCH=$(dpkg-deb -f "$DEB" Architecture)
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
"https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/$NAME/$VER/$ARCH" || true
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$DEB" \
"https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/upload"
done
echo "published arm64 client to $OWNER/debian $DISTRIBUTION/$COMPONENT"
- name: Attach the arm64 .deb to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v')
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
. scripts/ci/gitea-release.sh
RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
for DEB in dist/*.deb; do
upsert_asset "$RID" "$DEB"
done
+30 -24
View File
@@ -6,12 +6,17 @@
# #
# The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT # The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT
# need the Decky CLI (which requires Docker + rust-nightly only to compile native backends). # need the Decky CLI (which requires Docker + rust-nightly only to compile native backends).
# We build the frontend with pnpm and stage the store-layout tree with the SAME script local # We build the frontend with pnpm and assemble the store-layout zip by hand:
# builds use (clients/decky/scripts/package.sh) — the plugin's file list lives in exactly ONE #
# place, so a file added there (bin/, assets/, controller_config/, …) can never be silently # punktfunk.zip
# missing from the published build. (Hand-assembling the zip here is how the shipped plugin # punktfunk/ <- single top-level dir == plugin.json "name"
# lost the shortcut artwork + Steam Input layout for a while.) CI only adds `update.json` on # plugin.json [required]
# top: the {channel, manifest} pointer the plugin's self-update check polls. # package.json [required; CI stamps "version" — Decky reads the installed version here]
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
# #
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel # SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed # `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
@@ -85,27 +90,28 @@ jobs:
- name: Assemble store-layout zip - name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }} working-directory: ${{ gitea.workspace }}
run: | run: |
# node:22-bookworm ships python3 (a package.sh dep) but not zip; install both anyway apt-get update && apt-get install -y --no-install-recommends zip >/dev/null
# so an image change can't silently break the build. STAGE="$RUNNER_TEMP/decky"
apt-get update && apt-get install -y --no-install-recommends zip python3 >/dev/null DEST="$STAGE/$PLUGIN"
# Stage the canonical plugin tree (dist/, main.py, bin/, assets/, controller_config/, rm -rf "$STAGE"; mkdir -p "$DEST/dist" "$DEST/bin"
# LICENSE, …) with the same script local/sideload builds use — see the header comment. cp clients/decky/plugin.json "$DEST/"
# Runs AFTER the version stamp, so the staged package.json carries $VERSION. cp clients/decky/package.json "$DEST/"
bash clients/decky/scripts/package.sh cp clients/decky/main.py "$DEST/"
DEST="clients/decky/out/$PLUGIN" cp clients/decky/dist/index.js "$DEST/dist/"
# CI-only addition: the self-update channel pointer the backend reads (main.py cp clients/decky/README.md "$DEST/"
# check_update). It points at THIS channel's manifest.json (published below); that # The stream-launch wrapper (target of the Steam shortcut); keep it executable
# manifest in turn points at the immutable per-version zip, so its sha256 stays valid # (runner_info() also re-chmods at runtime in case the zip/extract drops the bit).
# across future alias re-uploads. cp clients/decky/bin/punktfunkrun.sh "$DEST/bin/"
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json" printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd clients/decky/out && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" ) ( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip" ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip" unzip -l "$RUNNER_TEMP/punktfunk.zip"
# Backstop against packaging drift: the runtime-loaded pieces MUST be in the zip.
for f in main.py dist/index.js bin/punktfunkrun.sh assets/grid.png \
controller_config/punktfunk.vdf update.json; do
unzip -l "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN/$f" >/dev/null || { echo "MISSING $f" >&2; exit 1; }
done
# The update manifest the plugin polls: the immutable per-version artifact + its # The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on # sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias). # mismatch — so it MUST be the per-version URL, never the mutable alias).
-36
View File
@@ -2,7 +2,6 @@
# punktfunk-web — management console (web/Dockerfile, repo-root context) # punktfunk-web — management console (web/Dockerfile, repo-root context)
# punktfunk-docs — documentation site (docs-site/Dockerfile) # punktfunk-docs — documentation site (docs-site/Dockerfile)
# punktfunk-rust-ci — Rust CI builder image consumed by ci.yml # punktfunk-rust-ci — Rust CI builder image consumed by ci.yml
# punktfunk-rust-ci-arm64cross — the above + an arm64 sysroot, for the aarch64 client legs
# punktfunk-fedora-rpm — Fedora 43 builder image consumed by rpm.yml (Bazzite RPM) # punktfunk-fedora-rpm — Fedora 43 builder image consumed by rpm.yml (Bazzite RPM)
# Host and clients are intentionally NOT containerized (see CLAUDE.md "What's left"). # Host and clients are intentionally NOT containerized (see CLAUDE.md "What's left").
# #
@@ -81,41 +80,6 @@ jobs:
docker push "$REGISTRY/$OWNER/${{ matrix.image }}:latest" docker push "$REGISTRY/$OWNER/${{ matrix.image }}:latest"
case "$GITHUB_REF" in refs/tags/v*) docker push "$REGISTRY/$OWNER/${{ matrix.image }}:${GITHUB_REF_NAME}" ;; esac case "$GITHUB_REF" in refs/tags/v*) docker push "$REGISTRY/$OWNER/${{ matrix.image }}:${GITHUB_REF_NAME}" ;; esac
# The aarch64 CROSS builder — a SEPARATE job because it is `FROM punktfunk-rust-ci:latest`
# and so must not race the matrix entry that publishes that base. Consumed by the arm64
# client legs in deb.yml/rpm.yml/arch.yml. Root context: it needs rust-toolchain.toml to
# install the target against the toolchain the workspace actually pins.
build-push-arm64cross:
runs-on: ubuntu-24.04
needs: build-push
timeout-minutes: 45
env:
IMAGE: punktfunk-rust-ci-arm64cross
steps:
- uses: actions/checkout@v4
- name: Login to registry
run: |
echo "${{ secrets.REGISTRY_TOKEN }}" \
| docker login "$REGISTRY" -u enricobuehler --password-stdin
- name: Build
run: |
EXTRA=""
case "$GITHUB_REF" in refs/tags/v*) EXTRA="-t $REGISTRY/$OWNER/$IMAGE:${GITHUB_REF_NAME}" ;; esac
docker build --pull \
-f ci/rust-ci-arm64cross.Dockerfile \
-t "$REGISTRY/$OWNER/$IMAGE:latest" \
-t "$REGISTRY/$OWNER/$IMAGE:sha-${GITHUB_SHA::8}" \
$EXTRA \
.
- name: Push
run: |
docker push "$REGISTRY/$OWNER/$IMAGE:sha-${GITHUB_SHA::8}"
docker push "$REGISTRY/$OWNER/$IMAGE:latest"
case "$GITHUB_REF" in refs/tags/v*) docker push "$REGISTRY/$OWNER/$IMAGE:${GITHUB_REF_NAME}" ;; esac
# Deploy the docs site to unom-1, the DMZ services VM website/cms also deploy to # Deploy the docs site to unom-1, the DMZ services VM website/cms also deploy to
# (docs.punktfunk.unom.io via Caddy on home-reverse-proxy-1 -> :3220). Same secret set # (docs.punktfunk.unom.io via Caddy on home-reverse-proxy-1 -> :3220). Same secret set
# as unom/website's deploy: DEPLOY_HOST/DEPLOY_USER/DEPLOY_PORT/DEPLOY_SSH_KEY (the # as unom/website's deploy: DEPLOY_HOST/DEPLOY_USER/DEPLOY_PORT/DEPLOY_SSH_KEY (the
+2 -15
View File
@@ -73,21 +73,8 @@ jobs:
# sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM # sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM
# trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied # trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied
# there, right before the first `flatpak` network call. # there, right before the first `flatpak` network call.
- name: Fix container DNS (drop nss-resolve) - name: Fix container DNS (drop nss-resolve — no systemd-resolved in CI)
run: | run: sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf
sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf
# History: this step used to ALSO force glibc onto TCP DNS (`options use-vc`) because
# the runner fleet's Docker embedded resolver dropped UDP lookups under parallel-job
# load (investigated 2026-07-11; v0.15.0/v0.16.0 each burned retry.sh's whole budget).
# That root cause is now fixed at the infra level (2026-07-22): the runner host runs a
# local dnsmasq cache on the docker bridge and daemon.json points every job container
# at it, so lookups terminate on-box instead of crossing the saturated uplink — the
# UDP path is reliable again. The TCP path through the same chain proved FLAKY under
# fleet concurrency (flatpak remote-add failed 10/10 with instant NXDOMAIN while dnf
# in the same container resolved fine), so `use-vc` flipped from mitigation to sole
# cause of this leg's failures — removed. retry.sh (10×) stays as the backstop for
# genuine upstream blips.
cat /etc/resolv.conf || true
# fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step # fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step
# executes via the container shell (no node needed), so install node BEFORE checkout. # executes via the container shell (no node needed), so install node BEFORE checkout.
-69
View File
@@ -1,69 +0,0 @@
# Publish the plugin framework (@punktfunk/plugin-kit) to the Gitea npm registry
# (https://git.unom.io/api/packages/unom/npm/).
#
# Trigger: push a tag `plugin-kit-vX.Y.Z` (must equal plugin-kit/package.json "version"),
# or run manually. Versions independently of the app's `v*` and the SDK's `sdk-v*` tags.
#
# The kit's devDependency on @punktfunk/host is `file:../sdk`, so the SDK's dist must be
# built BEFORE the kit's `bun install` copies it.
#
# Auth: REGISTRY_TOKEN — the same repo Actions secret sdk-publish.yml uses.
name: plugin-kit-publish
on:
push:
tags: ['plugin-kit-v*']
workflow_dispatch:
jobs:
publish:
runs-on: ubuntu-24.04
container:
image: oven/bun:1
timeout-minutes: 15
steps:
# oven/bun's slim base ships neither git, a CA bundle, nor node — actions/checkout's HTTPS
# fetch needs git + ca-certificates, and the version-guard step below uses node.
- name: Install git + node + CA certs
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates git nodejs
- uses: actions/checkout@v4
- name: Build the SDK (file:../sdk dependency source)
working-directory: sdk
run: |
bun install --frozen-lockfile --ignore-scripts
bun run build
- name: Install dependencies
working-directory: plugin-kit
run: bun install --frozen-lockfile --ignore-scripts
- name: Typecheck
working-directory: plugin-kit
run: bun run typecheck
- name: Test
working-directory: plugin-kit
run: bun test
- name: Build (dist/ JS + .d.ts + theme.css)
working-directory: plugin-kit
run: bun run build
- name: Tag matches package version
if: startsWith(github.ref, 'refs/tags/')
working-directory: plugin-kit
run: |
TAG="${GITHUB_REF_NAME#plugin-kit-v}"
PKG="$(node -p "require('./package.json').version")"
test "$TAG" = "$PKG" || { echo "tag $GITHUB_REF_NAME does not match package version $PKG"; exit 1; }
- name: Publish to Gitea registry
working-directory: plugin-kit
env:
NODE_AUTH_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
test -n "$NODE_AUTH_TOKEN" || { echo "REGISTRY_TOKEN secret is empty"; exit 1; }
printf '//git.unom.io/api/packages/unom/npm/:_authToken=%s\n' "$NODE_AUTH_TOKEN" >> .npmrc
bun publish
-94
View File
@@ -149,26 +149,6 @@ jobs:
# inherits this from the env during the xcframework build). # inherits this from the env during the xcframework build).
echo "CMAKE_POLICY_VERSION_MINIMUM=3.5" >> "$GITHUB_ENV" echo "CMAKE_POLICY_VERSION_MINIMUM=3.5" >> "$GITHUB_ENV"
- name: Pin + prune Xcode DerivedData
# Without -derivedDataPath, xcodebuild derives its DerivedData directory name from the
# PROJECT'S ABSOLUTE PATH — and act_runner rotates its workspace
# (~/.cache/act/<hash>/hostexecutor), so each rotation minted a brand new ~760 MB tree
# under ~/Library that nothing ever collected. 31 of them piled up in three days
# (~32 GB with the shared ModuleCache), filled the runner's boot volume, and failed
# v0.16.0's xcframework build with "No space left on device". Pinning one path makes the
# tree REUSED instead of multiplied — it also keeps the module cache warm between runs.
run: |
DD="$HOME/ci/derived-data/release"
mkdir -p "$DD"
echo "DERIVED_DATA=$DD" >> "$GITHUB_ENV"
# Safety net for trees the pin does not own: the legacy per-path ones from before this
# change, and anything another job leaves in the default root. Untouched for a week ⇒ gone.
if [ -d "$HOME/Library/Developer/Xcode/DerivedData" ]; then
find "$HOME/Library/Developer/Xcode/DerivedData" -mindepth 1 -maxdepth 1 \
-mtime +7 -exec rm -rf {} + 2>/dev/null || true
fi
echo "disk after prune:"; df -h /System/Volumes/Data | tail -1
- name: Build PunktfunkCore.xcframework (mac + iOS + tvOS) - name: Build PunktfunkCore.xcframework (mac + iOS + tvOS)
# tvOS is a tier-3 target (nightly -Zbuild-std): slow on the first build, then cached on # tvOS is a tier-3 target (nightly -Zbuild-std): slow on the first build, then cached on
# the self-hosted runner. Built on canary too so the tvOS archive/upload below runs on the # the self-hosted runner. Built on canary too so the tvOS archive/upload below runs on the
@@ -196,7 +176,6 @@ jobs:
-project "$PROJECT" -scheme Punktfunk \ -project "$PROJECT" -scheme Punktfunk \
-destination 'generic/platform=macOS' \ -destination 'generic/platform=macOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-macos.xcarchive" \ -archivePath "$RUNNER_TEMP/Punktfunk-macos.xcarchive" \
-derivedDataPath "$DERIVED_DATA" \
-skipMacroValidation -skipPackagePluginValidation \ -skipMacroValidation -skipPackagePluginValidation \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \ MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGNING_ALLOWED=NO CODE_SIGNING_ALLOWED=NO
@@ -294,7 +273,6 @@ jobs:
-project "$PROJECT" -scheme Punktfunk \ -project "$PROJECT" -scheme Punktfunk \
-destination 'generic/platform=macOS' \ -destination 'generic/platform=macOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-macos-appstore.xcarchive" \ -archivePath "$RUNNER_TEMP/Punktfunk-macos-appstore.xcarchive" \
-derivedDataPath "$DERIVED_DATA" \
-skipMacroValidation -skipPackagePluginValidation \ -skipMacroValidation -skipPackagePluginValidation \
-allowProvisioningUpdates \ -allowProvisioningUpdates \
-authenticationKeyPath "$RUNNER_TEMP/asc.p8" \ -authenticationKeyPath "$RUNNER_TEMP/asc.p8" \
@@ -358,7 +336,6 @@ jobs:
-project "$PROJECT" -scheme Punktfunk-iOS \ -project "$PROJECT" -scheme Punktfunk-iOS \
-destination 'generic/platform=iOS' \ -destination 'generic/platform=iOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-ios.xcarchive" \ -archivePath "$RUNNER_TEMP/Punktfunk-ios.xcarchive" \
-derivedDataPath "$DERIVED_DATA" \
-skipMacroValidation -skipPackagePluginValidation \ -skipMacroValidation -skipPackagePluginValidation \
-allowProvisioningUpdates \ -allowProvisioningUpdates \
-authenticationKeyPath "$RUNNER_TEMP/asc.p8" \ -authenticationKeyPath "$RUNNER_TEMP/asc.p8" \
@@ -393,76 +370,6 @@ jobs:
-authenticationKeyID "${{ secrets.ASC_API_KEY_ID }}" \ -authenticationKeyID "${{ secrets.ASC_API_KEY_ID }}" \
-authenticationKeyIssuerID "${{ secrets.ASC_API_ISSUER_ID }}" -authenticationKeyIssuerID "${{ secrets.ASC_API_ISSUER_ID }}"
- name: iOS — export .ipa (Gitea release + run artifact)
# The TestFlight step above uploads straight to App Store Connect (destination=upload) and
# leaves NO .ipa on disk. Re-export the SAME archive with destination=export to get an
# App Store distribution-signed .ipa for the Gitea release + the run artifacts. Same gate as
# that archive; a warn+skip (never fails the best-effort iOS leg) if the archive is absent,
# e.g. a workflow_dispatch with testflight=false. NOTE: an App Store-signed .ipa installs
# only via TestFlight/App Store, not by direct sideload — it's a release/archival artifact.
if: gitea.event_name != 'workflow_dispatch' || inputs.testflight == 'true'
id: ios_ipa
run: |
ARCHIVE="$RUNNER_TEMP/Punktfunk-ios.xcarchive"
if [ ! -d "$ARCHIVE" ]; then
echo "::warning::iOS archive not found — skipping .ipa export"
exit 0
fi
PROFILE="Punktfunk iOS App Store Distribution"
WIDGET_PROFILE="Punktfunk iOS Widgets App Store Distribution"
# destination=export writes the .ipa to -exportPath; otherwise identical manual signing to
# the upload plist (both profiles, Apple Distribution). No ASC key needed — no network.
cat > "$RUNNER_TEMP/export-appstore-ipa.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>method</key><string>app-store-connect</string>
<key>destination</key><string>export</string>
<key>teamID</key><string>$TEAM_ID</string>
<key>signingStyle</key><string>manual</string>
<key>signingCertificate</key><string>Apple Distribution</string>
<key>provisioningProfiles</key>
<dict>
<key>io.unom.punktfunk</key><string>$PROFILE</string>
<key>io.unom.punktfunk.widgets</key><string>$WIDGET_PROFILE</string>
</dict>
</dict>
</plist>
EOF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild -exportArchive \
-archivePath "$ARCHIVE" \
-exportOptionsPlist "$RUNNER_TEMP/export-appstore-ipa.plist" \
-exportPath "$RUNNER_TEMP/export-ipa"
SRC=$(ls "$RUNNER_TEMP/export-ipa/"*.ipa 2>/dev/null | head -1)
[ -n "$SRC" ] || { echo "::warning::no .ipa was produced by export"; exit 0; }
mkdir -p "$GITHUB_WORKSPACE/dist"
IPA="$GITHUB_WORKSPACE/dist/Punktfunk-$VERSION.ipa"
mv "$SRC" "$IPA"
echo "IPA=$IPA" >> "$GITHUB_ENV"
echo "ipa=dist/Punktfunk-$VERSION.ipa" >> "$GITHUB_OUTPUT"
echo "exported $IPA"
- name: Attach .ipa to the workflow run
if: steps.ios_ipa.outputs.ipa != ''
# v3, not v4: Gitea's artifact backend identifies as GHES, which upload-artifact@v4 refuses
# (same reason as android.yml / apple.yml). Download is a zip of the .ipa.
uses: actions/upload-artifact@v3
with:
name: punktfunk-ios-ipa
path: ${{ steps.ios_ipa.outputs.ipa }}
if-no-files-found: warn
retention-days: 30
- name: Attach .ipa to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v') && steps.ios_ipa.outputs.ipa != ''
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
. scripts/ci/gitea-release.sh
RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
upsert_asset "$RID" "$IPA" "Punktfunk-$VERSION.ipa"
- name: tvOS — archive + upload to TestFlight - name: tvOS — archive + upload to TestFlight
# Canary + stable, the same track as iOS/macOS — the tvOS xcframework slice is now built # Canary + stable, the same track as iOS/macOS — the tvOS xcframework slice is now built
# on every apple push (above), so this matches the iOS step's gate exactly. # on every apple push (above), so this matches the iOS step's gate exactly.
@@ -487,7 +394,6 @@ jobs:
-project "$PROJECT" -scheme Punktfunk-tvOS \ -project "$PROJECT" -scheme Punktfunk-tvOS \
-destination 'generic/platform=tvOS' \ -destination 'generic/platform=tvOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-tvos.xcarchive" \ -archivePath "$RUNNER_TEMP/Punktfunk-tvos.xcarchive" \
-derivedDataPath "$DERIVED_DATA" \
-skipMacroValidation -skipPackagePluginValidation \ -skipMacroValidation -skipPackagePluginValidation \
-allowProvisioningUpdates \ -allowProvisioningUpdates \
-authenticationKeyPath "$RUNNER_TEMP/asc.p8" \ -authenticationKeyPath "$RUNNER_TEMP/asc.p8" \
-32
View File
@@ -50,23 +50,6 @@ on:
# builds — without these, encoder changes only reached this workflow via Cargo.lock luck. # builds — without these, encoder changes only reached this workflow via Cargo.lock luck.
- 'crates/pf-encode/**' - 'crates/pf-encode/**'
- 'crates/libvpl-sys/**' - 'crates/libvpl-sys/**'
# …and the rest of the W6 subsystem crates this build compiles. pf-encode was listed while
# the crates it speaks (pf-frame's CapturedFrame/PixelFormat/dxgi vocabulary, pf-gpu's
# adapter selection, pf-zerocopy, pf-host-config) were not, so a change that broke the
# Windows host through one of THEM reached main with no Windows build at all — the same
# Cargo.lock-luck gap the two lines above were added to close.
- 'crates/pf-frame/**'
- 'crates/pf-gpu/**'
- 'crates/pf-zerocopy/**'
- 'crates/pf-host-config/**'
- 'crates/pf-capture/**'
- 'crates/pf-win-display/**'
- 'crates/pf-vdisplay/**'
- 'crates/pf-inject/**'
- 'crates/pf-paths/**'
- 'crates/pf-driver-proto/**'
- 'crates/pf-clipboard/**'
- 'crates/pyrowave-sys/**'
- 'packaging/windows/**' - 'packaging/windows/**'
- 'scripts/windows/**' - 'scripts/windows/**'
- 'web/**' - 'web/**'
@@ -171,23 +154,8 @@ jobs:
# build minutes earlier). Linting in release reuses those native build-script artifacts (no # build minutes earlier). Linting in release reuses those native build-script artifacts (no
# openh264 rebuild), and keeps everything in one C:\t\release tree. Same reason # openh264 rebuild), and keeps everything in one C:\t\release tree. Same reason
# pf-vkhdr-layer's clippy below runs --release. # pf-vkhdr-layer's clippy below runs --release.
#
# pf-encode is linted SEPARATELY with --all-targets so its Windows `#[cfg(test)]` modules
# are type-checked — the AMF C-ABI layout assertions (`variant_layout_matches_c` and
# friends, which are the only guard on a hand-mirrored vtable ABI), the QSV tests, and the
# PyroWave-Windows smoke test. The host lint above cannot cover them: `-p punktfunk-host`
# only builds pf-encode as a dependency, so its test targets are never compiled, and that
# blind spot is what let the Linux twin's tests rot to the wrong arity unnoticed.
# NOTE: clippy (a check, no link step) is deliberately the vehicle here — `cargo test`
# with `nvenc` cannot LINK on MSVC: nvidia-video-codec-sdk link-imports
# NvEncodeAPICreateInstance / NvEncodeAPIGetMaxSupportedVersion, which resolve only against
# the driver's import lib. (On Linux the same crate dlopens them, so ci.yml can and does
# run the tests there.) Running them here would need an `--features amf-qsv,qsv` build
# without `nvenc`, i.e. a third full dep tree on a runner that already trips C1069 — not
# worth it while ci.yml executes the same tests.
run: | run: |
cargo clippy --release -p punktfunk-host --features nvenc,amf-qsv,qsv -- -D warnings; if ($LASTEXITCODE) { throw "host clippy" } cargo clippy --release -p punktfunk-host --features nvenc,amf-qsv,qsv -- -D warnings; if ($LASTEXITCODE) { throw "host clippy" }
cargo clippy --release -p pf-encode --all-targets --features nvenc,amf-qsv,qsv -- -D warnings; if ($LASTEXITCODE) { throw "pf-encode clippy" }
cargo clippy --release -p punktfunk-tray -- -D warnings; if ($LASTEXITCODE) { throw "tray clippy" } cargo clippy --release -p punktfunk-tray -- -D warnings; if ($LASTEXITCODE) { throw "tray clippy" }
- name: Build + lint the HDR Vulkan layer (pf-vkhdr-layer) - name: Build + lint the HDR Vulkan layer (pf-vkhdr-layer)
Generated
+27 -98
View File
@@ -656,30 +656,6 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "chacha20poly1305"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35"
dependencies = [
"aead",
"chacha20",
"cipher",
"poly1305",
"zeroize",
]
[[package]] [[package]]
name = "ciborium" name = "ciborium"
version = "0.2.2" version = "0.2.2"
@@ -715,7 +691,6 @@ checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [ dependencies = [
"crypto-common", "crypto-common",
"inout", "inout",
"zeroize",
] ]
[[package]] [[package]]
@@ -1459,16 +1434,6 @@ dependencies = [
"version_check", "version_check",
] ]
[[package]]
name = "gethostname"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bd49230192a3797a9a4d6abe9b3eed6f7fa4c8a8a4947977c6f80025f92cbd8"
dependencies = [
"rustix",
"windows-link 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]] [[package]]
name = "getrandom" name = "getrandom"
version = "0.2.17" version = "0.2.17"
@@ -2194,7 +2159,7 @@ dependencies = [
[[package]] [[package]]
name = "latency-probe" name = "latency-probe"
version = "0.19.2" version = "0.15.0"
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
@@ -2299,7 +2264,7 @@ dependencies = [
[[package]] [[package]]
name = "libvpl-sys" name = "libvpl-sys"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -2334,7 +2299,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]] [[package]]
name = "loss-harness" name = "loss-harness"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"punktfunk-core", "punktfunk-core",
] ]
@@ -2823,7 +2788,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]] [[package]]
name = "pf-capture" name = "pf-capture"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2839,12 +2804,11 @@ dependencies = [
"tokio", "tokio",
"tracing", "tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)", "windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"x11rb",
] ]
[[package]] [[package]]
name = "pf-client-core" name = "pf-client-core"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2868,7 +2832,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-clipboard" name = "pf-clipboard"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2886,7 +2850,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-console-ui" name = "pf-console-ui"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2907,7 +2871,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-encode" name = "pf-encode"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2917,7 +2881,6 @@ dependencies = [
"libvpl-sys", "libvpl-sys",
"nvidia-video-codec-sdk", "nvidia-video-codec-sdk",
"openh264", "openh264",
"pf-capture",
"pf-frame", "pf-frame",
"pf-gpu", "pf-gpu",
"pf-host-config", "pf-host-config",
@@ -2931,7 +2894,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-ffvk" name = "pf-ffvk"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"ash", "ash",
"bindgen", "bindgen",
@@ -2940,7 +2903,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-frame" name = "pf-frame"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"libc", "libc",
@@ -2952,7 +2915,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-gpu" name = "pf-gpu"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-host-config", "pf-host-config",
@@ -2966,11 +2929,11 @@ dependencies = [
[[package]] [[package]]
name = "pf-host-config" name = "pf-host-config"
version = "0.19.2" version = "0.15.0"
[[package]] [[package]]
name = "pf-inject" name = "pf-inject"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2981,7 +2944,6 @@ dependencies = [
"pf-driver-proto", "pf-driver-proto",
"pf-host-config", "pf-host-config",
"pf-paths", "pf-paths",
"pf-win-display",
"punktfunk-core", "punktfunk-core",
"reis", "reis",
"tokio", "tokio",
@@ -2999,14 +2961,14 @@ dependencies = [
[[package]] [[package]]
name = "pf-paths" name = "pf-paths"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "pf-presenter" name = "pf-presenter"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -3021,11 +2983,10 @@ dependencies = [
[[package]] [[package]]
name = "pf-vdisplay" name = "pf-vdisplay"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
"bitflags",
"bytemuck", "bytemuck",
"futures-util", "futures-util",
"hex", "hex",
@@ -3052,7 +3013,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-win-display" name = "pf-win-display"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-paths", "pf-paths",
@@ -3064,7 +3025,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-zerocopy" name = "pf-zerocopy"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -3073,7 +3034,6 @@ dependencies = [
"libloading", "libloading",
"serde", "serde",
"serde_json", "serde_json",
"tempfile",
"tracing", "tracing",
] ]
@@ -3176,17 +3136,6 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "poly1305"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf"
dependencies = [
"cpufeatures",
"opaque-debug",
"universal-hash",
]
[[package]] [[package]]
name = "polyval" name = "polyval"
version = "0.6.2" version = "0.6.2"
@@ -3272,7 +3221,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-android" name = "punktfunk-client-android"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"jni", "jni",
@@ -3288,7 +3237,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-linux" name = "punktfunk-client-linux"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -3304,7 +3253,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-session" name = "punktfunk-client-session"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-client-core", "pf-client-core",
@@ -3319,7 +3268,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-windows" name = "punktfunk-client-windows"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"async-channel", "async-channel",
"ffmpeg-next", "ffmpeg-next",
@@ -3338,12 +3287,11 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-core" name = "punktfunk-core"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"aes-gcm", "aes-gcm",
"bytes", "bytes",
"cbindgen", "cbindgen",
"chacha20poly1305",
"criterion", "criterion",
"fec-rs", "fec-rs",
"hmac", "hmac",
@@ -3370,7 +3318,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-host" name = "punktfunk-host"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"aes", "aes",
"aes-gcm", "aes-gcm",
@@ -3415,13 +3363,11 @@ dependencies = [
"rand 0.8.6", "rand 0.8.6",
"rcgen", "rcgen",
"reis", "reis",
"ring",
"roxmltree", "roxmltree",
"rsa", "rsa",
"rusqlite", "rusqlite",
"rustls", "rustls",
"rusty_enet", "rusty_enet",
"semver",
"serde", "serde",
"serde_json", "serde_json",
"sha2", "sha2",
@@ -3454,7 +3400,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-probe" name = "punktfunk-probe"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"mdns-sd", "mdns-sd",
@@ -3468,7 +3414,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-tray" name = "punktfunk-tray"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ksni", "ksni",
@@ -3491,7 +3437,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]] [[package]]
name = "pyrowave-sys" name = "pyrowave-sys"
version = "0.19.2" version = "0.15.0"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -5900,23 +5846,6 @@ version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4" checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
[[package]]
name = "x11rb"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9993aa5be5a26815fe2c3eacfc1fde061fc1a1f094bf1ad2a18bf9c495dd7414"
dependencies = [
"gethostname",
"rustix",
"x11rb-protocol",
]
[[package]]
name = "x11rb-protocol"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea6fc2961e4ef194dcbfe56bb845534d0dc8098940c7e5c012a258bfec6701bd"
[[package]] [[package]]
name = "x509-parser" name = "x509-parser"
version = "0.16.0" version = "0.16.0"
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" } ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package] [workspace.package]
version = "0.19.2" version = "0.15.0"
edition = "2021" edition = "2021"
rust-version = "1.82" rust-version = "1.82"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
+4 -1136
View File
File diff suppressed because it is too large Load Diff
-45
View File
@@ -1,45 +0,0 @@
#!/bin/bash
# Host-side C compiler wrapper for the aarch64 cross image (ci/rust-ci-arm64cross.Dockerfile).
#
# Why this exists: ffmpeg-sys-next's build script compiles a probe it intends to RUN — it
# executes the binary to read the libav* version macros — so it forces `.target(HOST)` with
# the comment "don't cross-compile this", but still hands that host compile the TARGET's
# pkg-config include paths. `-I/usr/include/aarch64-linux-gnu` then shadows the host's own
# multiarch libc headers and the x86 compiler dies inside bits/math-vector.h on NEON/SVE
# types it has never heard of.
#
# Prepending the host's multiarch dir does NOT fix it: GCC drops a `-I` that duplicates a
# directory already on its system include path (keeping it in the original, later position),
# so the arm64 dir stays in front. The reliable fix is to remove the target include dirs from
# the host compile entirely — the probe only wants FFmpeg's version macros, and the amd64
# libav*-dev headers are installed and on the default search path, at the same version (both
# come from this Ubuntu release).
#
# Scope: only ever invoked as CC for the HOST triple (CC_x86_64_unknown_linux_gnu). Target
# compiles go to aarch64-linux-gnu-gcc and never pass through here.
set -euo pipefail
declare -a out=()
while (($#)); do
case "$1" in
# `-I dir` as two arguments — the form cc's Command building and ffmpeg-sys both emit.
-I)
if [[ ${2-} == *aarch64-linux-gnu* ]]; then
shift 2
continue
fi
out+=("$1" "${2-}")
shift 2
;;
# `-Idir` glued into one argument.
-I*aarch64-linux-gnu*)
shift
;;
*)
out+=("$1")
shift
;;
esac
done
exec /usr/bin/cc "${out[@]}"
-81
View File
@@ -1,81 +0,0 @@
# Cross-compiling CI builder: amd64 host toolchain + an arm64 multiarch sysroot, for the
# aarch64 Linux CLIENT artifacts (punktfunk-client + punktfunk-session).
#
# docker build -f ci/rust-ci-arm64cross.Dockerfile -t punktfunk-rust-ci-arm64cross .
#
# Derived from punktfunk-rust-ci so the Rust toolchain, clang, and CMake are byte-identical
# to the amd64 legs — this image only adds the target side. Kept as a SEPARATE image rather
# than folded into the base because the :arm64 dev libs are ~1 GB that every other CI job
# would otherwise pull for nothing.
#
# Client only: the Linux HOST stays amd64 (its encode stack is NVENC/QSV/AMF), so none of the
# host's CUDA/GBM link deps are mirrored here.
#
# Ubuntu splits archives by architecture: amd64 lives on archive.ubuntu.com, every port
# (arm64 included) on ports.ubuntu.com. Both stanzas therefore have to be pinned with an
# explicit `Architectures:` or apt tries to fetch arm64 from the amd64 mirror and 404s.
#
# Built from the REPO ROOT context (not ci/) — see the rust-toolchain.toml copy below.
FROM git.unom.io/unom/punktfunk-rust-ci:latest
ENV DEBIAN_FRONTEND=noninteractive
# 1. Pin the stock sources to amd64, add ports.ubuntu.com for arm64.
RUN sed -i 's|^Types: deb$|Types: deb\nArchitectures: amd64|' /etc/apt/sources.list.d/ubuntu.sources \
&& . /etc/os-release \
&& printf 'Types: deb\nArchitectures: arm64\nURIs: http://ports.ubuntu.com/ubuntu-ports/\nSuites: %s %s-updates %s-backports %s-security\nComponents: main universe restricted multiverse\nSigned-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg\n' \
"$VERSION_CODENAME" "$VERSION_CODENAME" "$VERSION_CODENAME" "$VERSION_CODENAME" \
> /etc/apt/sources.list.d/ubuntu-ports-arm64.sources \
&& dpkg --add-architecture arm64
# 2. The cross toolchain + every arm64 dev lib the client links. Mirrors the client half of
# rust-ci.Dockerfile's list (FFmpeg, PipeWire, Opus, SDL3, GTK4/libadwaita, xkbcommon,
# Vulkan headers for pf-ffvk's bindgen over hwcontext_vulkan.h).
RUN apt-get update && apt-get install -y --no-install-recommends \
crossbuild-essential-arm64 \
libavcodec-dev:arm64 libavformat-dev:arm64 libavutil-dev:arm64 libswscale-dev:arm64 \
libavfilter-dev:arm64 libavdevice-dev:arm64 \
libpipewire-0.3-dev:arm64 libopus-dev:arm64 \
libsdl3-dev:arm64 libgtk-4-dev:arm64 libadwaita-1-dev:arm64 \
libwayland-dev:arm64 libxkbcommon-dev:arm64 libvulkan-dev:arm64 \
&& rm -rf /var/lib/apt/lists/*
# 3. The Rust target — installed against the toolchain the WORKSPACE pins, not the image's
# default. The base image bakes whatever `stable` was at its build time, while every build
# in the repo switches to the exact channel in rust-toolchain.toml; adding the target to
# the default toolchain instead leaves the pinned one without an aarch64 std, and the build
# dies on `can't find crate for core` a few hundred crates in. Running rustup from a
# directory that contains the pin file resolves the right toolchain (and pre-downloads it,
# which every workspace job would otherwise pay for on first use).
COPY rust-toolchain.toml /opt/pf-toolchain/
WORKDIR /opt/pf-toolchain
RUN rustup target add aarch64-unknown-linux-gnu && rustup show
WORKDIR /
# 4. Cross wiring. Everything in this image is a cross build, so the plain (un-suffixed)
# variables are safe and cover the crates that roll their own pkg-config/bindgen calls
# instead of going through the target-scoped lookups.
# * PKG_CONFIG uses Debian's multiarch wrapper, which resolves the arm64 .pc files and
# rewrites -I/-L into the sysroot without per-crate cooperation.
# * BINDGEN_EXTRA_CLANG_ARGS: clang defaults to the host triple, so bindgen would parse
# arm64 headers with amd64 type layouts (silently wrong, not a build error) — the
# explicit --target plus the multiarch include dir is what keeps the layouts honest.
# * CC_x86_64_unknown_linux_gnu routes HOST-targeted compiles through a wrapper that
# strips the arm64 include dirs — see ci/pf-host-cc for the ffmpeg-sys-next probe it
# exists for.
COPY ci/pf-host-cc /usr/local/bin/pf-host-cc
RUN chmod 0755 /usr/local/bin/pf-host-cc
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++ \
AR_aarch64_unknown_linux_gnu=aarch64-linux-gnu-ar \
CC_x86_64_unknown_linux_gnu=/usr/local/bin/pf-host-cc \
PKG_CONFIG=aarch64-linux-gnu-pkg-config \
PKG_CONFIG_ALLOW_CROSS=1 \
BINDGEN_EXTRA_CLANG_ARGS="--target=aarch64-unknown-linux-gnu -I/usr/include/aarch64-linux-gnu"
# Fail the BUILD, not some later CI job, if the wrapper or a sysroot .pc is missing.
RUN command -v aarch64-linux-gnu-pkg-config \
&& aarch64-linux-gnu-pkg-config --cflags libavcodec sdl3 gtk4 libpipewire-0.3 \
&& aarch64-linux-gnu-gcc -dumpmachine | grep -q aarch64
@@ -1,107 +0,0 @@
package io.unom.punktfunk
import android.content.ClipData
import android.content.ClipboardManager
import android.content.Context
import android.os.Handler
import android.os.Looper
import io.unom.punktfunk.kit.NativeBridge
/**
* Text clipboard sync for the active session (the desktop-client model, text-only v1):
* * **Device → host**: a local copy (the primary-clip listener, plus one probe at start) is
* announced as a lazy offer — the text crosses only when the host actually pastes (a
* `fetch:` event, answered with the clipboard's current content).
* * **Host → device**: a host copy arrives as an `offer:` event and is fetched eagerly into
* the system clipboard (Android apps can't lazily materialize a paste from the network
* without a content-provider round-trip that isn't worth it here).
*
* Loop guard: text set from a host fetch is remembered ([lastFromHost]) so the resulting
* primary-clip-changed callback doesn't bounce it straight back as a new offer. Clipboard reads
* happen while the stream is foreground (Android only allows focused-app reads). The native
* events are drained on a dedicated thread and applied on the main thread; [stop] joins it.
*/
class ClipboardSync(
private val context: Context,
private val handle: Long,
) {
private val main = Handler(Looper.getMainLooper())
private val cm = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
@Volatile private var running = true
private var seq = 0
private var lastOffered: String? = null
private var lastFromHost: String? = null
private var pendingFetch = -1
private var thread: Thread? = null
private val clipListener = ClipboardManager.OnPrimaryClipChangedListener { offerLocal() }
fun start() {
NativeBridge.nativeClipControl(handle, true)
cm.addPrimaryClipChangedListener(clipListener)
thread = Thread({ pollLoop() }, "pf-clipboard").also { it.start() }
offerLocal() // whatever is already on the clipboard is pasteable host-side right away
}
fun stop() {
running = false
cm.removePrimaryClipChangedListener(clipListener)
thread?.join(600) // one poll timeout (250 ms) + slack
thread = null
}
/** Announce the current local text (if it's new and not an echo of a host copy). */
private fun offerLocal() {
if (!running) return
val text = currentClipText() ?: return
if (text == lastOffered || text == lastFromHost) return
lastOffered = text
seq += 1
NativeBridge.nativeClipOfferText(handle, seq)
}
private fun currentClipText(): String? = runCatching {
cm.primaryClip?.takeIf { it.itemCount > 0 }?.getItemAt(0)
?.coerceToText(context)?.toString()?.takeIf { it.isNotEmpty() }
}.getOrNull()
private fun pollLoop() {
while (running) {
val ev = NativeBridge.nativeNextClip(handle) ?: continue
if (ev == "closed") return
main.post { handleEvent(ev) }
}
}
private fun handleEvent(ev: String) {
if (!running) return
val parts = ev.split(":", limit = 3)
when (parts[0]) {
"offer" -> {
val offerSeq = parts.getOrNull(1)?.toIntOrNull() ?: return
if (parts.getOrNull(2) == "1") {
pendingFetch = NativeBridge.nativeClipFetchText(handle, offerSeq)
}
}
"fetch" -> {
val req = parts.getOrNull(1)?.toIntOrNull() ?: return
val text = currentClipText()
if (text != null) {
NativeBridge.nativeClipServeText(handle, req, text)
} else {
NativeBridge.nativeClipCancel(handle, req)
}
}
"data" -> {
val xfer = parts.getOrNull(1)?.toIntOrNull() ?: return
if (xfer != pendingFetch) return // stale/unknown transfer
pendingFetch = -1
val text = parts.getOrNull(2)?.takeIf { it.isNotEmpty() } ?: return
lastFromHost = text
runCatching { cm.setPrimaryClip(ClipData.newPlainText("Punktfunk", text)) }
}
// "state"/"cancel"/"error": nothing to drive in the text-only v1.
}
}
}
@@ -54,21 +54,6 @@ class MainActivity : ComponentActivity() {
var padKeyProbe: ((KeyEvent) -> Boolean)? = null var padKeyProbe: ((KeyEvent) -> Boolean)? = null
var padMotionProbe: ((MotionEvent) -> Boolean)? = null var padMotionProbe: ((MotionEvent) -> Boolean)? = null
/**
* Physical-mouse forwarder for the active session (built/released by StreamScreen, like
* [gamepadRouter]): uncaptured hover/click/wheel forwards as absolute cursor input, captured
* ([android.view.View.requestPointerCapture]) raw deltas as relative mouse-look. The dispatch
* overrides below route every SOURCE_MOUSE event here while streaming. Null while not streaming.
*/
var mouseForwarder: MouseForwarder? = null
/**
* TV remote-as-pointer for the active session (StreamScreen builds it on TV devices only):
* hold SELECT to toggle, then the D-pad glides the host cursor. Consulted first for
* non-gamepad keys while streaming. Null while not streaming or not a TV.
*/
var remotePointer: RemotePointer? = null
/** /**
* Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a * Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a
* couch user with no keyboard/Back can always leave a stream. * couch user with no keyboard/Back can always leave a stream.
@@ -339,37 +324,9 @@ class MainActivity : ComponentActivity() {
return true // consumed return true // consumed
} }
} }
// TV remote-as-pointer sees non-gamepad keys first (SELECT long-press toggles it;
// while active it owns the D-pad/SELECT/PLAY-PAUSE/BACK).
if (!event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
remotePointer?.let { if (it.onKey(event)) return true }
}
// Ctrl+Alt+Shift+Q — the cross-client pointer-capture toggle chord. Swallow both
// edges of the Q (the modifiers already went over the wire, exactly like desktop).
if (event.keyCode == KeyEvent.KEYCODE_Q &&
event.isCtrlPressed && event.isAltPressed && event.isShiftPressed
) {
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) {
mouseForwarder?.toggleCapture()
}
return true
}
when (event.keyCode) { when (event.keyCode) {
// A mouse's back/forward buttons already go over the wire as X1/X2 via their
// BUTTON_* motion edges — but Android ALSO delivers them as key events: the input
// reader synthesizes KEYCODE_BACK/FORWARD (stamped SOURCE_MOUSE) unconditionally,
// and a view-level FALLBACK BACK appears when the BUTTON_* press goes unconsumed.
// Swallow every such duplicate or it doubles as Android navigation and yanks the
// user out of the stream. A remote/keyboard BACK is never mouse-sourced, so it
// still falls through to the BackHandler and exits.
KeyEvent.KEYCODE_BACK, KeyEvent.KEYCODE_FORWARD ->
if (event.isFromSource(InputDevice.SOURCE_MOUSE) ||
event.flags and KeyEvent.FLAG_FALLBACK != 0
) {
return true
}
// Leave these to the system even while streaming. // Leave these to the system even while streaming.
// (BACK above → BackHandler leaves the stream.) KeyEvent.KEYCODE_BACK, // → BackHandler leaves the stream
KeyEvent.KEYCODE_VOLUME_UP, KeyEvent.KEYCODE_VOLUME_UP,
KeyEvent.KEYCODE_VOLUME_DOWN, KeyEvent.KEYCODE_VOLUME_DOWN,
KeyEvent.KEYCODE_VOLUME_MUTE, KeyEvent.KEYCODE_VOLUME_MUTE,
@@ -437,10 +394,6 @@ class MainActivity : ComponentActivity() {
override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean { override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
if (streamHandle != 0L) { if (streamHandle != 0L) {
if (gamepadRouter?.onMotion(event) == true) return true if (gamepadRouter?.onMotion(event) == true) return true
// Physical mouse (uncaptured): hover motion, wheel, button edges.
if (event.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onGenericMotion(event)) return true }
}
return super.dispatchGenericMotionEvent(event) return super.dispatchGenericMotionEvent(event)
} }
// The Controllers debug screen sees pad motion before the stick→D-pad synthesis below. // The Controllers debug screen sees pad motion before the stick→D-pad synthesis below.
@@ -478,24 +431,6 @@ class MainActivity : ComponentActivity() {
return super.dispatchGenericMotionEvent(event) return super.dispatchGenericMotionEvent(event)
} }
/**
* Mouse clicks/drags ride the TOUCH stream (the pointer is "down"). While streaming they
* belong to the mouse forwarder, never to the Compose touch-gesture layer — a physical
* mouse click must be a real click at the cursor, not a synthesized trackpad tap.
*/
override fun dispatchTouchEvent(ev: MotionEvent): Boolean {
if (streamHandle != 0L && ev.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onTouchEvent(ev)) return true }
}
return super.dispatchTouchEvent(ev)
}
/** The OS is the source of truth for pointer capture (it releases on focus loss). */
override fun onPointerCaptureChanged(hasCapture: Boolean) {
super.onPointerCaptureChanged(hasCapture)
mouseForwarder?.onCaptureChanged(hasCapture)
}
/** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */ /** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */
private fun isConsoleNavKey(kc: Int): Boolean = when (kc) { private fun isConsoleNavKey(kc: Int): Boolean = when (kc) {
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT,
@@ -1,206 +0,0 @@
package io.unom.punktfunk
import android.view.InputDevice
import android.view.MotionEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.roundToInt
/** True when any connected input device is a pointer (USB/BT mouse, or a touchpad driving one). */
fun hasPhysicalMouse(): Boolean = InputDevice.getDeviceIds().any { id ->
InputDevice.getDevice(id)?.supportsSource(InputDevice.SOURCE_MOUSE) == true
}
/**
* Physical mouse → wire, in two modes (the iPadOS/desktop model):
* * **uncaptured** (default): hover/drag positions forward as absolute cursor moves
* (`MouseMoveAbs`, host-normalized against the window size) — desktop-style pointing. The
* local cursor is hidden over the stream (StreamScreen sets a TYPE_NULL pointer icon); the
* host's own cursor, composited into the video, is the one you see.
* * **captured**: the OS pointer is grabbed ([android.view.View.requestPointerCapture]) and raw
* relative deltas forward as `MouseMove` — FPS mouse-look. Engaged at stream start / by
* clicking into the stream when the "Capture pointer for games" setting is on, and toggled
* any time by Ctrl+Alt+Shift+Q (the cross-client chord). Focus loss releases it (the OS
* guarantees that); a click re-engages.
*
* Buttons ride [MotionEvent.ACTION_BUTTON_PRESS]/RELEASE edges (left/middle/right/back/forward →
* wire 1/2/3/4/5), the wheel rides [MotionEvent.ACTION_SCROLL] with fractional accumulation so
* high-resolution wheels don't lose sub-notch travel. Held buttons are tracked and flushed on
* capture loss / stream exit so nothing sticks on the host. Events reach this class from
* MainActivity's dispatch overrides (uncaptured) and the capture view's captured-pointer listener.
*/
class MouseForwarder(
private val handle: Long,
private val invertScroll: Boolean,
private val captureWanted: Boolean,
private val surfaceSize: () -> Pair<Int, Int>,
) {
/** Capture plumbing, owned by StreamScreen (the focusable capture view). */
var onRequestCapture: (() -> Unit)? = null
var onReleaseCapture: (() -> Unit)? = null
/** Live capture state, updated from [android.app.Activity.onPointerCaptureChanged]. */
var captured = false
private set
/** Chord-released: no auto re-engage (start / click) until the user opts back in. */
private var userReleased = false
private val heldButtons = mutableSetOf<Int>()
private var scrollAccV = 0f
private var scrollAccH = 0f
private var moveAccX = 0f
private var moveAccY = 0f
/** Uncaptured mouse events on the TOUCH stream (position while a button is down). */
fun onTouchEvent(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_DOWN -> {
if (captureWanted && !captured && !userReleased) {
// The engaging click: grab the pointer and swallow the click (desktop
// parity — the click that captures never reaches the host). The paired
// BUTTON_RELEASE is dropped by the held-set guard in [button].
onRequestCapture?.invoke()
return true
}
sendAbs(ev)
}
MotionEvent.ACTION_MOVE -> sendAbs(ev)
// Button edges are documented on the generic stream, but be robust to either.
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
}
return true
}
/** Uncaptured mouse events on the GENERIC stream (hover motion, wheel, button edges). */
fun onGenericMotion(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_HOVER_MOVE -> sendAbs(ev)
MotionEvent.ACTION_SCROLL -> wheel(ev)
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_EXIT -> {}
else -> return false
}
return true
}
/**
* Captured-pointer events (the view holds [android.view.View.requestPointerCapture]): x/y ARE
* the relative deltas ([InputDevice.SOURCE_MOUSE_RELATIVE]), batched samples included. A
* captured touchpad reports absolute finger coordinates instead — not handled (the touch
* gesture layer is the touchpad story); returning false leaves those to the framework.
*/
fun onCapturedPointer(ev: MotionEvent): Boolean {
if (!ev.isFromSource(InputDevice.SOURCE_MOUSE_RELATIVE)) return false
when (ev.actionMasked) {
MotionEvent.ACTION_MOVE -> {
var dx = 0f
var dy = 0f
for (i in 0 until ev.historySize) {
dx += ev.getHistoricalX(i)
dy += ev.getHistoricalY(i)
}
dx += ev.x
dy += ev.y
moveAccX += dx
moveAccY += dy
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept w/ sign
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_SCROLL -> wheel(ev)
}
return true
}
/** Ctrl+Alt+Shift+Q: release the grab, or (re-)engage it — works even when auto-capture is off. */
fun toggleCapture() {
if (captured) {
userReleased = true
onReleaseCapture?.invoke()
} else {
userReleased = false
onRequestCapture?.invoke()
}
}
/** Auto-engage at stream start (setting on + a mouse actually present). */
fun engageFromStart() {
if (captureWanted && !captured && !userReleased && hasPhysicalMouse()) {
onRequestCapture?.invoke()
}
}
/** From [android.app.Activity.onPointerCaptureChanged] — the OS is the source of truth. */
fun onCaptureChanged(has: Boolean) {
captured = has
// Losing the grab (focus loss, chord) must not leave buttons held on the host.
if (!has) flushButtons()
}
/** Stream teardown: lift anything held and let the grab go. */
fun release() {
flushButtons()
if (captured) onReleaseCapture?.invoke()
}
private fun sendAbs(ev: MotionEvent) {
val (w, h) = surfaceSize()
if (w <= 0 || h <= 0) return
NativeBridge.nativeSendPointerAbs(
handle,
ev.x.roundToInt().coerceIn(0, w - 1),
ev.y.roundToInt().coerceIn(0, h - 1),
w,
h,
)
}
private fun wheel(ev: MotionEvent) {
val dir = if (invertScroll) -1f else 1f
// Android: AXIS_VSCROLL + = up/away, AXIS_HSCROLL + = right — the wire's convention too.
scrollAccV += ev.getAxisValue(MotionEvent.AXIS_VSCROLL) * 120f * dir
scrollAccH += ev.getAxisValue(MotionEvent.AXIS_HSCROLL) * 120f * dir
val v = scrollAccV.toInt()
if (v != 0) {
NativeBridge.nativeSendScroll(handle, 0, v)
scrollAccV -= v
}
val h = scrollAccH.toInt()
if (h != 0) {
NativeBridge.nativeSendScroll(handle, 1, h)
scrollAccH -= h
}
}
private fun button(actionButton: Int, down: Boolean) {
val b = when (actionButton) {
MotionEvent.BUTTON_PRIMARY -> 1
MotionEvent.BUTTON_TERTIARY -> 2
MotionEvent.BUTTON_SECONDARY -> 3
MotionEvent.BUTTON_BACK -> 4
MotionEvent.BUTTON_FORWARD -> 5
else -> return
}
if (down) {
heldButtons.add(b)
NativeBridge.nativeSendPointerButton(handle, b, true)
} else if (heldButtons.remove(b)) {
// Only release what we pressed — drops the release of a swallowed engaging click
// and anything that raced a capture transition.
NativeBridge.nativeSendPointerButton(handle, b, false)
}
}
private fun flushButtons() {
heldButtons.forEach { NativeBridge.nativeSendPointerButton(handle, it, false) }
heldButtons.clear()
}
}
@@ -1,193 +0,0 @@
package io.unom.punktfunk
import android.os.Handler
import android.os.Looper
import android.view.Choreographer
import android.view.KeyEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.hypot
// Hold this long on SELECT (pointer-mode toggle) / PLAY-PAUSE (keyboard toggle) for the long-press
// action instead of the tap action.
private const val LONG_PRESS_MS = 800L
// D-pad glide ballistics, in screen-widths per second: start slow enough to hit a close button,
// ramp over RAMP_S seconds of continuous hold so crossing the desktop doesn't take all day.
private const val SPEED_MIN = 0.14f
private const val SPEED_MAX = 0.70f
private const val RAMP_S = 1.2f
/**
* Android TV remote as a pointer — the Android analogue of the Apple client's Siri-remote pointer,
* adapted for D-pad-only remotes (most Android TV remotes have no touch surface). For the
* "TV as a desktop client" use case, where a plain remote is often the only thing in hand.
*
* While streaming on a TV, **hold SELECT ≈ 0.8 s** to toggle pointer mode. While active:
* * D-pad (held) glides the host cursor with ramping acceleration (relative `MouseMove`,
* Choreographer-paced, diagonal-normalized);
* * SELECT tap = left click; PLAY/PAUSE tap = right click (Siri-remote parity);
* * PLAY/PAUSE held = toggle the on-screen keyboard; BACK = leave pointer mode
* (a second BACK then leaves the stream as usual).
* While inactive, everything except the SELECT long-press passes through untouched (D-pad =
* arrow keys, SELECT tap = Enter — synthesized on release, since the down was held back to
* disambiguate the long-press).
*
* Only consulted for non-gamepad key events on TV devices (MainActivity gates the calls); all
* state lives on the main thread.
*/
class RemotePointer(
private val handle: Long,
private val surfaceWidth: () -> Int,
private val onActiveChanged: (Boolean) -> Unit,
private val onKeyboardToggle: () -> Unit,
) {
var active = false
private set
private val handler = Handler(Looper.getMainLooper())
private val held = mutableSetOf<Int>() // D-pad keycodes currently down
private var moveAccX = 0f
private var moveAccY = 0f
private var lastFrameNs = 0L
private var rampSec = 0f
private var tickerRunning = false
private var centerLongFired = false
private var playLongFired = false
private val centerLong = Runnable {
centerLongFired = true
toggle()
}
private val playLong = Runnable {
playLongFired = true
onKeyboardToggle()
}
private val frame = object : Choreographer.FrameCallback {
override fun doFrame(nowNs: Long) {
if (!tickerRunning) return
if (held.isEmpty() || !active) {
tickerRunning = false
return
}
val dt = if (lastFrameNs == 0L) {
1f / 60f
} else {
((nowNs - lastFrameNs) / 1e9f).coerceIn(0.001f, 0.1f)
}
lastFrameNs = nowNs
rampSec += dt
var vx = 0f
var vy = 0f
if (KeyEvent.KEYCODE_DPAD_LEFT in held) vx -= 1f
if (KeyEvent.KEYCODE_DPAD_RIGHT in held) vx += 1f
if (KeyEvent.KEYCODE_DPAD_UP in held) vy -= 1f
if (KeyEvent.KEYCODE_DPAD_DOWN in held) vy += 1f
val mag = hypot(vx, vy)
if (mag > 0f) {
val w = surfaceWidth().coerceAtLeast(640)
val speed = w * (SPEED_MIN + (SPEED_MAX - SPEED_MIN) * (rampSec / RAMP_S).coerceAtMost(1f))
moveAccX += vx / mag * speed * dt
moveAccY += vy / mag * speed * dt
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
Choreographer.getInstance().postFrameCallback(this)
}
}
/** One remote key event; true = consumed. Ignore key repeats — the ticker owns motion. */
fun onKey(event: KeyEvent): Boolean {
val down = event.action == KeyEvent.ACTION_DOWN
when (event.keyCode) {
KeyEvent.KEYCODE_DPAD_CENTER -> {
if (down) {
if (event.repeatCount == 0) {
centerLongFired = false
handler.postDelayed(centerLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(centerLong)
if (!centerLongFired) {
if (active) {
click(1)
} else {
// The down was held back to disambiguate the long-press, so the
// normal path never saw it — synthesize the Enter here instead.
NativeBridge.nativeSendKey(handle, 0x0D, true, 0)
NativeBridge.nativeSendKey(handle, 0x0D, false, 0)
}
}
}
return true
}
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN,
KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_RIGHT,
-> {
if (!active) return false
if (down) {
if (held.add(event.keyCode) && held.size == 1) startTicker()
} else {
held.remove(event.keyCode)
}
return true
}
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE -> {
if (!active) return false // inactive: the media-key VK path owns it
if (down) {
if (event.repeatCount == 0) {
playLongFired = false
handler.postDelayed(playLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(playLong)
if (!playLongFired) click(3)
}
return true
}
KeyEvent.KEYCODE_BACK -> {
if (!active) return false
if (!down) toggle() // leave pointer mode; the next BACK leaves the stream
return true
}
else -> return false
}
}
/** Stream teardown: stop timers/ticker; nothing wire-held to flush (clicks are edges). */
fun release() {
handler.removeCallbacks(centerLong)
handler.removeCallbacks(playLong)
active = false
held.clear()
tickerRunning = false
}
private fun toggle() {
active = !active
if (!active) {
held.clear()
tickerRunning = false
}
onActiveChanged(active)
}
private fun startTicker() {
rampSec = 0f
lastFrameNs = 0L
if (!tickerRunning) {
tickerRunning = true
Choreographer.getInstance().postFrameCallback(frame)
}
}
private fun click(button: Int) {
NativeBridge.nativeSendPointerButton(handle, button, true)
NativeBridge.nativeSendPointerButton(handle, button, false)
}
}
@@ -109,27 +109,6 @@ data class Settings(
* setup where the OS-level pad (lizard mode) is preferred. * setup where the OS-level pad (lizard mode) is preferred.
*/ */
val sc2Capture: Boolean = true, val sc2Capture: Boolean = true,
/**
* Lock a physical mouse to the stream ([android.view.View.requestPointerCapture]) and forward
* raw relative motion — FPS mouse-look, the iPad "Capture pointer for games" twin. Engages at
* stream start and on a click into the stream; Ctrl+Alt+Shift+Q toggles it live (the chord
* works even with this off). Off (default): a mouse points absolutely, desktop-style.
*/
val pointerCapture: Boolean = false,
/**
* Flip scroll direction — the mouse wheel and the two-finger touch scroll both. Parity with
* the Apple/GTK clients' "Invert scroll direction".
*/
val invertScroll: Boolean = false,
/**
* Sync text copied on this device to the host and vice versa while streaming (the desktop
* clients' shared clipboard, text-only here). Only effective when the host advertises the
* clipboard capability; the protocol is opt-in per session either way.
*/
val clipboardSync: Boolean = true,
) )
/** [Settings.touchMode] values; persisted by name. */ /** [Settings.touchMode] values; persisted by name. */
@@ -193,9 +172,6 @@ class SettingsStore(context: Context) {
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true), autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false), rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true), sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
pointerCapture = prefs.getBoolean(K_POINTER_CAPTURE, false),
invertScroll = prefs.getBoolean(K_INVERT_SCROLL, false),
clipboardSync = prefs.getBoolean(K_CLIPBOARD_SYNC, true),
) )
fun save(s: Settings) { fun save(s: Settings) {
@@ -219,9 +195,6 @@ class SettingsStore(context: Context) {
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled) .putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone) .putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture) .putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.putBoolean(K_POINTER_CAPTURE, s.pointerCapture)
.putBoolean(K_INVERT_SCROLL, s.invertScroll)
.putBoolean(K_CLIPBOARD_SYNC, s.clipboardSync)
.apply() .apply()
} }
@@ -260,9 +233,6 @@ class SettingsStore(context: Context) {
const val K_AUTO_WAKE = "auto_wake_enabled" const val K_AUTO_WAKE = "auto_wake_enabled"
const val K_RUMBLE_ON_PHONE = "rumble_on_phone" const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_SC2_CAPTURE = "sc2_capture" const val K_SC2_CAPTURE = "sc2_capture"
const val K_POINTER_CAPTURE = "pointer_capture"
const val K_INVERT_SCROLL = "invert_scroll"
const val K_CLIPBOARD_SYNC = "clipboard_sync"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */ /** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode" const val K_TRACKPAD = "trackpad_mode"
@@ -412,27 +412,6 @@ private fun ControlsSettings(s: Settings, update: (Settings) -> Unit, onOpenCont
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant,
) )
ToggleRow(
title = "Capture pointer for games",
subtitle = "Lock a connected mouse to the stream and send raw relative motion " +
"(mouse-look). Ctrl+Alt+Shift+Q toggles it live; click the stream to re-capture. " +
"Off: the mouse points at the desktop directly",
checked = s.pointerCapture,
onCheckedChange = { on -> update(s.copy(pointerCapture = on)) },
)
ToggleRow(
title = "Invert scroll direction",
subtitle = "Flip the mouse wheel and two-finger touch scrolling",
checked = s.invertScroll,
onCheckedChange = { on -> update(s.copy(invertScroll = on)) },
)
ToggleRow(
title = "Shared clipboard",
subtitle = "Text copied here pastes on the host and vice versa (hosts with " +
"clipboard sharing enabled)",
checked = s.clipboardSync,
onCheckedChange = { on -> update(s.copy(clipboardSync = on)) },
)
} }
SettingsCard { SettingsCard {
SettingDropdown( SettingDropdown(
@@ -13,7 +13,6 @@ import android.net.wifi.WifiManager
import android.os.Build import android.os.Build
import android.text.InputType import android.text.InputType
import android.util.Log import android.util.Log
import android.view.KeyEvent
import android.view.SurfaceHolder import android.view.SurfaceHolder
import android.view.SurfaceView import android.view.SurfaceView
import android.view.View import android.view.View
@@ -50,9 +49,6 @@ import androidx.core.content.ContextCompat
import androidx.core.view.WindowCompat import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat import androidx.core.view.WindowInsetsControllerCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.LifecycleOwner
import io.unom.punktfunk.kit.GamepadFeedback import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.GamepadRouter import io.unom.punktfunk.kit.GamepadRouter
import io.unom.punktfunk.kit.deviceBodyVibrator import io.unom.punktfunk.kit.deviceBodyVibrator
@@ -180,14 +176,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// "hold to quit" hint overlay. Set from the router's onExitArmed (main thread). // "hold to quit" hint overlay. Set from the router's onExitArmed (main thread).
var exitArming by remember { mutableStateOf(false) } var exitArming by remember { mutableStateOf(false) }
// True while the TV remote is acting as a pointer (hold SELECT toggles) — drives the mode hint.
var remotePointerOn by remember { mutableStateOf(false) }
// Focus anchor the soft keyboard is summoned onto AND the pointer-capture grab target (a grab
// needs a focusable view; captured-pointer events land on it). Declared before the effect
// below so the capture callbacks can reach the view once it exists.
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
DisposableEffect(handle) { DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON) window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
wifiLocks.forEach { lock -> wifiLocks.forEach { lock ->
@@ -233,54 +221,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual // Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread. // exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed } router.onExitArmed = { armed -> exitArming = armed }
// Physical mouse: uncaptured hover/click/wheel forwards as absolute pointing; captured
// (setting or the Ctrl+Alt+Shift+Q chord) raw deltas forward as relative mouse-look.
// The local cursor is hidden over the stream — the host's own cursor, composited into
// the video, is the one the user sees (twin of the desktop clients' hidden cursor).
val decor = window?.decorView
val priorPointerIcon = decor?.pointerIcon
decor?.pointerIcon = android.view.PointerIcon.getSystemIcon(
context,
android.view.PointerIcon.TYPE_NULL,
)
val mouse = MouseForwarder(
handle,
invertScroll = initialSettings.invertScroll,
captureWanted = initialSettings.pointerCapture,
surfaceSize = { (decor?.width ?: 0) to (decor?.height ?: 0) },
)
mouse.onRequestCapture = {
// The grab needs the (focusable) capture view: focus it, then ask. Posted so a
// request racing view attach/focus settles on the next frame.
keyCapture?.let { v ->
v.post {
v.requestFocus()
v.requestPointerCapture()
}
}
}
mouse.onReleaseCapture = { keyCapture?.releasePointerCapture() }
activity?.mouseForwarder = mouse
// TV remote-as-pointer: hold SELECT ≈ 0.8 s to toggle; the D-pad then glides the host
// cursor (see RemotePointer). TV only — a phone's remote-less keys stay on the VK path.
val remote = if (isTv) {
RemotePointer(
handle,
surfaceWidth = { decor?.width ?: 1920 },
onActiveChanged = { on -> remotePointerOn = on },
onKeyboardToggle = { keyCapture?.let { it.setImeVisible(!it.imeShown) } },
)
} else {
null
}
activity?.remotePointer = remote
// Shared clipboard (text v1): only when the user setting is on AND the host has a
// working clipboard service. Protocol-level opt-in + the poll thread live in the sync.
val clip = if (initialSettings.clipboardSync && NativeBridge.nativeClipSupported(handle)) {
ClipboardSync(context, handle).also { it.start() }
} else {
null
}
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad // Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad
// index via the router; poll threads stopped + joined before the router is released and the // index via the router; poll threads stopped + joined before the router is released and the
@@ -346,7 +286,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
} }
onDispose { onDispose {
closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it
clip?.stop() // stop + join the clipboard poll thread BEFORE the handle is freed
feedback.onHidRaw = null feedback.onHidRaw = null
feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed
sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } } sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
@@ -354,12 +293,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
mouse.release()
activity?.mouseForwarder = null
remote?.release()
activity?.remotePointer = null
decor?.pointerIcon = priorPointerIcon
activity?.streamHandle = 0L activity?.streamHandle = 0L
activity?.requestStreamExit = null activity?.requestStreamExit = null
// Back in the menus: the SC2 (if present) resumes driving the console UI. // Back in the menus: the SC2 (if present) resumes driving the console UI.
@@ -387,32 +320,8 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger). // Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() } BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Leaving the app (Home, task switch, screen off) MUST end the session. Android does not // Focus anchor the three-finger keyboard swipe summons the IME onto (see KeyCaptureView).
// suspend a process for going to background, so without this the native worker kept running and var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
// its QUIC connection kept answering the host's keep-alives — the user was long gone but the
// host still saw a live client and held the session (and its display + encoder) open until the
// OS eventually reclaimed the process, which on a TV box is effectively never.
//
// Route it through `onDisconnect()` so the composable's `onDispose` above runs the one real
// teardown path. Deliberately NOT a `nativeDisconnectQuit`: backgrounding isn't a user "quit",
// so the host should linger the display and make coming straight back a fast reconnect.
DisposableEffect(handle) {
val lifecycle = (context as? LifecycleOwner)?.lifecycle
val obs = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_STOP) {
onDisconnect()
}
}
lifecycle?.addObserver(obs)
onDispose { lifecycle?.removeObserver(obs) }
}
// Auto-engage pointer capture at stream start (setting on + a mouse actually present).
// Delayed a beat: the grab needs window focus and the capture view attached.
LaunchedEffect(handle) {
delay(400)
activity?.mouseForwarder?.engageFromStart()
}
Box(modifier = Modifier.fillMaxSize()) { Box(modifier = Modifier.fillMaxSize()) {
AndroidView( AndroidView(
@@ -470,49 +379,23 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (exitArming) { if (exitArming) {
ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp)) ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
} }
// Remote-pointer mode hint — the remote's keys are remapped while it's on, so say so. // Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe
if (remotePointerOn) { // up in the mouse modes) — it never draws or takes touches, it just owns IME focus.
RemotePointerHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe up
// in the mouse modes) AND the pointer-capture grab target — it never draws or takes
// touches, it just owns IME focus and receives captured-pointer events.
AndroidView( AndroidView(
modifier = Modifier.size(1.dp), modifier = Modifier.size(1.dp),
factory = { ctx -> factory = { ctx -> KeyCaptureView(ctx).also { keyCapture = it } },
KeyCaptureView(ctx).also { v ->
keyCapture = v
// Real IME text path when the host types committed text (see KeyCaptureView).
v.textHandle =
if (NativeBridge.nativeTextInputSupported(handle)) handle else 0L
v.setOnCapturedPointerListener { _, ev ->
(ctx as? MainActivity)?.mouseForwarder?.onCapturedPointer(ev) ?: false
}
}
},
) )
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture // Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no // vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
// keyboard gesture: its fingers belong to the host verbatim (a swipe there may BE a // keyboard gesture: its fingers belong to the host verbatim (a swipe there may BE a
// host-OS gesture), so intercepting three fingers would corrupt real multi-touch. // host-OS gesture), so intercepting three fingers would corrupt real multi-touch.
// Stylus lane (design/pen-tablet-input.md §7): against a HOST_CAP_PEN host a stylus
// splits out of BOTH touch models onto the pen plane; its heartbeat coroutine keeps a
// stationary held stroke alive (and its cancellation lifts everything on teardown).
val stylus = remember(handle) {
if (NativeBridge.nativeHostSupportsPen(handle)) StylusStream(handle) else null
}
if (stylus != null) {
LaunchedEffect(stylus) { stylus.heartbeatLoop() }
}
Box( Box(
Modifier.fillMaxSize().pointerInput(handle, touchMode) { Modifier.fillMaxSize().pointerInput(handle, touchMode) {
when (touchMode) { when (touchMode) {
TouchMode.TOUCH -> streamTouchPassthrough(handle, stylus) TouchMode.TOUCH -> streamTouchPassthrough(handle)
else -> streamTouchInput( else -> streamTouchInput(
handle, handle,
stylus,
trackpad = touchMode == TouchMode.TRACKPAD, trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() }, onCycleStats = { statsVerbosity = statsVerbosity.next() },
onKeyboard = { show -> keyCapture?.setImeVisible(show) }, onKeyboard = { show -> keyCapture?.setImeVisible(show) },
) )
@@ -540,35 +423,14 @@ private fun ExitChordHint(modifier: Modifier = Modifier) {
) )
} }
/**
* The remote-pointer mode cue: while active the remote's keys are remapped (D-pad glides the host
* cursor, SELECT clicks), so the overlay both confirms the toggle and teaches the vocabulary.
*/
@Composable
private fun RemotePointerHint(modifier: Modifier = Modifier) {
Text(
"Remote pointer — SELECT click · play/pause right-click · hold SELECT to exit",
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/** /**
* Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME * Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME
* onto this view. Two IME models, picked by the host's capabilities: * onto this view. `TYPE_NULL` puts the IME in "dumb keyboard" mode — it delivers raw [KeyEvent]s
* * **Text path** ([textHandle] set — the host advertised `HOST_CAP_TEXT_INPUT`): a real * (no composing text, no autocorrect), which flow through `MainActivity.dispatchKeyEvent` →
* editable [HostTextConnection], so the IME gives autocorrect, gesture typing, non-Latin * `Keymap.toVk` → the host, the exact path a hardware keyboard takes. Text an IME insists on
* composition and emoji, all mirrored to the host as committed text + diffs. * committing instead still arrives: the non-editable [BaseInputConnection] synthesizes KeyEvents
* * **Fallback** (older host): `TYPE_NULL` puts the IME in "dumb keyboard" mode — raw * for it via `KeyCharacterMap` (with Shift carried as meta state — see the IME-shift wrap in
* [KeyEvent]s flow through `MainActivity.dispatchKeyEvent` → `Keymap.toVk` → the host, the * `MainActivity.dispatchKeyEvent`).
* exact path a hardware keyboard takes (with the IME-shift wrap documented there).
*
* Doubles as the pointer-capture grab target: a grab needs a focusable view, and captured-pointer
* events are delivered to it (routed to [MouseForwarder.onCapturedPointer] via the listener the
* stream screen installs).
*/ */
private class KeyCaptureView(context: Context) : View(context) { private class KeyCaptureView(context: Context) : View(context) {
init { init {
@@ -576,171 +438,22 @@ private class KeyCaptureView(context: Context) : View(context) {
isFocusableInTouchMode = true isFocusableInTouchMode = true
} }
/** The session handle when the host types committed text; `0` = VK-only fallback. */ override fun onCheckIsTextEditor(): Boolean = true
var textHandle: Long = 0L
/** Whether [setImeVisible] last showed the IME — for toggle-style callers (remote pointer). */ override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
var imeShown = false
private set
override fun onCheckIsTextEditor(): Boolean = imeShown
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection? {
// Only an editor while the user has SUMMONED the keyboard (gesture / remote toggle).
// This view holds focus for the whole stream (it's the capture anchor), and with an
// always-live editable connection the IME counts input as active on it — TV IMEs then
// pop their UI the moment a PHYSICAL keyboard key arrives. With no connection, hardware
// typing stays on the raw dispatchKeyEvent → Keymap → wire path and no keyboard appears.
if (!imeShown) return null
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return if (textHandle != 0L) {
outAttrs.inputType = InputType.TYPE_CLASS_TEXT or
InputType.TYPE_TEXT_FLAG_AUTO_CORRECT or InputType.TYPE_TEXT_FLAG_MULTI_LINE
HostTextConnection(this, textHandle)
} else {
outAttrs.inputType = InputType.TYPE_NULL outAttrs.inputType = InputType.TYPE_NULL
BaseInputConnection(this, false) outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN
} return BaseInputConnection(this, false)
} }
fun setImeVisible(show: Boolean) { fun setImeVisible(show: Boolean) {
val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager
?: return ?: return
imeShown = show
if (show) { if (show) {
requestFocus() requestFocus()
// The view may already be focused from a null-connection state — restart so the
// framework re-queries onCreateInputConnection with the gate now open.
imm.restartInput(this)
imm.showSoftInput(this, 0) imm.showSoftInput(this, 0)
} else { } else {
imm.hideSoftInputFromWindow(windowToken, 0) imm.hideSoftInputFromWindow(windowToken, 0)
imm.restartInput(this) // gate closed — drop the editable connection
} }
} }
/**
* BACK while the summoned keyboard is up: the IME consumes it pre-IME to dismiss itself, so
* [setImeVisible] never hears about it — sync the gate here or a stale `imeShown` leaves the
* editable connection live and physical typing re-pops the keyboard.
*/
override fun onKeyPreIme(keyCode: Int, event: KeyEvent): Boolean {
if (keyCode == KeyEvent.KEYCODE_BACK && imeShown && event.action == KeyEvent.ACTION_UP) {
imeShown = false
(context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager)
?.restartInput(this)
}
return super.onKeyPreIme(keyCode, event)
}
}
/**
* IME → host text bridge (the `HOST_CAP_TEXT_INPUT` path): a real **editable** connection, so
* the IME runs its full machinery (autocorrect, gesture typing, non-Latin composition), mirrored
* to the host as it happens. The one piece of host-side state tracked is *what the host currently
* shows of the active composition* ([sentComposition]): composing updates send a common-prefix
* diff (backspaces + the new suffix) so corrections materialize live on the host; a commit
* settles it. [setComposingRegion] adopts already-committed text as the active composition
* (autocorrect-revert / backspace-into-word flows), so the next update diffs against it instead
* of retyping. Newlines become Enter taps; [deleteSurroundingText] becomes Backspace/Delete taps.
*
* Known approximation: diff lengths are counted in Unicode scalars, assuming one host Backspace
* deletes one scalar — true for the composition text IMEs actually produce (emoji and other
* multi-unit graphemes commit directly rather than composing).
*/
private class HostTextConnection(
view: KeyCaptureView,
private val handle: Long,
) : BaseInputConnection(view, true) {
/** What the host currently shows of the active composition ("" = none). */
private var sentComposition = ""
override fun commitText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
sentComposition = ""
val ok = super.commitText(text, newCursorPosition)
trimEditable()
return ok
}
override fun setComposingText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
return super.setComposingText(text, newCursorPosition)
}
override fun finishComposingText(): Boolean {
// The composition text stands as committed — the host already shows it verbatim.
sentComposition = ""
return super.finishComposingText()
}
override fun setComposingRegion(start: Int, end: Int): Boolean {
val e = editable
if (e != null) {
val a = start.coerceIn(0, e.length)
val b = end.coerceIn(0, e.length)
sentComposition = e.subSequence(minOf(a, b), maxOf(a, b)).toString()
}
return super.setComposingRegion(start, end)
}
override fun deleteSurroundingText(beforeLength: Int, afterLength: Int): Boolean {
repeat(beforeLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_BACK) }
repeat(afterLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_DELETE) }
return super.deleteSurroundingText(beforeLength, afterLength)
}
override fun performEditorAction(actionCode: Int): Boolean {
tapVk(VK_RETURN)
return true
}
/** Replace the host's view of the composition with [text] via a common-prefix diff. */
private fun retype(text: String) {
var common = sentComposition.commonPrefixWith(text)
// Never split a surrogate pair mid-diff — back off to the pair boundary.
if (common.isNotEmpty() && common.last().isHighSurrogate()) {
common = common.dropLast(1)
}
val stale = sentComposition.substring(common.length)
repeat(stale.codePointCount(0, stale.length).coerceAtMost(MAX_TAPS)) { tapVk(VK_BACK) }
sendText(text.substring(common.length))
sentComposition = text
}
/** Forward literal text, turning newlines into Enter taps (control chars never ride text). */
private fun sendText(s: String) {
var chunk = StringBuilder()
for (ch in s) {
if (ch == '\n') {
if (chunk.isNotEmpty()) {
NativeBridge.nativeSendText(handle, chunk.toString())
chunk = StringBuilder()
}
tapVk(VK_RETURN)
} else {
chunk.append(ch)
}
}
if (chunk.isNotEmpty()) NativeBridge.nativeSendText(handle, chunk.toString())
}
private fun tapVk(vk: Int) {
NativeBridge.nativeSendKey(handle, vk, true, 0)
NativeBridge.nativeSendKey(handle, vk, false, 0)
}
/** Bound the mirror buffer: once nothing is composing, old text serves no purpose. */
private fun trimEditable() {
val e = editable ?: return
if (getComposingSpanStart(e) == -1 && e.length > 4000) e.clear()
}
private companion object {
const val VK_BACK = 0x08
const val VK_RETURN = 0x0D
const val VK_DELETE = 0x2E
const val MAX_TAPS = 256
}
} }
@@ -1,195 +0,0 @@
package io.unom.punktfunk
import android.view.MotionEvent
import androidx.compose.ui.ExperimentalComposeUiApi
import androidx.compose.ui.input.pointer.PointerEvent
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.unit.IntSize
import io.unom.punktfunk.kit.NativeBridge
import kotlinx.coroutines.delay
// Wire PEN_* state bits (punktfunk_core::quic::pen; mirrored, asserted by the Rust shim's docs).
private const val PEN_IN_RANGE = 1f
private const val PEN_TOUCHING = 2f
private const val PEN_BARREL1 = 4f
private const val PEN_BARREL2 = 8f
private const val STRIDE = 10
private const val MAX_SAMPLES = 8
/**
* Android stylus → the state-full pen plane (design/pen-tablet-input.md §7): pressure, tilt
* (`AXIS_TILT`, radians from the surface normal), azimuth (`AXIS_ORIENTATION` — Android's 0 =
* "pointed away from the user" IS the wire's north, no offset needed), hover with
* `AXIS_DISTANCE`, the eraser tool, both stylus barrel buttons, and historical (coalesced)
* samples batched oldest-first for full capture-rate fidelity. Android has no barrel-roll
* axis — roll stays unknown on this client.
*
* Both touch loops call [intercept] first; stylus/eraser pointers are consumed here (against a
* pen-capable host) and never reach the finger paths, independent of the touch-input mode.
* [heartbeatLoop] implements the ≤100 ms keepalive wire contract: a stationary held stylus is
* silent in Android's input pipeline, and the host force-releases a stroke after 200 ms
* without samples.
*/
internal class StylusStream(private val handle: Long) {
private var inRange = false
private var touching = false
private var sawHover = false
private val last = FloatArray(STRIDE)
private val batch = FloatArray(MAX_SAMPLES * STRIDE)
init {
idle(last)
}
/**
* Consume the event's stylus pointers into pen samples. Returns true when this event
* carried any (the caller's finger/gesture handling must then skip those changes).
*/
@OptIn(ExperimentalComposeUiApi::class)
fun intercept(ev: PointerEvent, size: IntSize): Boolean {
val stylusChanges = ev.changes.filter {
it.type == PointerType.Stylus || it.type == PointerType.Eraser
}
if (stylusChanges.isEmpty()) return false
stylusChanges.forEach { it.consume() }
val me = ev.motionEvent ?: return true
if (size.width <= 0 || size.height <= 0) return true
// At most one stylus exists — find its pointer index by tool type.
val idx = (0 until me.pointerCount).firstOrNull {
me.getToolType(it) == MotionEvent.TOOL_TYPE_STYLUS ||
me.getToolType(it) == MotionEvent.TOOL_TYPE_ERASER
} ?: return true
when (me.actionMasked) {
MotionEvent.ACTION_DOWN, MotionEvent.ACTION_POINTER_DOWN,
MotionEvent.ACTION_MOVE,
-> {
touching = true
inRange = true
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_MOVE -> {
sawHover = true
inRange = true
touching = false
emitSamples(me, idx, size)
}
MotionEvent.ACTION_UP, MotionEvent.ACTION_POINTER_UP -> {
touching = false
// Hover-capable hardware keeps proximity (HOVER_EXIT owns the leave);
// anything else leaves range on lift — the host never parks a phantom pen.
inRange = sawHover
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_EXIT, MotionEvent.ACTION_CANCEL -> release()
else -> {}
}
return true
}
/** Session/composition teardown: leave range so the host lifts anything still inked. */
fun reset() {
if (inRange || touching) release()
sawHover = false
}
/** The ≤100 ms keepalive (80 ms leaves headroom for one lost datagram). Runs until
* cancelled; resends the last state-full sample while the pen is in range. */
suspend fun heartbeatLoop() {
try {
while (true) {
delay(80)
if (inRange || touching) {
last[9] = 0f // dt
NativeBridge.nativeSendPen(handle, last, 1)
}
}
} finally {
reset()
}
}
private fun release() {
touching = false
inRange = false
last[0] = 0f // state: out of range
last[4] = 0f // pressure
NativeBridge.nativeSendPen(handle, last, 1)
}
/** Historical (coalesced) samples oldest-first, then the current one — a single batch. */
private fun emitSamples(me: MotionEvent, idx: Int, size: IntSize) {
val history = minOf(me.historySize, MAX_SAMPLES - 1)
var count = 0
var prevT = if (history > 0) me.getHistoricalEventTime(0) else me.eventTime
for (h in (me.historySize - history) until me.historySize) {
val t = me.getHistoricalEventTime(h)
fill(
batch, count * STRIDE, size,
x = me.getHistoricalX(idx, h), y = me.getHistoricalY(idx, h),
pressure = me.getHistoricalPressure(idx, h),
tiltRad = me.getHistoricalAxisValue(MotionEvent.AXIS_TILT, idx, h),
orientRad = me.getHistoricalAxisValue(MotionEvent.AXIS_ORIENTATION, idx, h),
distance = me.getHistoricalAxisValue(MotionEvent.AXIS_DISTANCE, idx, h),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((t - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
prevT = t
count++
}
fill(
batch, count * STRIDE, size,
x = me.getX(idx), y = me.getY(idx), pressure = me.getPressure(idx),
tiltRad = me.getAxisValue(MotionEvent.AXIS_TILT, idx),
orientRad = me.getAxisValue(MotionEvent.AXIS_ORIENTATION, idx),
distance = me.getAxisValue(MotionEvent.AXIS_DISTANCE, idx),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((me.eventTime - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
count++
batch.copyInto(last, 0, (count - 1) * STRIDE, count * STRIDE)
NativeBridge.nativeSendPen(handle, batch, count)
}
private fun fill(
out: FloatArray,
off: Int,
size: IntSize,
x: Float,
y: Float,
pressure: Float,
tiltRad: Float,
orientRad: Float,
distance: Float,
buttons: Int,
tool: Int,
dtUs: Float,
) {
var state = 0f
if (inRange || touching) state += PEN_IN_RANGE
if (touching) state += PEN_TOUCHING
if (buttons and MotionEvent.BUTTON_STYLUS_PRIMARY != 0) state += PEN_BARREL1
if (buttons and MotionEvent.BUTTON_STYLUS_SECONDARY != 0) state += PEN_BARREL2
out[off + 0] = state
out[off + 1] = if (tool == MotionEvent.TOOL_TYPE_ERASER) 1f else 0f
out[off + 2] = (x / (size.width - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 3] = (y / (size.height - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 4] = if (touching) pressure.coerceIn(0f, 1f) else 0f
// AXIS_DISTANCE units are device-arbitrary; 0..1 covers real hardware, and 0 while
// hovering legitimately means "at the hover floor".
out[off + 5] = if (touching) 0f else distance.coerceIn(0f, 1f)
out[off + 6] = Math.toDegrees(tiltRad.toDouble()).toFloat().coerceIn(0f, 90f)
// AXIS_ORIENTATION: 0 = pointed away from the user (= wire north), clockwise, −π..π.
out[off + 7] = ((Math.toDegrees(orientRad.toDouble()) + 360.0) % 360.0).toFloat()
out[off + 8] = -1f // no barrel-roll axis on Android
out[off + 9] = dtUs
}
private fun idle(out: FloatArray) {
out.fill(0f)
out[5] = -1f // distance unknown
out[6] = -1f // tilt unknown
out[7] = -1f // azimuth unknown
out[8] = -1f // roll unknown
}
}
@@ -1,11 +1,9 @@
package io.unom.punktfunk package io.unom.punktfunk
import androidx.compose.foundation.gestures.awaitEachGesture import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.ui.input.pointer.AwaitPointerEventScope import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.ui.input.pointer.PointerId import androidx.compose.ui.input.pointer.PointerId
import androidx.compose.ui.input.pointer.PointerInputChange
import androidx.compose.ui.input.pointer.PointerInputScope import androidx.compose.ui.input.pointer.PointerInputScope
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.input.pointer.changedToDownIgnoreConsumed import androidx.compose.ui.input.pointer.changedToDownIgnoreConsumed
import androidx.compose.ui.input.pointer.changedToUpIgnoreConsumed import androidx.compose.ui.input.pointer.changedToUpIgnoreConsumed
import androidx.compose.ui.input.pointer.positionChanged import androidx.compose.ui.input.pointer.positionChanged
@@ -58,26 +56,7 @@ private const val ACCEL_MAX = 3.0f
* normalizes and maps into the output). On teardown (stream leaves composition) every still-held * normalizes and maps into the output). On teardown (stream leaves composition) every still-held
* contact is lifted so nothing stays stuck on the host. * contact is lifted so nothing stays stuck on the host.
*/ */
/** Whether this change belongs to the stylus lane (only when a pen-capable host is live). */ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
private fun isStylus(c: PointerInputChange, stylus: StylusStream?): Boolean =
stylus != null && (c.type == PointerType.Stylus || c.type == PointerType.Eraser)
/** [awaitFirstDown] with the stylus lane split out: pen events feed [stylus] and never start a
* mouse/touch gesture. Toward a pen-less host ([stylus] == null) a stylus stays a finger. */
private suspend fun AwaitPointerEventScope.awaitFirstFingerDown(
stylus: StylusStream?,
): PointerInputChange {
while (true) {
val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val down = ev.changes.firstOrNull {
it.changedToDownIgnoreConsumed() && !isStylus(it, stylus)
}
if (down != null) return down
}
}
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, stylus: StylusStream?) {
val ids = mutableMapOf<PointerId, Int>() val ids = mutableMapOf<PointerId, Int>()
fun alloc(p: PointerId): Int { fun alloc(p: PointerId): Int {
var id = 0 var id = 0
@@ -89,12 +68,10 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, styl
awaitPointerEventScope { awaitPointerEventScope {
while (true) { while (true) {
val ev = awaitPointerEvent() val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val sw = size.width val sw = size.width
val sh = size.height val sh = size.height
if (sw <= 0 || sh <= 0) continue if (sw <= 0 || sh <= 0) continue
for (c in ev.changes) { for (c in ev.changes) {
if (isStylus(c, stylus)) continue // the pen plane owns it
val x = c.position.x.roundToInt().coerceIn(0, sw - 1) val x = c.position.x.roundToInt().coerceIn(0, sw - 1)
val y = c.position.y.roundToInt().coerceIn(0, sh - 1) val y = c.position.y.roundToInt().coerceIn(0, sh - 1)
when { when {
@@ -121,13 +98,10 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, styl
internal suspend fun PointerInputScope.streamTouchInput( internal suspend fun PointerInputScope.streamTouchInput(
handle: Long, handle: Long,
stylus: StylusStream?,
trackpad: Boolean, trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit, onCycleStats: () -> Unit,
onKeyboard: (show: Boolean) -> Unit, onKeyboard: (show: Boolean) -> Unit,
) { ) {
val scrollDir = if (invertScroll) -1 else 1
var lastTapUp = 0L var lastTapUp = 0L
var lastTapX = 0f var lastTapX = 0f
var lastTapY = 0f var lastTapY = 0f
@@ -144,7 +118,7 @@ internal suspend fun PointerInputScope.streamTouchInput(
) )
} }
awaitEachGesture { awaitEachGesture {
val down = awaitFirstFingerDown(stylus) val down = awaitFirstDown(requireUnconsumed = false)
val startX = down.position.x val startX = down.position.x
val startY = down.position.y val startY = down.position.y
// A touch landing just after a quick tap nearby = tap-and-drag: hold the left // A touch landing just after a quick tap nearby = tap-and-drag: hold the left
@@ -181,8 +155,7 @@ internal suspend fun PointerInputScope.streamTouchInput(
while (true) { while (true) {
val ev = awaitPointerEvent() val ev = awaitPointerEvent()
stylus?.intercept(ev, size) val pressed = ev.changes.filter { it.pressed }
val pressed = ev.changes.filter { it.pressed && !isStylus(it, stylus) }
if (pressed.isEmpty()) { if (pressed.isEmpty()) {
upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime
break break
@@ -211,12 +184,12 @@ internal suspend fun PointerInputScope.streamTouchInput(
val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up
val sx = ((cx - prevCx) / SCROLL_DIV).toInt() val sx = ((cx - prevCx) / SCROLL_DIV).toInt()
if (sy != 0) { if (sy != 0) {
NativeBridge.nativeSendScroll(handle, 0, sy * 120 * scrollDir) NativeBridge.nativeSendScroll(handle, 0, sy * 120)
prevCy = cy prevCy = cy
moved = true moved = true
} }
if (sx != 0) { if (sx != 0) {
NativeBridge.nativeSendScroll(handle, 1, sx * 120 * scrollDir) NativeBridge.nativeSendScroll(handle, 1, sx * 120)
prevCx = cx prevCx = cx
moved = true moved = true
} }
@@ -106,17 +106,6 @@ object Keymap {
KeyEvent.KEYCODE_DPAD_UP -> 0x26 KeyEvent.KEYCODE_DPAD_UP -> 0x26
KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27 KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27
KeyEvent.KEYCODE_DPAD_DOWN -> 0x28 KeyEvent.KEYCODE_DPAD_DOWN -> 0x28
// TV-remote SELECT = Enter (a gamepad's press routes via SOURCE_GAMEPAD before this).
KeyEvent.KEYCODE_DPAD_CENTER -> 0x0D
// Consumer/media keys — forwarded to the host while streaming (volume stays local:
// MainActivity's pass-through list wins before the map is consulted).
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE,
KeyEvent.KEYCODE_MEDIA_PLAY,
KeyEvent.KEYCODE_MEDIA_PAUSE -> 0xB3 // VK_MEDIA_PLAY_PAUSE
KeyEvent.KEYCODE_MEDIA_NEXT -> 0xB0 // VK_MEDIA_NEXT_TRACK
KeyEvent.KEYCODE_MEDIA_PREVIOUS -> 0xB1 // VK_MEDIA_PREV_TRACK
KeyEvent.KEYCODE_MEDIA_STOP -> 0xB2 // VK_MEDIA_STOP
// Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant) // Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant)
KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0 KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0
@@ -287,66 +287,6 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */ /** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int) external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised full-fidelity stylus injection (`HOST_CAP_PEN`) — the gate
* for splitting stylus pointers out of the touch path onto the pen plane. False on `0`.
*/
external fun nativeHostSupportsPen(handle: Long): Boolean
/**
* One stylus batch of STATE-FULL samples (the pen plane; design/pen-tablet-input.md §7):
* [count] × 10 floats, oldest first — `[state, tool, x, y, pressure, distance, tilt_deg,
* azimuth_deg, roll_deg, dt_us]`. `state` = the wire in-range/touching/barrel bits; `tool`
* 0=pen 1=eraser; x/y/pressure/distance normalized 0..1; distance/tilt/azimuth/roll < 0 =
* unknown. Send only when [nativeHostSupportsPen]; repeat the last sample ≤100 ms while the
* pen is in range (the host force-releases a silent stroke after 200 ms).
*/
external fun nativeSendPen(handle: Long, samples: FloatArray, count: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) — its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
* gesture typing, non-Latin scripts) over the TYPE_NULL raw-key fallback. False on `0`.
*/
external fun nativeTextInputSupported(handle: Long): Boolean
/**
* Committed IME text → one `TextInput` wire event per Unicode scalar, in order. Control
* characters are skipped natively (Enter/Backspace ride [nativeSendKey]). Only meaningful
* when [nativeTextInputSupported] returned true — older hosts ignore the events.
*/
external fun nativeSendText(handle: Long, text: String)
// ---- Shared clipboard (text v1): Kotlin drives ClipboardManager, Rust the protocol ----
// Opt-in per session (nativeClipControl). Local copies are announced as lazy offers; bytes
// cross only when the host pastes (a "fetch:" event answered by nativeClipServeText). Host
// copies arrive as "offer:" events, fetched eagerly into the system clipboard.
/** Whether the host advertised a working shared-clipboard service (HOST_CAP_CLIPBOARD). */
external fun nativeClipSupported(handle: Long): Boolean
/** Session-level clipboard opt-in/out; nothing happens until enabled=true crosses. */
external fun nativeClipControl(handle: Long, enabled: Boolean)
/** Announce "this device's clipboard now holds text". [seq]: monotonic, newest wins. */
external fun nativeClipOfferText(handle: Long, seq: Int)
/** Pull the text of the host's offer [seq] → transfer id echoed on "data:"/"error:", or -1. */
external fun nativeClipFetchText(handle: Long, seq: Int): Int
/** Answer a "fetch:" event with the clipboard's current text (the host is pasting). */
external fun nativeClipServeText(handle: Long, reqId: Int, text: String)
/** Abort a clipboard transfer by id (either direction). */
external fun nativeClipCancel(handle: Long, id: Int)
/**
* Block ≤250 ms for the next clipboard event, as a compact string: `state:<0|1>` ·
* `offer:<seq>:<hasText>` · `fetch:<reqId>` · `data:<xferId>:<text>` · `cancel:<id>` ·
* `error:<id>:<code>` · `closed` (session gone) — null on timeout. Dedicated poll thread.
*/
external fun nativeNextClip(handle: Long): String?
// ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ---- // ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ----
// The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0, // The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0,
// so its wire is byte-identical to the old single-pad path. The core folds the per-transition // so its wire is byte-identical to the old single-pad path. The core folds the per-transition
@@ -404,14 +404,7 @@ fn feeder_loop(
// stage is consumed: the HUD, or the ABR decode signal (`measure_decode`). The // stage is consumed: the HUD, or the ABR decode signal (`measure_decode`). The
// HUD-only `received` point + host/network split stay gated on the overlay. // HUD-only `received` point + host/network split stay gated on the overlay.
if stats.enabled() || measure_decode { if stats.enabled() || measure_decode {
// Core reassembly-completion stamp (ABI v9), NOT the pull instant: stamping let received_ns = now_realtime_ns();
// here would fold the hand-off queue wait into the network latency figure
// (a client-side standing backlog masquerading as network). 0 = older core.
let received_ns = if frame.received_ns > 0 {
frame.received_ns as i128
} else {
now_realtime_ns()
};
{ {
let mut g = in_flight let mut g = in_flight
.lock() .lock()
@@ -221,13 +221,7 @@ pub(super) fn run_sync(
// samplers (`received` point, host/network split) stay gated on the overlay so // samplers (`received` point, host/network split) stay gated on the overlay so
// the hidden steady state adds only a wall-clock read + the receipt push. // the hidden steady state adds only a wall-clock read + the receipt push.
if stats.enabled() || measure_decode { if stats.enabled() || measure_decode {
// Core reassembly-completion stamp (ABI v9), not the pull instant — see let received_ns = now_realtime_ns();
// async_loop: a pull stamp folds hand-off queue wait into "network".
let received_ns = if frame.received_ns > 0 {
frame.received_ns as i128
} else {
now_realtime_ns()
};
in_flight.push_back((frame.pts_ns / 1000, received_ns)); in_flight.push_back((frame.pts_ns / 1000, received_ns));
if in_flight.len() > IN_FLIGHT_CAP { if in_flight.len() > IN_FLIGHT_CAP {
in_flight.pop_front(); // stale — codec never echoed it back in_flight.pop_front(); // stale — codec never echoed it back
@@ -1,182 +0,0 @@
//! Shared-clipboard plane (text-only v1): Kotlin drives the Android `ClipboardManager`, these
//! shims drive [`punktfunk_core::client::NativeClient`]'s clipboard surface.
//!
//! Model (mirrors the desktop clients): opt-in via `nativeClipControl(true)`; local copies are
//! announced lazily as format-list offers (`nativeClipOfferText`) and the bytes cross only when
//! the host pastes (a `fetch` event answered by `nativeClipServeText`); a host copy arrives as an
//! `offer` event, which the Kotlin side fetches eagerly (Android's clipboard has no lazy provider
//! path worth the complexity) and lands in the system clipboard on the `data` event.
//!
//! Events cross to Kotlin as compact strings from the blocking `nativeNextClip` poll (drained on
//! a dedicated thread, same pattern as `nativeNextRumble`):
//! `state:<0|1>` · `offer:<seq>:<has_text 0|1>` · `fetch:<req_id>` · `data:<xfer_id>:<text>` ·
//! `cancel:<id>` · `error:<id>:<code>` · `closed` — null on a poll timeout. Non-text fetch
//! requests are cancelled natively (only text is ever offered, so they shouldn't occur).
use std::time::Duration;
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint, jlong, jstring};
use jni::JNIEnv;
use punktfunk_core::clipboard::ClipEventCore;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::quic::{ClipKind, CLIP_FILE_INDEX_NONE, HOST_CAP_CLIPBOARD};
use super::SessionHandle;
/// The portable wire MIME both ends map to their platform text type.
const TEXT_MIME: &str = "text/plain;charset=utf-8";
/// Deref the opaque handle (`0` → `None`).
///
/// SAFETY: live handle per the nativeConnect/nativeClose contract; every method used is `&self`
/// on the `Sync` connector.
fn client(handle: jlong) -> Option<&'static SessionHandle> {
if handle == 0 {
return None;
}
// SAFETY: see the function docs — the Kotlin side guarantees the handle outlives the call.
Some(unsafe { &*(handle as *const SessionHandle) })
}
/// `NativeBridge.nativeClipSupported(handle)` — the host advertised `HOST_CAP_CLIPBOARD`.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
client(handle).map_or(0, |h| {
u8::from(h.client.host_caps() & HOST_CAP_CLIPBOARD != 0)
})
}
/// `NativeBridge.nativeClipControl(handle, enabled)` — session-level opt-in/out. Nothing
/// clipboard-related happens on either side until an `enabled: true` crosses.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipControl(
_env: JNIEnv,
_this: JObject,
handle: jlong,
enabled: jboolean,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_control(enabled != 0, 0);
}
}
/// `NativeBridge.nativeClipOfferText(handle, seq)` — announce "the Android clipboard now holds
/// text" (format list only; bytes cross when the host fetches). `seq` is Kotlin's monotonic
/// counter, newest wins.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipOfferText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_offer(
seq as u32,
vec![ClipKind {
mime: TEXT_MIME.into(),
size_hint: 0,
}],
);
}
}
/// `NativeBridge.nativeClipFetchText(handle, seq)` — pull the text of the host's offer `seq`.
/// Returns the transfer id echoed on the matching `data:`/`error:` event, or 1.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipFetchText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) -> jint {
client(handle)
.and_then(|h| {
h.client
.clip_fetch(seq as u32, TEXT_MIME.into(), CLIP_FILE_INDEX_NONE)
.ok()
})
.map_or(-1, |xfer| xfer as jint)
}
/// `NativeBridge.nativeClipServeText(handle, reqId, text)` — answer a `fetch:` event with the
/// clipboard's current text (the host is pasting our offer).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipServeText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
req_id: jint,
text: JString,
) {
let Some(h) = client(handle) else { return };
let Ok(s) = env.get_string(&text) else {
let _ = h.client.clip_cancel(req_id as u32);
return;
};
let _ = h
.client
.clip_serve(req_id as u32, String::from(s).into_bytes(), true);
}
/// `NativeBridge.nativeClipCancel(handle, id)` — abort a transfer (either direction).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipCancel(
_env: JNIEnv,
_this: JObject,
handle: jlong,
id: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_cancel(id as u32);
}
}
/// `NativeBridge.nativeNextClip(handle)` — block ≤250 ms for the next clipboard event, encoded
/// as a compact string (module docs); null on timeout, `"closed"` once the session is gone.
/// Call from a dedicated poll thread.
///
/// Text payloads ride `data:<xfer_id>:<text>` decoded lossily — safe because the phase-0
/// clipboard task delivers a whole payload in ONE event (`last = true`), so a chunk boundary
/// can never split a UTF-8 sequence.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextClip(
env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jstring {
let Some(h) = client(handle) else {
return std::ptr::null_mut();
};
let msg = match h.client.next_clip(Duration::from_millis(250)) {
Ok(ClipEventCore::State { enabled, .. }) => format!("state:{}", u8::from(enabled)),
Ok(ClipEventCore::RemoteOffer { seq, kinds }) => {
let has_text = kinds.iter().any(|k| k.mime.starts_with("text/plain"));
format!("offer:{seq}:{}", u8::from(has_text))
}
Ok(ClipEventCore::FetchRequest { req_id, mime, .. }) => {
if mime.starts_with("text/plain") {
format!("fetch:{req_id}")
} else {
// We only ever offer text; cancel anything else rather than stall the host.
let _ = h.client.clip_cancel(req_id);
return std::ptr::null_mut();
}
}
Ok(ClipEventCore::Data { xfer_id, bytes, .. }) => {
format!("data:{xfer_id}:{}", String::from_utf8_lossy(&bytes))
}
Ok(ClipEventCore::Cancelled { id }) => format!("cancel:{id}"),
Ok(ClipEventCore::Error { id, code }) => format!("error:{id}:{code}"),
Err(PunktfunkError::NoFrame) => return std::ptr::null_mut(),
Err(_) => "closed".into(),
};
env.new_string(msg)
.map(|s| s.into_raw())
.unwrap_or(std::ptr::null_mut())
}
@@ -201,9 +201,6 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// No display-volume forwarding from Android yet (the panel tone-maps PQ itself via the // No display-volume forwarding from Android yet (the panel tone-maps PQ itself via the
// Surface dataspace + static metadata) — the host keeps its virtual-display EDID defaults. // Surface dataspace + static metadata) — the host keeps its virtual-display EDID defaults.
None, None,
// No non-video caps: this client does not render the host cursor locally (no shape/state
// planes in the jni surface), so advertising CLIENT_CAP_CURSOR would stream cursor-less.
0,
launch, // a store-qualified library id to boot into a game, or None for the desktop launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous) identity, // owned (cert, key) PEM, or None (anonymous)
+2 -131
View File
@@ -6,14 +6,11 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal, //! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side). //! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JFloatArray, JObject, JString}; use jni::objects::{JByteBuffer, JObject};
use jni::sys::{jboolean, jint, jlong}; use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv; use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{ use punktfunk_core::quic::{RichInput, HID_REPORT_MAX};
PenSample, PenTool, RichInput, HID_REPORT_MAX, HOST_CAP_PEN, HOST_CAP_TEXT_INPUT,
PEN_ANGLE_UNKNOWN, PEN_BATCH_MAX, PEN_DISTANCE_UNKNOWN, PEN_TILT_UNKNOWN,
};
use super::SessionHandle; use super::SessionHandle;
@@ -148,132 +145,6 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
send_event(handle, kind, vk as u32, 0, 0, mods as u32); send_event(handle, kind, vk as u32, 0, 0, mods as u32);
} }
/// `NativeBridge.nativeTextInputSupported(handle)` — whether the host advertised
/// `HOST_CAP_TEXT_INPUT` (its inject backend types committed text), so the Kotlin side can pick
/// the real IME `InputConnection` over the TYPE_NULL raw-key fallback. `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeHostSupportsPen(handle)` — the host advertised `HOST_CAP_PEN`, so the
/// Kotlin side splits stylus pointers out of the touch path onto the pen plane
/// (design/pen-tablet-input.md §7). `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostSupportsPen(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_PEN != 0)
}
/// Floats per sample in the `nativeSendPen` flat array.
const PEN_JNI_STRIDE: usize = 10;
/// `NativeBridge.nativeSendPen(handle, samples, count)` — one stylus batch of STATE-FULL
/// samples, `count` × [`PEN_JNI_STRIDE`] floats, oldest first:
/// `[state, tool, x, y, pressure, distance, tilt_deg, azimuth_deg, roll_deg, dt_us]`.
/// `state` = the wire `PEN_*` bits; `tool` 0=pen 1=eraser; `x`/`y`/`pressure`/`distance`
/// normalized 0..1; `distance`/`tilt_deg`/`azimuth_deg`/`roll_deg` < 0 = unknown. Call only
/// against a [`nativeHostSupportsPen`] host; the client heartbeats the last sample ≤100 ms
/// while in range (Kotlin side — see `StylusStream`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPen(
env: JNIEnv,
_this: JObject,
handle: jlong,
samples: JFloatArray,
count: jint,
) {
if handle == 0 || count <= 0 {
return;
}
let count = (count as usize).min(PEN_BATCH_MAX);
let mut buf = [0f32; PEN_BATCH_MAX * PEN_JNI_STRIDE];
let flat = &mut buf[..count * PEN_JNI_STRIDE];
if env.get_float_array_region(&samples, 0, flat).is_err() {
return; // short array — a bridge bug, never worth a crash on the input path
}
let mut batch = [PenSample::default(); PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(flat.chunks_exact(PEN_JNI_STRIDE)) {
if !s[2].is_finite() || !s[3].is_finite() {
return; // never forward a NaN coordinate
}
*slot = PenSample {
state: s[0] as u8,
tool: if s[1] as u8 == 1 {
PenTool::Eraser
} else {
PenTool::Pen
},
x: s[2].clamp(0.0, 1.0),
y: s[3].clamp(0.0, 1.0),
pressure: (s[4].clamp(0.0, 1.0) * 65535.0) as u16,
distance: if s[5] < 0.0 {
PEN_DISTANCE_UNKNOWN
} else {
(s[5].clamp(0.0, 1.0) * 65534.0) as u16
},
tilt_deg: if s[6] < 0.0 {
PEN_TILT_UNKNOWN
} else {
(s[6].clamp(0.0, 90.0)) as u8
},
azimuth_deg: if s[7] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[7] as u16) % 360
},
roll_deg: if s[8] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[8] as u16) % 360
},
dt_us: s[9].clamp(0.0, 65535.0) as u16,
};
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_pen is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_pen(&batch[..count]);
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
/// [`Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported`] returned true.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
text: JString,
) {
if handle == 0 {
return;
}
let Ok(s) = env.get_string(&text) else {
return;
};
for ch in String::from(s).chars().filter(|c| !c.is_control()) {
send_event(handle, InputKind::TextInput, ch as u32, 0, 0, 0);
}
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input --------------- // ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried // Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried
// in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a // in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a
@@ -17,7 +17,6 @@
//! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode //! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode
//! renegotiation. Port the remaining orchestration from `clients/linux`. //! renegotiation. Port the remaining orchestration from `clients/linux`.
mod clipboard;
mod connect; mod connect;
mod input; mod input;
mod planes; mod planes;
@@ -144,26 +144,14 @@ struct ContentView: View {
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a // tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
// parallel session this drives the one `model` ContentView owns. // parallel session this drives the one `model` ContentView owns.
.onOpenURL { handleDeepLink($0) } .onOpenURL { handleDeepLink($0) }
#if os(iOS) || os(tvOS) #if os(iOS)
// Backgrounding driver. Only .background/.active matter; .inactive (a transient peek) is // Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a
// ignored so neither branch fires for a Control-Center pull. // transient peek) is ignored so the disconnect timer never starts for a Control-Center pull.
//
// Backgrounding MUST end the session one way or the other: the app keeps running while
// streaming (the `audio` background mode plus a live audio session), so its QUIC connection
// keeps answering the host's keep-alives with the user long gone the host has no way to
// tell that apart from someone watching, and the session survived indefinitely. Either hold
// it under the opt-in keep-alive (bounded by that path's own auto-disconnect timer) or end
// it here.
.onChange(of: scenePhase) { _, phase in .onChange(of: scenePhase) { _, phase in
switch phase { switch phase {
case .background: case .background:
guard model.phase == .streaming else { break } if backgroundKeepAlive, model.phase == .streaming {
if backgroundKeepAlive {
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes) model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
} else {
// Not deliberate: the user may come straight back, so let the host linger the
// display for a fast reconnect instead of tearing it down.
model.disconnect(deliberate: false)
} }
case .active: case .active:
model.exitBackground() model.exitBackground()
@@ -171,11 +159,7 @@ struct ContentView: View {
break break
} }
} }
#endif // Live Activity lifecycle, driven from the model's published state.
#if os(iOS)
// Live Activity lifecycle, driven from the model's published state. iPhone/iPad only
// ActivityKit (and so `liveActivity`) does not exist on tvOS, which is why this stays in its
// own os(iOS) block rather than riding the backgrounding driver's.
.onChange(of: model.phase) { _, phase in .onChange(of: model.phase) { _, phase in
switch phase { switch phase {
case .streaming: case .streaming:
@@ -595,21 +579,13 @@ struct ContentView: View {
model?.disconnect() // the captured-state D combo model?.disconnect() // the captured-state D combo
}, },
onFrame: { [meter = model.meter, latency = model.latency, onFrame: { [meter = model.meter, latency = model.latency,
split = model.latencySplit, queue = model.clientQueue, split = model.latencySplit, offset = conn.clockOffsetNs] au in
offset = conn.clockOffsetNs] au in
meter.note(byteCount: au.data.count) meter.note(byteCount: au.data.count)
latency.record(ptsNs: au.ptsNs, offsetNs: offset) latency.record(ptsNs: au.ptsNs, offsetNs: offset)
// The same receipt, keyed by pts, awaiting its 0xCF host timing (the // The same receipt, keyed by pts, awaiting its 0xCF host timing (the
// host/network split drained by the 1 s stats tick). receivedNs is // host/network split drained by the 1 s stats tick).
// the core's reassembly stamp (ABI v9), so the split's network term no
// longer contains the client-queue wait...
split.recordReceipt( split.recordReceipt(
ptsNs: au.ptsNs, receivedNs: au.receivedNs, offsetNs: offset) ptsNs: au.ptsNs, receivedNs: au.receivedNs, offsetNs: offset)
// ...which is measured as its own term instead (receiptpull, both
// client-local).
queue.record(
ptsNs: UInt64(bitPattern: au.receivedNs), atNs: au.pulledNs,
offsetNs: 0)
}, },
onSessionEnd: { [weak model] in onSessionEnd: { [weak model] in
Task { @MainActor in model?.sessionEnded() } Task { @MainActor in model?.sessionEnded() }
@@ -1,97 +0,0 @@
// Keeps the local display awake for the duration of a streaming session.
//
// A stream is not "user activity" to the OS: the pixels arrive over the network and the input that
// drives them is often a game controller, which does NOT feed the HID idle timer on any Apple
// platform. So a controller-only session reliably idles the panel out from under the user the
// same reason the Android client holds FLAG_KEEP_SCREEN_ON while streaming (StreamScreen.kt).
//
// Held by SessionModel from `beginStreaming` to `disconnect`, so it is scoped to the session and
// never leaks past it (including a host-ended or timed-out background session, which both land in
// `disconnect`).
import Foundation
#if os(macOS)
import IOKit.pwr_mgt
#else
import UIKit
#endif
@MainActor
final class DisplaySleepGuard {
#if os(macOS)
/// The `beginActivity` token; non-nil exactly while held.
private var activity: NSObjectProtocol?
/// Re-used across heartbeats so the whole session shares one assertion instead of
/// accumulating one per tick.
private var userActivityAssertion: IOPMAssertionID = IOPMAssertionID(0)
private var heartbeat: Timer?
/// The power assertion defers DISPLAY SLEEP but not the screen saver that runs off the
/// HID idle timer, which a controller-only session never touches. Declaring user activity
/// on an interval well under the shortest selectable screen-saver delay (1 minute) keeps
/// that timer from ever reaching it. Side effect, and the intended one: an idle-lock
/// configured to follow the screen saver is deferred too, for the session only.
private static let heartbeatInterval: TimeInterval = 30
#endif
private(set) var isHeld = false
/// Idempotent a second acquire while held is a no-op.
func acquire() {
guard !isHeld else { return }
isHeld = true
#if os(macOS)
// The high-level Foundation API over IOKit power assertions: `.idleDisplaySleepDisabled`
// is the panel, `.userInitiated` also holds off idle SYSTEM sleep and sudden termination
// for a session the user is watching in real time.
activity = ProcessInfo.processInfo.beginActivity(
options: [.userInitiated, .idleDisplaySleepDisabled],
reason: "Punktfunk streaming session")
declareUserActivity()
let timer = Timer.scheduledTimer(withTimeInterval: Self.heartbeatInterval, repeats: true) {
[weak self] _ in
MainActor.assumeIsolated { self?.declareUserActivity() }
}
// The stream runs under a tracking run-loop mode while a menu or a window resize is up;
// .common keeps the heartbeat ticking through those.
RunLoop.main.add(timer, forMode: .common)
heartbeat = timer
#else
// iOS/iPadOS/tvOS: app-wide, and ignored while backgrounded the background keep-alive
// (audio-only, video dropped) correctly lets the device sleep without touching this.
UIApplication.shared.isIdleTimerDisabled = true
#endif
}
/// Idempotent safe to call when not held (`disconnect` runs on paths that never streamed).
func release() {
guard isHeld else { return }
isHeld = false
#if os(macOS)
heartbeat?.invalidate()
heartbeat = nil
if let activity {
ProcessInfo.processInfo.endActivity(activity)
self.activity = nil
}
if userActivityAssertion != IOPMAssertionID(0) {
IOPMAssertionRelease(userActivityAssertion)
userActivityAssertion = IOPMAssertionID(0)
}
#else
UIApplication.shared.isIdleTimerDisabled = false
#endif
}
#if os(macOS)
/// Resets the HID idle timer (see `heartbeatInterval`). `kIOPMUserActiveLocal` = activity at
/// this Mac's own display, which is what a stream being watched here is.
private func declareUserActivity() {
IOPMAssertionDeclareUserActivity(
"Punktfunk streaming session" as CFString,
kIOPMUserActiveLocal,
&userActivityAssertion)
}
#endif
}
@@ -102,12 +102,6 @@ final class SessionModel: ObservableObject {
@Published var decodeValid = false @Published var decodeValid = false
@Published var displayP50Ms = 0.0 @Published var displayP50Ms = 0.0
@Published var displayValid = false @Published var displayValid = false
/// Client-queue wait: core reassembly receipt the pump's pull (`AccessUnit.pulledNs
/// receivedNs`, ABI v9 receipt split the 2026-07 two-pair investigation). ~0 on a healthy
/// stream; a persistent value is a client-side standing backlog that used to hide inside
/// "network". Shown in the detailed tier only when it says something ( ~2 ms).
@Published var clientQueueP50Ms = 0.0
@Published var clientQueueValid = false
/// The measured OS present floor (design/apple-presentation-rebuild.md): the deadline /// The measured OS present floor (design/apple-presentation-rebuild.md): the deadline
/// engine's vendglass pipeline depth an OS property no client can pace under (~2 refresh /// engine's vendglass pipeline depth an OS property no client can pace under (~2 refresh
/// intervals composited; would read ~1 under direct-to-display). The HUD subtracts it from /// intervals composited; would read ~1 under direct-to-display). The HUD subtracts it from
@@ -153,9 +147,6 @@ final class SessionModel: ObservableObject {
let endToEnd = LatencyMeter() let endToEnd = LatencyMeter()
let decodeStage = LatencyMeter() let decodeStage = LatencyMeter()
let displayStage = LatencyMeter() let displayStage = LatencyMeter()
/// Client-queue sampler (see `clientQueueP50Ms`) fed per AU by the stream view's onFrame,
/// drained by the same 1 s tick as the stage meters.
let clientQueue = LatencyMeter()
/// The OS present floor sampler (see `osFloorP50Ms`) fed one sample per display-link /// The OS present floor sampler (see `osFloorP50Ms`) fed one sample per display-link
/// update by the deadline engine, drained by the same 1 s tick as the stage meters. /// update by the deadline engine, drained by the same 1 s tick as the stage meters.
let presentFloor = LatencyMeter() let presentFloor = LatencyMeter()
@@ -196,11 +187,6 @@ final class SessionModel: ObservableObject {
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`. /// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
private var backgroundTimer: DispatchSourceTimer? private var backgroundTimer: DispatchSourceTimer?
/// Holds off display sleep (and, on macOS, the screen saver) for the life of a session
/// nothing about watching a stream looks like user activity to the OS, least of all a
/// controller-only session. Acquired in `beginStreaming`, released in `disconnect`.
private let displaySleepGuard = DisplaySleepGuard()
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true /// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host /// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
/// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair` /// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
@@ -307,26 +293,13 @@ final class SessionModel: ObservableObject {
if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported { if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported {
videoCodecs |= PunktfunkConnection.codecPyroWave videoCodecs |= PunktfunkConnection.codecPyroWave
} }
// Cursor channel (remote-desktop-sweep M2, macOS): sessions STARTING in the desktop
// mouse model advertise local cursor rendering the host then stops compositing
// the pointer and forwards shape/state, which StreamView draws as the real
// NSCursor. Capture-mode sessions keep today's composited pointer.
#if os(macOS)
let clientCaps: UInt8 =
(MouseInputMode(
rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? "")
?? .capture) == .desktop ? 0x01 : 0
#else
let clientCaps: UInt8 = 0
#endif
let result = Result { try PunktfunkConnection( let result = Result { try PunktfunkConnection(
host: host.address, port: host.port, host: host.address, port: host.port,
width: width, height: height, refreshHz: hz, width: width, height: height, refreshHz: hz,
pinSHA256: pin, identity: identity, compositor: compositor, pinSHA256: pin, identity: identity, compositor: compositor,
gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps, gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps,
audioChannels: audioChannels, audioChannels: audioChannels,
videoCodecs: videoCodecs, preferredCodec: preferredCodec, videoCodecs: videoCodecs, preferredCodec: preferredCodec, launchID: launchID,
clientCaps: clientCaps, launchID: launchID,
// Delegated approval: the host holds this connect open until the operator approves // Delegated approval: the host holds this connect open until the operator approves
// it (~180 s) outwait that window so a slow approval still lands here. Normal // it (~180 s) outwait that window so a slow approval still lands here. Normal
// connects keep the snappy default. // connects keep the snappy default.
@@ -460,8 +433,6 @@ final class SessionModel: ObservableObject {
func disconnect(deliberate: Bool = true) { func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate() statsTimer?.invalidate()
statsTimer = nil statsTimer = nil
// No-op when this session never reached `.streaming` (a refused/aborted connect).
displaySleepGuard.release()
// Drop any armed background keep-alive (incl. the timeout that just fired us). // Drop any armed background keep-alive (incl. the timeout that just fired us).
backgroundTimer?.cancel() backgroundTimer?.cancel()
backgroundTimer = nil backgroundTimer = nil
@@ -518,7 +489,6 @@ final class SessionModel: ObservableObject {
endToEndValid = false endToEndValid = false
decodeValid = false decodeValid = false
displayValid = false displayValid = false
clientQueueValid = false
osFloorValid = false osFloorValid = false
lostFrames = 0 lostFrames = 0
lostPct = 0 lostPct = 0
@@ -557,7 +527,6 @@ final class SessionModel: ObservableObject {
// Input capture itself is owned by StreamView (engaged by the captureEnabled // Input capture itself is owned by StreamView (engaged by the captureEnabled
// flip this phase change causes, released/re-engaged by the user from there). // flip this phase change causes, released/re-engaged by the user from there).
phase = .streaming phase = .streaming
displaySleepGuard.acquire()
// Audio starts with streaming, not during the trust prompt no host sound (or // Audio starts with streaming, not during the trust prompt no host sound (or
// mic uplink!) before the user trusted the host. Devices come from Settings; // mic uplink!) before the user trusted the host. Devices come from Settings;
// "" = system default. // "" = system default.
@@ -710,12 +679,6 @@ final class SessionModel: ObservableObject {
} else { } else {
self.osFloorValid = false self.osFloorValid = false
} }
if let q = self.clientQueue.drain() {
self.clientQueueP50Ms = q.p50Ms
self.clientQueueValid = true
} else {
self.clientQueueValid = false
}
// Mirror the window to the unified log (see statsLog) one line per second, // Mirror the window to the unified log (see statsLog) one line per second,
// stages in ms, only while frames actually flowed. `fps` counts RECEIVED AUs; // stages in ms, only while frames actually flowed. `fps` counts RECEIVED AUs;
// `presents` counts frames that reached glass (the display meter's sample count) // `presents` counts frames that reached glass (the display meter's sample count)
@@ -726,12 +689,9 @@ final class SessionModel: ObservableObject {
// captured before the 2026-07 floor policy); the appended trio carries the // captured before the 2026-07 floor policy); the appended trio carries the
// measured OS present floor and the floor-shaved values the HUD displays. // measured OS present floor and the floor-shaved values the HUD displays.
let line = String( let line = String(
// Swift Int is 64-bit %lld, NOT %d (which is a 32-bit C int); macOS 26's format: "fps=%d presents=%d e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
// strict String(format:) validator rejects the %d/Int mismatch and drops + "decode_p50=%.1f display_p50=%.1f lost=%d "
// the whole line (a cascade error that also mis-blames the float args). + "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f",
format: "fps=%lld presents=%lld e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%lld "
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f",
frames, frames,
displayWindow?.count ?? 0, displayWindow?.count ?? 0,
self.endToEndValid ? self.endToEndP50Ms : -1, self.endToEndValid ? self.endToEndP50Ms : -1,
@@ -742,8 +702,7 @@ final class SessionModel: ObservableObject {
lost, lost,
self.osFloorValid ? self.osFloorP50Ms : -1, self.osFloorValid ? self.osFloorP50Ms : -1,
self.displayValid ? self.displayAdjP50Ms : -1, self.displayValid ? self.displayAdjP50Ms : -1,
self.endToEndValid ? self.endToEndAdjP50Ms : -1, self.endToEndValid ? self.endToEndAdjP50Ms : -1)
self.clientQueueValid ? self.clientQueueP50Ms : -1)
statsLog.info("\(line, privacy: .public)") statsLog.info("\(line, privacy: .public)")
} }
} }
@@ -118,16 +118,6 @@ struct StreamHUDView: View {
.font(.system(.caption2, design: .monospaced)) .font(.system(.caption2, design: .monospaced))
.foregroundStyle(.tertiary) .foregroundStyle(.tertiary)
} }
// Client-queue wait (reassembly receipt decode pull, ABI v9 split): ~0 on
// a healthy stream and hidden as noise; shown from 2 ms a persistent value
// is a client-side standing backlog that pre-split builds displayed as
// "network" (the 2026-07 two-pair plateau). The core's standing-latency
// bleed logs alongside when it acts on the same state.
if model.clientQueueValid && model.clientQueueP50Ms >= 2 {
Text("client queue +\(model.clientQueueP50Ms, specifier: "%.1f") (receive backlog — standing if it persists)")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.tertiary)
}
} }
} else if model.hostNetworkValid { } else if model.hostNetworkValid {
// Stage-1 fallback presenter: the layer decodes + presents internally with no // Stage-1 fallback presenter: the layer decodes + presents internally with no
@@ -43,9 +43,6 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.presentPriority) private var presentPriority = @AppStorage(DefaultsKey.presentPriority) private var presentPriority =
SettingsOptions.presentPriorityDefault SettingsOptions.presentPriorityDefault
@AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0 @AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0
#if os(macOS)
@AppStorage(DefaultsKey.windowedSafePresent) private var windowedSafePresent = true
#endif
#if os(iOS) #if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false @AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
#endif #endif
@@ -348,22 +345,6 @@ struct GamepadSettingsView: View {
detail: "Turn off to use the touch interface even with a controller connected.", detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled), value: $gamepadUIEnabled),
] ]
#if os(macOS)
// The windowed safe-present toggle slots in after "Smoothness buffer" (staying inside
// the Video group) macOS only, mirroring the touch SettingsView's Presentation row
// (the DCP swapID-panic mitigation; see DefaultsKey.windowedSafePresent).
if let at = list.firstIndex(where: { $0.id == "smoothBuffer" }) {
list.insert(
toggleRow(
id: "windowedSafePresent", icon: "macwindow.badge.plus",
label: "Safe windowed presentation",
detail: "Windowed streams present in step with the compositor — avoids a "
+ "macOS display-driver crash on high-refresh displays, at a small "
+ "latency cost. Fullscreen always uses the fastest path.",
value: $windowedSafePresent),
at: at + 1)
}
#endif
#if os(iOS) #if os(iOS)
// The device-rumble mirror slots in after "Controller type" (staying inside the // The device-rumble mirror slots in after "Controller type" (staying inside the
// Controller group the next row carries the "Interface" header). iPhone only in // Controller group the next row carries the "Interface" header). iPhone only in
@@ -300,18 +300,6 @@ extension SettingsView {
+ "of added latency. Off shows frames as soon as they're ready.") { + "of added latency. Off shows frames as soon as they're ready.") {
Toggle("V-Sync", isOn: $vsync) Toggle("V-Sync", isOn: $vsync)
} }
// The DCP swapID-panic mitigation's user handle (see DefaultsKey.windowedSafePresent
// for the saga). Default ON: turning it off re-arms a WHOLE-MACHINE kernel panic on
// affected setups, so the caption says so in plain words.
described(windowedSafePresent
? "Windowed streams present in step with the system compositor — avoids a macOS "
+ "display-driver crash seen on high-refresh displays, at a small latency "
+ "cost. Fullscreen always uses the fastest path."
: "Windowed streams use the fastest present path. On some high-refresh setups "
+ "this can crash macOS itself (kernel panic) — turn back on if your Mac "
+ "restarts during windowed streaming.") {
Toggle("Safe windowed presentation", isOn: $windowedSafePresent)
}
#endif #endif
} }
} }
@@ -447,14 +435,6 @@ extension SettingsView {
/// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path). /// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path).
@ViewBuilder var inputSection: some View { @ViewBuilder var inputSection: some View {
Section("Keyboard & mouse") { Section("Keyboard & mouse") {
#if os(macOS)
described(mouseModeDescription) {
Picker("Mouse input", selection: $mouseMode) {
Text("Capture (games)").tag(MouseInputMode.capture.rawValue)
Text("Desktop (absolute)").tag(MouseInputMode.desktop.rawValue)
}
}
#endif
described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) { described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) {
Picker("Modifier keys", selection: $modifierLayout) { Picker("Modifier keys", selection: $modifierLayout) {
ForEach(ModifierLayout.allCases, id: \.self) { layout in ForEach(ModifierLayout.allCases, id: \.self) { layout in
@@ -467,20 +447,6 @@ extension SettingsView {
} }
} }
} }
#if os(macOS)
/// The SELECTED mouse model explained dynamic, like the touch-mode caption.
private var mouseModeDescription: String {
switch MouseInputMode(rawValue: mouseMode) ?? .capture {
case .capture:
return "The pointer locks to the stream and sends relative motion — best for "
+ "games. ⌃⌥⇧M switches live; applies from the next capture otherwise."
case .desktop:
return "The pointer moves freely in and out of the stream and sends absolute "
+ "positions — best for remote desktop work. Unavailable on gamescope hosts."
}
}
#endif
#endif #endif
// MARK: - Audio // MARK: - Audio
@@ -40,7 +40,6 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0 @AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false @AppStorage(DefaultsKey.vsync) var vsync = false
@AppStorage(DefaultsKey.windowedSafePresent) var windowedSafePresent = true
#endif #endif
#if !os(tvOS) #if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true @AppStorage(DefaultsKey.allowVRR) var allowVRR = true
@@ -89,7 +88,6 @@ struct SettingsView: View {
@State var customMode = false @State var customMode = false
#endif #endif
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.mouseMode) var mouseMode = MouseInputMode.capture.rawValue
@AppStorage(DefaultsKey.speakerUID) var speakerUID = "" @AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = "" @AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0 @AppStorage(DefaultsKey.micChannel) var micChannel = 0
@@ -35,31 +35,10 @@ public struct AccessUnit: Sendable {
public let ptsNs: UInt64 public let ptsNs: UInt64
public let frameIndex: UInt32 public let frameIndex: UInt32
public let flags: UInt32 public let flags: UInt32
/// Client `CLOCK_REALTIME` instant the AU finished reassembly in the core (post-FEC, /// Client `CLOCK_REALTIME` instant the AU was handed over by the core (post-FEC, decrypted)
/// decrypted `PunktfunkFrame.received_ns`, ABI v9) the **received** measurement point of /// the **received** measurement point of design/stats-unification.md. The decode stage is
/// design/stats-unification.md. NOT the pull instant: stamping at the pull folded the /// `decodedNs - receivedNs`, both client-local (no skew offset applies).
/// pre-decode hand-off wait into the network term, which is how the 2026-07 two-pair
/// standing-latency plateau hid as "network". The decode stage is `decodedNs - receivedNs`,
/// both client-local (no skew offset applies).
public let receivedNs: Int64 public let receivedNs: Int64
/// Client `CLOCK_REALTIME` instant this pull returned. `pulledNs - receivedNs` is the
/// client-queue wait (kernel hand-off + FrameChannel dwell) the term the HUD splits out
/// so a client-side standing backlog can never masquerade as network latency again.
public let pulledNs: Int64
/// `pulledNs` defaults to `receivedNs` (zero queue wait) for callers with no pull instant
/// the synthetic probe AUs and decode tests, where the split is meaningless.
public init(
data: Data, ptsNs: UInt64, frameIndex: UInt32, flags: UInt32,
receivedNs: Int64, pulledNs: Int64? = nil
) {
self.data = data
self.ptsNs = ptsNs
self.frameIndex = frameIndex
self.flags = flags
self.receivedNs = receivedNs
self.pulledNs = pulledNs ?? receivedNs
}
} }
/// One Opus audio packet (48 kHz stereo, 5 ms frames) decode with AVAudioConverter /// One Opus audio packet (48 kHz stereo, 5 ms frames) decode with AVAudioConverter
@@ -221,9 +200,6 @@ public final class PunktfunkConnection {
/// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too: /// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too:
/// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple. /// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple.
private let clipboardLock = NSLock() private let clipboardLock = NSLock()
/// Serializes the (single) cursor pull thread against close() both cursor planes are
/// drained by ONE thread, so one lock covers them.
private let cursorLock = NSLock()
/// Negotiated session mode (host-confirmed). /// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0 public private(set) var width: UInt32 = 0
@@ -384,106 +360,6 @@ public final class PunktfunkConnection {
public var hostSupportsClipboard: Bool { public var hostSupportsClipboard: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0 hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0
} }
/// The host answered `HOST_CAP_CURSOR`: it stopped compositing the pointer and forwards
/// shape/state on the cursor planes the client MUST draw the cursor locally.
/// `0x08` the bit moved when `HOST_CAP_TEXT_INPUT` claimed `0x04` on main; testing the
/// old bit would mistake a text-input-capable host (e.g. Windows) for a cursor grant.
public var hostSupportsCursor: Bool {
hostCaps & 0x08 != 0
}
/// The host injects full-fidelity stylus input (`HOST_CAP_PEN`) the gate for splitting
/// Apple Pencil out of the touch path onto the pen plane (``sendPen(_:)``).
public var hostSupportsPen: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_PEN) != 0
}
/// One forwarded host-cursor shape (the cursor channel, ABI v11): straight-alpha RGBA,
/// `rgba.count == width * height * 4`, hotspot within the bitmap. Cache by `serial`
/// states reference shapes by it and a re-shown serial never resends pixels.
public struct CursorShapeEvent: Sendable {
public let serial: UInt32
public let width: Int
public let height: Int
public let hotX: Int
public let hotY: Int
public let rgba: Data
}
/// Per-host-tick cursor state: position (host video px, the pointer/hotspot point),
/// visibility, and the host-driven relative-mode hint (an app grabbed/hid the pointer
/// run captured relative; clear absolute, reappearing at `x`/`y`). Latest-wins.
public struct CursorStateEvent: Sendable {
public let serial: UInt32
public let visible: Bool
public let relativeHint: Bool
public let x: Int32
public let y: Int32
}
/// Pull the next forwarded cursor SHAPE (nil = timeout). Only a session connected with
/// `clientCaps` cursor bit against a `hostSupportsCursor` host receives any. Drain shape
/// AND state from ONE dedicated cursor thread (they share a lock).
public func nextCursorShape(timeoutMs: UInt32 = 0) throws -> CursorShapeEvent? {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkCursorShape()
let rc = punktfunk_connection_next_cursor_shape(h, &out, timeoutMs)
switch rc {
case statusOK:
// Copy out of the ABI borrow (valid until the next shape call) immediately.
let bytes = out.rgba.map { Data(bytes: $0, count: Int(out.len)) } ?? Data()
return CursorShapeEvent(
serial: out.serial, width: Int(out.w), height: Int(out.h),
hotX: Int(out.hot_x), hotY: Int(out.hot_y), rgba: bytes)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Pull the next cursor STATE (nil = timeout). Latest-wins drain the queue and apply
/// only the newest. Same thread + gate as [`nextCursorShape`].
public func nextCursorState(timeoutMs: UInt32 = 0) throws -> CursorStateEvent? {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkCursorState()
let rc = punktfunk_connection_next_cursor_state(h, &out, timeoutMs)
switch rc {
case statusOK:
return CursorStateEvent(
serial: out.serial,
visible: out.flags & 0x01 != 0,
relativeHint: out.flags & 0x02 != 0,
x: out.x, y: out.y)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Tell the host who renders the pointer (the §8 mid-stream mouse-model flip, ABI v12):
/// `clientDraws = true` this client draws it locally (the desktop mouse model; the host
/// excludes the pointer from the video and forwards shape/state); `false` the host
/// composites it into the video (the capture model, full fidelity). Idempotent,
/// latest-wins; harmless against hosts without the cursor cap. Fire-and-forget errors
/// are swallowed (a closed session is the only failure and it moots the flip).
public func setCursorRender(clientDraws: Bool) {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_set_cursor_render(h, clientDraws)
}
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing /// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack). /// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) } public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
@@ -520,7 +396,6 @@ public final class PunktfunkConnection {
audioChannels: UInt8 = 2, audioChannels: UInt8 = 2,
videoCodecs: UInt8 = 0x02, // PUNKTFUNK_CODEC_HEVC the codecs this client can decode videoCodecs: UInt8 = 0x02, // PUNKTFUNK_CODEC_HEVC the codecs this client can decode
preferredCodec: UInt8 = 0, // 0 = auto; else PUNKTFUNK_CODEC_* soft preference preferredCodec: UInt8 = 0, // 0 = auto; else PUNKTFUNK_CODEC_* soft preference
clientCaps: UInt8 = 0, // ABI v11: PUNKTFUNK_CLIENT_CAP_CURSOR = render the host cursor locally
launchID: String? = nil, launchID: String? = nil,
timeoutMs: UInt32 = 10_000 timeoutMs: UInt32 = 10_000
) throws { ) throws {
@@ -540,18 +415,18 @@ public final class PunktfunkConnection {
withOptionalCString(launchID) { launch in withOptionalCString(launchID) { launch in
if let pin = pinSHA256 { if let pin = pinSHA256 {
return pin.withUnsafeBytes { p in return pin.withUnsafeBytes { p in
punktfunk_connect_ex9( punktfunk_connect_ex8(
cs, port, width, height, refreshHz, compositor.rawValue, cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, clientCaps, launch, videoCodecs, preferredCodec, launch,
p.bindMemory(to: UInt8.self).baseAddress, &observed, p.bindMemory(to: UInt8.self).baseAddress, &observed,
cert, key, timeoutMs, &connectStatus) cert, key, timeoutMs, &connectStatus)
} }
} }
return punktfunk_connect_ex9( return punktfunk_connect_ex8(
cs, port, width, height, refreshHz, compositor.rawValue, cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, clientCaps, launch, videoCodecs, preferredCodec, launch,
nil, &observed, cert, key, timeoutMs, &connectStatus) nil, &observed, cert, key, timeoutMs, &connectStatus)
} }
} }
@@ -787,16 +662,11 @@ public final class PunktfunkConnection {
let data = Data(bytes: base, count: Int(frame.len)) // copy: ptr valid only until next call let data = Data(bytes: base, count: Int(frame.len)) // copy: ptr valid only until next call
var ts = timespec() var ts = timespec()
clock_gettime(CLOCK_REALTIME, &ts) clock_gettime(CLOCK_REALTIME, &ts)
let pulledNs = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec) let receivedNs = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec)
// Receipt = the core's reassembly-completion stamp (ABI v9); the pull instant is
// kept separately so the client-queue wait is its own measured term. 0 would mean a
// pre-v9 core impossible here (core and Kit ship in one binary), but fall back to
// the pull instant rather than record a 1970 receipt.
let receivedNs = frame.received_ns > 0 ? Int64(frame.received_ns) : pulledNs
return AccessUnit( return AccessUnit(
data: data, ptsNs: frame.pts_ns, data: data, ptsNs: frame.pts_ns,
frameIndex: frame.frame_index, flags: frame.flags, frameIndex: frame.frame_index, flags: frame.flags,
receivedNs: receivedNs, pulledNs: pulledNs) receivedNs: receivedNs)
case statusNoFrame: case statusNoFrame:
return nil return nil
case statusClosed: case statusClosed:
@@ -1141,19 +1011,6 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_input(h, &ev) _ = punktfunk_connection_send_input(h, &ev)
} }
/// Send one stylus sample batch ( `PUNKTFUNK_PEN_BATCH_MAX`, oldest first) on the pen
/// plane. Gate on ``hostSupportsPen`` the core refuses toward a host without the cap.
/// Thread-safe; silently dropped after close (input is lossy by design).
public func sendPen(_ samples: [PunktfunkPenSample]) {
guard !samples.isEmpty else { return }
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return }
samples.withUnsafeBufferPointer { buf in
_ = punktfunk_connection_send_pen(h, buf.baseAddress, UInt32(buf.count))
}
}
/// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with /// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with
/// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the /// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the
/// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a /// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a
@@ -1176,12 +1033,10 @@ public final class PunktfunkConnection {
feedbackLock.lock() feedbackLock.lock()
statsLock.lock() statsLock.lock()
clipboardLock.lock() clipboardLock.lock()
cursorLock.lock()
abiLock.lock() abiLock.lock()
let h = handle let h = handle
handle = nil handle = nil
abiLock.unlock() abiLock.unlock()
cursorLock.unlock()
clipboardLock.unlock() clipboardLock.unlock()
statsLock.unlock() statsLock.unlock()
feedbackLock.unlock() feedbackLock.unlock()
@@ -110,11 +110,11 @@ public final class InputCapture {
/// event itself is swallowed). Main queue. /// event itself is swallowed). Main queue.
public var onToggleCapture: (() -> Void)? public var onToggleCapture: (() -> Void)?
/// Fired on M (the mouse-model flip, capture desktop cross-client parity with the /// Fired on C (the client-side-cursor toggle flips between the captured/disassociated
/// SDL clients' Ctrl+Alt+Shift+M; detected here, like , so it works regardless of the /// relative path and the visible-cursor absolute path; detected here, like , so it works
/// current capture state and the event itself is swallowed). macOS only; the /// regardless of the current capture state and the event itself is swallowed). macOS only;
/// absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue. /// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleMouseMode: (() -> Void)? public var onToggleCursor: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS /// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which /// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
@@ -245,14 +245,13 @@ public final class InputCapture {
self.onToggleCapture?() self.onToggleCapture?()
return nil return nil
} }
// M flips the mouse model (capture desktop the SDL clients' identical // C toggles the client-side cursor (visible-cursor absolute path vs the
// chord). Detected in both capture states, like , so the model can be set // captured relative path). keyCode 8 = kVK_ANSI_C; layout-independent so it
// before engaging. keyCode 46 = kVK_ANSI_M; layout-independent. Suppress the M // fires the same on any keyboard. Suppress the C (latched like 's Esc) so it
// (latched like 's Esc) so it doesn't type into the host, and swallow the // doesn't type into the host, and swallow the event so it doesn't beep.
// event so it doesn't beep. if event.keyCode == 8 /* C */, flags == [.command, .shift] {
if event.keyCode == 46 /* M */, flags == [.control, .option, .shift] { self.suppressedVK = 0x43 // VK_C the same physical C is en route via GC
self.suppressedVK = 0x4D // VK_M the same physical M is en route via GC self.onToggleCursor?()
self.onToggleMouseMode?()
return nil return nil
} }
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other // The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
@@ -1,12 +0,0 @@
/// How a physical mouse drives the host the cross-client mouse model (the SDL clients'
/// `MouseMode` / `Settings::mouse_mode`, design/remote-desktop-sweep.md M1). Stored stringly
/// under `DefaultsKey.mouseMode`.
public enum MouseInputMode: String, CaseIterable, Sendable {
/// Pointer capture (disassociated, hidden cursor, relative deltas) the game model,
/// and the default: the only cursor you see is the host's.
case capture
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion is forwarded as absolute positions through the letterbox. The remote desktop
/// model. Requires a host injector with absolute support (not gamescope).
case desktop
}
@@ -23,34 +23,6 @@ import os
private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter") private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
#if os(macOS)
/// HOW a windowed (composited) macOS session pushes finished frames to glass the DCP
/// "mismatched swapID's" kernel-panic saga's mechanism picker. Fullscreen always presents
/// `async` (direct-scanout promotion, lowest latency, no panic reports there); the windowed
/// mechanism is resolved per session by SessionPresenter (user setting +
/// PUNKTFUNK_WINDOWED_PRESENT env override) and routed here via `setWindowedPresent`.
///
/// - `async`: the CAMetalLayer image queue (`commandBuffer.present`) the fastest composited
/// path and the PANIC TRIGGER on high-refresh displays (the out-of-band swaps race
/// WindowServer's compositor; it survived glass pacing and every codec).
/// - `transaction`: `CAMetalLayer.presentsWithTransaction` the swap commits WITH the layer
/// tree, in lockstep with the compositor (Apple's documented remedy; validated no-panic on
/// the 240 Hz repro machine). The present is committed from the RENDER thread inside an
/// explicit CATransaction + flush see `encodePresent` for why that beats the original
/// main-thread hop.
/// - `surface`: no image queue at all render into a pooled IOSurface and swap it into a plain
/// CALayer's `contents` (the f407f418 PyroWave mitigation, resurrected format-aware:
/// rgba16Float + PQ tagging keeps HDR). WindowServer treats it as ordinary layer damage on
/// its own composite cadence. PROTOTYPE: whether the compositor honors PQ/EDR for plain-layer
/// IOSurface contents still needs an on-glass eyeball the metal layer stays underneath with
/// `wantsExtendedDynamicRangeContent` as the EDR anchor.
enum WindowedPresentMode: String, Sendable {
case async
case transaction
case surface
}
#endif
/// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the /// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors /// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the /// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
@@ -226,8 +198,8 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.) // in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
// The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084 // The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
// EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff // EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff
// BT.709 OETF. Used by the tvOS biplanar tone-map and the tvOS planar (PyroWave) tone-map (the // BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map the
// no-HDR-headroom fallback). macOS keeps real HDR windowed now see `WindowedPresentMode`. // latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only.
static inline float3 pqToSdr(float3 pq) { static inline float3 pqToSdr(float3 pq) {
const float m1 = 2610.0/16384.0; const float m1 = 2610.0/16384.0;
const float m2 = 78.84375; const float m2 = 78.84375;
@@ -258,7 +230,8 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows // PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
// fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave // fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave
// stream must land on an 8-bit surface: tvOS without HDR headroom. The passthrough planar // stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions
// (the IOSurface present path the DCP-panic mitigation is BGRA8). The passthrough planar
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the // HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation). // layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]], fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
@@ -286,51 +259,21 @@ public final class MetalVideoPresenter {
public let layer: CAMetalLayer public let layer: CAMetalLayer
#if os(macOS) #if os(macOS)
/// WINDOWED-mode present coordination the macOS DCP KERNEL PANIC mitigation. /// The WINDOWED-mode PyroWave present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it), fed IOSurfaces via `contents` inside ordinary CATransactions.
/// ///
/// The panic ("mismatched swapID's" @UnifiedPipeline.cpp, WindowServer dies, machine reboots): /// Why this exists the macOS DCP KERNEL PANIC ("mismatched swapID's" @UnifiedPipeline.cpp,
/// the CAMetalLayer's ASYNCHRONOUS image queue (`commandBuffer.present(drawable)` an /// WindowServer dies, machine reboots): out-of-band CAMetalLayer image-queue swaps into a
/// out-of-band flip, mandatory with `displaySyncEnabled=false`) diverges from WindowServer's /// COMPOSITED (windowed) session race WindowServer's own swap submissions on high-refresh
/// compositor on a high-refresh COMPOSITED (windowed) session the compositor's notion of the /// displays, and the race survives glass pacing a fully serialized one-in-flight present
/// current swap and the layer's queued swap disagree, and the DCP asserts. It survived glass /// stream still panicked a 240 Hz Mac Studio (2026-07-18, twice). So in windowed mode we stop
/// pacing: a fully serialized one-in-flight present stream still panicked a 240 Hz Mac Studio /// using the image queue entirely and present the way video players do: render the planar CSC
/// (2026-07-18, PyroWave), and a windowed HEVC session panicked the same machine 2026-07-21 /// into an IOSurface pool and swap `contents` on main WindowServer treats it as ordinary
/// so it is the async image queue itself, at any pacing or codec, not a present rate. /// damage on its own composite cadence, coalescing faster-than-refresh updates instead of
/// /// latching queue swaps mid-cycle. Fullscreen keeps the CAMetalLayer path (direct-scanout
/// The fix keeps the full render path (rgba16Float / PQ / EDR real HDR is preserved) and /// promotion, no compositing, no panic reports). Contents updates are transparent to the
/// only changes HOW the drawable is presented: `CAMetalLayer.presentsWithTransaction`. With it /// layer below when nil, so flipping modes just covers/uncovers the metal layer.
/// set, we don't hand the drawable to the command buffer; we commit, wait until scheduled, then public let surfaceLayer: CALayer = {
/// call `drawable.present()` INSIDE a CATransaction the present is enrolled in Core
/// Animation's transaction and committed together with the layer tree, so the swap stays in
/// lockstep with the compositor instead of racing it (Apple's documented remedy for Metal
/// presentation drifting out of sync with CA). Fullscreen keeps the async path (direct-scanout
/// promotion, lowest latency, no compositor and no panic reports there).
///
/// 2026-07-21 latency rework: the mitigation MECHANISM is now a three-way pick
/// (`WindowedPresentMode`) and the transactional present commits from the RENDER thread
/// see `encodePresent`. Staged under `stagingLock` (main pushes it via
/// `setComposited``setWindowedPresent`); the render thread drains it and toggles the layer
/// property + present style. `Active` is the render-thread copy so the layer property flips
/// exactly once per mode change.
private var windowedPresentStaged: WindowedPresentMode = .async
private var windowedPresentActive: WindowedPresentMode = .async
/// PUNKTFUNK_TXN_PRESENT=main the ORIGINAL transactional present (commit
/// waitUntilScheduled hop to the MAIN thread and present inside its CATransaction), kept
/// as a field A/B lever. The default is the render-thread commit: the present harness
/// (2026-07-21, this saga) measured the main hop landing a runloop turn late on a busy main
/// thread, and an ACTIVE implicit transaction there NESTS the explicit one presents batch
/// at runloop-iteration rate (the field's presents=55 @ fps=240, display_p50 18.6 ms).
/// Off-main commits measured immune to main-thread churn (~10 ms glass p50 at 240 Hz
/// full-size vs 14+ ms under a churned main hop).
private let txnPresentOnMain =
ProcessInfo.processInfo.environment["PUNKTFUNK_TXN_PRESENT"] == "main"
/// The WINDOWED-mode `surface` present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it by SessionPresenter), fed IOSurfaces via `contents` inside explicit
/// CATransactions. Transparent (nil contents) whenever surface mode is off, so the metal
/// layer below shows through. See `WindowedPresentMode.surface`.
let surfaceLayer: CALayer = {
let l = CALayer() let l = CALayer()
l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout
l.isOpaque = true l.isOpaque = true
@@ -338,8 +281,8 @@ public final class MetalVideoPresenter {
return l return l
}() }()
/// One IOSurface-backed render target of the windowed surface-present pool. All pool state /// One IOSurface-backed render target of the windowed present pool. All pool state is
/// is RENDER-THREAD confined; only the immutable surface refs cross threads (contents swap). /// RENDER-THREAD confined; only the immutable surface refs cross to main (contents swap).
private struct SurfaceSlot { private struct SurfaceSlot {
let surface: IOSurfaceRef let surface: IOSurfaceRef
let texture: MTLTexture let texture: MTLTexture
@@ -349,52 +292,15 @@ public final class MetalVideoPresenter {
private var surfacePool: [SurfaceSlot] = [] private var surfacePool: [SurfaceSlot] = []
private var surfacePoolSize: CGSize = .zero private var surfacePoolSize: CGSize = .zero
private var surfacePoolHDR = false
private var surfaceSeq: UInt64 = 0 private var surfaceSeq: UInt64 = 0
/// Index of the slot most recently handed to the layer never rewritten next, even if its /// Index of the slot most recently handed to the layer never rewritten next, even if its
/// use count already dropped (the compositor may still be scanning out the previous frame). /// use count already dropped (the compositor may still be scanning out the previous frame).
private var lastHandedOff: Int? private var lastHandedOff: Int?
/// Staged (under `stagingLock`, like every cross-thread input): the hosting view's windowed
/// Once-per-second decomposition of the ACTIVE windowed present path (the field-diagnosis /// vs fullscreen state, pushed from main via `setSurfacePresents`. Drained in `renderPlanar`.
/// half of the DCP-latency work): scheduled/completed wait + commit/flush cost per present, private var surfacePresentsStaged = false
/// and how many presents/swaps were issued. The pf-present line shows the GLASS side /// Render-thread copy, so pool teardown happens exactly once on a mode flip.
/// (latchMs / dropped); this shows the ISSUE side. Logged via `presenterLog` only while a private var surfacePresentsActive = false
/// windowed mechanism is active (zero cost fullscreen). Lock-guarded: transaction mode
/// records from the render thread, surface mode from Metal completion threads.
private final class WindowedPresentDiag: @unchecked Sendable {
private let lock = NSLock()
private var presents = 0
private var schedMs: [Double] = []
private var commitMs: [Double] = []
private var last = CACurrentMediaTime()
func record(schedMs sched: Double, commitMs commit: Double, mode: WindowedPresentMode) {
lock.lock()
presents += 1
schedMs.append(sched)
commitMs.append(commit)
let now = CACurrentMediaTime()
guard now - last >= 1 else {
lock.unlock()
return
}
last = now
let sSched = schedMs.sorted()
let sCommit = commitMs.sorted()
let line = String(
format: "pf-windowed mode=%@ presents=%d schedMs p50=%.2f max=%.2f "
+ "commitMs p50=%.2f max=%.2f",
mode.rawValue, presents, sSched[sSched.count / 2], sSched.last ?? 0,
sCommit[sCommit.count / 2], sCommit.last ?? 0)
presents = 0
schedMs.removeAll(keepingCapacity: true)
commitMs.removeAll(keepingCapacity: true)
lock.unlock()
presenterLog.info("\(line, privacy: .public)")
}
}
private let windowedDiag = WindowedPresentDiag()
#endif #endif
private let device: MTLDevice private let device: MTLDevice
@@ -410,7 +316,7 @@ public final class MetalVideoPresenter {
private let pipelinePlanar: MTLRenderPipelineState private let pipelinePlanar: MTLRenderPipelineState
/// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour /// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour
/// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm /// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm
/// bgra8; tvOS without HDR headroom). /// bgra8; tvOS without headroom + macOS windowed IOSurface presents).
private let pipelinePlanarHDR: MTLRenderPipelineState private let pipelinePlanarHDR: MTLRenderPipelineState
private let pipelinePlanarToneMap: MTLRenderPipelineState private let pipelinePlanarToneMap: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache? private var textureCache: CVMetalTextureCache?
@@ -685,14 +591,13 @@ public final class MetalVideoPresenter {
} }
#if os(macOS) #if os(macOS)
/// Park the windowed present mechanism (MAIN thread the hosting view pushes its window /// Park the windowed-vs-fullscreen present routing (MAIN thread the hosting view pushes its
/// state on every layout; SessionPresenter resolves the mechanism per session). `.async` = /// window state on every layout). true = PyroWave frames present via `surfaceLayer` contents
/// FULLSCREEN (or the user opted out of the mitigation): the image queue. `.transaction` / /// (the DCP swapID-panic mitigation see `surfaceLayer`); false = the CAMetalLayer path.
/// `.surface` = COMPOSITED (windowed) mitigation mechanisms see `WindowedPresentMode`.
/// Applied by the render thread on the next frame, like every other staged value here. /// Applied by the render thread on the next frame, like every other staged value here.
func setWindowedPresent(_ mode: WindowedPresentMode) { public func setSurfacePresents(_ on: Bool) {
stagingLock.lock() stagingLock.lock()
windowedPresentStaged = mode surfacePresentsStaged = on
stagingLock.unlock() stagingLock.unlock()
} }
#endif #endif
@@ -829,12 +734,36 @@ public final class MetalVideoPresenter {
) -> Bool { ) -> Bool {
stagingLock.lock() stagingLock.lock()
let targetFromLayout = drawableTarget let targetFromLayout = drawableTarget
#if os(macOS)
let surfaceMode = surfacePresentsStaged
#endif
stagingLock.unlock() stagingLock.unlock()
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path // A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path;
// including macOS windowed sessions, which keep real HDR (the DCP mitigation is the // macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader).
// transactional present in `encodePresent`, not a colour downgrade). #if os(macOS)
configure(hdr: planes.pq && !surfaceMode)
#else
configure(hdr: planes.pq) configure(hdr: planes.pq)
#endif
var csc = planes.csc var csc = planes.csc
#if os(macOS)
if surfaceMode != surfacePresentsActive {
surfacePresentsActive = surfaceMode
presenterLog.info(
"stage2: windowed surface presents \(surfaceMode ? "ON" : "OFF", privacy: .public) (PyroWave DCP-panic mitigation)")
if !surfaceMode {
// Back to the metal path (fullscreen): drop the pool at 5K it holds >100 MB,
// and re-entering windowed mode rebuilds it in one frame.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
}
if surfaceMode {
return renderPlanarToSurface(
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
}
#endif
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still) // PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
// 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps // 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps
// in-shader instead (the pipeline must match the drawable's pixel format). // in-shader instead (the pipeline must match the drawable's pixel format).
@@ -863,6 +792,118 @@ public final class MetalVideoPresenter {
} }
} }
#if os(macOS)
/// The windowed-mode present tail (see `surfaceLayer` for why this path exists): render the
/// planar CSC into a pooled IOSurface and hand it to `surfaceLayer.contents` on MAIN inside a
/// plain CATransaction an ordinary damaged-layer update on WindowServer's own composite
/// cadence, no CAMetalLayer image-queue swap anywhere. `presentAtMediaTime` doesn't apply
/// (the compositor paces); `onPresented` fires after the contents swap is committed, stamped
/// with CLOCK_REALTIME then the closest observable analogue of "reached glass" here (the
/// composite follows within a refresh, so the meters' display stage reads slightly optimistic).
private func renderPlanarToSurface(
_ planes: WaveletPlanes, targetFromLayout: CGSize, csc: inout CscUniform,
onPresented: ((Int64?) -> Void)?
) -> Bool {
let decodedSize = CGSize(width: planes.width, height: planes.height)
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
ensureSurfacePool(size: targetSize)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let keepAlive: [Any] = [planes.y, planes.cb, planes.cr]
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // ring textures pinned until the GPU finished sampling
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
onPresented?(
Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size` 4 BGRA8 IOSurface render targets (one on glass, one queued
/// in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the pool empty;
/// the caller returns false and the ring's putBack + display-link retry take over.
private func ensureSurfacePool(size: CGSize) {
guard size != surfacePoolSize else { return }
surfacePool.removeAll()
surfacePoolSize = size
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// 256-byte row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerRow = ((w * 4) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: 4,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one /// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one
/// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule /// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule
/// the present and the on-glass callback. /// the present and the on-glass callback.
@@ -895,36 +936,6 @@ public final class MetalVideoPresenter {
#if DEBUG #if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize) logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
#endif #endif
#if os(macOS)
// Windowed (composited) the DCP swapID-panic mitigation mechanism (see
// `WindowedPresentMode`). Toggle the layer property BEFORE vending a drawable so the
// vend matches how it will be presented; drained here on the render thread, flipped
// exactly once per mode change.
stagingLock.lock()
let windowedMode = windowedPresentStaged
stagingLock.unlock()
if windowedMode != windowedPresentActive {
windowedPresentActive = windowedMode
layer.presentsWithTransaction = windowedMode == .transaction
if windowedMode != .surface, !surfacePool.isEmpty {
// Leaving surface mode (fullscreen entry / mechanism A/B): drop the pool at 5K
// it holds >100 MB, and re-entering rebuilds it in one frame. SessionPresenter
// clears the surface layer's contents on main.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
presenterLog.info(
"stage2: windowed present mode \(windowedMode.rawValue, privacy: .public) (DCP swapID-panic mitigation)")
}
if windowedMode == .surface {
// No image queue at all: render into a pooled IOSurface and swap it into the
// sibling layer's contents. The drawable/queue tail below never runs.
return encodeToSurface(
targetSize: targetSize, pipeline: pipeline, onPresented: onPresented,
keepAlive: keepAlive, bind: bind)
}
#endif
if let providedDrawable, if let providedDrawable,
providedDrawable.texture.pixelFormat != layer.pixelFormat { providedDrawable.texture.pixelFormat != layer.pixelFormat {
return false // config outran the vend (HDR flip) next vend has the new format return false // config outran the vend (HDR flip) next vend has the new format
@@ -963,61 +974,6 @@ public final class MetalVideoPresenter {
} }
#endif #endif
} }
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
#if os(macOS)
if windowedPresentActive == .transaction {
// Windowed DCP mitigation: present the drawable THROUGH a Core Animation transaction
// (`presentsWithTransaction`, set above) instead of the async image queue, so the swap
// commits with the layer tree and stays in lockstep with the compositor (no out-of-band
// flip to race WindowServer's swaps). Wait until the GPU work is scheduled (contents
// will be ready p50 ~0.1 ms), then present inside an EXPLICIT CATransaction ON THIS
// RENDER THREAD and `flush()`. `presentAtMediaTime` does not apply the transaction
// paces.
//
// Threading history, because BOTH failure modes shipped or nearly shipped:
// A bare `present()` from this thread (no transaction) never flushes nothing
// commits a runloop-less thread's implicit transaction, so drawables are never
// released; after maximumDrawableCount vends `nextDrawable()` blocks forever and
// the stream FREEZES (the fullscreenwindowed switch did exactly this).
// The explicit begin/commit alone is NOT enough either: this thread has an ACTIVE
// implicit transaction (the layer mutations above drawableSize/colour created
// it), so the explicit transaction NESTS inside it and its commit defers to the
// implicit one that never comes. The harness reproduced the exact freeze: every
// present reported presentedTime=0, nothing reached glass. `CATransaction.flush()`
// pushes the implicit transaction (present included) to the render server NOW.
// The original fix hopped to MAIN and presented there correct, but slow in the
// field (presents=55 @ fps=240, display_p50 18.6 ms on the 240 Hz Studio): each
// present lands a runloop turn late, and main's own implicit transaction batches
// enrolled presents at runloop-iteration rate. Kept as PUNKTFUNK_TXN_PRESENT=main.
// The off-main commit measured immune to main-thread churn in the harness
// (2026-07-21: glass p50 ~10 ms at 240 Hz full-size, cadence a clean 4.17 ms).
commandBuffer.commit()
let schedStart = CACurrentMediaTime()
commandBuffer.waitUntilScheduled()
let schedMs = (CACurrentMediaTime() - schedStart) * 1000
let commitStart = CACurrentMediaTime()
if txnPresentOnMain {
let presentedDrawable = drawable
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
presentedDrawable.present()
CATransaction.commit()
}
} else {
CATransaction.begin()
CATransaction.setDisableActions(true)
drawable.present()
CATransaction.commit()
CATransaction.flush()
}
windowedDiag.record(
schedMs: schedMs, commitMs: (CACurrentMediaTime() - commitStart) * 1000,
mode: .transaction)
return true
}
#endif
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment); // Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing. // immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime { if let presentAtMediaTime {
@@ -1025,146 +981,12 @@ public final class MetalVideoPresenter {
} else { } else {
commandBuffer.present(drawable) commandBuffer.present(drawable)
} }
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
commandBuffer.commit() commandBuffer.commit()
return true return true
} }
#if os(macOS)
/// The WINDOWED `surface` present tail (see `WindowedPresentMode.surface`): render with the
/// same per-frame pipeline into a pooled IOSurface and hand it to `surfaceLayer.contents`
/// from the command buffer's COMPLETION handler, inside an explicit CATransaction + flush
/// (the same off-main commit discipline as the transactional present an ordinary
/// damaged-layer update on WindowServer's own composite cadence, no image queue anywhere).
/// RENDER THREAD. `onPresented` is stamped right after the contents swap commits the
/// closest observable analogue of "reached glass" here (the composite follows within a
/// refresh, so the display-stage meters read slightly OPTIMISTIC in this mode).
///
/// The pool tracks `hdrActive`: bgra8 for SDR, rgba16Float tagged BT.2100 PQ for HDR
/// `configure` already ran, so the caller's `pipeline` attachment format always matches.
/// HDR OPEN RISK (why this whole mode is a prototype): whether the compositor honors the
/// PQ tag + EDR for plain-CALayer IOSurface contents needs an on-glass eyeball; the metal
/// layer underneath keeps `wantsExtendedDynamicRangeContent` as the EDR anchor (the harness
/// measured the display's EDR headroom engaging with this arrangement).
private func encodeToSurface(
targetSize: CGSize, pipeline: MTLRenderPipelineState,
onPresented: ((Int64?) -> Void)?,
keepAlive: [Any], bind: (MTLRenderCommandEncoder) -> Void
) -> Bool {
ensureSurfacePool(size: targetSize, hdr: hdrActive)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(pipeline)
bind(encoder)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let diag = windowedDiag
let commitStamp = CACurrentMediaTime()
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // sources pinned until the GPU finished sampling
let completedAt = CACurrentMediaTime()
// Swap on THIS Metal completion thread: explicit transaction + flush, so the commit
// reaches the render server now, independent of main (completion handlers for one
// queue fire in execution order, so swaps can't reorder).
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
CATransaction.flush()
diag.record(
schedMs: (completedAt - commitStamp) * 1000,
commitMs: (CACurrentMediaTime() - completedAt) * 1000, mode: .surface)
onPresented?(Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size`/`hdr` 4 IOSurface render targets (one on glass, one
/// committed in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the
/// pool empty; the caller returns false and the ring's putBack + display-link retry take
/// over.
private func ensureSurfacePool(size: CGSize, hdr: Bool) {
guard size != surfacePoolSize || hdr != surfacePoolHDR else { return }
surfacePool.removeAll()
surfacePoolSize = size
surfacePoolHDR = hdr
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// rgba16Float (8 B/px) carries the PQ-encoded HDR samples; bgra8 the SDR ones. 256-byte
// row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerElement = hdr ? 8 : 4
let bytesPerRow = ((w * bytesPerElement) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: bytesPerElement,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: hdr
? kCVPixelFormatType_64RGBAHalf : kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: hdr ? .rgba16Float : .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
if hdr, let name = CGColorSpace(name: CGColorSpace.itur_2100_PQ)?.name {
// Tag the surface BT.2100 PQ so the compositor interprets the half-float
// samples as PQ-encoded HDR (the CALayer-contents analogue of the metal
// layer's colorspace).
IOSurfaceSetValue(surface, "IOSurfaceColorSpace" as CFString, name)
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
// The EDR request rides the SURFACE layer too (its contents are what composite); the
// metal layer underneath keeps its own from configureColor as the anchor. Layer flags
// are committed by the next swap's transaction flush.
surfaceLayer.wantsExtendedDynamicRangeContent = hdr
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the /// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// draw the MTLTexture is only valid while its CVMetalTexture is retained. /// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture( private func makeTexture(
@@ -142,17 +142,20 @@ enum PresentPriority: Equatable {
final class SessionPresenter { final class SessionPresenter {
/// Present pacing for this session. Stage-3 always means glass gating; under the stage-2 /// Present pacing for this session. Stage-3 always means glass gating; under the stage-2
/// default, macOS PyroWave sessions ALSO get glass gating for SMOOTHNESS, not as the panic /// default, macOS PyroWave sessions ALSO get glass gating a kernel-panic mitigation, not a
/// fix (that is the windowed transactional present see `setComposited`). PyroWave's wavelet /// latency tweak. macOS's DCP panics ("mismatched swapID's" @UnifiedPipeline.cpp, the whole
/// decode is near-instant Metal compute, so a network clump presents within the same /// machine dies) when WindowServer's swap submissions race, and the reliable trigger is
/// millisecond, and it is the codec that sustains stream rates above the panel's refresh; the /// out-of-band CAMetalLayer presents (displaySyncEnabled=false mandatory for us, see
/// glass gate admits one presented-but-undisplayed swap at a time (serialized on the on-glass /// MetalVideoPresenter's init) arriving faster than the compositor latches them in a
/// callback, 100 ms stale backstop) so those bursts coalesce in the newest-wins ring instead /// COMPOSITED (windowed) session. Arrival pacing does exactly that with PyroWave: the wavelet
/// of flooding the queue. (Glass pacing was ALSO the original DCP-panic mitigation attempt /// decode is near-instant Metal compute, so a network clump of frames presents within the
/// disproven: a fully serialized stream still panicked, which is why the real fix moved to the /// same millisecond, and PyroWave is the codec that sustains stream rates above the panel's
/// present mechanism.) An explicit stage-2 pick (setting/env) still forces arrival pacing /// refresh. The glass gate admits one presented-but-undisplayed swap at a time (serialized on
/// that A/B lever must stay honest. VideoToolbox codecs keep arrival pacing: decode latency /// the on-glass callback, 100 ms stale backstop), which removes the racing pattern outright;
/// spaces their presents. /// frames the panel couldn't have shown anyway coalesce in the newest-wins ring. An explicit
/// stage-2 pick (setting/env) still forces arrival pacing that A/B lever must stay honest.
/// VideoToolbox codecs keep arrival pacing: decode latency spaces their presents, and years
/// of stage-2 defaults there predate any panic report.
static func pacing( static func pacing(
for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec
) -> PresentPacing { ) -> PresentPacing {
@@ -175,25 +178,10 @@ final class SessionPresenter {
/// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing /// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing
/// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate. /// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate.
/// ///
#if os(macOS)
/// Resolve the windowed (composited) present MECHANISM for this session the DCP
/// swapID-panic mitigation picker (see `WindowedPresentMode`). The
/// `PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface` env lever wins (dev A/B);
/// otherwise the user's safe-present setting: ON/unset `.transaction` (the validated
/// mitigation), OFF `.async` (the fast pre-mitigation path the panic returns on
/// affected high-refresh setups; the Settings caption says so). `.surface` is currently
/// env-only (prototype HDR-composite verification owed). Fullscreen always presents
/// async regardless (`setComposited`). Internal (not private) for unit tests.
static func windowedPresentMode(setting: Bool?, env: String?) -> WindowedPresentMode {
if let env, let mode = WindowedPresentMode(rawValue: env) { return mode }
return (setting ?? true) ? .transaction : .async
}
#endif
/// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder /// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder
/// stays reproducible on-device; macOS is pinned to 1, env ignored a deeper gate only builds /// stays reproducible on-device; macOS is pinned to 1, env ignored glass pacing exists
/// a standing queue (see above), and macOS glass pacing exists for PyroWave smoothness /// there as the DCP swapID kernel-panic mitigation (see `pacing`), and STRICT present
/// (see `pacing`), where depth 1 is the point. Internal (not private) for unit tests. /// serialization is its point. Internal (not private) for unit tests.
static func gateDepth(env: String?) -> Int { static func gateDepth(env: String?) -> Int {
#if os(macOS) #if os(macOS)
return 1 return 1
@@ -208,16 +196,10 @@ final class SessionPresenter {
private var stage2Link: CADisplayLink? private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer? private var metalLayer: CAMetalLayer?
#if os(macOS) #if os(macOS)
/// The windowed present MECHANISM this session runs while composited (resolved once per /// The windowed-mode PyroWave present target (sibling above `metalLayer`) and the last
/// session in `start` the user's safe-present setting + the PUNKTFUNK_WINDOWED_PRESENT /// routing pushed to the pipeline see `setComposited`. Main-thread only, like all of this.
/// dev override) and the routing last pushed to the pipeline see `setComposited` (the DCP
/// swapID-panic mitigation). Main-thread only, like all of this.
private var windowedMode: WindowedPresentMode = .transaction
private var windowedPresentApplied: WindowedPresentMode = .async
/// The windowed `surface` present target (sibling above `metalLayer`, transparent while
/// unused) installed whenever stage-2 runs so a mechanism flip never has to mutate the
/// layer tree mid-session.
private var surfaceLayer: CALayer? private var surfaceLayer: CALayer?
private var surfacePresentsActive = false
#endif #endif
private var connection: PunktfunkConnection? private var connection: PunktfunkConnection?
/// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's /// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's
@@ -301,17 +283,11 @@ final class SessionPresenter {
baseLayer.addSublayer(metal) baseLayer.addSublayer(metal)
metalLayer = metal metalLayer = metal
#if os(macOS) #if os(macOS)
windowedPresentApplied = .async // The windowed-PyroWave present target sits ABOVE the metal layer: transparent (nil
// Resolve THIS session's windowed mechanism once (setting + dev env lever) // contents) while the metal path presents, covering it while surface presents run.
// `setComposited` routes between it and fullscreen-async from every layout.
windowedMode = Self.windowedPresentMode(
setting: UserDefaults.standard.object(
forKey: DefaultsKey.windowedSafePresent) as? Bool,
env: ProcessInfo.processInfo.environment["PUNKTFUNK_WINDOWED_PRESENT"])
// The surface present target sits ABOVE the metal layer: transparent (nil contents)
// unless the surface mechanism actually presents, covering it while it does.
baseLayer.addSublayer(pipeline.surfaceLayer) baseLayer.addSublayer(pipeline.surfaceLayer)
surfaceLayer = pipeline.surfaceLayer surfaceLayer = pipeline.surfaceLayer
surfacePresentsActive = false
#endif #endif
stage2 = pipeline stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger // The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
@@ -456,23 +432,19 @@ final class SessionPresenter {
#if os(macOS) #if os(macOS)
/// Route presents for the window's composited state (MAIN thread the view pushes it on /// Route presents for the window's composited state (MAIN thread the view pushes it on
/// every layout, which fullscreen transitions always trigger). A COMPOSITED (windowed) /// every layout, which fullscreen transitions always trigger). PyroWave sessions in a
/// session presents through this session's resolved mitigation mechanism (`windowedMode` /// COMPOSITED (windowed) session present via `surfaceLayer` contents instead of the
/// transactional by default, see `windowedPresentMode`) instead of the async image queue /// CAMetalLayer image queue the DCP "mismatched swapID's" kernel-panic mitigation (see
/// the DCP "mismatched swapID's" kernel-panic mitigation (see `MetalVideoPresenter`; the /// `MetalVideoPresenter.surfaceLayer`; the metal-swap race survives glass pacing, so pacing
/// async-swap race survives glass pacing, so pacing alone was not enough). ALL codecs: /// alone was not enough). VT codecs keep the metal path: no panic reports there, and their
/// PyroWave hit it 2026-07-18 and windowed HEVC hit the same 240 Hz Mac Studio 2026-07-21 /// HDR/EDR presentation has no surface-contents equivalent wired.
/// it is the async image queue itself, not any codec or present rate. Fullscreen keeps the
/// async path (direct scanout, lowest latency, no panic there). The full HDR/EDR render
/// path is preserved in every mechanism.
func setComposited(_ composited: Bool) { func setComposited(_ composited: Bool) {
guard let stage2 else { return } guard let stage2, let connection else { return }
let mode: WindowedPresentMode = composited ? windowedMode : .async let wantsSurface = composited && connection.videoCodec == .pyrowave
guard mode != windowedPresentApplied else { return } guard wantsSurface != surfacePresentsActive else { return }
let wasSurface = windowedPresentApplied == .surface surfacePresentsActive = wantsSurface
windowedPresentApplied = mode stage2.setSurfacePresents(wantsSurface)
stage2.setWindowedPresent(mode) if !wantsSurface {
if wasSurface {
// Uncover the metal layer NOW (its last drawable is still attached, so fullscreen // Uncover the metal layer NOW (its last drawable is still attached, so fullscreen
// entry shows the previous frame until the next present no black flash). // entry shows the previous frame until the next present no black flash).
CATransaction.begin() CATransaction.begin()
@@ -499,7 +471,7 @@ final class SessionPresenter {
#if os(macOS) #if os(macOS)
surfaceLayer?.removeFromSuperlayer() surfaceLayer?.removeFromSuperlayer()
surfaceLayer = nil surfaceLayer = nil
windowedPresentApplied = .async surfacePresentsActive = false
#endif #endif
connection = nil connection = nil
} }
@@ -1114,15 +1114,15 @@ public final class Stage2Pipeline {
} }
#if os(macOS) #if os(macOS)
/// Forward the windowed present mechanism (MAIN thread see /// The windowed-mode PyroWave present target (see `MetalVideoPresenter.surfaceLayer` the
/// `MetalVideoPresenter.setWindowedPresent`, the DCP swapID-panic mitigation). /// DCP swapID-panic mitigation). The hosting view installs it as a sibling above `layer`.
func setWindowedPresent(_ mode: WindowedPresentMode) { public var surfaceLayer: CALayer { presenter.surfaceLayer }
presenter.setWindowedPresent(mode)
}
/// The windowed `surface` present target the hosting SessionPresenter installs as a sibling /// Forward the windowed-vs-fullscreen present routing (MAIN thread see
/// ABOVE `layer` (transparent while unused see `MetalVideoPresenter.surfaceLayer`). /// `MetalVideoPresenter.setSurfacePresents`).
var surfaceLayer: CALayer { presenter.surfaceLayer } public func setSurfacePresents(_ on: Bool) {
presenter.setSurfacePresents(on)
}
#endif #endif
/// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen` /// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen`
@@ -1213,9 +1213,7 @@ public final class Stage2Pipeline {
let chunkAligned = let chunkAligned =
au.flags & PunktfunkConnection.userFlagChunkAligned != 0 au.flags & PunktfunkConnection.userFlagChunkAligned != 0
let ptsNs = au.ptsNs let ptsNs = au.ptsNs
// Decode stage starts at the PULL (matching the VT path's FrameContext let receivedNs = au.receivedNs
// receiptpull is the HUD's separate client-queue term, ABI v9 split).
let receivedNs = au.pulledNs
let flags = au.flags let flags = au.flags
let submitted = decoder.decode( let submitted = decoder.decode(
au: au.data, chunkAligned: chunkAligned, windowSize: windowSize au: au.data, chunkAligned: chunkAligned, windowSize: windowSize
@@ -32,12 +32,9 @@ public enum ReadyImage: @unchecked Sendable {
public struct ReadyFrame: @unchecked Sendable { public struct ReadyFrame: @unchecked Sendable {
/// Host capture clock (the AU's pts), in nanoseconds. /// Host capture clock (the AU's pts), in nanoseconds.
public let ptsNs: UInt64 public let ptsNs: UInt64
/// Client `CLOCK_REALTIME` instant the AU left `nextAU` (`AccessUnit.pulledNs`, threaded /// Client `CLOCK_REALTIME` instant the AU was received (`AccessUnit.receivedNs`, threaded
/// through the decode via the frame refcon), in nanoseconds the decode stage's start /// through the decode via the frame refcon), in nanoseconds. 0 when unknown (a caller that
/// point. (Named for its historical role; since the ABI v9 receipt split the true /// didn't stamp receipt) the decode-stage meter then drops the sample via its sanity guard.
/// reassembly receipt lives on `AccessUnit.receivedNs`, and receiptpull is the HUD's own
/// client-queue term.) 0 when unknown (a caller that didn't stamp) the decode-stage meter
/// then drops the sample via its sanity guard.
public let receivedNs: Int64 public let receivedNs: Int64
/// Client `CLOCK_REALTIME` instant decode completed, in nanoseconds. /// Client `CLOCK_REALTIME` instant decode completed, in nanoseconds.
public let decodedNs: Int64 public let decodedNs: Int64
@@ -170,11 +167,7 @@ public final class VideoDecoder: @unchecked Sendable {
var infoOut = VTDecodeInfoFlags() var infoOut = VTDecodeInfoFlags()
// The AU's receipt instant + wire flags ride through as a retained context; the output // The AU's receipt instant + wire flags ride through as a retained context; the output
// callback reclaims it. Retain immediately before submit so no early return can leak it. // callback reclaims it. Retain immediately before submit so no early return can leak it.
// The decode stage starts at the PULL (the AU leaving nextAU), not the reassembly let ctx = FrameContext(receivedNs: au.receivedNs, flags: au.flags)
// receipt: both consumers the decode-stage meter and the ABR decode signal are
// specified from the pull, and the receiptpull wait is the HUD's separate client-queue
// term (see AccessUnit.pulledNs).
let ctx = FrameContext(receivedNs: au.pulledNs, flags: au.flags)
let refcon = Unmanaged.passRetained(ctx).toOpaque() let refcon = Unmanaged.passRetained(ctx).toOpaque()
let status = VTDecompressionSessionDecodeFrame( let status = VTDecompressionSessionDecodeFrame(
session, session,
@@ -1,268 +0,0 @@
// Apple Pencil state-full wire pen samples (design/pen-tablet-input.md §7).
//
// Every sample carries the COMPLETE pen state (in-range/touching/buttons + all axes) the
// host diffs consecutive samples and synthesizes down/up/button transitions itself, so a lost
// datagram self-heals and this file never sends edge events. Three sources feed one stream:
// UITouch contacts (with coalesced samples for full 240 Hz fidelity), the hover gesture
// (zOffset > 0 distinguishes a hovering Pencil from a trackpad pointer), and
// UIPencilInteraction (squeeze held barrel 1, double-tap a momentary barrel 2
// Apple Pencil has no hardware eraser end or barrel buttons, so these mappings are how
// host-side apps get their stylus button/eraser affordances).
//
// HEARTBEAT (wire contract see `PunktfunkPenSample` in punktfunk_core.h): while the pen is
// in range or touching, the last sample repeats every 100 ms even when nothing changed.
// UIKit is silent for a stationary Pencil, and the host force-releases the stroke after
// 200 ms without samples (its dead-client failsafe) the timer keeps a held stroke alive.
#if os(iOS)
import PunktfunkCore
import UIKit
final class PencilStream: NSObject, UIPencilInteractionDelegate {
enum Phase { case down, move, up, cancel }
/// One assembled batch ( `PUNKTFUNK_PEN_BATCH_MAX` samples) ready for the connection.
var send: (([PunktfunkPenSample]) -> Void)?
/// View-space point normalized [0,1] video coordinates (the letterbox mapping the
/// touch path already uses). nil until a mode is negotiated samples are dropped then.
var videoNorm: ((CGPoint) -> (Float, Float)?)?
private var inRange = false
private var touching = false
/// Squeeze held (mapped to wire BARREL1).
private var squeezeHeld = false
/// Whether this device/Pencil pair has demonstrated hover decides what a lift means:
/// hover-capable hardware keeps proximity (the hover recognizer owns the exit), anything
/// else leaves range on lift so the host never parks a phantom hovering pen.
private var sawHover = false
/// A hover gesture is live right now (routes ended-state hover callbacks to us even when
/// the recognizer's final zOffset reads 0).
private(set) var hoverActive = false
private var last = PencilStream.idleSample()
private var heartbeat: Timer?
// MARK: - Contact path (UITouch, `.pencil` only)
func touches(_ touches: Set<UITouch>, event: UIEvent?, phase: Phase, in view: UIView) {
// At most one Pencil exists; a set with several is UIKit batching phases of the same
// stylus the last one carries the freshest state.
guard let touch = touches.max(by: { $0.timestamp < $1.timestamp }) else { return }
switch phase {
case .down, .move:
touching = true
inRange = true
// Coalesced samples restore the Pencil's full capture rate (UIKit delivers at
// display cadence); oldest first, `dt_us` preserving their spacing.
let raw = event?.coalescedTouches(for: touch) ?? [touch]
var batch: [PunktfunkPenSample] = []
var prevTs: TimeInterval?
for t in raw.suffix(Int(PUNKTFUNK_PEN_BATCH_MAX)) {
guard let s = contactSample(t, in: view, prevTs: prevTs) else { continue }
prevTs = t.timestamp
batch.append(s)
}
emit(batch)
case .up:
touching = false
// Hover-capable hardware: lift back to hover, the recognizer exits range later.
// Otherwise a lift IS the range exit (mirror of the host's GameStream heuristic).
inRange = sawHover
var s = last
s.pressure = 0
s.state = stateBits()
if let posSample = contactSample(touch, in: view, prevTs: nil) {
s.x = posSample.x
s.y = posSample.y
}
emit([s])
case .cancel:
release()
}
}
// MARK: - Hover path (forwarded from the view's hover recognizer)
/// Returns whether the event was consumed as Pencil hover; `false` hands it back to the
/// pointer path. A hovering Pencil reports `zOffset > 0`; trackpad/mouse hover is 0.
func maybeHover(_ r: UIHoverGestureRecognizer, in view: UIView) -> Bool {
switch r.state {
case .began, .changed:
guard r.zOffset > 0 || hoverActive else { return false }
hoverActive = true
sawHover = true
inRange = true
touching = false
guard let (x, y) = videoNorm?(r.location(in: view)) else { return true }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
s.distance = UInt16((r.zOffset.clamped(to: 0...1) * 65534).rounded())
s.tilt_deg = Self.tiltDeg(altitude: r.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(r.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(r.rollAngle) }
emit([s])
return true
case .ended, .cancelled, .failed:
guard hoverActive else { return false }
hoverActive = false
if !touching { release() }
return true
default:
return hoverActive
}
}
// MARK: - UIPencilInteractionDelegate (squeeze barrel 1 held, tap barrel 2 click)
@available(iOS 17.5, *)
func pencilInteraction(
_ interaction: UIPencilInteraction, didReceiveSqueeze squeeze: UIPencilInteraction.Squeeze
) {
switch squeeze.phase {
case .began:
squeezeHeld = true
case .ended, .cancelled:
squeezeHeld = false
default:
return
}
guard inRange || touching else { return }
var s = last
s.state = stateBits()
emit([s])
}
func pencilInteractionDidTap(_ interaction: UIPencilInteraction) {
guard inRange || touching else { return }
// A momentary barrel-2 click: press + release as two state-full samples in ONE batch
// the host's tracker emits the button press and release in order.
var press = last
press.state = stateBits() | UInt8(PUNKTFUNK_PEN_BARREL2)
var releaseS = last
releaseS.state = stateBits()
emit([press, releaseS])
}
// MARK: - Lifecycle
/// Session stop / view teardown: leave range so the host lifts anything held.
func reset() {
if inRange || touching { release() }
sawHover = false
hoverActive = false
squeezeHeld = false
}
private func release() {
touching = false
inRange = false
var s = last
s.pressure = 0
s.state = 0
emit([s])
}
// MARK: - Sample assembly
private func contactSample(
_ t: UITouch, in view: UIView, prevTs: TimeInterval?
) -> PunktfunkPenSample? {
guard let (x, y) = videoNorm?(t.location(in: view)) else { return nil }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
// maximumPossibleForce is 0 until the system knows the stylus full force then
// (binary-stylus semantics, matching the host's unknown-pressure rule).
let maxForce = t.maximumPossibleForce
s.pressure =
maxForce > 0
? UInt16((Double(t.force / maxForce).clamped(to: 0...1) * 65535).rounded())
: UInt16.max
s.distance = 0
s.tilt_deg = Self.tiltDeg(altitude: t.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(t.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(t.rollAngle) }
if let prevTs {
s.dt_us = UInt16(((t.timestamp - prevTs) * 1_000_000).clamped(to: 0...65535))
}
return s
}
private func stateBits() -> UInt8 {
var bits: UInt8 = 0
if inRange || touching { bits |= UInt8(PUNKTFUNK_PEN_IN_RANGE) }
if touching { bits |= UInt8(PUNKTFUNK_PEN_TOUCHING) }
if squeezeHeld { bits |= UInt8(PUNKTFUNK_PEN_BARREL1) }
return bits
}
private func emit(_ batch: [PunktfunkPenSample]) {
guard !batch.isEmpty else { return }
last = batch[batch.count - 1]
last.dt_us = 0
send?(batch)
armHeartbeat()
}
/// The 100 ms keepalive while in range (see the file header). 80 ms leaves headroom
/// under the host's 200 ms failsafe even with one lost datagram.
private func armHeartbeat() {
heartbeat?.invalidate()
guard inRange || touching else {
heartbeat = nil
return
}
heartbeat = Timer.scheduledTimer(withTimeInterval: 0.08, repeats: true) {
[weak self] _ in
guard let self, self.inRange || self.touching else {
self?.heartbeat?.invalidate()
self?.heartbeat = nil
return
}
self.send?([self.last])
}
}
// MARK: - Angle conversions
/// Altitude (π/2 = perpendicular) wire tilt-from-normal in degrees, 0...90.
private static func tiltDeg(altitude: CGFloat) -> UInt8 {
UInt8((90 - altitude * 180 / .pi).rounded().clamped(to: 0...90))
}
/// Apple azimuth (0 along the view's +x axis, clockwise, y-down) wire azimuth
/// (0 = north/up on screen, clockwise): +90° offset.
private static func azimuthDeg(_ apple: CGFloat) -> UInt16 {
let deg = (apple * 180 / .pi + 90).truncatingRemainder(dividingBy: 360)
return UInt16((deg + 360).truncatingRemainder(dividingBy: 360).rounded()) % 360
}
/// Pencil Pro roll (radians, π...π) wire barrel roll 0...359°.
private static func rollDeg(_ roll: CGFloat) -> UInt16 {
let deg = (roll * 180 / .pi).truncatingRemainder(dividingBy: 360)
return UInt16(((deg + 360).truncatingRemainder(dividingBy: 360)).rounded()) % 360
}
/// All-unknown baseline: sentinel angles/distance, tool = pen (the eraser is host-side
/// state driven by the squeeze/tap mappings, not a hardware end).
private static func idleSample() -> PunktfunkPenSample {
PunktfunkPenSample(
x: 0, y: 0, pressure: 0,
distance: UInt16(PUNKTFUNK_PEN_DISTANCE_UNKNOWN),
azimuth_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
roll_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
dt_us: 0, state: 0,
tool: UInt8(PUNKTFUNK_PEN_TOOL_PEN),
tilt_deg: UInt8(PUNKTFUNK_PEN_TILT_UNKNOWN),
_reserved: (0, 0, 0))
}
}
extension Comparable {
fileprivate func clamped(to range: ClosedRange<Self>) -> Self {
min(max(self, range.lowerBound), range.upperBound)
}
}
#endif
@@ -38,11 +38,10 @@ private let streamInputDebug =
/// dragged deltas become the relative motion StreamLayerView forwards), and hide it. /// dragged deltas become the relative motion StreamLayerView forwards), and hide it.
/// hide/unhide and associate are balanced via `captured`. /// hide/unhide and associate are balanced via `captured`.
/// ///
/// In the DESKTOP mouse model (absolute pointer, remote-desktop-sweep M1) this is a no-op: /// In CLIENT-SIDE-CURSOR mode (gamescope, whose capture carries no host cursor) this is a
/// the pointer stays free (entering and leaving the stream at will) and StreamLayerView /// no-op: the local cursor stays visible and free, and StreamLayerView forwards ABSOLUTE
/// forwards ABSOLUTE positions instead; the local cursor is hidden only while over the view /// positions instead the visible system cursor IS the on-screen cursor. `disassociate`
/// (cursor rects). `disassociate` selects between the two; `release()` only undoes what /// selects between the two; `release()` only undoes what `capture` actually did.
/// `capture` actually did.
private final class CursorCapture { private final class CursorCapture {
private var captured = false private var captured = false
/// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode), /// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode),
@@ -208,43 +207,14 @@ public final class StreamLayerView: NSView {
/// forwarded). Main-thread only. /// forwarded). Main-thread only.
public private(set) var captured = false public private(set) var captured = false
/// Desktop (absolute) mouse model remote-desktop-sweep M1: when true the pointer is /// Client-side-cursor mode: when true the local system cursor stays VISIBLE over the
/// never disassociated (it enters and leaves the stream freely) and the mouse monitor /// stream and the mouse monitor forwards ABSOLUTE positions (the visible cursor is the
/// forwards ABSOLUTE positions through the letterbox; the local cursor is hidden only /// on-screen cursor gamescope draws none, so no double cursor); when false the existing
/// while over this view (cursor rects the host's composited cursor, tracking our /// captured/disassociated relative path runs unchanged. Initialized at session start from
/// sends, is the one you see) and reappears the moment it leaves. When false the /// the `cursorMode` setting + the host's resolved compositor, toggled live by C. A live
/// captured/disassociated relative path runs unchanged. Initialized at session start /// flip re-engages capture in the new mode so disassociation + the abs/rel choice swap
/// from the `mouseMode` setting gated by the host's resolved compositor (gamescope's /// atomically. Main-thread only.
/// EIS is relative-only absolute sends would be dropped, so it pins to capture); private var cursorVisible = false
/// flipped live by M. A live flip re-engages capture in the new model so
/// disassociation + the abs/rel choice swap atomically. Main-thread only.
private var desktopMouse = false
/// Cursor channel (M2): the host forwards shape/state and WE draw the pointer. Active
/// when the Welcome carried `HOST_CAP_CURSOR` (only sessions that advertised the client
/// cap get it). Shapes cache by serial; state is latest-wins. Main-thread only.
private var cursorChannelActive = false
/// A forwarded host cursor shape, cached RAW (not as a finished `NSCursor`) so the pointer can be
/// (re)built at the CURRENT video-fit scale see `scaledCursor`. The host forwards the bitmap in
/// host FRAMEBUFFER pixels, whose size tracks the host's display scaling (32 px at 100%, 96 px at
/// 300% DPI); scaling by the video fit keeps the pointer sized to the streamed desktop at any host
/// scaling instead of ballooning on a high-DPI host.
private struct HostCursorShape {
let cg: CGImage
let width: Int
let height: Int
let hotX: Int
let hotY: Int
}
private var hostCursors: [UInt32: HostCursorShape] = [:]
private var cursorState: PunktfunkConnection.CursorStateEvent?
/// Last `CursorRenderMode.clientDraws` told to the host (the §8 mid-stream render flip);
/// nil = nothing sent yet. Edge-detected by [`reconcileCursorRender`] from the live mouse
/// model, so the chord, engage/release, and session start all reconcile through one path.
private var sentClientDraws: Bool?
/// M3 hint tracking: edge-triggered so a manual M isn't fought the override latch
/// holds until the HOST's intent next changes.
private var lastHint: Bool?
private var hintOverride = false
/// One-shot auto-engage request (stream start, trust confirmed) attempted as soon /// One-shot auto-engage request (stream start, trust confirmed) attempted as soon
/// as the view is in a window with real bounds, then dropped, so it can never fire /// as the view is in a window with real bounds, then dropped, so it can never fire
/// surprisingly later (e.g. on a resize). /// surprisingly later (e.g. on a resize).
@@ -470,9 +440,9 @@ public final class StreamLayerView: NSView {
// If the cursor grab is refused (e.g. the reactivating click arrives before the app is // If the cursor grab is refused (e.g. the reactivating click arrives before the app is
// frontmost), stay released so the NEXT click retries never latch captured=true over // frontmost), stay released so the NEXT click retries never latch captured=true over
// a free cursor, which would make mouseDown's `!captured` guard reject every later click. // a free cursor, which would make mouseDown's `!captured` guard reject every later click.
// In the desktop mouse model there is no grab (the pointer stays free) capture // In client-side-cursor mode there is no grab (the cursor stays visible) capture
// always engages and the monitor forwards absolute positions instead. // always engages and the monitor forwards absolute positions instead.
guard cursorCapture.capture(in: self, disassociate: !desktopMouse) else { return } guard cursorCapture.capture(in: self, disassociate: !cursorVisible) else { return }
inputCapture?.setForwarding(true, suppressClick: fromClick) inputCapture?.setForwarding(true, suppressClick: fromClick)
// Install AFTER the warp + setForwarding: the engage warp generates no forwarded // Install AFTER the warp + setForwarding: the engage warp generates no forwarded
// delta (the monitor isn't up yet), and the engage click's suppression latch is // delta (the monitor isn't up yet), and the engage click's suppression latch is
@@ -480,9 +450,7 @@ public final class StreamLayerView: NSView {
installMouseMonitor() installMouseMonitor()
captured = true captured = true
window?.makeFirstResponder(self) window?.makeFirstResponder(self)
window?.invalidateCursorRects(for: self) // desktop model: hide-over-view engages
notifyCaptureChange(true) notifyCaptureChange(true)
reconcileCursorRender()
} }
private func releaseCapture() { private func releaseCapture() {
@@ -491,194 +459,7 @@ public final class StreamLayerView: NSView {
cursorCapture.release() cursorCapture.release()
inputCapture?.setForwarding(false) inputCapture?.setForwarding(false)
captured = false captured = false
window?.invalidateCursorRects(for: self)
notifyCaptureChange(false) notifyCaptureChange(false)
reconcileCursorRender() // released the host composites the pointer again
}
/// A fully transparent cursor for the desktop mouse model's hide-over-view rect
/// an empty 1×1 image draws nothing.
private static let invisibleCursor = NSCursor(
image: NSImage(size: NSSize(width: 1, height: 1)), hotSpot: .zero)
/// Desktop mouse model: the local cursor is hidden while over the stream (the host's
/// composited cursor, tracking our absolute sends, is the one you see) and reappears
/// the moment it leaves the view AppKit applies/removes the rect's cursor for us,
/// so there is no hide/unhide balancing to get wrong. Capture model instead hides
/// globally via `CursorCapture` (the pointer can't leave the view there).
override public func resetCursorRects() {
if captured && desktopMouse {
// Cursor channel active: wear the HOST's pointer shape (it is no longer in the
// video); hidden host pointer (or no shape yet) = invisible. Without the channel,
// M1 behavior: invisible local cursor, the composited host cursor is the visible one.
if cursorChannelActive, let st = cursorState, st.visible,
let shape = hostCursors[st.serial] {
addCursorRect(bounds, cursor: scaledCursor(shape))
} else {
addCursorRect(bounds, cursor: Self.invisibleCursor)
}
} else {
super.resetCursorRects()
}
}
/// Tell the host who renders the pointer (the §8 mid-stream render flip): we draw it only
/// while the DESKTOP model is engaged (the local OS cursor wears the host shape); under
/// the capture model and while released the host composites it into the video (full
/// fidelity, the pre-channel look). One edge-detected reconciler, called from every
/// transition (chord, engage/release, session start).
private func reconcileCursorRender() {
guard cursorChannelActive, let connection else { return }
let clientDraws = captured && desktopMouse
guard sentClientDraws != clientDraws else { return }
sentClientDraws = clientDraws
connection.setCursorRender(clientDraws: clientDraws)
}
/// Flip the mouse model with the atomic release/re-engage swap; `reappearAt` (host video
/// px the M3 hand-back position) warps the local pointer so leaving relative lands the
/// cursor exactly where the host last had it.
private func setDesktopMouse(_ on: Bool, reappearAt: (x: Int32, y: Int32)?) {
guard desktopMouse != on else { return }
let wasCaptured = captured
if wasCaptured { releaseCapture() }
desktopMouse = on
if wasCaptured { engageCapture(fromClick: false) }
window?.invalidateCursorRects(for: self)
if on, let p = reappearAt, let sp = cgScreenPoint(forHostX: p.x, p.y) {
CGWarpMouseCursorPosition(sp)
}
reconcileCursorRender()
}
/// The single cursor pull thread (both planes share the connection's cursor lock):
/// latest-wins state at a short timeout + a non-blocking shape poll per iteration.
/// Exits when the connection closes; events hop to main where all cursor state lives.
private func startCursorPump(_ connection: PunktfunkConnection) {
let thread = Thread { [weak self] in
while true {
do {
var newest: PunktfunkConnection.CursorStateEvent?
if let st = try connection.nextCursorState(timeoutMs: 100) {
newest = st
while let more = try connection.nextCursorState(timeoutMs: 0) {
newest = more // drain latest wins
}
}
while let shape = try connection.nextCursorShape(timeoutMs: 0) {
DispatchQueue.main.async { self?.applyCursorShape(shape) }
}
if let st = newest {
DispatchQueue.main.async { self?.applyCursorState(st) }
}
} catch {
return // connection closed the session is over
}
if self == nil { return }
}
}
thread.name = "pf-cursor-pump"
thread.start()
}
private func applyCursorShape(_ ev: PunktfunkConnection.CursorShapeEvent) {
guard let shape = Self.makeShape(ev) else {
streamInputLog.warning("cursor shape rejected (\(ev.width)x\(ev.height)) — keeping the previous cursor")
return
}
if hostCursors.count >= 64 { hostCursors.removeAll() } // degenerate host: reset
hostCursors[ev.serial] = shape
if cursorState?.serial == ev.serial {
window?.invalidateCursorRects(for: self)
}
}
private func applyCursorState(_ ev: PunktfunkConnection.CursorStateEvent) {
let prev = cursorState
cursorState = ev
if prev?.visible != ev.visible || prev?.serial != ev.serial {
window?.invalidateCursorRects(for: self)
}
// M3 host-driven auto-flip is DISABLED: `relative_hint` is derived from host cursor
// VISIBILITY, and Windows hides the pointer for ordinary desktop activity (clicking,
// typing) not just when a game grabs it. Acting on those transients flipped
// desktopcapturedesktop, which warped the cursor to view-centre and flushed held
// buttons (a spurious button-up ~200 ms into every press broke window drags). Until
// the host exposes a real pointer-LOCK signal (ClipCursor/raw-input, not visibility),
// the mouse model is user-driven only (M). The hint still rides the wire, unused.
_ = (lastHint, hintOverride)
}
/// Decode a forwarded straight-alpha RGBA shape into a CGImage + hotspot. The on-screen SIZE is
/// NOT baked in here it is applied per-use in `scaledCursor` from the live video-fit scale, so
/// the same shape re-fits across window resizes / retina moves without a re-forward.
private static func makeShape(_ ev: PunktfunkConnection.CursorShapeEvent) -> HostCursorShape? {
let (w, h) = (ev.width, ev.height)
guard w > 0, h > 0, ev.rgba.count >= w * h * 4,
let provider = CGDataProvider(data: ev.rgba as CFData),
let cg = CGImage(
width: w, height: h, bitsPerComponent: 8, bitsPerPixel: 32,
bytesPerRow: w * 4, space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue),
provider: provider, decode: nil, shouldInterpolate: false,
intent: .defaultIntent)
else { return nil }
return HostCursorShape(
cg: cg, width: w, height: h,
hotX: min(ev.hotX, w - 1), hotY: min(ev.hotY, h - 1))
}
/// Points-per-host-pixel: the exact factor the video frame is aspect-fit into the view (the same
/// `AVMakeRect` fit `hostPoint`/`cgScreenPoint` use). The host forwards the pointer bitmap in host
/// framebuffer pixels the mode we drive is in the client's BACKING pixels, so on retina this is
/// ~1/backingScale and the pointer lands at its TRUE size relative to the streamed desktop
/// (crisp, 1:1 with the video) rather than the 2×-inflated pixel-as-points it used to be. Because
/// the bitmap grows with the host's display scaling (96 px at 300% DPI), scaling by this is what
/// keeps a high-DPI host from forwarding a giant pointer. Falls back to 1 before the first
/// mode/layout.
private func cursorFitScale() -> CGFloat {
guard let connection else { return 1 }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0, bounds.width > 0, bounds.height > 0 else { return 1 }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)), insideRect: bounds)
guard fit.width > 0 else { return 1 }
return fit.width / CGFloat(mode.width)
}
/// Build the `NSCursor` for a cached shape at the CURRENT video-fit scale (see `cursorFitScale`).
/// Both the image size and the hotspot scale together so the click point stays true.
private func scaledCursor(_ shape: HostCursorShape) -> NSCursor {
let scale = cursorFitScale()
let sw = max(1, (CGFloat(shape.width) * scale).rounded())
let sh = max(1, (CGFloat(shape.height) * scale).rounded())
let image = NSImage(cgImage: shape.cg, size: NSSize(width: sw, height: sh))
let hot = NSPoint(
x: min(CGFloat(shape.hotX) * scale, sw - 1),
y: min(CGFloat(shape.hotY) * scale, sh - 1))
return NSCursor(image: image, hotSpot: hot)
}
/// Host video px CG GLOBAL screen coordinates (top-left origin, the
/// `CGWarpMouseCursorPosition` convention `CursorCapture` established) through the
/// aspect-fit letterbox the inverse direction of `hostPoint(from:)`.
private func cgScreenPoint(forHostX hx: Int32, _ hy: Int32) -> CGPoint? {
guard let connection, let window else { return nil }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0 else { return nil }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
guard fit.width > 0, fit.height > 0 else { return nil }
let u = (CGFloat(hx) / CGFloat(mode.width)).clamped(to: 0...1)
let v = (CGFloat(hy) / CGFloat(mode.height)).clamped(to: 0...1)
let videoMinYTop = bounds.height - fit.maxY
let pTop = CGPoint(x: fit.minX + u * fit.width, y: videoMinYTop + v * fit.height)
let inView = CGPoint(x: pTop.x, y: bounds.height - pTop.y)
let inWindow = convert(inView, to: nil)
let onScreen = window.convertPoint(toScreen: inWindow)
let primaryHeight = NSScreen.screens.first?.frame.height ?? 0
return CGPoint(x: onScreen.x, y: primaryHeight - onScreen.y)
} }
/// A single local monitor for motion + buttons, installed only while captured. A local /// A single local monitor for motion + buttons, installed only while captured. A local
@@ -692,12 +473,12 @@ public final class StreamLayerView: NSView {
/// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're /// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're
/// inert locally. /// inert locally.
/// ///
/// In the desktop mouse model the cursor is NOT frozen, so bare `.mouseMoved` events are /// In client-side-cursor mode the cursor is NOT frozen, so bare `.mouseMoved` events are
/// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and /// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and
/// restore it on removal so absolute hover-motion keeps flowing without a click held. /// restore it on removal so absolute hover-motion keeps flowing without a click held.
private func installMouseMonitor() { private func installMouseMonitor() {
guard mouseEventMonitor == nil else { return } guard mouseEventMonitor == nil else { return }
if desktopMouse { if cursorVisible {
savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents
window?.acceptsMouseMovedEvents = true window?.acceptsMouseMovedEvents = true
} }
@@ -709,8 +490,8 @@ public final class StreamLayerView: NSView {
guard let self, self.captured, let ic = self.inputCapture else { return event } guard let self, self.captured, let ic = self.inputCapture else { return event }
switch event.type { switch event.type {
case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged: case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged:
if self.desktopMouse { if self.cursorVisible {
// Desktop mouse model: forward the ABSOLUTE position (mapped through the // Client-side cursor: forward the ABSOLUTE position (mapped through the
// aspect-fit letterbox into host pixels), the same path the iPad pointer // aspect-fit letterbox into host pixels), the same path the iPad pointer
// fallback uses. Events in the letterbox bars are dropped (nil host point). // fallback uses. Events in the letterbox bars are dropped (nil host point).
if let p = self.hostPoint(from: event) { if let p = self.hostPoint(from: event) {
@@ -828,25 +609,14 @@ public final class StreamLayerView: NSView {
// be a cursor trap with dead input. // be a cursor trap with dead input.
self?.releaseCapture() self?.releaseCapture()
} }
// M flips the mouse model (capture desktop) live the SDL clients' identical // C flips the client-side cursor live. Only the key window's stream owns it (same
// chord. Only the key window's stream owns it (same guard as the capture toggle). // guard as the capture toggle). Re-engage capture in the new mode so disassociation
// Re-engage capture in the new model so disassociation and the absolute/relative // and the absolute/relative forwarding choice swap atomically releaseCapture restores
// forwarding choice swap atomically releaseCapture restores the old model's grab // the old mode's grab (if any), engageCapture installs the new one.
// (if any), engageCapture installs the new one. On a gamescope host the chord is a // C would flip the client-side cursor live NEUTERED while the feature is disabled
// no-op: its EIS grants only a relative pointer, so the desktop model's absolute // (see the cursorVisible resolution below): toggling it on under gamescope's relative-only
// sends would be silently dropped (pointer stuck = "all input dead"). // input traps the pointer. Restore this body when absolute/synthetic-cursor support lands.
capture.onToggleMouseMode = { [weak self] in capture.onToggleCursor = {}
guard let self, self.window?.isKeyWindow == true,
let conn = self.connection else { return }
guard conn.resolvedCompositor != .gamescope else {
streamInputLog.info("mouse-mode chord ignored: gamescope host is relative-only")
return
}
// A manual flip outranks the standing host hint until the hint next CHANGES.
self.hintOverride = true
self.setDesktopMouse(!self.desktopMouse, reappearAt: nil)
streamInputLog.info("chord: mouse mode \(self.desktopMouse ? "desktop" : "capture", privacy: .public)")
}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by // The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout. // the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in capture.onReleaseCapture = { [weak self] in
@@ -873,26 +643,15 @@ public final class StreamLayerView: NSView {
capture.start() capture.start()
inputCapture = capture inputCapture = capture
// Desktop (absolute) mouse model resolved at session start from the mouseMode // Client-side cursor is TEMPORARILY DISABLED. It positions the host cursor with ABSOLUTE
// setting, gated by the host's compositor: gamescope's input socket (EIS) grants // events, but gamescope's input socket (EIS) grants only a relative pointer, so those are
// only a relative pointer, so absolute sends would be silently dropped there // silently dropped the pointer never moves and clicks/scroll land on the stuck position
// (pointer stuck = "all input dead") pinned to capture. M flips it live. // (looks like "all input dead"). gamescope is exactly the compositor Auto enabled it for.
let mode = MouseInputMode( // Forced off until per-compositor gating (KWin/GNOME/Sway have absolute) or a synthetic-
rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? "" // cursor-over-relative path lands; the resolution logic below is kept for that. See the
) ?? .capture // C handler (also neutered) and the cursorMode setting (hidden).
let absOK = connection.resolvedCompositor != .gamescope cursorVisible = false
desktopMouse = mode == .desktop && absOK _ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent)
if mode == .desktop && !absOK {
streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture")
}
// Cursor channel (M2): the host stopped compositing the pointer drain its shape/
// state planes and draw the pointer as the real NSCursor (plus the M3 auto-flip).
if connection.hostSupportsCursor {
cursorChannelActive = true
streamInputLog.info("cursor channel negotiated — host cursor renders locally")
startCursorPump(connection)
reconcileCursorRender() // initial render mode (a capture-model start composites)
}
// Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2 // Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by // (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
@@ -941,9 +700,9 @@ public final class StreamLayerView: NSView {
private func layoutPresenter() { private func layoutPresenter() {
presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1) presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1)
// Present routing tracks the window's composited state (fullscreen transitions always // Present routing tracks the window's composited state (fullscreen transitions always
// re-layout, so this stays current): a windowed session presents through a Core Animation // re-layout, so this stays current): windowed PyroWave presents via surface contents
// transaction the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited). // the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited). A view
// A view not yet in a window counts as composited (the safe default). // not yet in a window counts as composited (the safe default).
presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false)) presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false))
// Feed the follower only once in a window (backing scale is real then) and with real // Feed the follower only once in a window (backing scale is real then) and with real
// bounds a pre-window layout would report point-sized dimensions. // bounds a pre-window layout would report point-sized dimensions.
@@ -952,11 +711,6 @@ public final class StreamLayerView: NSView {
matchFollower?.noteSize( matchFollower?.noteSize(
widthPx: Int(px.width.rounded()), heightPx: Int(px.height.rounded())) widthPx: Int(px.width.rounded()), heightPx: Int(px.height.rounded()))
} }
// The video-fit scale just changed (resize / retina move); rebuild the worn host pointer at
// the new scale so it tracks the video instead of freezing at its build-time size.
if captured, desktopMouse, cursorChannelActive {
window?.invalidateCursorRects(for: self)
}
} }
public override func viewDidChangeBackingProperties() { public override func viewDidChangeBackingProperties() {
@@ -987,14 +741,6 @@ public final class StreamLayerView: NSView {
matchFollower = nil matchFollower = nil
lastDecodedContentSize = nil // the next session re-derives it from its first frame lastDecodedContentSize = nil // the next session re-derives it from its first frame
connection = nil connection = nil
// Cursor-channel state is per-session: without this reset a next session against a
// host WITHOUT the cap would wear this session's stale shapes (`cursorChannelActive`
// stayed latched true across sessions).
cursorChannelActive = false
cursorState = nil
hostCursors.removeAll()
sentClientDraws = nil
window?.invalidateCursorRects(for: self)
} }
deinit { deinit {
@@ -333,13 +333,6 @@ public final class StreamViewController: StreamViewControllerBase {
guard self?.captureEnabled == true else { return } guard self?.captureEnabled == true else { return }
connection?.send(event) connection?.send(event)
} }
// Apple Pencil the stylus plane, only against a pen-capable host (elsewhere the
// Pencil stays a finger, exactly as before). Same trust gate as touch.
streamView.penEnabled = connection.hostSupportsPen
streamView.onPenBatch = { [weak self, weak connection] batch in
guard self?.captureEnabled == true else { return }
connection?.sendPen(batch)
}
// Indirect pointer (mouse/trackpad) WITHOUT a lock absolute cursor + buttons + scroll. // Indirect pointer (mouse/trackpad) WITHOUT a lock absolute cursor + buttons + scroll.
// While the scene is pointer-LOCKED the GCMouse path owns motion AND buttons AND scroll, so // While the scene is pointer-LOCKED the GCMouse path owns motion AND buttons AND scroll, so
// the whole UIKit indirect path is gated off here (`gcMouseForwarding`). The trackpad and a // the whole UIKit indirect path is gated off here (`gcMouseForwarding`). The trackpad and a
@@ -506,8 +499,6 @@ public final class StreamViewController: StreamViewControllerBase {
// onTouchEvent can still deliver the button-up. // onTouchEvent can still deliver the button-up.
streamView.resetTouchInput() streamView.resetTouchInput()
streamView.onTouchEvent = nil streamView.onTouchEvent = nil
streamView.onPenBatch = nil // after reset the pen's leave-range sample rides it
streamView.penEnabled = false
streamView.onPointerMoveAbs = nil streamView.onPointerMoveAbs = nil
streamView.onPointerButton = nil streamView.onPointerButton = nil
streamView.onScroll = nil streamView.onScroll = nil
@@ -701,12 +692,6 @@ final class StreamLayerUIView: UIView {
/// Direct fingers / Pencil wire events: real touches in passthrough mode, or the /// Direct fingers / Pencil wire events: real touches in passthrough mode, or the
/// touch-driven mouse events (`TouchMouse`) in the trackpad/pointer modes. /// touch-driven mouse events (`TouchMouse`) in the trackpad/pointer modes.
var onTouchEvent: ((PunktfunkInputEvent) -> Void)? var onTouchEvent: ((PunktfunkInputEvent) -> Void)?
/// Apple Pencil state-full pen sample batches (the stylus plane). Active only while
/// `penEnabled`; without it the Pencil stays on the finger path exactly as before.
var onPenBatch: (([PunktfunkPenSample]) -> Void)?
/// The host advertised `HOST_CAP_PEN`, so Pencil input splits out of the finger path onto
/// the pen plane independent of the touch-input mode (drawing must not depend on it).
var penEnabled = false
/// Indirect pointer (mouse/trackpad with no lock) absolute cursor moves. /// Indirect pointer (mouse/trackpad with no lock) absolute cursor moves.
var onPointerMoveAbs: ((HostPoint) -> Void)? var onPointerMoveAbs: ((HostPoint) -> Void)?
/// Indirect-pointer buttons (GameStream ids: 1=left 3=right); `down` = press. /// Indirect-pointer buttons (GameStream ids: 1=left 3=right); `down` = press.
@@ -730,21 +715,10 @@ final class StreamLayerUIView: UIView {
/// The finger route latched at gesture start a Settings change mid-gesture applies to /// The finger route latched at gesture start a Settings change mid-gesture applies to
/// the NEXT touch, so one gesture never splits across input models. /// the NEXT touch, so one gesture never splits across input models.
private var fingerRoute: TouchInputMode? private var fingerRoute: TouchInputMode?
/// The Apple Pencil pipeline (contacts + hover + squeeze/tap pen samples).
private lazy var pencil: PencilStream = {
let stream = PencilStream()
stream.send = { [weak self] batch in self?.onPenBatch?(batch) }
stream.videoNorm = { [weak self] point in
guard let h = self?.hostPoint(from: point) else { return nil }
return (Float(h.x) / Float(max(h.w - 1, 1)), Float(h.y) / Float(max(h.h - 1, 1)))
}
return stream
}()
/// Release anything the touch-driven mouse holds and forget gesture state session stop. /// Release anything the touch-driven mouse holds and forget gesture state session stop.
func resetTouchInput() { func resetTouchInput() {
touchMouse.reset() touchMouse.reset()
pencil.reset() // leaves range the host lifts anything still inked
fingerRoute = nil fingerRoute = nil
setSoftKeyboardVisible(false) // a stream that's gone takes its keyboard with it setSoftKeyboardVisible(false) // a stream that's gone takes its keyboard with it
} }
@@ -781,11 +755,6 @@ final class StreamLayerUIView: UIView {
scrollPan.allowedScrollTypesMask = .all scrollPan.allowedScrollTypesMask = .all
scrollPan.allowedTouchTypes = [] scrollPan.allowedTouchTypes = []
addGestureRecognizer(scrollPan) addGestureRecognizer(scrollPan)
// Pencil squeeze / double-tap the pen plane's barrel buttons (no-op while
// `penEnabled` is false PencilStream ignores interactions out of range).
let pencilInteraction = UIPencilInteraction()
pencilInteraction.delegate = pencil
addInteraction(pencilInteraction)
#endif #endif
backgroundColor = .black backgroundColor = .black
} }
@@ -810,32 +779,17 @@ final class StreamLayerUIView: UIView {
private enum TouchKind { case down, move, up, cancel } private enum TouchKind { case down, move, up, cancel }
/// Split a touch batch by kind: an INDIRECT POINTER (mouse/trackpad with no lock) drives /// Split a touch batch by kind: an INDIRECT POINTER (mouse/trackpad with no lock) drives
/// the host cursor as an absolute mouse; a Pencil goes to the pen plane when the host /// the host cursor as an absolute mouse; everything else (direct finger, Pencil) is a host
/// supports it; everything else (direct finger and the Pencil toward a pen-less host) /// touch. Mixed batches are possible, so partition rather than branch on the first touch.
/// is a host touch. Mixed batches are possible, so partition rather than branch on the
/// first touch.
private func route(_ touches: Set<UITouch>, event: UIEvent?, kind: TouchKind) { private func route(_ touches: Set<UITouch>, event: UIEvent?, kind: TouchKind) {
var fingers: Set<UITouch> = [] var fingers: Set<UITouch> = []
var pencilTouches: Set<UITouch> = []
for touch in touches { for touch in touches {
if touch.type == .indirectPointer { if touch.type == .indirectPointer {
handleIndirectPointer(touch, event: event, kind: kind) handleIndirectPointer(touch, event: event, kind: kind)
} else if penEnabled, touch.type == .pencil {
pencilTouches.insert(touch)
} else { } else {
fingers.insert(touch) fingers.insert(touch)
} }
} }
if !pencilTouches.isEmpty {
let phase: PencilStream.Phase =
switch kind {
case .down: .down
case .move: .move
case .up: .up
case .cancel: .cancel
}
pencil.touches(pencilTouches, event: event, phase: phase, in: self)
}
if !fingers.isEmpty { forwardFingers(fingers, kind: kind) } if !fingers.isEmpty { forwardFingers(fingers, kind: kind) }
} }
@@ -907,11 +861,8 @@ final class StreamLayerUIView: UIView {
} }
} }
/// Button-less mouse/trackpad movement (no lock) absolute cursor move unless it is a /// Button-less mouse/trackpad movement (no lock) absolute cursor move.
/// hovering PENCIL (`zOffset > 0`) on a pen-capable host, which becomes in-range pen
/// samples (hover preview with distance/tilt/azimuth) instead of a cursor move.
@objc private func handleHover(_ recognizer: UIHoverGestureRecognizer) { @objc private func handleHover(_ recognizer: UIHoverGestureRecognizer) {
if penEnabled, pencil.maybeHover(recognizer, in: self) { return }
switch recognizer.state { switch recognizer.state {
case .began, .changed: case .began, .changed:
if let h = hostPoint(from: recognizer.location(in: self)) { onPointerMoveAbs?(h) } if let h = hostPoint(from: recognizer.location(in: self)) { onPointerMoveAbs?(h) }
@@ -70,16 +70,6 @@ public enum DefaultsKey {
/// (lowest latency the default, OFF). Resolved once per session; /// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header. /// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync" public static let vsync = "punktfunk.vsync"
/// macOS: present WINDOWED sessions in lockstep with the system compositor (the DCP
/// "mismatched swapID's" kernel-panic mitigation see SessionPresenter.windowedPresentMode
/// and the MetalVideoPresenter saga notes). ON/unset (the default): windowed presents ride
/// a Core Animation transaction validated panic-free on the 240 Hz repro machine, at a
/// small display-latency cost vs the raw path. OFF: windowed sessions keep the fast async
/// image queue ON AFFECTED SETUPS (high-refresh displays) THAT PATH KERNEL-PANICS THE
/// WHOLE MAC, which is why the default is ON. Fullscreen always presents async (fast path)
/// regardless. Resolved once per session; PUNKTFUNK_WINDOWED_PRESENT=async|transaction|
/// surface overrides it for dev A/B.
public static let windowedSafePresent = "punktfunk.windowedSafePresent"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor, /// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical /// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off, /// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
@@ -94,11 +84,8 @@ public enum DefaultsKey {
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth. /// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444" public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts" public static let hosts = "punktfunk.hosts"
/// Physical-mouse model (macOS): "capture" (pointer lock + relative, the default) or /// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never".
/// "desktop" (uncaptured absolute pointer) the cross-client `mouse_mode`. Replaces the public static let cursorMode = "punktfunk.cursorMode"
/// never-shipped "punktfunk.cursorMode" (auto/always/never client-side-cursor setting,
/// which was hidden while disabled and had no readers).
public static let mouseMode = "punktfunk.mouseMode"
/// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by /// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by
/// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is /// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is
/// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across /// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across
@@ -316,41 +316,6 @@ final class PresentPacingTests: XCTestCase {
SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline) SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline)
} }
// MARK: - Windowed present mechanism (the macOS DCP swapID-panic mitigation picker)
#if os(macOS)
/// The safe-present setting: ON/unset the validated transactional mitigation; an explicit
/// OFF the fast async path (the user accepted the affected-setup panic risk). The
/// PUNKTFUNK_WINDOWED_PRESENT env lever overrides both ways, `surface` is env-only (the
/// prototype mechanism), and garbage/empty env values are "unset", not an override.
func testWindowedPresentModeResolution() {
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: nil), .transaction,
"unset defaults to the panic mitigation")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: nil), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: nil), .async,
"an explicit opt-out gets the fast async path")
// The dev env lever wins over the setting, both directions.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "async"), .async)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "transaction"),
.transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "surface"), .surface,
"the surface prototype is reachable via env only")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "surface"), .surface)
// Garbage/empty env = unset.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: "garbage"), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: ""), .async)
}
#endif
// MARK: - Glass-gate depth // MARK: - Glass-gate depth
/// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue /// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue
+2 -11
View File
@@ -16,8 +16,6 @@
# PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games) # PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games)
# PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect) # PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect)
# PF_MGMT management-API port for --browse (optional; client defaults to 47990) # PF_MGMT management-API port for --browse (optional; client defaults to 47990)
# PF_CONNECT_TIMEOUT connect budget in seconds (optional; the plugin stretches it after
# firing Wake-on-LAN so the connect survives the host's resume)
# PF_APPID flatpak app id (default io.unom.Punktfunk) # PF_APPID flatpak app id (default io.unom.Punktfunk)
# PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH) # PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH)
# #
@@ -63,17 +61,10 @@ if [ -z "${PF_HOST:-}" ]; then
echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2 echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2
exit 2 exit 2
fi fi
# Trailing args shared by both streaming execs. A stretched connect budget rides along when the
# plugin set one (it just fired Wake-on-LAN, so the host may still be resuming); an older flatpak
# without --connect-timeout ignores the flag harmlessly (hand-scanned argv).
set -- --fullscreen
if [ -n "${PF_CONNECT_TIMEOUT:-}" ]; then
set -- --connect-timeout "$PF_CONNECT_TIMEOUT" "$@"
fi
if [ -n "${PF_LAUNCH:-}" ]; then if [ -n "${PF_LAUNCH:-}" ]; then
# A pinned game: the id rides the session Hello and the host launches that title. # A pinned game: the id rides the session Hello and the host launches that title.
echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" "$@" exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" --fullscreen
fi fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" "$@" exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen
+3 -33
View File
@@ -10,17 +10,9 @@ import {
showModal, showModal,
staticClasses, staticClasses,
} from "@decky/ui"; } from "@decky/ui";
import { definePlugin, routerHook, toaster } from "@decky/api"; import { definePlugin, routerHook } from "@decky/api";
import { FC } from "react"; import { FC } from "react";
import { import { FaDownload, FaLock, FaLockOpen, FaPlay, FaSyncAlt, FaTv } from "react-icons/fa";
FaDownload,
FaLock,
FaLockOpen,
FaPlay,
FaPlus,
FaSyncAlt,
FaTv,
} from "react-icons/fa";
import { PluginErrorBoundary } from "./boundary"; import { PluginErrorBoundary } from "./boundary";
import { import {
applyUpdate, applyUpdate,
@@ -39,19 +31,7 @@ import {
import { streamPin } from "./library"; import { streamPin } from "./library";
import { PunktfunkRoute, ROUTE } from "./page"; import { PunktfunkRoute, ROUTE } from "./page";
import { PairModal } from "./pair"; import { PairModal } from "./pair";
import { ensureGamepadUiShortcut, recreateShortcuts } from "./steam"; import { ensureGamepadUiShortcut } from "./steam";
// Recovery action for "the Punktfunk library entry vanished" — recreates the visible shortcut.
// Deleting the shortcut (optionally + reinstalling the plugin) leaves a stale appId in Steam's
// CEF localStorage that self-heal fixes on the next mount, but this gives an in-session button
// that works even without a reload. Always ends in a toast so the tap has feedback.
async function recreatePunktfunkShortcut(): Promise<void> {
const appId = await recreateShortcuts();
toaster.toast({
title: "Punktfunk",
body: appId != null ? "Shortcut restored to your library" : "Couldn't create the shortcut",
});
}
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
// QAM panel — quick status + entry into the full page + one-tap stream for known hosts // QAM panel — quick status + entry into the full page + one-tap stream for known hosts
@@ -210,16 +190,6 @@ const QamPanel: FC = () => {
{checking ? "Checking…" : "Check for updates"} {checking ? "Checking…" : "Check for updates"}
</ButtonItem> </ButtonItem>
</PanelSectionRow> </PanelSectionRow>
<PanelSectionRow>
<ButtonItem
layout="below"
description="Missing the Punktfunk entry in your library? This puts it back."
onClick={() => void recreatePunktfunkShortcut()}
>
<FaPlus style={{ marginRight: "0.5em" }} />
Recreate library shortcut
</ButtonItem>
</PanelSectionRow>
</PanelSection> </PanelSection>
</> </>
); );
+9 -86
View File
@@ -55,29 +55,6 @@ declare const collectionStore:
| { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void } | { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void }
| undefined; | undefined;
// SteamUI's appStore indexes every registered app/shortcut by appId; a remembered appId whose
// overview is gone was deleted out from under us (the user removed the library entry). We must
// verify this because the remembered appId lives in Steam's CEF localStorage — which survives a
// plugin UNINSTALL/REINSTALL — so a manually-deleted shortcut otherwise leaves a dangling appId
// that the reuse path below silently repoints (SetShortcut* on a dead id is a no-op), and the
// entry never comes back.
declare const appStore:
| { GetAppOverviewByAppID?: (appId: number) => unknown | null }
| undefined;
/** True if a remembered appId still maps to a live Steam shortcut. When appStore is unavailable
* we can't tell, so assume it exists better to keep reusing than risk a duplicate library
* entry from a false "missing". A confident null means the shortcut was deleted recreate. */
function shortcutStillExists(appId: number): boolean {
try {
const get = appStore?.GetAppOverviewByAppID;
if (!get) return true; // no way to verify — preserve the reuse path
return get(appId) != null;
} catch {
return true;
}
}
/** Set a shortcut's library visibility (best-effort, deferred the overview registers a moment /** Set a shortcut's library visibility (best-effort, deferred the overview registers a moment
* after AddShortcut). Hides the stateful stream shortcut; keeps the gamepad-UI one visible. */ * after AddShortcut). Hides the stateful stream shortcut; keeps the gamepad-UI one visible. */
function setShortcutHidden(appId: number, hidden: boolean): void { function setShortcutHidden(appId: number, hidden: boolean): void {
@@ -93,9 +70,7 @@ function setShortcutHidden(appId: number, hidden: boolean): void {
}; };
// Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId). // Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId).
// v3: CI zips through 0.17.1 shipped no assets/ at all, yet v2 was still recorded as applied const ART_VERSION = 2;
// on those installs — the bump makes them re-apply once on the first build that has the files.
const ART_VERSION = 3;
function artKey(appId: number): string { function artKey(appId: number): string {
return `punktfunk:shortcutArt:${appId}`; return `punktfunk:shortcutArt:${appId}`;
} }
@@ -104,7 +79,7 @@ function artKey(appId: number): string {
* Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per * Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per
* appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call. * appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call.
*/ */
async function applyArtwork(appId: number, isRetry = false): Promise<void> { async function applyArtwork(appId: number): Promise<void> {
try { try {
if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) { if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) {
return; return;
@@ -116,29 +91,16 @@ async function applyArtwork(appId: number, isRetry = false): Promise<void> {
[art.logo, 2], [art.logo, 2],
[art.gridwide, 3], [art.gridwide, 3],
]; ];
let applied = false;
for (const [data, assetType] of assets) { for (const [data, assetType] of assets) {
if (data) { if (data) {
await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType); await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType);
applied = true;
} }
} }
if (art.icon_path) { if (art.icon_path) {
SteamClient.Apps.SetShortcutIcon(appId, art.icon_path); SteamClient.Apps.SetShortcutIcon(appId, art.icon_path);
applied = true;
} }
// Only record "done" when something actually landed — a plugin build whose assets/ is
// missing/empty must keep retrying on later mounts instead of poisoning the marker.
if (applied) {
localStorage.setItem(artKey(appId), `${ART_VERSION}`); localStorage.setItem(artKey(appId), `${ART_VERSION}`);
}
} catch (e) { } catch (e) {
// A shortcut fresh out of AddShortcut may not be registered yet (the same race
// setShortcutHidden defers around) — one deferred second attempt, then leave it to
// the next mount.
if (!isRetry) {
setTimeout(() => void applyArtwork(appId, true), 2500);
}
console.warn("punktfunk: shortcut artwork not applied", e); console.warn("punktfunk: shortcut artwork not applied", e);
} }
} }
@@ -195,9 +157,7 @@ async function ensureControllerConfig(): Promise<void> {
return; return;
} }
const r = await applyControllerConfig(SHORTCUT_NAME); const r = await applyControllerConfig(SHORTCUT_NAME);
// `ok` alone isn't done: with zero account configset dirs (fresh Steam) the backend if (r?.ok) {
// succeeds without pointing any account at the template — keep retrying until one lands.
if (r?.ok && (r.applied ?? []).some((a) => a.startsWith("configset:"))) {
localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`); localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`);
} else { } else {
console.warn("punktfunk: controller config not fully applied", r); console.warn("punktfunk: controller config not fully applied", r);
@@ -222,10 +182,8 @@ async function ensureStreamShortcut(): Promise<{ appId: number; runner: string }
const startDir = info.runner.replace(/\/[^/]*$/, ""); // the plugin's bin/ dir const startDir = info.runner.replace(/\/[^/]*$/, ""); // the plugin's bin/ dir
void ensureControllerConfig(); // fire-and-forget — never blocks the launch void ensureControllerConfig(); // fire-and-forget — never blocks the launch
// Reuse the remembered shortcut only if it still exists — a stale appId (shortcut deleted, key
// outlived it across a reinstall) must fall through to AddShortcut, not be silently repointed.
const remembered = recall(STORAGE_KEY_STREAM); const remembered = recall(STORAGE_KEY_STREAM);
if (remembered != null && shortcutStillExists(remembered)) { if (remembered != null) {
SteamClient.Apps.SetShortcutExe(remembered, SHELL); SteamClient.Apps.SetShortcutExe(remembered, SHELL);
SteamClient.Apps.SetShortcutStartDir(remembered, startDir); SteamClient.Apps.SetShortcutStartDir(remembered, startDir);
SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME); SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME);
@@ -261,11 +219,8 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
// home). %command% expands to the shortcut exe (/bin/sh); the wrapper rides behind as an arg. // home). %command% expands to the shortcut exe (/bin/sh); the wrapper rides behind as an arg.
const launchOpts = `PF_BROWSE=1 %command% "${info.runner}"`; const launchOpts = `PF_BROWSE=1 %command% "${info.runner}"`;
// Reuse the remembered entry only if it still exists; a stale appId (deleted shortcut whose
// localStorage key survived a plugin reinstall) falls through to AddShortcut so the visible
// library entry actually comes back instead of repointing a dead id.
let appId = recall(STORAGE_KEY_UI); let appId = recall(STORAGE_KEY_UI);
if (appId != null && shortcutStillExists(appId)) { if (appId != null) {
SteamClient.Apps.SetShortcutExe(appId, SHELL); SteamClient.Apps.SetShortcutExe(appId, SHELL);
SteamClient.Apps.SetShortcutStartDir(appId, startDir); SteamClient.Apps.SetShortcutStartDir(appId, startDir);
SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME); SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME);
@@ -284,30 +239,6 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
} }
} }
/**
* Force the visible "Punktfunk" library entry back into existence the recovery button for
* "my shortcut disappeared". Drops any remembered appId that no longer maps to a live shortcut
* (so it can't shadow a fresh AddShortcut), then re-ensures. Safe to press anytime: a shortcut
* that still exists is left in place (no duplicate); a missing one is recreated. Covers the case
* self-heal-on-mount can't deleting the shortcut WITHOUT reinstalling (no mount no ensure).
* Returns the (new or existing) visible appId, or null on failure.
*/
export async function recreateShortcuts(): Promise<number | null> {
for (const key of [STORAGE_KEY_STREAM, STORAGE_KEY_UI]) {
const id = recall(key);
if (id != null && !shortcutStillExists(id)) {
try {
localStorage.removeItem(artKey(id)); // stale art marker for the dead appId
localStorage.removeItem(key);
} catch {
/* ignore */
}
}
}
// Recreate the visible entry now; the hidden stream shortcut re-registers lazily on next launch.
return ensureGamepadUiShortcut();
}
/** Launch the stateless gamepad-UI shortcut (console home) from the plugin, e.g. a QAM button. */ /** Launch the stateless gamepad-UI shortcut (console home) from the plugin, e.g. a QAM button. */
export async function launchGamepadUi(): Promise<void> { export async function launchGamepadUi(): Promise<void> {
const appId = await ensureGamepadUiShortcut(); const appId = await ensureGamepadUiShortcut();
@@ -352,21 +283,13 @@ export async function launchStream(
opts: LaunchOpts = {}, opts: LaunchOpts = {},
): Promise<void> { ): Promise<void> {
// Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now // Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now
// so it races with the shortcut setup (near-zero added latency); its outcome is needed below // so it races with the shortcut setup (near-zero added latency), and awaited just before RunGame.
// (the connect budget), and RunGame follows the await either way, so nothing is slower for it.
// Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is // Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is
// known), and the connect that follows has its own retry window, so a failure never blocks launch. // known), and the connect that follows has its own retry window, so a failure never blocks launch.
const waking = wake(host, port).catch(() => ({ ok: false })); const waking = wake(host, port).catch(() => ({ ok: false }));
const [{ appId, runner }, woke] = await Promise.all([ensureStreamShortcut(), waking]); const { appId, runner } = await ensureStreamShortcut();
const target = port && port !== 9777 ? `${host}:${port}` : host; const target = port && port !== 9777 ? `${host}:${port}` : host;
const env = [`PF_HOST=${target}`]; const env = [`PF_HOST=${target}`];
// A magic packet actually went out (a MAC was known), so the host may be mid-resume from
// suspend — that takes far longer than the client's default 15 s connect budget. Stretch the
// budget so the client's wake-tolerant dial keeps retrying across the resume; against an
// already-awake host the connect still lands in under a second, so this costs nothing.
if (woke.ok) {
env.push("PF_CONNECT_TIMEOUT=75");
}
if (opts.browse) { if (opts.browse) {
env.push("PF_BROWSE=1"); env.push("PF_BROWSE=1");
if (opts.mgmt) { if (opts.mgmt) {
@@ -380,9 +303,9 @@ export async function launchStream(
env.push(`PF_LAUNCH=${opts.launchId}`); env.push(`PF_LAUNCH=${opts.launchId}`);
} }
// KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper // KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper
// script rides behind it as an argument and reads PF_* from the environment. The wake was // script rides behind it as an argument and reads PF_* from the environment.
// awaited above, so the magic packet is out before the connect attempt.
SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`); SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`);
await waking; // ensure the magic packet is out before the connect attempt
SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100); SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100);
} }
-1
View File
@@ -547,7 +547,6 @@ impl AppModel {
crate::video::decodable_codecs(), // codecs (unused by the probe, but honest) crate::video::decodable_codecs(), // codecs (unused by the probe, but honest)
0, // preferred_codec: no preference 0, // preferred_codec: no preference
None, // display_hdr: probe connect, nothing presents None, // display_hdr: probe connect, nothing presents
0, // client_caps: probe connect, nothing renders a cursor
None, // launch: probe connect, no game None, // launch: probe connect, no game
pin, pin,
Some(identity), Some(identity),
+13 -1
View File
@@ -3,7 +3,7 @@
//! scenes. //! scenes.
use crate::app::AppModel; use crate::app::AppModel;
use crate::trust::{forget_placeholder, KnownHost, KnownHosts}; use crate::trust::{KnownHost, KnownHosts};
use crate::ui_hosts::{ConnectRequest, HostsMsg}; use crate::ui_hosts::{ConnectRequest, HostsMsg};
use gtk::glib; use gtk::glib;
use gtk::prelude::*; use gtk::prelude::*;
@@ -252,6 +252,18 @@ fn probe_all(hosts: &[KnownHost]) -> Vec<bool> {
) )
} }
/// Drop an fp-less placeholder for `addr:port` (see `headless_pair`). No-op when none exists.
fn forget_placeholder(addr: &str, port: u16) {
let mut known = KnownHosts::load();
let before = known.hosts.len();
known
.hosts
.retain(|h| !(h.fp_hex.is_empty() && h.addr == addr && h.port == port));
if known.hosts.len() != before {
let _ = known.save();
}
}
/// `--list-hosts [--probe]` — the saved known-hosts store as JSON (the store the Decky plugin /// `--list-hosts [--probe]` — the saved known-hosts store as JSON (the store the Decky plugin
/// renders). With `--probe`, each host carries an `online` bool from a live reachability probe /// renders). With `--probe`, each host carries an `online` bool from a live reachability probe
/// (mDNS-independent); without it, `online` is `null` (unknown — the caller falls back to its /// (mDNS-independent); without it, `online` is `null` (unknown — the caller falls back to its
+1 -40
View File
@@ -69,14 +69,6 @@ const TOUCH_MODE_CAPTIONS: &[&str] = &[
"The cursor jumps to your finger — a tap clicks there", "The cursor jumps to your finger — a tap clicks there",
"Real multi-touch reaches the host — for touch-native apps", "Real multi-touch reaches the host — for touch-native apps",
]; ];
/// Physical-mouse model values (persisted) + labels + dynamic captions — same idiom as
/// the touch rows. Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[&str] = &["capture", "desktop"];
const MOUSE_MODE_LABELS: &[&str] = &["Capture (games)", "Desktop (absolute)"];
const MOUSE_MODE_CAPTIONS: &[&str] = &[
"Pointer locks to the stream — relative motion, best for games",
"Pointer moves freely in and out — best for remote desktop work",
];
/// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page. /// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page.
const APP_LICENSE: &str = concat!( const APP_LICENSE: &str = concat!(
@@ -550,20 +542,6 @@ pub fn show(
set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]); set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]);
}); });
} }
let mouse_row = ChoiceRow::new(
&dialog,
inline,
"Mouse input",
MOUSE_MODE_CAPTIONS[0],
MOUSE_MODE_LABELS,
);
{
let w = mouse_row.widget().clone();
mouse_row.connect_changed(move |i| {
let i = (i as usize).min(MOUSE_MODE_CAPTIONS.len() - 1);
set_row_subtitle(&w, MOUSE_MODE_CAPTIONS[i]);
});
}
let inhibit_row = adw::SwitchRow::builder() let inhibit_row = adw::SwitchRow::builder()
.title("Capture system shortcuts") .title("Capture system shortcuts")
.subtitle("Forward Alt+Tab, Super, … to the host while input is captured") .subtitle("Forward Alt+Tab, Super, … to the host while input is captured")
@@ -740,12 +718,6 @@ pub fn show(
touch_row.set_selected(touch_i as u32); touch_row.set_selected(touch_i as u32);
// set_selected never fires the changed hook, so seed the dynamic caption directly. // set_selected never fires the changed hook, so seed the dynamic caption directly.
set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]); set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]);
let mouse_i = MOUSE_MODES
.iter()
.position(|&m| m == s.mouse_mode)
.unwrap_or(0);
mouse_row.set_selected(mouse_i as u32);
set_row_subtitle(mouse_row.widget(), MOUSE_MODE_CAPTIONS[mouse_i]);
let comp_i = COMPOSITORS let comp_i = COMPOSITORS
.iter() .iter()
.position(|&c| c == s.compositor) .position(|&c| c == s.compositor)
@@ -816,7 +788,6 @@ pub fn show(
touch_group.add(touch_row.widget()); touch_group.add(touch_row.widget());
// Group titles are Pango markup — the ampersand must be an entity. // Group titles are Pango markup — the ampersand must be an entity.
let kbm_group = group("Keyboard &amp; mouse", ""); let kbm_group = group("Keyboard &amp; mouse", "");
kbm_group.add(mouse_row.widget());
kbm_group.add(&inhibit_row); kbm_group.add(&inhibit_row);
kbm_group.add(&invert_row); kbm_group.add(&invert_row);
input.add(&touch_group); input.add(&touch_group);
@@ -885,19 +856,9 @@ pub fn show(
s.render_scale = s.render_scale =
RENDER_SCALES[(scale_row.selected() as usize).min(RENDER_SCALES.len() - 1)]; RENDER_SCALES[(scale_row.selected() as usize).min(RENDER_SCALES.len() - 1)];
s.bitrate_kbps = (bitrate_row.value() * 1000.0) as u32; s.bitrate_kbps = (bitrate_row.value() * 1000.0) as u32;
// Keep a stored preference this table doesn't list (e.g. "switchpro" — valid to the s.gamepad = GAMEPADS[(pad_row.selected() as usize).min(GAMEPADS.len() - 1)].to_string();
// session, hand-edited or written by another client): it displays as "Automatic", and
// writing that back would silently erase it just by opening + closing the dialog.
// Persist the row only when the user picked a non-Auto entry or the stored value was
// a listed one to begin with.
let pad_sel = (pad_row.selected() as usize).min(GAMEPADS.len() - 1);
if pad_sel != 0 || GAMEPADS.contains(&s.gamepad.as_str()) {
s.gamepad = GAMEPADS[pad_sel].to_string();
}
s.touch_mode = s.touch_mode =
TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string(); TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string();
s.mouse_mode =
MOUSE_MODES[(mouse_row.selected() as usize).min(MOUSE_MODES.len() - 1)].to_string();
s.forward_pad = chosen_pin.borrow().clone(); s.forward_pad = chosen_pin.borrow().clone();
s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)] s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)]
.to_string(); .to_string();
+13 -29
View File
@@ -458,11 +458,7 @@ async fn session(args: Args) -> Result<()> {
), ),
(None, None) => tracing::info!(%remote, "punktfunk/1 connected"), (None, None) => tracing::info!(%remote, "punktfunk/1 connected"),
} }
let (mut send, recv) = conn.open_bi().await.context("open control stream")?; let (mut send, mut recv) = conn.open_bi().await.context("open control stream")?;
// Frame every read on the control stream through the resumable reader, exactly as the client
// pump does: `clock_sync` bounds each read with a timeout, and a frame straddling two wakeups
// would otherwise leave the stream permanently misaligned for the rest of the run.
let mut recv = io::MsgReader::new(recv);
io::write_msg( io::write_msg(
&mut send, &mut send,
@@ -487,24 +483,14 @@ async fn session(args: Args) -> Result<()> {
// host/network split is exactly what it exists to report. Old hosts ignore the bit. // host/network split is exactly what it exists to report. Old hosts ignore the bit.
// PROBE_SEQ: the shared-core reassembler windows probe-space frames, so the probe // PROBE_SEQ: the shared-core reassembler windows probe-space frames, so the probe
// qualifies for `--speed-test` bursts; without the bit the host declines them. // qualifies for `--speed-test` bursts; without the bit the host declines them.
// STREAMED_AU: the same shared reassembler accepts sentinel-headed streamed
// blocks, and the probe is exactly the tool that measures the overlap win.
let mut caps = punktfunk_core::quic::VIDEO_CAP_HOST_TIMING let mut caps = punktfunk_core::quic::VIDEO_CAP_HOST_TIMING
| punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ | punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ;
| punktfunk_core::quic::VIDEO_CAP_STREAMED_AU;
if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() { if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_10BIT; caps |= punktfunk_core::quic::VIDEO_CAP_10BIT;
} }
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() { if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444; caps |= punktfunk_core::quic::VIDEO_CAP_444;
} }
// PUNKTFUNK_CLIENT_CHACHA20=1 advertises VIDEO_CAP_CHACHA20 — drives the
// host's ChaCha20-Poly1305 session-cipher resolution (the soft-AES armv7
// negotiation, design/chacha20-session-cipher.md §7) without a webOS build;
// the negotiated cipher is reported in the welcome log line below.
if std::env::var_os("PUNKTFUNK_CLIENT_CHACHA20").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_CHACHA20;
}
caps caps
}, },
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the // `--audio-channels` (default stereo); the probe multistream-decodes + validates the
@@ -523,15 +509,12 @@ async fn session(args: Args) -> Result<()> {
// writes it into the virtual display's EDID (CTA HDR block), so the EDID-forwarding // writes it into the virtual display's EDID (CTA HDR block), so the EDID-forwarding
// path can be validated headlessly (check the host's monitor caps / ADD log line). // path can be validated headlessly (check the host's monitor caps / ADD log line).
display_hdr: punktfunk_core::client::display_hdr_env_override(), display_hdr: punktfunk_core::client::display_hdr_env_override(),
// No CLIENT_CAP_CURSOR: this headless tool renders nothing — advertising it would
// just strip the pointer from the dumped bitstream.
client_caps: 0,
} }
.encode(), .encode(),
) )
.await?; .await?;
let welcome = let welcome = Welcome::decode(&io::read_msg(&mut recv).await?)
Welcome::decode(&recv.read_msg().await?).map_err(|e| anyhow!("Welcome decode: {e:?}"))?; .map_err(|e| anyhow!("Welcome decode: {e:?}"))?;
tracing::info!( tracing::info!(
mode = ?welcome.mode, mode = ?welcome.mode,
fec = ?welcome.fec, fec = ?welcome.fec,
@@ -545,11 +528,6 @@ async fn session(args: Args) -> Result<()> {
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444, chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format, chroma_format_idc = welcome.chroma_format,
codec = codec_ext(welcome.codec), codec = codec_ext(welcome.codec),
cipher = if welcome.cipher == punktfunk_core::quic::CIPHER_CHACHA20_POLY1305 {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
"session offer" "session offer"
); );
@@ -651,7 +629,10 @@ async fn session(args: Args) -> Result<()> {
tracing::error!("Reconfigure write failed"); tracing::error!("Reconfigure write failed");
return; return;
} }
match rr.read_msg().await.map(|b| Reconfigured::decode(&b)) { match io::read_msg(&mut rr)
.await
.map(|b| Reconfigured::decode(&b))
{
Ok(Ok(ack)) if ack.accepted => { Ok(Ok(ack)) if ack.accepted => {
tracing::info!(mode = ?ack.mode, "mode switch ACCEPTED") tracing::info!(mode = ?ack.mode, "mode switch ACCEPTED")
} }
@@ -704,7 +685,10 @@ async fn session(args: Args) -> Result<()> {
tracing::error!("SetBitrate write failed"); tracing::error!("SetBitrate write failed");
return; return;
} }
match rr.read_msg().await.map(|b| BitrateChanged::decode(&b)) { match io::read_msg(&mut rr)
.await
.map(|b| BitrateChanged::decode(&b))
{
Ok(Ok(ack)) => tracing::info!( Ok(Ok(ack)) => tracing::info!(
applied_kbps = ack.bitrate_kbps, applied_kbps = ack.bitrate_kbps,
"BITRATE CHANGE acked by host" "BITRATE CHANGE acked by host"
@@ -766,7 +750,7 @@ async fn session(args: Args) -> Result<()> {
tracing::error!("ProbeRequest write failed"); tracing::error!("ProbeRequest write failed");
return; return;
} }
let res = match sr.read_msg().await.map(|b| ProbeResult::decode(&b)) { let res = match io::read_msg(&mut sr).await.map(|b| ProbeResult::decode(&b)) {
Ok(Ok(r)) => r, Ok(Ok(r)) => r,
other => { other => {
tracing::error!(?other, "bad ProbeResult"); tracing::error!(?other, "bad ProbeResult");
+1 -9
View File
@@ -8,7 +8,6 @@ presenter of the Linux client re-architecture (punktfunk-planning:
``` ```
punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen] [--stats] punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen] [--stats]
punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen] punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen]
punktfunk-session --pair <PIN> --connect host[:port] [--name LABEL]
``` ```
`--browse` opens the console game library (the Skia coverflow over the animated aurora) `--browse` opens the console game library (the Skia coverflow over the animated aurora)
@@ -18,16 +17,9 @@ pairing is the desktop client / Decky plugin's job. `PUNKTFUNK_FAKE_LIBRARY=<fil
feeds canned entries with no host (portrait paths starting with `/` load from disk). feeds canned entries with no host (portrait paths starting with `/` load from disk).
Reads the same identity / known-hosts / settings stores as the desktop client Reads the same identity / known-hosts / settings stores as the desktop client
(`punktfunk-client`), so enrolling on either side makes the other work; this binary never (`punktfunk-client`) — pair there (or via its headless `--pair`) first; this binary never
connects to a host it has no pinned fingerprint for (`--fp HEX` overrides the store). connects to a host it has no pinned fingerprint for (`--fp HEX` overrides the store).
`--pair <PIN> --connect host[:port]` runs the SPAKE2 ceremony with no window and no
toolkit, prints `paired <addr>:<port> fp=<hex>`, and exits — the route for a machine that
has only SSH (an embedded/kiosk client, an image being provisioned). `--name` sets the
label the host files this client under, defaulting to the hostname. It is in the
`--no-default-features` build too: enrolling must never be the reason a minimal image has
to pull in Skia.
Stdout is the machine interface: `{"ready":true}` after the first presented frame, Stdout is the machine interface: `{"ready":true}` after the first presented frame,
`stats: …` once per second while the overlay tier isn't Off (always the full detailed `stats: …` once per second while the overlay tier isn't Off (always the full detailed
text, whatever the OSD shows; `--stats` forces the overlay on), one text, whatever the OSD shows; `--stats` forces the overlay on), one
+17 -2
View File
@@ -105,7 +105,7 @@ pub fn run(target: Option<&str>) -> u8 {
}; };
let opts = ConsoleOptions { let opts = ConsoleOptions {
device_name: trust::device_name(), device_name: device_name(),
deck: is_steam_deck(), deck: is_steam_deck(),
}; };
let (overlay, handles) = match SkiaOverlay::console(opts, entry) { let (overlay, handles) = match SkiaOverlay::console(opts, entry) {
@@ -158,7 +158,6 @@ pub fn run(target: Option<&str>) -> u8 {
v => v, v => v,
}, },
touch_mode: settings_at_start.touch_mode(), touch_mode: settings_at_start.touch_mode(),
mouse_mode: settings_at_start.mouse_mode(),
invert_scroll: settings_at_start.invert_scroll, invert_scroll: settings_at_start.invert_scroll,
json_status, json_status,
on_connected: Some(Box::new(move |fingerprint: [u8; 32]| { on_connected: Some(Box::new(move |fingerprint: [u8; 32]| {
@@ -251,6 +250,22 @@ pub fn run(target: Option<&str>) -> u8 {
} }
} }
/// The machine's name — what the host lists this client as after pairing.
fn device_name() -> String {
#[cfg(target_os = "linux")]
if let Ok(s) = std::fs::read_to_string("/etc/hostname") {
let s = s.trim();
if !s.is_empty() {
return s.to_string();
}
}
std::env::var("COMPUTERNAME")
.or_else(|_| std::env::var("HOSTNAME"))
.ok()
.filter(|s| !s.trim().is_empty())
.unwrap_or_else(|| "This device".into())
}
fn host_display_name(name: &str, addr: &str) -> String { fn host_display_name(name: &str, addr: &str) -> String {
if name.trim().is_empty() { if name.trim().is_empty() {
addr.to_string() addr.to_string()
+4 -68
View File
@@ -5,9 +5,8 @@
//! One stream session per invocation: `--connect host[:port]` (+ `--fp HEX`, //! One stream session per invocation: `--connect host[:port]` (+ `--fp HEX`,
//! `--launch id`, `--fullscreen`), exits when the session ends. Reads the same identity //! `--launch id`, `--fullscreen`), exits when the session ends. Reads the same identity
//! / known-hosts / settings stores as the desktop shell on each OS — the GTK client //! / known-hosts / settings stores as the desktop shell on each OS — the GTK client
//! (`punktfunk-client`) on Linux, the WinUI client on Windows — so pairing on either side //! (`punktfunk-client`) on Linux, the WinUI client on Windows — so pairing there (or
//! makes the other connect silently. `--pair <PIN> --connect host` runs the ceremony here, //! via the shell's headless `--pair`) makes this binary connect silently.
//! with no window and no toolkit, for machines that have only a shell.
//! //!
//! Stdout is the machine interface (the shell↔session contract): `{"ready":true}` after //! Stdout is the machine interface (the shell↔session contract): `{"ready":true}` after
//! the first presented frame, `stats:` lines per 1 s window, one `{"error": …}` / //! the first presented frame, `stats:` lines per 1 s window, one `{"error": …}` /
@@ -62,54 +61,6 @@ mod session_main {
Some((x.trim().parse().ok()?, y.trim().parse().ok()?)) Some((x.trim().parse().ok()?, y.trim().parse().ok()?))
} }
/// `--pair <PIN> --connect host[:port]` — the SPAKE2 PIN ceremony with no window, no GTK
/// and no console UI, so a machine that has only SSH can be enrolled: an embedded/kiosk
/// client, a headless box, an image being provisioned. Writes the verified host into the
/// same known-hosts store `--connect` reads, so pairing here is exactly what makes the
/// later stream connect silently.
///
/// Deliberately identical in shape and output to `punktfunk-client --pair` (which stays
/// the desktop route) — the difference is only that this binary carries no toolkit, so it
/// is the one a minimal image installs. Present in the `--no-default-features` build too:
/// enrolment must not be the reason an embedded image has to pull in Skia.
fn headless_pair(pin: &str) -> u8 {
let Some(target) = arg_value("--connect") else {
eprintln!("--pair requires --connect host[:port]");
return EXIT_CONNECT_FAILED;
};
let (addr, port) = parse_host_port(&target);
// The label the HOST files this client under. A headless box has nobody to ask, so
// the hostname is the only name that will mean anything in the paired-devices list.
let name = arg_value("--name").unwrap_or_else(trust::device_name);
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
eprintln!("client identity: {e:#}");
return EXIT_CONNECT_FAILED;
}
};
match trust::pair_with_host(&addr, port, &identity, pin, &name) {
Ok(fp) => {
let fp_hex = trust::hex(&fp);
trust::persist_host(
&arg_value("--host-label").unwrap_or_else(|| addr.clone()),
&addr,
port,
&fp_hex,
true,
);
trust::forget_placeholder(&addr, port);
println!("paired {addr}:{port} fp={fp_hex}");
0
}
Err(e) => {
eprintln!("pairing failed: {} ({e:?})", trust::pair_error_message(&e));
EXIT_TRUST_REJECTED
}
}
}
/// `host[:port]`, port defaulting to the native 9777. /// `host[:port]`, port defaulting to the native 9777.
pub(crate) fn parse_host_port(target: &str) -> (String, u16) { pub(crate) fn parse_host_port(target: &str) -> (String, u16) {
match target.rsplit_once(':') { match target.rsplit_once(':') {
@@ -221,11 +172,6 @@ mod session_main {
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session // defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
// pump) pins one manually. // pump) pins one manually.
display_hdr: None, display_hdr: None,
// The presenter renders the host cursor locally in desktop mouse mode (M2 cursor
// channel); capture-mode sessions keep the composited cursor, so only advertise
// when the session STARTS in desktop mode. The host gates further (Linux portal
// compositors only).
cursor_forward: settings.mouse_mode() == trust::MouseMode::Desktop,
mic_enabled: settings.mic_enabled, mic_enabled: settings.mic_enabled,
clipboard, clipboard,
// The Settings preference (auto → VAAPI where it exists; the presenter // The Settings preference (auto → VAAPI where it exists; the presenter
@@ -361,13 +307,6 @@ mod session_main {
}; };
} }
// `--pair <PIN>`: enrol this machine against a host and exit. Sits with the other
// non-streaming subcommands, above every graphics call — the box doing this may have
// no display at all.
if let Some(pin) = arg_value("--pair") {
return headless_pair(&pin);
}
// Before any Vulkan call: make RADV expose its video-decode queue + extensions so the // Before any Vulkan call: make RADV expose its video-decode queue + extensions so the
// decoder's `auto` path prefers Vulkan Video over VAAPI (Steam Deck, and any gated RADV). // decoder's `auto` path prefers Vulkan Video over VAAPI (Steam Deck, and any gated RADV).
// Windows drivers (NVIDIA/AMD Adrenalin) expose theirs unconditionally. // Windows drivers (NVIDIA/AMD Adrenalin) expose theirs unconditionally.
@@ -425,14 +364,12 @@ mod session_main {
eprintln!( eprintln!(
"usage: punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen]\n\ "usage: punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen]\n\
\x20 punktfunk-session --browse [host[:port]] [--mgmt PORT] [--fullscreen] [--json-status]\n\ \x20 punktfunk-session --browse [host[:port]] [--mgmt PORT] [--fullscreen] [--json-status]\n\
\x20 punktfunk-session --pair <PIN> --connect host[:port] [--name LABEL]\n\
\n\ \n\
Streams from a paired punktfunk host in a Vulkan window. --browse opens the\n\ Streams from a paired punktfunk host in a Vulkan window. --browse opens the\n\
gamepad console instead: bare --browse is the host list (discovery, PIN\n\ gamepad console instead: bare --browse is the host list (discovery, PIN\n\
pairing, settings, wake-on-LAN); with a target it opens that host's game\n\ pairing, settings, wake-on-LAN); with a target it opens that host's game\n\
library. --connect never dials a host it has no pinned fingerprint for \n\ library. --connect never dials a host it has no pinned fingerprint for \n\
enrol with --pair (no display needed), in the console, or from the desktop\n\ pair in the console or via `punktfunk-client --pair <PIN> --connect `."
client."
); );
return EXIT_CONNECT_FAILED; return EXIT_CONNECT_FAILED;
}; };
@@ -464,7 +401,7 @@ mod session_main {
"error", "error",
&format!( &format!(
"no pinned fingerprint for {addr}:{port} — pair first \ "no pinned fingerprint for {addr}:{port} — pair first \
(punktfunk-session --pair <PIN> --connect {addr}:{port}) or pass --fp HEX" (punktfunk-client --pair <PIN> --connect {addr}:{port}) or pass --fp HEX"
), ),
Some(true), Some(true),
); );
@@ -492,7 +429,6 @@ mod session_main {
v => v, v => v,
}, },
touch_mode: settings.touch_mode(), touch_mode: settings.touch_mode(),
mouse_mode: settings.mouse_mode(),
invert_scroll: settings.invert_scroll, invert_scroll: settings.invert_scroll,
json_status: true, json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| { on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
-18
View File
@@ -90,13 +90,6 @@ const TOUCH_MODES: &[(&str, &str)] = &[
("pointer", "Direct pointer"), ("pointer", "Direct pointer"),
("touch", "Touch passthrough"), ("touch", "Touch passthrough"),
]; ];
/// Physical-mouse presets: `(stored value, display label)` — capture (pointer lock,
/// relative, for games) vs desktop (uncaptured absolute pointer, for remote desktop
/// work). Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[(&str, &str)] = &[
("capture", "Capture (games)"),
("desktop", "Desktop (absolute)"),
];
/// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to /// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to
/// auto-detect when the choice is unavailable. Only meaningful against a Linux host. /// auto-detect when the choice is unavailable. Only meaningful against a Linux host.
const COMPOSITORS: &[(&str, &str)] = &[ const COMPOSITORS: &[(&str, &str)] = &[
@@ -401,10 +394,6 @@ pub(crate) fn settings_page(
let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| { let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| {
s.touch_mode = TOUCH_MODES[i].0.to_string(); s.touch_mode = TOUCH_MODES[i].0.to_string();
}); });
let (mouse_names, mouse_i) = presets(MOUSE_MODES, |v| *v == s.mouse_mode);
let mouse_combo = setting_combo(ctx, "Mouse input", mouse_names, mouse_i, |s, i| {
s.mouse_mode = MOUSE_MODES[i].0.to_string();
});
let invert_scroll_toggle = let invert_scroll_toggle =
setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| { setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| {
s.invert_scroll = on s.invert_scroll = on
@@ -553,13 +542,6 @@ pub(crate) fn settings_page(
out.extend(group( out.extend(group(
Some("Keyboard & mouse"), Some("Keyboard & mouse"),
vec![ vec![
described(
mouse_combo,
"Capture locks the pointer to the stream and sends relative motion — \
best for games. Desktop leaves the pointer free to enter and leave \
the stream and sends absolute positions best for remote desktop \
work. Ctrl+Alt+Shift+M switches live.",
),
described( described(
shortcuts_toggle, shortcuts_toggle,
"Alt+Tab, the Windows key and friends reach the host while the stream \ "Alt+Tab, the Windows key and friends reach the host while the stream \
-1
View File
@@ -56,7 +56,6 @@ pub fn run_speed_probe(
decodable_codecs(), decodable_codecs(),
0, // preferred_codec: no preference 0, // preferred_codec: no preference
None, // display_hdr: probe connect, nothing presents None, // display_hdr: probe connect, nothing presents
0, // client_caps: probe connect, nothing renders a cursor
None, // launch: no game None, // launch: no game
pin, pin,
Some(identity), Some(identity),
-8
View File
@@ -30,12 +30,6 @@ pipewire = "0.9"
libc = "0.2" libc = "0.2"
# ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane). # ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane).
tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] } tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
# XFixes cursor source for gamescope (remote-desktop-sweep Phase C): gamescope paints no
# `SPA_META_Cursor`, so the pointer never reaches the PipeWire node. We read the shape/hotspot/
# visibility from gamescope's nested Xwayland via XFixes instead and feed the existing cursor slot.
# `RustConnection` is the pure-Rust default (no libxcb link → no new C dependency on the host); the
# `xfixes` feature (auto-pulls `render` + `shape`) is what exposes GetCursorImage/SelectCursorInput.
x11rb = { version = "0.13", default-features = false, features = ["xfixes"] }
[target.'cfg(target_os = "windows")'.dependencies] [target.'cfg(target_os = "windows")'.dependencies]
# The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header). # The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header).
@@ -49,9 +43,7 @@ windows = { version = "0.62", features = [
"Win32_Graphics_Direct3D_Fxc", "Win32_Graphics_Direct3D_Fxc",
"Win32_Graphics_Dxgi", "Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common", "Win32_Graphics_Dxgi_Common",
"Win32_Graphics_Gdi",
"Win32_System_LibraryLoader", "Win32_System_LibraryLoader",
"Win32_System_StationsAndDesktops",
"Win32_System_Memory", "Win32_System_Memory",
"Win32_System_Threading", "Win32_System_Threading",
"Win32_UI_HiDpi", "Win32_UI_HiDpi",
-70
View File
@@ -24,16 +24,6 @@ use pf_frame::DmabufFrame;
pub trait Capturer: Send { pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>; fn next_frame(&mut self) -> Result<CapturedFrame>;
/// [`next_frame`](Self::next_frame) with a caller-chosen first-frame budget instead of the
/// backend's default. The pipeline retry loop shortens its FIRST attempt's wait: a PipeWire
/// stream connected while gamescope re-inits its headless takeover can negotiate a format,
/// reach `Streaming`, and still never receive a buffer — a fresh connect then delivers within
/// ~0.5 s, so waiting out the full default budget on a doomed stream just delays the retry
/// that fixes it. Backends without an internal wait budget ignore it (the default delegates).
fn next_frame_within(&mut self, _budget: std::time::Duration) -> Result<CapturedFrame> {
self.next_frame()
}
/// Non-blocking: the freshest frame available since the last call, or `None` if none has /// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default /// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal /// just produces a frame each call — fine for instant synthetic sources; the portal
@@ -69,34 +59,6 @@ pub trait Capturer: Send {
/// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet). /// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet).
/// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram), /// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram),
/// so the two stay a single source of truth. May change mid-session if the source is regraded. /// so the two stay a single source of truth. May change mid-session if the source is regraded.
/// The capture source's LIVE cursor state, when it arrives out-of-band from the frames
/// (the Windows IddCx hardware-cursor channel). Polled by the encode loop every tick and
/// preferred over `CapturedFrame::cursor` — with a hardware cursor, pointer-only moves
/// produce NO new frame, so the frame-attached overlay would go stale on a static desktop.
/// Default `None`: the Linux portal path attaches its cursor to frames instead.
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
None
}
/// LIVE cursor-render flip for a cursor-forward session (design/remote-desktop-sweep.md §8):
/// `on = true` — the client draws the pointer, keep it OUT of the video; `on = false` —
/// the capture mouse model, the pointer must be IN the video again. The Windows IDD
/// capturer implements the composite side ITSELF (slot-copy + alpha-blended quad from the
/// GDI poller) — a declared IddCx hardware cursor is irrevocable, so DWM can never be
/// handed the job back. Called every encode tick (implementations cache; steady state is
/// one compare). Default no-op: the Linux portal never bakes the pointer into frames —
/// the encode loop blends its overlay instead.
fn set_cursor_forward(&mut self, _on: bool) {}
/// Attach a gamescope cursor source (remote-desktop-sweep Phase C). gamescope paints no
/// `SPA_META_Cursor`, so [`cursor`](Self::cursor)'s slot stays empty — this hands the Linux
/// portal capturer gamescope's nested Xwayland `(DISPLAY, XAUTHORITY)` targets (it may run
/// several — one per `--xwayland-count`) so it reads the pointer shape/position over X11
/// (XFixes + QueryPointer), following whichever display is focused, and publishes it into that
/// same slot. Called once, after the capturer is built, only for gamescope sessions. Default
/// no-op: every non-gamescope capturer already has a cursor source.
fn attach_gamescope_cursor(&mut self, _targets: Vec<(String, Option<String>)>) {}
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> { fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None None
} }
@@ -287,12 +249,6 @@ pub struct ZeroCopyPolicy {
/// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only /// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only
/// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`. /// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`.
pub pyrowave_session: bool, pub pyrowave_session: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (the Linux raw Vulkan
/// Video backend on an H265/AV1 session — resolved by the host facade via
/// `pf_encode::linux_native_nv12_ok`). Gates whether the negotiation PREFERS gamescope's
/// producer-side NV12 pod: libav VAAPI (H264's backend) would misread the two-plane buffer,
/// so H264/GameStream/PyroWave sessions must never see NV12 frames.
pub native_nv12_session: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc, /// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA, /// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise. /// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
@@ -395,26 +351,6 @@ pub type FrameChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync, dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync,
>; >;
/// Delivery closure for the v5 hardware-cursor channel (`IOCTL_SET_CURSOR_CHANNEL`) — same
/// facade contract as [`FrameChannelSender`]. `Some` also OPTS THE SESSION IN: the capturer
/// creates + delivers the cursor section only when the host hands it a sender (the negotiated
/// cursor-forward sessions), and the driver only declares the hardware cursor once that
/// delivery lands — so a plain session keeps DWM's composited pointer untouched.
#[cfg(target_os = "windows")]
pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
/// The mid-stream cursor-render flip (`IOCTL_SET_CURSOR_FORWARD`, proto v6) as a host-facade
/// closure — same contract as [`CursorChannelSender`]. `bool` = declare the IddCx hardware
/// cursor (`true`) or stand it down (`false`; the host facade additionally forces the same-mode
/// re-commit that actualises the OS's software-cursor default). The capturer drives this from
/// its secure-desktop watch: UAC/Winlogon render only through the software-cursor path, so a
/// path pinned to the hardware cursor never presents them (the 0.18.0 secure-desktop
/// regression).
#[cfg(target_os = "windows")]
pub type CursorForwardSender = std::sync::Arc<dyn Fn(bool) -> Result<()> + Send + Sync>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads. // One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub mod pwinit; pub mod pwinit;
@@ -472,7 +408,6 @@ pub fn open_virtual_output(
allow_zerocopy: bool, allow_zerocopy: bool,
want_444: bool, want_444: bool,
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
expect_exact_dims: bool,
) -> Result<Box<dyn Capturer>> { ) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::from_virtual_output( linux::PortalCapturer::from_virtual_output(
remote_fd, remote_fd,
@@ -482,7 +417,6 @@ pub fn open_virtual_output(
allow_zerocopy, allow_zerocopy,
want_444, want_444,
policy, policy,
expect_exact_dims,
) )
.map(|c| Box::new(c) as Box<dyn Capturer>) .map(|c| Box::new(c) as Box<dyn Capturer>)
} }
@@ -500,8 +434,6 @@ pub fn open_idd_push(
pyrowave: bool, pyrowave: bool,
keepalive: Box<dyn Send>, keepalive: Box<dyn Send>,
sender: FrameChannelSender, sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
cursor_forward: Option<CursorForwardSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> { ) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open( idd_push::IddPushCapturer::open(
target, target,
@@ -511,8 +443,6 @@ pub fn open_idd_push(
pyrowave, pyrowave,
keepalive, keepalive,
sender, sender,
cursor_sender,
cursor_forward,
) )
.map(|c| Box::new(c) as Box<dyn Capturer>) .map(|c| Box::new(c) as Box<dyn Capturer>)
} }
+108 -447
View File
@@ -22,10 +22,6 @@
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy}; use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy};
use anyhow::{anyhow, Context, Result}; use anyhow::{anyhow, Context, Result};
// gamescope cursor source (remote-desktop-sweep Phase C) — feeds `cursor_live` from XFixes when
// the PipeWire node carries no `SPA_META_Cursor` (gamescope's does not).
mod xfixes_cursor;
use std::os::fd::OwnedFd; use std::os::fd::OwnedFd;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, TryRecvError}; use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, TryRecvError};
@@ -61,12 +57,6 @@ pub struct PortalCapturer {
/// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream /// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream
/// is `Streaming`. /// is `Streaming`.
stall_since: Option<std::time::Instant>, stall_since: Option<std::time::Instant>,
/// The LIVE cursor overlay, published by the PipeWire thread from every buffer's
/// `SPA_META_Cursor` — including the cursor-only "corrupted" buffers that never become
/// frames. [`Capturer::cursor`] serves it so the encode loop's forwarder tracks pointer-only
/// motion on a static desktop; the frame-attached overlay alone goes stale between damage
/// frames (the same gap the Windows IddCx channel fills, and why the tick prefers LIVE).
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
/// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If /// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If
/// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches /// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline /// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
@@ -92,12 +82,6 @@ pub struct PortalCapturer {
/// is, releasing the compositor-side output via the keepalive's own `Drop`. `None` for the /// is, releasing the compositor-side output via the keepalive's own `Drop`. `None` for the
/// portal source (its session ends with the portal thread's zbus connection). /// portal source (its session ends with the portal thread's zbus connection).
_keepalive: Option<Box<dyn Send>>, _keepalive: Option<Box<dyn Send>>,
/// The gamescope XFixes cursor reader (remote-desktop-sweep Phase C), when this capturer
/// serves a gamescope node. `Some` after
/// [`attach_gamescope_cursor`](Capturer::attach_gamescope_cursor); its `Drop` stops the reader
/// thread, so it lives exactly as long as the capturer. `None` on the portal path (its cursor
/// comes from `SPA_META_Cursor`).
_gs_cursor: Option<xfixes_cursor::XFixesCursorSource>,
} }
impl PortalCapturer { impl PortalCapturer {
@@ -130,17 +114,10 @@ impl PortalCapturer {
"ScreenCast portal session started; connecting PipeWire" "ScreenCast portal session started; connecting PipeWire"
); );
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before. // This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok(spawn_pipewire( Ok(
Some(fd), spawn_pipewire(Some(fd), node_id, None, true, false, want_hdr, policy)?
node_id, .into_capturer(node_id, None),
None, )
true,
false,
want_hdr,
policy,
false,
)?
.into_capturer(node_id, None))
} }
/// Build a capturer from an already-created virtual output's PipeWire node. The host facade /// Build a capturer from an already-created virtual output's PipeWire node. The host facade
@@ -160,13 +137,11 @@ impl PortalCapturer {
allow_zerocopy: bool, allow_zerocopy: bool,
want_444: bool, want_444: bool,
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
expect_exact_dims: bool,
) -> Result<PortalCapturer> { ) -> Result<PortalCapturer> {
tracing::info!( tracing::info!(
node_id, node_id,
allow_zerocopy, allow_zerocopy,
want_444, want_444,
expect_exact_dims,
"connecting PipeWire to virtual output" "connecting PipeWire to virtual output"
); );
// Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit // Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit
@@ -179,7 +154,6 @@ impl PortalCapturer {
want_444, want_444,
false, false,
policy, policy,
expect_exact_dims,
)? )?
.into_capturer(node_id, Some(keepalive))) .into_capturer(node_id, Some(keepalive)))
} }
@@ -202,8 +176,6 @@ struct PwHandles {
hdr_offer: bool, hdr_offer: bool,
/// See [`PortalCapturer::hdr_negotiated`]. /// See [`PortalCapturer::hdr_negotiated`].
hdr_negotiated: Arc<AtomicBool>, hdr_negotiated: Arc<AtomicBool>,
/// See [`PortalCapturer::cursor_live`].
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
quit: ::pipewire::channel::Sender<()>, quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>, join: thread::JoinHandle<()>,
} }
@@ -223,12 +195,10 @@ impl PwHandles {
vaapi_dmabuf: self.vaapi_dmabuf, vaapi_dmabuf: self.vaapi_dmabuf,
hdr_offer: self.hdr_offer, hdr_offer: self.hdr_offer,
hdr_negotiated: self.hdr_negotiated, hdr_negotiated: self.hdr_negotiated,
cursor_live: self.cursor_live,
node_id, node_id,
quit: Some(self.quit), quit: Some(self.quit),
join: Some(self.join), join: Some(self.join),
_keepalive: keepalive, _keepalive: keepalive,
_gs_cursor: None,
} }
} }
} }
@@ -236,7 +206,6 @@ impl PwHandles {
/// Spawn the PipeWire consumer thread for `node_id` (fd `Some` = portal remote, `None` = /// Spawn the PipeWire consumer thread for `node_id` (fd `Some` = portal remote, `None` =
/// default daemon) and return its [`PwHandles`]. `preferred` seeds the format negotiation's /// default daemon) and return its [`PwHandles`]. `preferred` seeds the format negotiation's
/// default size/framerate — for Mutter virtual monitors this is what actually sizes the monitor. /// default size/framerate — for Mutter virtual monitors this is what actually sizes the monitor.
#[allow(clippy::too_many_arguments)]
fn spawn_pipewire( fn spawn_pipewire(
fd: Option<OwnedFd>, fd: Option<OwnedFd>,
node_id: u32, node_id: u32,
@@ -255,12 +224,6 @@ fn spawn_pipewire(
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way // Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6). // capture→encode edge (plan §W6).
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
// The producer's FIRST negotiation is for a sacrificial mode and a renegotiation to
// `preferred`'s dims is guaranteed to follow (KWin virtual outputs — see kwin.rs `create`):
// skip whole buffers until the negotiated size matches, so the pipeline never builds against
// the doomed birth mode. `false` everywhere else (Mutter SIZES the monitor from negotiation,
// gamescope fixates its own — gating those would starve legitimate first frames).
expect_exact_dims: bool,
) -> Result<PwHandles> { ) -> Result<PwHandles> {
// Frames flow from the pipewire thread over a small bounded channel. // Frames flow from the pipewire thread over a small bounded channel.
let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8); let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8);
@@ -274,8 +237,6 @@ fn spawn_pipewire(
let broken_cb = broken.clone(); let broken_cb = broken.clone();
let hdr_negotiated = Arc::new(AtomicBool::new(false)); let hdr_negotiated = Arc::new(AtomicBool::new(false));
let hdr_negotiated_cb = hdr_negotiated.clone(); let hdr_negotiated_cb = hdr_negotiated.clone();
let cursor_live = Arc::new(std::sync::Mutex::new(None::<pf_frame::CursorOverlay>));
let cursor_live_cb = cursor_live.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the // pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the // sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope. // inner `mod pipewire` shadows the crate name at this scope.
@@ -311,14 +272,12 @@ fn spawn_pipewire(
streaming_cb, streaming_cb,
broken_cb, broken_cb,
hdr_negotiated_cb, hdr_negotiated_cb,
cursor_live_cb,
zerocopy, zerocopy,
want_444, want_444,
want_hdr, want_hdr,
preferred, preferred,
quit_rx, quit_rx,
policy, policy,
expect_exact_dims,
) { ) {
tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed"); tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed");
} }
@@ -333,7 +292,6 @@ fn spawn_pipewire(
vaapi_dmabuf, vaapi_dmabuf,
hdr_offer: want_hdr, hdr_offer: want_hdr,
hdr_negotiated, hdr_negotiated,
cursor_live,
quit: quit_tx, quit: quit_tx,
join, join,
}) })
@@ -341,29 +299,29 @@ fn spawn_pipewire(
impl Capturer for PortalCapturer { impl Capturer for PortalCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> { fn next_frame(&mut self) -> Result<CapturedFrame> {
self.frame_within(Duration::from_secs(10)) // First frame can lag behind format negotiation; later frames arrive at ~fps. Wait in
// short slices so a GPU-import poison (worker death) fails the capture within ~0.5 s
// instead of sitting out the full first-frame budget.
let deadline = std::time::Instant::now() + Duration::from_secs(10);
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
} }
if let Some(f) = self.pending.take() {
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> { return Ok(f); // a wait_arrival stash outranks the channel (it's older)
// The PipeWire thread's live cursor slot (fed by every buffer's meta, frames or not) — }
// lets the forwarder track pointer-only motion on a static desktop. See `cursor_live`. let slice = Duration::from_millis(500)
// On a gamescope node the meta never arrives; the XFixes source (attached below) fills .min(deadline.saturating_duration_since(std::time::Instant::now()));
// the same slot instead. match self.frames.recv_timeout(slice) {
self.cursor_live.lock().ok().and_then(|slot| slot.clone()) Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e),
} }
fn attach_gamescope_cursor(&mut self, targets: Vec<(String, Option<String>)>) {
// gamescope paints no `SPA_META_Cursor`, so `cursor_live` would stay empty. Spawn the
// XFixes reader to publish gamescope's pointer into that SAME slot — `cursor()` above then
// serves it and the encode loop composites it, exactly like the portal path. It connects
// to every nested Xwayland and follows the focused one's pointer. A failure (no Xwayland /
// no XFixes) logs and leaves the slot empty = today's cursorless gamescope.
self._gs_cursor =
xfixes_cursor::XFixesCursorSource::spawn(targets, Arc::clone(&self.cursor_live));
} }
fn next_frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
self.frame_within(budget)
} }
fn supports_arrival_wait(&self) -> bool { fn supports_arrival_wait(&self) -> bool {
@@ -459,41 +417,9 @@ impl Capturer for PortalCapturer {
} }
impl PortalCapturer { impl PortalCapturer {
/// The blocking first-frame wait behind [`Capturer::next_frame`] / /// The [`Capturer::next_frame`] budget expired (or the thread ended) — turn it into the
/// [`Capturer::next_frame_within`]. First frame can lag behind format negotiation; later /// diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
/// frames arrive at ~fps. Wait in short slices so a GPU-import poison (worker death) fails fn next_frame_timed_out(&self, err: RecvTimeoutError) -> Result<CapturedFrame> {
/// the capture within ~0.5 s instead of sitting out the full first-frame budget.
fn frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
let deadline = std::time::Instant::now() + budget;
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e, budget),
}
}
}
/// The [`frame_within`](Self::frame_within) budget expired (or the thread ended) — turn it
/// into the diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(
&self,
err: RecvTimeoutError,
budget: Duration,
) -> Result<CapturedFrame> {
let within = budget.as_secs_f32();
match err { match err {
RecvTimeoutError::Timeout => { RecvTimeoutError::Timeout => {
// Split the two black-screen root causes apart so the operator gets a cause, not // Split the two black-screen root causes apart so the operator gets a cause, not
@@ -501,10 +427,9 @@ impl PortalCapturer {
// not (no acceptable format / node never emitted a param)? // not (no acceptable format / node never emitted a param)?
if self.negotiated.load(Ordering::Relaxed) { if self.negotiated.load(Ordering::Relaxed) {
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within {within}s (node {}): format negotiated but no \ "no PipeWire frame within 10s (node {}): format negotiated but no buffers \
buffers arrived the compositor produced no frames (virtual output \ arrived the compositor produced no frames (virtual output idle/unmapped, \
idle/unmapped, capture never started, or a stream bound during a \ or capture never started)",
compositor (re)start that will never deliver a reconnect fixes that)",
self.node_id self.node_id
)) ))
} else if self.hdr_offer { } else if self.hdr_offer {
@@ -515,10 +440,10 @@ impl PortalCapturer {
// auto-reconnects) negotiates SDR instead of re-running this same timeout. // auto-reconnects) negotiates SDR instead of re-running this same timeout.
super::note_hdr_capture_failed(); super::note_hdr_capture_failed();
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within {within}s (node {}): the compositor never \ "no PipeWire frame within 10s (node {}): the compositor never accepted \
accepted the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored \ the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored monitor in \
monitor in HDR mode on GNOME 50+? Downgrading this host to SDR capture; \ HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \
reconnect to stream SDR", to stream SDR",
self.node_id self.node_id
)) ))
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() { } else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
@@ -527,15 +452,14 @@ impl PortalCapturer {
// retries on the CPU offer instead of failing this same negotiation forever. // retries on the CPU offer instead of failing this same negotiation forever.
pf_zerocopy::note_vaapi_dmabuf_failed(); pf_zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within {within}s (node {}): the compositor never \ "no PipeWire frame within 10s (node {}): the compositor never accepted \
accepted the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this \ the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this host to the \
host to the CPU capture path; the pipeline rebuild will renegotiate \ CPU capture path; the pipeline rebuild will renegotiate without dmabuf",
without dmabuf",
self.node_id self.node_id
)) ))
} else { } else {
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within {within}s (node {}): format negotiation never \ "no PipeWire frame within 10s (node {}): format negotiation never \
completed the compositor offered no format this consumer accepts \ completed the compositor offered no format this consumer accepts \
(pixel-format/modifier mismatch) or the node never emitted a Format param", (pixel-format/modifier mismatch) or the node never emitted a Format param",
self.node_id self.node_id
@@ -900,7 +824,6 @@ mod pipewire {
VideoFormat::RGBA => PixelFormat::Rgba, VideoFormat::RGBA => PixelFormat::Rgba,
VideoFormat::RGB => PixelFormat::Rgb, VideoFormat::RGB => PixelFormat::Rgb,
VideoFormat::BGR => PixelFormat::Bgr, VideoFormat::BGR => PixelFormat::Bgr,
VideoFormat::NV12 => PixelFormat::Nv12,
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by // The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020). // the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10, VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
@@ -928,21 +851,13 @@ mod pipewire {
/// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves, /// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves,
/// so the GPU encoder re-uploads its cursor texture only on change. /// so the GPU encoder re-uploads its cursor texture only on change.
serial: u64, serial: u64,
/// The compositor-reported hotspot — carried on the overlay for the cursor-forward
/// channel (the blend path uses the pre-adjusted `x`/`y` and never reads it).
hot_x: i32,
hot_y: i32,
} }
impl CursorState { impl CursorState {
/// A shareable overlay for the encode/forward paths, or `None` before the first bitmap /// A shareable overlay for the GPU encode paths (blended at encode time), or `None` when
/// arrived. A HIDDEN pointer still yields `Some` (with `visible: false`): the /// there is nothing to draw. Cheap: clones an `Arc` + a few scalars.
/// cursor-forward channel needs "known but hidden" — an app grabbed the pointer, the
/// client's relative-mode hint (M3) — which is a different fact from "no cursor yet".
/// The encode loop strips invisible overlays before any blend path sees the frame.
/// Cheap: clones an `Arc` + a few scalars.
fn overlay(&self) -> Option<pf_frame::CursorOverlay> { fn overlay(&self) -> Option<pf_frame::CursorOverlay> {
if self.rgba.is_empty() { if !self.visible || self.rgba.is_empty() {
return None; return None;
} }
Some(pf_frame::CursorOverlay { Some(pf_frame::CursorOverlay {
@@ -952,9 +867,6 @@ mod pipewire {
h: self.bh, h: self.bh,
rgba: self.rgba.clone(), rgba: self.rgba.clone(),
serial: self.serial, serial: self.serial,
hot_x: self.hot_x.max(0) as u32,
hot_y: self.hot_y.max(0) as u32,
visible: self.visible,
}) })
} }
} }
@@ -1005,21 +917,6 @@ mod pipewire {
dbg_log_n: u64, dbg_log_n: u64,
/// Cursor-as-metadata state, composited into the CPU de-pad path (see `consume_frame`). /// Cursor-as-metadata state, composited into the CPU de-pad path (see `consume_frame`).
cursor: CursorState, cursor: CursorState,
/// LIVE overlay slot shared with [`super::PortalCapturer::cursor_live`] — refreshed after
/// every `update_cursor_meta`, including from cursor-only buffers that never become frames.
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
/// `Some((w, h))` while the producer's negotiated size is a sacrificial birth mode and a
/// renegotiation to these dims is guaranteed (KWin virtual outputs — kwin.rs `create`):
/// `.process` skips whole buffers until the negotiated size matches, then clears this
/// (self-disarming — later legitimate resizes are unaffected). `None` = no gating.
expect_dims: Option<(u32, u32)>,
/// Buffers skipped by the `expect_dims` gate (rate-limits its log).
gate_skips: u64,
/// When the gate first held a buffer — after [`GATE_DEADLINE`] with no renegotiation the
/// gate disarms and accepts what the producer serves (degraded dims beat a session wedged
/// into the first-frame-timeout retry loop; the promised renegotiation normally lands
/// within a frame or two).
gate_since: Option<std::time::Instant>,
} }
/// Consecutive tiled-import failures (worker alive, e.g. a per-buffer `EGL_BAD_MATCH`) before /// Consecutive tiled-import failures (worker alive, e.g. a per-buffer `EGL_BAD_MATCH`) before
@@ -1092,10 +989,10 @@ mod pipewire {
.into_inner()) .into_inner())
} }
/// Build a LINEAR/modifier DMA-BUF `EnumFormat` pod. Packed BGRx is the existing import path; /// Build a BGRx dmabuf `EnumFormat` pod advertising the EGL-importable `modifiers` as a
/// NV12 is gamescope's producer-side RGB→YUV path (opt-in during bring-up). /// mandatory enum Choice; the compositor fixates to one of them that it can allocate, which
/// we read back in `param_changed`.
fn build_dmabuf_format( fn build_dmabuf_format(
format: VideoFormat,
modifiers: &[u64], modifiers: &[u64],
preferred: Option<(u32, u32, u32)>, preferred: Option<(u32, u32, u32)>,
) -> Result<Vec<u8>> { ) -> Result<Vec<u8>> {
@@ -1106,7 +1003,7 @@ mod pipewire {
pw::spa::param::ParamType::EnumFormat, pw::spa::param::ParamType::EnumFormat,
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video), pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw), pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format), pw::spa::pod::property!(FormatProperties::VideoFormat, Id, VideoFormat::BGRx),
pw::spa::pod::property!( pw::spa::pod::property!(
FormatProperties::VideoSize, FormatProperties::VideoSize,
Choice, Choice,
@@ -1135,22 +1032,6 @@ mod pipewire {
pw::spa::utils::Fraction { num: 240, denom: 1 } pw::spa::utils::Fraction { num: 240, denom: 1 }
), ),
); );
if format == VideoFormat::NV12 {
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorMatrix,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_MATRIX_BT709,
)),
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorRange,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_RANGE_16_235,
)),
});
}
obj.properties.push(pw::spa::pod::Property { obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier, key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
flags: pw::spa::pod::PropertyFlags::MANDATORY, flags: pw::spa::pod::PropertyFlags::MANDATORY,
@@ -1398,17 +1279,10 @@ mod pipewire {
value: pw::spa::pod::Value::Choice(pw::spa::pod::ChoiceValue::Int( value: pw::spa::pod::Value::Choice(pw::spa::pod::ChoiceValue::Int(
pw::spa::utils::Choice( pw::spa::utils::Choice(
pw::spa::utils::ChoiceFlags::empty(), pw::spa::utils::ChoiceFlags::empty(),
// The max must cover the producer's offer or the Meta param silently
// fails to negotiate and NO buffer ever carries the meta region:
// Mutter offers a FIXED `SPA_POD_Int(CURSOR_META_SIZE(384, 384))`
// (meta-screen-cast-stream-src.c, GNOME 50) — a 256² max made the
// intersection empty, which cost the whole Linux cursor channel
// on-glass. 1024² is headroom, not an allocation: the negotiated
// region follows the producer's value.
pw::spa::utils::ChoiceEnum::Range { pw::spa::utils::ChoiceEnum::Range {
default: meta_size(64, 64), default: meta_size(64, 64),
min: meta_size(1, 1), min: meta_size(1, 1),
max: meta_size(1024, 1024), max: meta_size(256, 256),
}, },
), ),
)), )),
@@ -1436,25 +1310,21 @@ mod pipewire {
/// (which Mutter delivers as metadata-only "corrupted" buffers) still refresh the position. /// (which Mutter delivers as metadata-only "corrupted" buffers) still refresh the position.
fn update_cursor_meta(cursor: &mut CursorState, spa_buf: *mut spa::sys::spa_buffer) { fn update_cursor_meta(cursor: &mut CursorState, spa_buf: *mut spa::sys::spa_buffer) {
// SAFETY: `spa_buf` is the live buffer we still hold (dequeued, not yet requeued). // SAFETY: `spa_buf` is the live buffer we still hold (dequeued, not yet requeued).
// `spa_buffer_find_meta` returns the `spa_meta` (type + byte `size` + `data` pointer) for // `spa_buffer_find_meta_data` scans its metadata array for a `SPA_META_Cursor` of at least
// `SPA_META_Cursor`, or null. We take `find_meta` rather than `find_meta_data` specifically // `size_of::<spa_meta_cursor>()` bytes and returns a pointer into that buffer's metadata
// to obtain the region's real `size`: the bitmap offset, pixel offset and stride read below // (or null), valid until requeue. The size argument matches the struct the result is cast to.
// are ALL producer-written, and without a bound against the actual region they drive let cur = unsafe {
// out-of-bounds pointer arithmetic and an oversized `slice::from_raw_parts` — an OOB read spa::sys::spa_buffer_find_meta_data(
// that SIGSEGVs inside the PipeWire `.process` callback (a segfault `catch_unwind` cannot spa_buf,
// catch). Every offset below is validated against `region_size` with checked arithmetic, spa::sys::SPA_META_Cursor,
// mirroring the fd-length guard the main frame path already applies to xdg-desktop-portal-wlr. std::mem::size_of::<spa::sys::spa_meta_cursor>(),
let meta = unsafe { spa::sys::spa_buffer_find_meta(spa_buf, spa::sys::SPA_META_Cursor) }; ) as *const spa::sys::spa_meta_cursor
if meta.is_null() { };
if cur.is_null() {
return; return;
} }
// SAFETY: `meta` is non-null and points into the held buffer's metadata array. // SAFETY: `cur` is non-null and points to a `spa_meta_cursor` of at least its own size
let (region_size, data) = unsafe { ((*meta).size as usize, (*meta).data as *const u8) }; // inside the held buffer (guaranteed by the size arg above), so every field read is in bounds.
if data.is_null() || region_size < std::mem::size_of::<spa::sys::spa_meta_cursor>() {
return;
}
let cur = data as *const spa::sys::spa_meta_cursor;
// SAFETY: `region_size >= size_of::<spa_meta_cursor>()` checked above, so every field is in bounds.
let (id, pos_x, pos_y, hot_x, hot_y, bmp_off) = unsafe { let (id, pos_x, pos_y, hot_x, hot_y, bmp_off) = unsafe {
( (
(*cur).id, (*cur).id,
@@ -1466,35 +1336,24 @@ mod pipewire {
) )
}; };
if id == 0 { if id == 0 {
// SPA contract: id 0 = "no cursor information", NOT "cursor hidden". Mutter only // Compositor reports no visible pointer (e.g. a game grabbed/hid it).
// REWRITES a buffer's meta region when the cursor changed, so recycled buffers cursor.visible = false;
// between damage frames carry a stale id-0 meta — treating that as hidden flickered
// the cursor off between hovers (on-glass round 5). Keep the last-known state; a
// pointer that really left/hid simply stops producing updates. (The M3 hidden hint
// loses its Mutter signal — Windows has its own CURSOR_SUPPRESSED source.)
return; return;
} }
cursor.visible = true; cursor.visible = true;
cursor.x = pos_x - hot_x; cursor.x = pos_x - hot_x;
cursor.y = pos_y - hot_y; cursor.y = pos_y - hot_y;
cursor.hot_x = hot_x;
cursor.hot_y = hot_y;
if bmp_off == 0 { if bmp_off == 0 {
// Position-only update — keep the cached bitmap. // Position-only update — keep the cached bitmap.
return; return;
} }
let bmp_off = bmp_off as usize; // SAFETY: `bitmap_offset` is a byte offset from `cur` to a `spa_meta_bitmap`, which the
// The `spa_meta_bitmap` header must fit entirely inside the region before we read it — // producer placed inside the same meta region it sized for this cursor (>= the size we
// `bitmap_offset` is producer-controlled and otherwise reads past the metadata. // requested). The resulting pointer is in bounds and aligned for `spa_meta_bitmap`.
match bmp_off.checked_add(std::mem::size_of::<spa::sys::spa_meta_bitmap>()) { let bmp =
Some(end) if end <= region_size => {} unsafe { (cur as *const u8).add(bmp_off as usize) as *const spa::sys::spa_meta_bitmap };
_ => return, // SAFETY: `bmp` is the in-bounds, aligned `spa_meta_bitmap` pointer computed just above; the
} // producer fully initialized this header, so reading its scalar fields is sound.
// SAFETY: `bmp_off + size_of::<spa_meta_bitmap>() <= region_size` (checked directly above),
// so the header is fully in bounds; the producer places it aligned as before.
let bmp = unsafe { data.add(bmp_off) as *const spa::sys::spa_meta_bitmap };
// SAFETY: `bmp` is the in-bounds `spa_meta_bitmap` header validated just above; reading its
// scalar fields is sound.
let (vfmt, bw, bh, stride, pix_off) = unsafe { let (vfmt, bw, bh, stride, pix_off) = unsafe {
( (
(*bmp).format, (*bmp).format,
@@ -1504,34 +1363,16 @@ mod pipewire {
(*bmp).offset as usize, (*bmp).offset as usize,
) )
}; };
// Ignore empty or implausibly large bitmaps (the meta-size request covers <= 1024×1024; // Ignore empty or implausibly large bitmaps (we requested <= 256×256).
// real cursors are ≤96px — the cursor channel downscales >120px for the wire anyway). if bw == 0 || bh == 0 || bw > 256 || bh > 256 {
if bw == 0 || bh == 0 || bw > 1024 || bh > 1024 {
return; return;
} }
let row = bw as usize * 4; let row = bw as usize * 4;
let stride = if stride < row { row } else { stride }; let stride = if stride < row { row } else { stride };
// `span` is the exact byte extent the strided loop reads: `stride·(bh-1) + row`. Compute it let span = stride * (bh as usize - 1) + row;
// with checked arithmetic (a producer stride near `i32::MAX` would otherwise overflow) and // SAFETY: the bitmap pixels live at `bmp + pix_off` for `span` bytes, within the
// require the whole pixel block `[bmp_off + pix_off, +span)` to lie inside the region before // producer-sized meta region. `span` is the exact extent the strided copy below reads.
// fabricating the slice — this is the check whose absence made the read go out of bounds. let src = unsafe { std::slice::from_raw_parts((bmp as *const u8).add(pix_off), span) };
let span = match stride
.checked_mul(bh as usize - 1)
.and_then(|v| v.checked_add(row))
{
Some(s) => s,
None => return,
};
match bmp_off
.checked_add(pix_off)
.and_then(|v| v.checked_add(span))
{
Some(end) if end <= region_size => {}
_ => return,
}
// SAFETY: `bmp_off + pix_off + span <= region_size` (checked directly above), so the slice
// is fully within the producer's meta region; `span` is exactly the strided loop's extent.
let src = unsafe { std::slice::from_raw_parts(data.add(bmp_off + pix_off), span) };
let mut rgba = vec![0u8; bw as usize * bh as usize * 4]; let mut rgba = vec![0u8; bw as usize * bh as usize * 4];
for y in 0..bh as usize { for y in 0..bh as usize {
for x in 0..bw as usize { for x in 0..bw as usize {
@@ -1737,8 +1578,8 @@ mod pipewire {
} }
} }
// Raw DMA-BUF passthrough: packed RGB is imported for GPU CSC; producer-native NV12 can // VAAPI zero-copy passthrough: hand the raw dmabuf straight to the encoder, which imports
// be consumed by the Vulkan Video encoder without another color conversion. // it into a VA surface and does RGB→NV12 on the GPU video engine. No CUDA importer here.
if ud.vaapi_passthrough { if ud.vaapi_passthrough {
if let Some(fmt) = ud.format { if let Some(fmt) = ud.format {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf { if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
@@ -1746,41 +1587,9 @@ mod pipewire {
let chunk = datas[0].chunk(); let chunk = datas[0].chunk();
let offset = chunk.offset(); let offset = chunk.offset();
let stride = chunk.stride().max(0) as u32; let stride = chunk.stride().max(0) as u32;
// Native NV12 usually arrives as a two-plane SPA buffer over ONE buffer
// object; plane 1's chunk carries the REAL UV offset/stride (compositors
// may align the Y plane before UV). Pass it through instead of assuming
// contiguity. Each spa_data holds its own (dup'd) fd, so BO identity is
// by inode, not fd number; a genuinely two-BO frame cannot travel through
// the single-fd import — drop it with a diagnosis instead of streaming
// garbage chroma.
let plane1 =
if fmt == PixelFormat::Nv12 && datas.len() >= 2 && datas[1].fd() > 0 {
// SAFETY: zeroed `libc::stat` is a valid POD initializer; both fds are
// owned by the live PipeWire buffer for this callback, and `fstat`
// only writes the out-param structs, whose fields are read only after
// the `== 0` success checks.
let same_bo = unsafe {
let mut s0: libc::stat = std::mem::zeroed();
let mut s1: libc::stat = std::mem::zeroed();
libc::fstat(datas[0].fd() as i32, &mut s0) == 0
&& libc::fstat(datas[1].fd() as i32, &mut s1) == 0
&& (s0.st_dev, s0.st_ino) == (s1.st_dev, s1.st_ino)
};
if !same_bo {
warn_once(
"NV12 planes live in different buffer objects — frames \
dropped (single-fd import only)",
);
return;
}
let c1 = datas[1].chunk();
Some((c1.offset(), c1.stride().max(0) as u32))
} else {
None
};
// dup the fd so it survives the SPA buffer recycle — the encode thread // dup the fd so it survives the SPA buffer recycle — the encode thread
// imports it. Content stability across the brief import/encode window relies // imports it. (Content stability across the brief map+CSC window relies on
// on the compositor's buffer-pool depth, like any zero-copy capture. // the compositor's buffer-pool depth, like any zero-copy capture.)
// SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid // SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid
// for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd, // for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd,
// touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1). // touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1).
@@ -1807,10 +1616,9 @@ mod pipewire {
modifier: ud.modifier, modifier: ud.modifier,
offset, offset,
stride, stride,
plane1,
}), }),
// Cursor-as-metadata is blended only by RGB→NV12 backends. Gamescope // Cursor-as-metadata: the encoder blends this into its owned VA
// embeds its pointer in the produced pixels, so native NV12 has none. // surface (raw dmabuf never touched).
cursor: ud.cursor.overlay(), cursor: ud.cursor.overlay(),
}); });
static ONCE: std::sync::atomic::AtomicBool = static ONCE: std::sync::atomic::AtomicBool =
@@ -1821,12 +1629,7 @@ mod pipewire {
h, h,
modifier = ud.modifier, modifier = ud.modifier,
fourcc = format_args!("{:#010x}", fourcc), fourcc = format_args!("{:#010x}", fourcc),
source = if fmt == PixelFormat::Nv12 { "zero-copy: handing the raw dmabuf to the encoder (GPU import + CSC)"
"producer-native NV12"
} else {
"packed RGB (encoder GPU CSC)"
},
"zero-copy: handing the raw DMA-BUF to the encoder"
); );
} }
return; return;
@@ -2106,9 +1909,6 @@ mod pipewire {
streaming: Arc<AtomicBool>, streaming: Arc<AtomicBool>,
broken: Arc<AtomicBool>, broken: Arc<AtomicBool>,
hdr_negotiated: Arc<AtomicBool>, hdr_negotiated: Arc<AtomicBool>,
// LIVE cursor publisher (see `PortalCapturer::cursor_live`): refreshed from every
// dequeued buffer's cursor meta, frames or not.
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
zerocopy: bool, zerocopy: bool,
// 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert. // 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert.
want_444: bool, want_444: bool,
@@ -2120,9 +1920,6 @@ mod pipewire {
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way // Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6). // capture→encode edge (plan §W6).
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
// See `spawn_pipewire`: the first negotiation is for a sacrificial mode; hold frames
// until the producer renegotiates to `preferred`'s dims.
expect_exact_dims: bool,
) -> Result<()> { ) -> Result<()> {
crate::pwinit::ensure_init(); crate::pwinit::ensure_init();
@@ -2191,37 +1988,7 @@ mod pipewire {
// consumer imports raw dmabufs itself — the VAAPI backend (libva import + GPU CSC) or a // consumer imports raw dmabufs itself — the VAAPI backend (libva import + GPU CSC) or a
// PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw // PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw
// dmabuf straight to the encoder. // dmabuf straight to the encoder.
// let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough;
// ...unless the encoder already proved it cannot import them here. A driver that refuses
// the compositor's buffers refuses them identically on every retry, so without this the
// session died on its first frame and every reconnect repeated it. The latch (set by the
// encode side after consecutive import failures) is what turns that into one bad session
// followed by a working, if slower, host.
let raw_dmabuf_off = raw_passthrough && pf_zerocopy::raw_dmabuf_import_disabled();
let vaapi_passthrough =
zerocopy && !force_shm && importer.is_none() && raw_passthrough && !raw_dmabuf_off;
// Producer-side NV12 (default-on; PUNKTFUNK_PIPEWIRE_NV12=0 escapes): gamescope offers a
// one-fd LINEAR NV12 image when the consumer asks — its compositor pass does the RGB→YUV,
// and the Vulkan Video encoder imports the buffer as its encode source directly (no host
// CSC at all). `native_nv12_session` restricts this to sessions whose encoder can ingest
// it (Linux vulkan-encode H265/AV1 — never H264/libav-VAAPI, GameStream-resolve, or
// PyroWave, whose Vulkan compute CSC ingests packed RGB only). Raw passthrough is
// required because the CUDA importer expects packed RGB, and 4:4:4/HDR must not be
// silently subsampled/downconverted. Non-NV12 compositors (KWin/GNOME) simply match the
// packed-RGB fallback pod.
let prefer_native_nv12 = std::env::var("PUNKTFUNK_PIPEWIRE_NV12").as_deref() != Ok("0")
&& policy.native_nv12_session
&& backend_is_vaapi
&& vaapi_passthrough
&& !policy.pyrowave_session
&& !want_444
&& !want_hdr;
if prefer_native_nv12 {
tracing::info!(
"zero-copy: preferring gamescope producer-side NV12 LINEAR DMA-BUF (no host \
RGB CSC; PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation)"
);
}
// Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR // Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR
// (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via // (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via
// CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only: // CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only:
@@ -2257,19 +2024,11 @@ mod pipewire {
tracing::info!( tracing::info!(
"capture: PUNKTFUNK_FORCE_SHM — race-free SHM download path (no dmabuf, no zero-copy)" "capture: PUNKTFUNK_FORCE_SHM — race-free SHM download path (no dmabuf, no zero-copy)"
); );
} else if raw_dmabuf_off {
tracing::warn!(
"zero-copy raw-dmabuf passthrough disabled after repeated encoder import failures \
capturing CPU frames instead (this host's GPU driver would not import the \
compositor's buffers)"
);
} else if zerocopy && !want_dmabuf { } else if zerocopy && !want_dmabuf {
tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path"); tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path");
} else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() { } else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() {
tracing::info!( tracing::info!(
native_nv12_preferred = prefer_native_nv12, "zero-copy: advertising LINEAR dmabuf for direct VAAPI import (GPU CSC)"
"zero-copy: advertising LINEAR DMA-BUF for encoder import (native NV12 first \
when enabled, packed RGB fallback)"
); );
} else if want_dmabuf && !vaapi_passthrough { } else if want_dmabuf && !vaapi_passthrough {
tracing::info!( tracing::info!(
@@ -2321,14 +2080,6 @@ mod pipewire {
linear_nv12_failed: false, linear_nv12_failed: false,
dbg_log_n: 0, dbg_log_n: 0,
cursor: CursorState::default(), cursor: CursorState::default(),
cursor_live,
expect_dims: if expect_exact_dims {
preferred.map(|(w, h, _)| (w, h))
} else {
None
},
gate_skips: 0,
gate_since: None,
}; };
let stream = pw::stream::StreamBox::new( let stream = pw::stream::StreamBox::new(
@@ -2413,91 +2164,36 @@ mod pipewire {
} }
}) })
.process(|stream, ud| { .process(|stream, ud| {
// Latest-frame-only (OBS pattern): Mutter delivers buffers in bursts and recycles its // PipeWire dispatches this from a C trampoline with no catch_unwind; a
// pool; an older queued buffer carries a STALE frame. Drain all queued buffers, requeue // panic crossing that FFI boundary would abort the whole host. Contain it.
// the older ones, keep only the newest. This dequeue/requeue runs OUTSIDE the let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
// `catch_unwind` below — they are non-panicking C FFI pointer ops, and `newest` is // Latest-frame-only (OBS pattern): Mutter delivers buffers in bursts and
// requeued exactly once AFTER the panic-containing region. Previously the whole thing was // recycles its pool; an older queued buffer carries a STALE frame. Drain all
// inside the catch, so a caught panic (in `update_cursor_meta`/`consume_frame`) stranded // queued buffers, requeue the older ones, keep only the newest.
// `newest` forever, permanently shrinking the stream's fixed pool until capture wedged. // SAFETY: `stream` is the live stream PipeWire passes into this `.process` callback on
// SAFETY: `stream` is the live stream PipeWire passes into this `.process` callback on the // the loop thread, where `pw_stream_dequeue_buffer` is the documented call. It returns
// loop thread; `dequeue_raw_buffer` returns a stream-owned `*mut pw_buffer` or null // a `*mut pw_buffer` owned by the stream (or null when the queue is drained),
// (null-checked), single-threaded so no concurrent access. // null-checked before any use. The loop is single-threaded, so no concurrent access.
let mut newest = unsafe { stream.dequeue_raw_buffer() }; let mut newest = unsafe { stream.dequeue_raw_buffer() };
if newest.is_null() { if newest.is_null() {
return; return;
} }
let mut drained = 1u32; let mut drained = 1u32;
loop { loop {
// SAFETY: same stream/loop-thread contract; returns the next stream-owned buffer or null. // SAFETY: same stream/loop-thread contract as the dequeue above; each call returns
// the next stream-owned `*mut pw_buffer` or null (null-checked before use).
let next = unsafe { stream.dequeue_raw_buffer() }; let next = unsafe { stream.dequeue_raw_buffer() };
if next.is_null() { if next.is_null() {
break; break;
} }
// SAFETY: `newest` was dequeued from this stream and not yet requeued; we immediately // SAFETY: `newest` is a non-null `*mut pw_buffer` previously dequeued from this same
// overwrite it, so the requeued pointer is never touched again. // stream and not yet requeued; `pw_stream_queue_buffer` hands ownership back to the
// stream. We immediately overwrite `newest = next`, so the requeued pointer is never
// touched again (no use-after-requeue). Loop thread, single-threaded.
unsafe { stream.queue_raw_buffer(newest) }; unsafe { stream.queue_raw_buffer(newest) };
newest = next; newest = next;
drained += 1; drained += 1;
} }
// Sacrificial-mode gate (kwin.rs `create`): until the producer renegotiates to the
// expected dims, every buffer — frame AND cursor meta, whose positions are in the
// doomed mode's space — belongs to the birth mode; consuming one would build the
// pipeline at the wrong size. Self-disarms on the first matching negotiation, or
// after `GATE_DEADLINE` without one — degraded dims beat wedging the session into
// the first-frame-timeout retry loop when the promised renegotiation never comes.
if let Some((ew, eh)) = ud.expect_dims {
/// The renegotiation normally lands within a frame or two of recording; well
/// past that, the producer is not going to deliver it (the on-glass case: the
/// real mode never actually applied) — stop starving the pipeline.
const GATE_DEADLINE: std::time::Duration = std::time::Duration::from_secs(3);
let sz = ud.info.size();
if sz.width == ew && sz.height == eh {
tracing::info!(
skipped = ud.gate_skips,
width = ew,
height = eh,
"producer renegotiated to the expected mode — frames flow"
);
ud.expect_dims = None;
} else if ud
.gate_since
.get_or_insert_with(std::time::Instant::now)
.elapsed()
> GATE_DEADLINE
{
tracing::warn!(
negotiated_w = sz.width,
negotiated_h = sz.height,
expected_w = ew,
expected_h = eh,
skipped = ud.gate_skips,
"producer never renegotiated to the expected mode — accepting its \
dims (session runs degraded rather than wedged)"
);
ud.expect_dims = None;
} else {
ud.gate_skips += 1;
if ud.gate_skips == 1 || ud.gate_skips.is_power_of_two() {
tracing::info!(
negotiated_w = sz.width,
negotiated_h = sz.height,
expected_w = ew,
expected_h = eh,
n = ud.gate_skips,
"holding frames until the producer renegotiates to the expected mode"
);
}
// SAFETY: `newest` was dequeued from this stream and not yet requeued;
// requeued exactly once here, then never touched (mirrors the null path).
unsafe { stream.queue_raw_buffer(newest) };
return;
}
}
// PipeWire dispatches from a C trampoline with no catch_unwind; a panic crossing that FFI
// boundary would abort the whole host. Contain the inspect/consume work — the only Rust
// code here that can panic — and requeue `newest` unconditionally after it.
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
// SAFETY: `newest` is the non-null buffer we still own (dequeued, not requeued); // SAFETY: `newest` is the non-null buffer we still own (dequeued, not requeued);
// `.buffer` is a `*mut spa_buffer` field libpipewire populated. This is a single field // `.buffer` is a `*mut spa_buffer` field libpipewire populated. This is a single field
// load through a valid pointer — no mutation or aliasing. // load through a valid pointer — no mutation or aliasing.
@@ -2507,19 +2203,6 @@ mod pipewire {
// pointer-only movements as metadata-only "corrupted" buffers we drop for their // pointer-only movements as metadata-only "corrupted" buffers we drop for their
// frame, but their cursor meta is fresh and must still move our overlay. // frame, but their cursor meta is fresh and must still move our overlay.
update_cursor_meta(&mut ud.cursor, spa_buf); update_cursor_meta(&mut ud.cursor, spa_buf);
// Publish the LIVE overlay (frames or not) so the encode loop's forwarder
// tracks pointer-only motion on a static desktop — the frame-attached overlay
// alone stales between damage frames. ONLY when we actually have one: a
// gamescope node carries no `SPA_META_Cursor`, so `overlay()` is always `None`
// here, and writing that would clobber — at frame rate — the `Some` the
// attached XFixes source publishes into this SAME slot, strobing the
// composited pointer on/off. Portal cursors are `None` only before the first
// bitmap (nothing to drop), and a HIDDEN pointer is still `Some(visible:false)`.
if let Some(overlay) = ud.cursor.overlay() {
if let Ok(mut slot) = ud.cursor_live.lock() {
*slot = Some(overlay);
}
}
// Inspect the newest buffer's header + first chunk for the diagnostic and the // Inspect the newest buffer's header + first chunk for the diagnostic and the
// CORRUPTED skip. SPA_META_Header is optional — `hdr` may be null. // CORRUPTED skip. SPA_META_Header is optional — `hdr` may be null.
@@ -2589,18 +2272,19 @@ mod pipewire {
"capture: skipped a stale CORRUPTED/cursor buffer (GNOME)" "capture: skipped a stale CORRUPTED/cursor buffer (GNOME)"
); );
} }
// Skip this stale/cursor buffer — `newest` is requeued unconditionally below. // SAFETY: `newest` is the non-null buffer we own (dequeued, never requeued on this
// skip path); hand it back to the stream exactly once and return without touching it
// again. Loop thread inside `.process`.
unsafe { stream.queue_raw_buffer(newest) };
return; return;
} }
consume_frame(ud, spa_buf); consume_frame(ud, spa_buf);
})); // SAFETY: `consume_frame` has finished reading `spa_buf` (and the `datas` borrows derived
// Hand `newest` back to the stream exactly once, on EVERY path — normal, corrupted-skip, // from `newest`), so requeuing the owned `newest` exactly once here is sound — no
// or a caught panic in the closure above. This single requeue is what keeps the fixed // use-after-requeue. Loop thread inside `.process`.
// buffer pool from draining.
// SAFETY: all reads of `spa_buf`/`newest` (update_cursor_meta, consume_frame) completed
// inside the closure above; `newest` was dequeued from this stream and not yet requeued.
unsafe { stream.queue_raw_buffer(newest) }; unsafe { stream.queue_raw_buffer(newest) };
}));
if outcome.is_err() { if outcome.is_err() {
// In the per-frame `.process` callback: a deterministic panic (e.g. a bad // In the per-frame `.process` callback: a deterministic panic (e.g. a bad
// format) would fire this every frame, so power-of-two throttle it — enough to // format) would fire this every frame, so power-of-two throttle it — enough to
@@ -2666,18 +2350,6 @@ mod pipewire {
build_default_format_obj(preferred) build_default_format_obj(preferred)
}; };
// gamescope trap — the Steam overlay's presence in the stream is decided HERE by omission:
// gamescope's `paint_pipewire()` composites the overlay (Shift+Tab / Quick Access Menu) into
// the node it hands us ONLY when the consumer-negotiated `gamescope_focus_appid` is 0 — the
// default, and the "mirror the focused window + overlay" branch (gamescope ≥ 3.16.23; see
// `MIN_GAMESCOPE_OVERLAY`). None of the EnumFormat pods below advertise the
// `SPA_FORMAT_VIDEO_gamescope_focus_appid` property, so gamescope reads 0 and paints the
// overlay for us for free. DO NOT add a non-zero focus-appid (e.g. to "isolate the game" in a
// dedicated session) — that flips gamescope into the Remote-Play branch that deliberately
// drops the overlay (and all host chrome) back out of the capture. The cursor, external
// overlay (MangoHUD), and notifications are excluded from the node on EVERY gamescope
// version and are composited host-side instead (see `xfixes_cursor.rs`).
//
// When zero-copy is on, offer ONLY a BGRx dmabuf format with our EGL-importable modifiers // When zero-copy is on, offer ONLY a BGRx dmabuf format with our EGL-importable modifiers
// (offering shm too makes the compositor pick shm). The modifier list is advertised with // (offering shm too makes the compositor pick shm). The modifier list is advertised with
// DONT_FIXATE so the compositor's allocator chooses one; we re-emit the fixated format in // DONT_FIXATE so the compositor's allocator chooses one; we re-emit the fixated format in
@@ -2696,18 +2368,7 @@ mod pipewire {
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?, build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
] ]
} else if want_dmabuf { } else if want_dmabuf {
let mut pods = Vec::with_capacity(if prefer_native_nv12 { 2 } else { 1 }); vec![build_dmabuf_format(&modifiers, preferred)?]
if prefer_native_nv12 {
// First compatible consumer pod wins. Gamescope advertises NV12 and BGRx; pinning
// BT.709 limited here selects its RGB→NV12 shader with our bitstream colorimetry.
pods.push(build_dmabuf_format(VideoFormat::NV12, &[0], preferred)?);
}
pods.push(build_dmabuf_format(
VideoFormat::BGRx,
&modifiers,
preferred,
)?);
pods
} else { } else {
vec![serialize_pod(obj)?] vec![serialize_pod(obj)?]
}; };
@@ -1,567 +0,0 @@
//! XFixes cursor source for the gamescope capture path (remote-desktop-sweep Phase C).
//!
//! gamescope draws the pointer on a DRM hardware-cursor plane and its `paint_pipewire()`
//! deliberately excludes the cursor from the frame it feeds its built-in PipeWire node — so
//! `SPA_META_Cursor` never arrives and the ordinary [`cursor_live`](super::PortalCapturer) slot
//! stays empty (a KWin/GNOME session gets its cursor from that meta; gamescope can't embed one
//! either, its `set_hw_cursor` is inert). We instead read the pointer from gamescope's nested
//! Xwayland via X11 — the trick Sunshine uses — and publish a [`CursorOverlay`] into that same
//! slot, so the encoder blend composites the pointer into the video exactly like the portal path.
//!
//! **Multiple Xwaylands.** gamescope runs one Xwayland per `--xwayland-count` (Steam Gaming Mode
//! uses 2: one for Big Picture, one for the game). The pointer lives on whichever is FOCUSED — an
//! inactive display's pointer is frozen. So the source connects to ALL of them and publishes from
//! the one gamescope is actually drawing the pointer on; it reads that display's shape too, since
//! each Xwayland has its own current cursor. This is why a single-display read froze the pointer
//! the moment a game on the OTHER Xwayland took focus.
//!
//! Three X sources per display, split by cost (Sunshine's split, plus gamescope's own verdict):
//! * **Position** — core `QueryPointer` on the root, polled fast. Cheap (a few-byte reply, no
//! bitmap), so it can out-pace the stream fps and keep the composited pointer smooth.
//! * **Shape / hotspot** — `XFixesGetCursorImage`, refreshed only after an XFixes `CursorNotify`
//! (a real cursor change). A fully-transparent image reads as hidden.
//! * **Visibility + focus** — [`GAMESCOPE_CURSOR_VISIBLE_FEEDBACK`](GS_CURSOR_FEEDBACK) on the
//! root, read at connect and re-read on its `PropertyNotify`.
//!
//! **Why the feedback atom and not pointer motion.** gamescope hides its pointer by WARPING the X
//! pointer to the root's bottom-right corner pixel — it does NOT swap in a transparent X cursor, so
//! `XFixesGetCursorImage` keeps handing back the last opaque arrow. The original "follow whichever
//! display's pointer moved" heuristic therefore stuck to the parked display (a parked pointer never
//! moves again) and composited that arrow at `(w-1, h-1)`: a sliver of cursor welded to the corner
//! of the stream for the rest of the session, while the real pointer went undrawn. Reported
//! on-glass as "part of cursor shows up on bottom right … isn't where it really is", in every game
//! (a game grabs the pointer, so the hide is permanent). Measured on a live 1920x1080 Gaming Mode
//! session: real motion ⇒ pointer live + feedback `1`; 3 s idle (`--hide-cursor-delay 3000`) ⇒
//! pointer `(1919, 1079)` + feedback `0`. The atom answers BOTH questions correctly for a static
//! pointer, which motion cannot: which Xwayland owns it, and whether to draw it at all — so
//! honouring it also gives the stream gamescope's own idle auto-hide, which this source never had.
use std::sync::{
atomic::{AtomicBool, Ordering},
Arc, Mutex,
};
use std::time::Duration;
use pf_frame::CursorOverlay;
use x11rb::connection::Connection;
use x11rb::errors::ReplyError;
use x11rb::protocol::xfixes::{self, ConnectionExt as _, GetCursorImageReply};
use x11rb::protocol::xproto::{
Atom, AtomEnum, ChangeWindowAttributesAux, ConnectionExt as _, EventMask, QueryPointerReply,
Window,
};
use x11rb::protocol::Event;
use x11rb::rust_connection::RustConnection;
/// Serializes the brief `XAUTHORITY` env swap around a connect (the var is process-global). Only
/// ever contended if two gamescope sessions start at once — rare, and the swap is microseconds.
static XAUTH_LOCK: Mutex<()> = Mutex::new(());
/// Position out-paces the stream fps (`POLL`); shape rides `CursorNotify` events drained each tick.
/// 4 ms ≈ 250 Hz matches the Windows GDI poller — the polled position IS the composited position
/// and must out-run a 240 fps session or the pointer stutters.
const POLL: Duration = Duration::from_millis(4);
/// gamescope's own pointer verdict, published on EVERY nested Xwayland's root: `1` on the server
/// whose pointer gamescope is currently drawing, `0` on the others — and `0` on all of them once
/// the pointer is hidden (a game grabbed it, or `--hide-cursor-delay` fired). See the module docs
/// for why this, and not pointer motion, is the signal this source follows.
const GS_CURSOR_FEEDBACK: &str = "GAMESCOPE_CURSOR_VISIBLE_FEEDBACK";
/// Self-heal cadence for the feedback re-read: `PropertyNotify` drives it, this only covers a
/// missed/coalesced event (and a gamescope that publishes the atom after we connected). One
/// `GetProperty` per display per interval is nothing next to the 250 Hz pointer poll.
const FEEDBACK_RESYNC: Duration = Duration::from_millis(250);
/// A running XFixes cursor reader. Dropping it stops the worker thread and joins it, releasing the
/// X connections — so it lives exactly as long as the capturer that owns it.
pub(super) struct XFixesCursorSource {
stop: Arc<AtomicBool>,
join: Option<std::thread::JoinHandle<()>>,
}
impl XFixesCursorSource {
/// Connect to every gamescope nested Xwayland in `targets` (`(DISPLAY, XAUTHORITY)`) and start
/// publishing cursor overlays into `slot`, following the focused display's pointer. Returns
/// `None` — and logs — if NONE can be used (no X connection / no XFixes), so the caller
/// degrades to no gamescope cursor (today's behaviour) instead of failing the session.
pub(super) fn spawn(
targets: Vec<(String, Option<String>)>,
slot: Arc<Mutex<Option<CursorOverlay>>>,
) -> Option<Self> {
// Connect on the caller's thread so failures degrade cleanly and the displays are validated
// before we commit a thread.
let mut displays = Vec::new();
for (dpy, xauth) in targets {
match connect(&dpy, xauth.as_deref()) {
Ok((conn, root, feedback)) => {
displays.push(XDisplay::new(dpy, conn, root, feedback))
}
Err(e) => tracing::warn!(
dpy = %dpy,
error = %e,
"gamescope cursor: skipping a nested Xwayland we can't use"
),
}
}
if displays.is_empty() {
tracing::warn!(
"gamescope cursor: no usable nested Xwayland — no in-video pointer this session \
(falls back to today's cursorless gamescope stream)"
);
return None;
}
let names: Vec<&str> = displays.iter().map(|d| d.name.as_str()).collect();
let feedback = displays.iter().any(|d| d.gs_visible.is_some());
tracing::info!(
displays = ?names,
cursor_feedback = feedback,
"gamescope cursor: XFixes source live — following the Xwayland gamescope draws the \
pointer on (cursor_feedback=false this gamescope publishes no \
GAMESCOPE_CURSOR_VISIBLE_FEEDBACK, degrading to the pointer-motion heuristic)"
);
let stop = Arc::new(AtomicBool::new(false));
let stop_worker = Arc::clone(&stop);
let join = std::thread::Builder::new()
.name("pf-gs-cursor".into())
.spawn(move || run(displays, slot, stop_worker))
.ok()?;
Some(XFixesCursorSource {
stop,
join: Some(join),
})
}
}
impl Drop for XFixesCursorSource {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(j) = self.join.take() {
let _ = j.join();
}
}
}
/// Open the X connection, negotiate XFixes, and select cursor-change + root-property events —
/// returning the connection, root window and this display's initial
/// [`GAMESCOPE_CURSOR_FEEDBACK`](GS_CURSOR_FEEDBACK) reading (`(atom, value)`; the value is `None`
/// when gamescope publishes no such property here). `RustConnection` reads `XAUTHORITY` from the
/// env at connect time only, so set it under the lock (the host isn't a gamescope child), connect,
/// then restore.
type Connected = (RustConnection, Window, (Atom, Option<bool>));
fn connect(dpy: &str, xauthority: Option<&str>) -> Result<Connected, String> {
let (conn, screen_num) = {
let _g = XAUTH_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let prev = std::env::var_os("XAUTHORITY");
if let Some(x) = xauthority {
std::env::set_var("XAUTHORITY", x);
}
let out = RustConnection::connect(Some(dpy));
match (&prev, xauthority) {
(Some(p), _) => std::env::set_var("XAUTHORITY", p),
(None, Some(_)) => std::env::remove_var("XAUTHORITY"),
(None, None) => {}
}
out.map_err(|e| format!("connect: {e}"))?
};
// XFixes ≥ 1 gives GetCursorImage / SelectCursorInput; ask for a modern minor, take what we get.
conn.xfixes_query_version(5, 0)
.map_err(ReplyError::from)
.and_then(|c| c.reply())
.map_err(|e| format!("XFixes unavailable: {e}"))?;
let root = conn
.setup()
.roots
.get(screen_num)
.ok_or_else(|| format!("no X screen {screen_num}"))?
.root;
// Wake the worker's event drain whenever the cursor shape changes (incl. hide/show).
conn.xfixes_select_cursor_input(root, xfixes::CursorNotifyMask::DISPLAY_CURSOR)
.map_err(ReplyError::from)
.and_then(|c| c.check())
.map_err(|e| format!("SelectCursorInput: {e}"))?;
// …and whenever gamescope republishes its cursor verdict. Interned with `only_if_exists=false`
// so we hold a matchable atom id even on a gamescope that sets the property later; a failure to
// select PROPERTY_CHANGE is NOT fatal — the resync re-read still tracks it, just at 250 ms.
let feedback_atom = conn
.intern_atom(false, GS_CURSOR_FEEDBACK.as_bytes())
.map_err(ReplyError::from)
.and_then(|c| c.reply())
.map(|r| r.atom)
.unwrap_or(0);
if let Ok(c) = conn.change_window_attributes(
root,
&ChangeWindowAttributesAux::new().event_mask(EventMask::PROPERTY_CHANGE),
) {
let _ = c.check();
}
let _ = conn.flush();
let feedback = read_cursor_feedback(&conn, root, feedback_atom);
Ok((conn, root, (feedback_atom, feedback)))
}
/// Read `GAMESCOPE_CURSOR_VISIBLE_FEEDBACK` off `root`. `None` = the property is absent or
/// unreadable (not a gamescope that publishes it) — the caller then falls back to the pointer-motion
/// heuristic rather than blanking the cursor, so an older gamescope keeps today's behaviour.
fn read_cursor_feedback(conn: &RustConnection, root: Window, atom: Atom) -> Option<bool> {
if atom == 0 {
return None;
}
let reply = conn
.get_property(false, root, atom, AtomEnum::CARDINAL, 0, 1)
.ok()?
.reply()
.ok()?;
let value = reply.value32()?.next()?;
Some(value != 0)
}
/// One gamescope Xwayland the source tracks.
struct XDisplay {
name: String,
conn: RustConnection,
root: Window,
/// Last polled pointer position — a change since the previous tick marks this display FOCUSED.
last_pos: Option<(i32, i32)>,
/// Cached cursor shape, refreshed only after this display's XFixes `CursorNotify`.
shape: Shape,
/// A `CursorNotify` (or first read) is pending — fetch the shape when this display is active.
need_shape: bool,
/// Interned [`GS_CURSOR_FEEDBACK`] atom (`0` = intern failed — treated as absent).
feedback_atom: Atom,
/// gamescope's verdict for THIS display: `Some(true)` = it is drawing the pointer here,
/// `Some(false)` = it is not, `None` = this gamescope publishes no verdict at all.
gs_visible: Option<bool>,
/// The X connection died (game/Xwayland exited) — skip it.
dead: bool,
}
impl XDisplay {
fn new(
name: String,
conn: RustConnection,
root: Window,
(feedback_atom, gs_visible): (Atom, Option<bool>),
) -> Self {
XDisplay {
name,
conn,
root,
last_pos: None,
shape: Shape::default(),
need_shape: true,
feedback_atom,
gs_visible,
dead: false,
}
}
/// Re-read gamescope's verdict, keeping a previously-seen one if the read fails (a transient
/// failure must not look like "this gamescope has no feedback" and re-arm the motion heuristic).
fn resync_feedback(&mut self) {
let fresh = read_cursor_feedback(&self.conn, self.root, self.feedback_atom);
if fresh.is_some() || self.gs_visible.is_none() {
self.gs_visible = fresh;
}
}
}
/// Cached cursor shape for one display.
#[derive(Default)]
struct Shape {
/// Straight-alpha RGBA (`w*h*4`, bytes R,G,B,A); empty before the first image arrives.
rgba: Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
/// XFixes' own per-display cursor serial — bumps on every shape change.
serial: u64,
/// A hidden pointer arrives as an all-transparent image; kept so a position-only tick preserves
/// the last known visibility.
visible: bool,
}
fn run(
mut displays: Vec<XDisplay>,
slot: Arc<Mutex<Option<CursorOverlay>>>,
stop: Arc<AtomicBool>,
) {
let mut active = 0usize;
// The overlay serial must bump whenever the DRAWN cursor changes — either the active display's
// shape OR which display is active (per-display XFixes serials aren't comparable across
// displays, so switching could reuse a number and the encoder would keep the old texture).
let mut out_serial = 0u64;
let mut last_key = (usize::MAX, u64::MAX);
let mut warned_image = false;
let mut last_resync = std::time::Instant::now();
while !stop.load(Ordering::Relaxed) {
// A missed/coalesced PropertyNotify would otherwise strand the verdict — re-read on a slow
// cadence so the source always converges (also picks the atom up if gamescope adds it late).
let resync = last_resync.elapsed() >= FEEDBACK_RESYNC;
if resync {
last_resync = std::time::Instant::now();
}
// 1) Poll every display's pointer; note which moved since last tick (the fallback focus
// signal, used only when this gamescope publishes no cursor verdict).
let mut active_moved = false;
let mut other_moved: Option<usize> = None;
for (i, d) in displays.iter_mut().enumerate() {
if d.dead {
continue;
}
// Drain pending events. Two kinds are selected: XFixes CursorNotify (the shape
// changed) and root PropertyNotify (gamescope republished its cursor verdict, among
// the many other properties it keeps on the root — hence the atom match).
let mut need_feedback = resync;
loop {
match d.conn.poll_for_event() {
Ok(Some(Event::XfixesCursorNotify(_))) => d.need_shape = true,
Ok(Some(Event::PropertyNotify(ev))) => {
need_feedback |= d.feedback_atom != 0 && ev.atom == d.feedback_atom;
}
Ok(Some(_)) => {}
Ok(None) => break,
Err(_) => {
d.dead = true;
break;
}
}
}
if need_feedback && !d.dead {
d.resync_feedback();
}
match fetch_pointer(&d.conn, d.root) {
Ok(p) if p.same_screen => {
let pos = (i32::from(p.root_x), i32::from(p.root_y));
let moved = d.last_pos.is_some_and(|lp| lp != pos);
d.last_pos = Some(pos);
if moved {
if i == active {
active_moved = true;
} else if other_moved.is_none() {
other_moved = Some(i);
}
}
}
Ok(_) => {} // pointer on another screen — keep the last position.
Err(_) => d.dead = true,
}
}
// 2) Pick the display to publish from, and decide whether a pointer should be drawn at all.
let states: Vec<(bool, Option<bool>)> =
displays.iter().map(|d| (d.dead, d.gs_visible)).collect();
let hidden_by_gamescope;
(active, hidden_by_gamescope) = pick_active(&states, active, active_moved, other_moved);
if displays.get(active).is_none_or(|d| d.dead) {
match displays.iter().position(|d| !d.dead) {
Some(k) => active = k,
None => {
std::thread::sleep(POLL); // all connections dead — idle until Drop.
continue;
}
}
}
// 3) Fetch the active display's shape if a CursorNotify (or a focus switch) left it stale.
if displays[active].need_shape {
match fetch_cursor_image(&displays[active].conn) {
Ok(img) => {
update_shape(&mut displays[active].shape, &img);
displays[active].need_shape = false;
}
Err(e) => {
if !warned_image {
warned_image = true;
tracing::warn!(error = %e, "gamescope cursor: GetCursorImage failed — retrying");
}
}
}
}
// 4) Publish the ACTIVE display's pointer + shape (or clear the slot when it has no cursor
// of its own — so a focus switch never leaves the other display's stale pointer showing).
// A pointer gamescope is not drawing is published `visible: false`, NOT dropped: the
// encode loop overwrites the frame's overlay from this slot and strips invisible ones, so
// a `None` here would leave the last visible overlay standing on repeat frames.
let d = &displays[active];
let drawn = d.shape.visible && !hidden_by_gamescope;
let overlay = match (d.last_pos, d.shape.rgba.is_empty()) {
(Some((px, py)), false) => {
let key = (active, d.shape.serial);
if key != last_key {
out_serial += 1;
last_key = key;
}
Some(CursorOverlay {
// Top-left = pointer position hotspot (the overlay contract).
x: px - d.shape.hot_x as i32,
y: py - d.shape.hot_y as i32,
w: d.shape.w,
h: d.shape.h,
rgba: Arc::clone(&d.shape.rgba),
serial: out_serial,
hot_x: d.shape.hot_x,
hot_y: d.shape.hot_y,
visible: drawn,
})
}
_ => None,
};
if let Ok(mut s) = slot.lock() {
*s = overlay;
}
std::thread::sleep(POLL);
}
}
/// Which display to publish from, and whether gamescope is drawing NO pointer right now —
/// `(active, hidden)`. `states` is one `(dead, gs_visible)` per display, in `displays` order.
///
/// PREFERRED: gamescope's own verdict. It is authoritative for a STATIC pointer, which is exactly
/// the case motion cannot read — a game grabs the pointer, gamescope parks it in the bottom-right
/// corner, and "follow whichever moved" then never switches again (see the module docs).
///
/// FALLBACK, only when NO live display publishes the verdict: the original motion heuristic —
/// sticky to the active display while its pointer moves (no flapping), else follow another that
/// moved. `hidden` is never asserted on this path, so an older gamescope keeps today's behaviour.
fn pick_active(
states: &[(bool, Option<bool>)],
active: usize,
active_moved: bool,
other_moved: Option<usize>,
) -> (usize, bool) {
let live = |&(dead, _): &(bool, Option<bool>)| !dead;
if states.iter().filter(|s| live(s)).any(|(_, v)| v.is_some()) {
return match states.iter().position(|s| live(s) && s.1 == Some(true)) {
// gamescope is drawing the pointer here — publish from it.
Some(i) => (i, false),
// Drawing none of them (a game grabbed it, or the idle auto-hide fired). Keep `active`
// so the shape cache and last position stay warm for the re-show; the caller publishes
// the overlay `visible: false` instead of dropping it.
None => (active, true),
};
}
match other_moved {
Some(j) if !active_moved => (j, false),
_ => (active, false),
}
}
/// Update `shape` from a fresh `GetCursorImage` reply. A hidden pointer (all-transparent) keeps the
/// last bitmap (instant re-show) but flips visibility; the serial still bumps so the change shows.
fn update_shape(shape: &mut Shape, img: &GetCursorImageReply) {
let visible =
img.width > 0 && img.height > 0 && img.cursor_image.iter().any(|&p| (p >> 24) & 0xff != 0);
if visible {
shape.rgba = Arc::new(argb_premul_to_straight_rgba(&img.cursor_image));
shape.w = u32::from(img.width);
shape.h = u32::from(img.height);
shape.hot_x = u32::from(img.xhot);
shape.hot_y = u32::from(img.yhot);
}
shape.visible = visible;
shape.serial = u64::from(img.cursor_serial);
}
/// One request+reply — x11rb splits errors (the request is `ConnectionError`, `reply()` is
/// `ReplyError` which is `From<ConnectionError>`), so the request `?` converts into the reply error.
fn fetch_cursor_image(conn: &RustConnection) -> Result<GetCursorImageReply, ReplyError> {
conn.xfixes_get_cursor_image()?.reply()
}
fn fetch_pointer(conn: &RustConnection, root: Window) -> Result<QueryPointerReply, ReplyError> {
conn.query_pointer(root)?.reply()
}
/// XFixes cursor pixels are packed `0xAARRGGBB` with **premultiplied** alpha (the Xrender / Xcursor
/// convention). The overlay + both blend paths want **straight** alpha RGBA (R,G,B,A bytes), like
/// the `SPA_META_Cursor` path — so un-premultiply here. (If on-glass shows over-bright fringes the
/// source wasn't premultiplied after all; drop the divide.)
fn argb_premul_to_straight_rgba(argb: &[u32]) -> Vec<u8> {
let mut out = Vec::with_capacity(argb.len() * 4);
for &px in argb {
let a = (px >> 24) & 0xff;
let r = (px >> 16) & 0xff;
let g = (px >> 8) & 0xff;
let b = px & 0xff;
let (r, g, b) = match a {
0 => (0, 0, 0),
255 => (r, g, b),
a => (
((r * 255 + a / 2) / a).min(255),
((g * 255 + a / 2) / a).min(255),
((b * 255 + a / 2) / a).min(255),
),
};
out.extend_from_slice(&[r as u8, g as u8, b as u8, a as u8]);
}
out
}
#[cfg(test)]
mod tests {
use super::pick_active;
/// Steam Gaming Mode shape: display 0 = Big Picture's Xwayland, 1 = the game's.
const BPM: usize = 0;
const GAME: usize = 1;
#[test]
fn follows_the_display_gamescope_draws_on() {
// Verdict beats motion: gamescope says the game's Xwayland owns the pointer, so we publish
// from it even though only BPM's (parked) pointer looks like it moved.
let states = [(false, Some(false)), (false, Some(true))];
assert_eq!(pick_active(&states, BPM, false, Some(BPM)), (GAME, false));
}
/// THE REGRESSION: gamescope hides the pointer by warping it to the root's bottom-right corner
/// and leaves the opaque arrow as the X cursor. Nothing moves ever again, so the motion
/// heuristic stayed on the parked display and composited that arrow at (w-1, h-1) — a sliver of
/// cursor welded to the corner of the stream for the whole session, in every game.
#[test]
fn a_pointer_gamescope_draws_nowhere_is_hidden_not_parked() {
let states = [(false, Some(false)), (false, Some(false))];
// `active` is kept (shape cache + last position stay warm for the re-show) but hidden.
assert_eq!(pick_active(&states, GAME, false, None), (GAME, true));
}
#[test]
fn re_show_returns_to_the_drawing_display() {
let states = [(false, Some(true)), (false, Some(false))];
assert_eq!(pick_active(&states, GAME, false, None), (BPM, false));
}
#[test]
fn a_dead_displays_verdict_is_ignored() {
// The game's Xwayland exited mid-session with a stale `Some(true)`; BPM is the live one.
let states = [(false, Some(true)), (true, Some(true))];
assert_eq!(pick_active(&states, GAME, false, None), (BPM, false));
// …and a dead display's `Some` must not count as "this gamescope publishes a verdict",
// which would blank the cursor forever on a gamescope that publishes none.
let states = [(false, None), (true, Some(false))];
assert_eq!(pick_active(&states, BPM, false, Some(GAME)), (GAME, false));
}
#[test]
fn no_verdict_falls_back_to_the_motion_heuristic() {
let none = [(false, None), (false, None)];
// Sticky while the active display's own pointer moves…
assert_eq!(pick_active(&none, BPM, true, Some(GAME)), (BPM, false));
// …otherwise follow the one that moved…
assert_eq!(pick_active(&none, BPM, false, Some(GAME)), (GAME, false));
// …and never assert `hidden` on this path (that would regress an older gamescope to a
// cursorless stream).
assert_eq!(pick_active(&none, BPM, false, None), (BPM, false));
}
}
+14 -211
View File
@@ -161,7 +161,7 @@ pub fn install_gpu_pref_hook() {
}); });
} }
pub(crate) unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> { unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> {
let mut blob: Option<ID3DBlob> = None; let mut blob: Option<ID3DBlob> = None;
let mut errs: Option<ID3DBlob> = None; let mut errs: Option<ID3DBlob> = None;
let r = D3DCompile( let r = D3DCompile(
@@ -194,7 +194,7 @@ pub(crate) unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> R
} }
/// Fullscreen-triangle vertex shader for the HDR conversion pass (3 verts, no input layout). /// Fullscreen-triangle vertex shader for the HDR conversion pass (3 verts, no input layout).
pub(crate) const HDR_VS: &str = r" const HDR_VS: &str = r"
struct VOut { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; }; struct VOut { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; };
VOut main(uint vid : SV_VertexID) { VOut main(uint vid : SV_VertexID) {
float2 uv = float2((vid << 1) & 2, vid & 2); float2 uv = float2((vid << 1) & 2, vid & 2);
@@ -308,9 +308,8 @@ float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV /// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This /// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
/// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`] /// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`]
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format. (There is no runtime /// surfaces a clear error if `CreateRenderTargetView` rejects the plane format so the caller can fall
/// fallback — the error propagates through `try_consume` and ends the session; the "R10 path" the /// back to the existing R10 path.
/// original design referenced was never kept.)
pub(crate) struct HdrP010Converter { pub(crate) struct HdrP010Converter {
vs: ID3D11VertexShader, vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader, ps_y: ID3D11PixelShader,
@@ -738,157 +737,14 @@ fn p010_reference(r: f64, g: f64, b: f64) -> (f64, f64, f64) {
/// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL. /// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL.
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn hdr_p010_selftest() -> Result<()> { pub fn hdr_p010_selftest() -> Result<()> {
hdr_p010_selftest_at(64, 64, None) use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_HARDWARE;
} use windows::Win32::Graphics::Dxgi::IDXGIAdapter;
/// [`hdr_p010_selftest`] at an arbitrary even size and (optionally) on a specific GPU vendor // 64x64, even dims. A 4x4 grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform →
/// (PCI vendor id, e.g. `0x8086` Intel / `0x10de` NVIDIA / `0x1002` AMD). The size matters on // exact chroma comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus
/// top of the 64×64 default because the field sessions run at capture resolutions whose height // a couple of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
/// is NOT 16-aligned (1080 → the encoder's align16 pool seam) and a driver may treat the planar const W: u32 = 64;
/// RTVs differently at real sizes; the vendor pin matters on dual-GPU boxes where the default const H: u32 = 64;
/// adapter is not the one the session encodes on.
/// Test support (used by pf-encode's live e2e): the 8 sRGB colour bars (white/yellow/cyan/green/
/// magenta/red/blue/black, sRGB 1.0 = scRGB 1.0 = 80 nits) as a w×h FP16 scRGB texture on the
/// adapter with `luid`, converted through the REAL [`HdrP010Converter`] into a P010 texture with
/// **`BIND_RENDER_TARGET` only, `MiscFlags` 0 — the exact bind profile of the IDD out-ring** (the
/// CPU-upload encoder tests can't use that profile, so only this path exercises "RTV-written P010
/// → encoder ingest copy"). Returns `(device, p010)`; expected decoded codes per bar are the
/// bars_pq2020 fixture's: (490,512,512) (478,423,518) (464,525,473) (450,432,476) (350,584,585)
/// (325,448,598) (226,650,535) (64,512,512).
#[cfg(target_os = "windows")]
#[doc(hidden)]
pub fn hdr_p010_convert_bars_on_luid(
luid: [u8; 8],
w: u32,
h: u32,
) -> Result<(ID3D11Device, ID3D11Texture2D)> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("bars pattern needs even non-zero dimensions, got {w}x{h}");
}
// sRGB primaries at full/zero channels: sRGB EOTF(1.0)=1.0, (0)=0 → the scRGB pattern is
// pure 0/1 floats and the PQ/BT.2020 reference codes above are exact.
const BARS: [(f32, f32, f32); 8] = [
(1.0, 1.0, 1.0),
(1.0, 1.0, 0.0),
(0.0, 1.0, 1.0),
(0.0, 1.0, 0.0),
(1.0, 0.0, 1.0),
(1.0, 0.0, 0.0),
(0.0, 0.0, 1.0),
(0.0, 0.0, 0.0),
];
let bar_w = (w / 8).max(1) as usize;
let mut fp16 = vec![0u16; (w * h * 4) as usize];
for y in 0..h as usize {
for x in 0..w as usize {
let (r, g, b) = BARS[(x / bar_w).min(7)];
let i = (y * w as usize + x) * 4;
fp16[i] = f32_to_f16(r);
fp16[i + 1] = f32_to_f16(g);
fp16[i + 2] = f32_to_f16(b);
fp16[i + 3] = f32_to_f16(1.0);
}
}
// SAFETY: same single-device/single-thread contract as `hdr_p010_selftest_at`; the FP16
// initial-data Vec outlives the synchronous CreateTexture2D; the returned COM handles own
// their references.
unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("dxgi factory")?;
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).context("adapter by luid")?;
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice(luid) for bars convert")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
let src_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R16G16B16A16_FLOAT,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: fp16.as_ptr() as *const c_void,
SysMemPitch: w * 8,
SysMemSlicePitch: 0,
};
let mut src_tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&src_desc, Some(&init), Some(&mut src_tex))
.context("CreateTexture2D(fp16 bars)")?;
let src_tex = src_tex.context("null src tex")?;
let mut src_srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&src_tex, None, Some(&mut src_srv))
.context("CreateShaderResourceView(fp16 bars)")?;
let src_srv = src_srv.context("null src srv")?;
// The IDD out-ring's exact profile: P010, RENDER_TARGET only, MiscFlags 0.
let p010_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
..Default::default()
};
let mut p010: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&p010_desc, None, Some(&mut p010))
.context("CreateTexture2D(P010 bars dst)")?;
let p010 = p010.context("null p010 tex")?;
let conv = HdrP010Converter::new(&device)?;
conv.convert(&device, &context, &src_srv, &p010, w, h)?;
Ok((device, p010))
}
}
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_UNKNOWN};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("hdr-p010-selftest needs even non-zero dimensions, got {w}x{h}");
}
// A grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → exact chroma
// comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus a couple
// of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
#[allow(non_snake_case)]
let (W, H) = (w, h);
const BLK: u32 = 16; const BLK: u32 = 16;
// (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values. // (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values.
let named: [(&str, f32, f32, f32); 8] = [ let named: [(&str, f32, f32, f32); 8] = [
@@ -941,36 +797,12 @@ pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
// `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are // `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are
// proven individually at the `read_u16` closure below. // proven individually at the `read_u16` closure below.
unsafe { unsafe {
// Device on the requested vendor's adapter (dual-GPU boxes encode on a specific one), else // Hardware D3D11 device (no adapter pin — the default GPU is fine for the self-test).
// the default hardware GPU. Always says which adapter ran — a PASS is only meaningful for
// the GPU it actually tested.
let adapter: Option<IDXGIAdapter> = match vendor {
None => None,
Some(want) => {
let factory: IDXGIFactory1 = CreateDXGIFactory1().context("dxgi factory")?;
let mut found = None;
for i in 0.. {
let Ok(a) = factory.EnumAdapters(i) else {
break;
};
let desc = a.GetDesc().context("adapter desc")?;
if desc.VendorId == want {
found = Some(a);
break;
}
}
Some(found.with_context(|| format!("no adapter with vendor id {want:#x}"))?)
}
};
let mut device: Option<ID3D11Device> = None; let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None; let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice( D3D11CreateDevice(
adapter.as_ref(), None::<&IDXGIAdapter>,
if adapter.is_some() { D3D_DRIVER_TYPE_HARDWARE,
D3D_DRIVER_TYPE_UNKNOWN
} else {
D3D_DRIVER_TYPE_HARDWARE
},
HMODULE::default(), HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]), Some(&[D3D_FEATURE_LEVEL_11_0]),
@@ -982,22 +814,6 @@ pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
.context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?; .context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?;
let device = device.context("null device")?; let device = device.context("null device")?;
let context = context.context("null context")?; let context = context.context("null context")?;
{
let dxgi: windows::Win32::Graphics::Dxgi::IDXGIDevice =
device.cast().context("device -> IDXGIDevice")?;
let desc = dxgi.GetAdapter().context("GetAdapter")?.GetDesc()?;
let name = String::from_utf16_lossy(
&desc.Description[..desc
.Description
.iter()
.position(|&c| c == 0)
.unwrap_or(desc.Description.len())],
);
println!(
"adapter: {name} (vendor {:#06x}, luid {:08x}:{:08x})",
desc.VendorId, desc.AdapterLuid.HighPart, desc.AdapterLuid.LowPart
);
}
// Source FP16 texture (initialized) + SRV. // Source FP16 texture (initialized) + SRV.
let src_desc = D3D11_TEXTURE2D_DESC { let src_desc = D3D11_TEXTURE2D_DESC {
@@ -1359,16 +1175,3 @@ impl VideoConverter {
blt.context("VideoProcessorBlt") blt.context("VideoProcessorBlt")
} }
} }
#[cfg(test)]
mod hdr_selftests {
/// LIVE (needs the GPU): [`super::hdr_p010_selftest_at`] at the field capture size — 1080 is
/// NOT 16-aligned, and the planar-RTV write path is driver-specific per vendor. Pinned to the
/// Intel adapter (`0x8086`), so it runs on the Intel validation boxes and errors out cleanly
/// ("no adapter") elsewhere. `cargo test -p pf-capture -- --ignored hdr_p010 --nocapture`.
#[test]
#[ignore]
fn hdr_p010_selftest_intel_1080_live() {
super::hdr_p010_selftest_at(1920, 1080, Some(0x8086)).expect("hdr p010 selftest @1080");
}
}
+16 -496
View File
@@ -365,12 +365,6 @@ pub unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &str) ->
#[path = "idd_push/channel.rs"] #[path = "idd_push/channel.rs"]
mod channel; mod channel;
#[path = "idd_push/cursor.rs"]
mod cursor;
#[path = "idd_push/cursor_blend.rs"]
mod cursor_blend;
#[path = "idd_push/cursor_poll.rs"]
mod cursor_poll;
#[path = "idd_push/descriptor.rs"] #[path = "idd_push/descriptor.rs"]
mod descriptor; mod descriptor;
#[path = "idd_push/stall.rs"] #[path = "idd_push/stall.rs"]
@@ -392,64 +386,6 @@ pub struct IddPushCapturer {
/// The sealed channel's handle-duplication broker (WUDFHost process + control device); used at open /// The sealed channel's handle-duplication broker (WUDFHost process + control device); used at open
/// and again on every ring recreate to deliver fresh duplicates. /// and again on every ring recreate to deliver fresh duplicates.
broker: ChannelBroker, broker: ChannelBroker,
/// The v5 hardware-cursor channel's host end (`Some` = delivered; the driver declared the
/// hardware cursor and seqlock-publishes into it). Survives ring recreates — the section is
/// independent of the frame ring's generation. With the channel delivered, the driver's
/// hardware cursor keeps DWM from compositing ANY cursor into the frame; the SHAPE now comes
/// from [`cursor_poll::CursorPoller`] (the IddCx query is alpha-only — see cursor_poll.rs),
/// and this shm read is the fallback if that poller dies.
cursor_shared: Option<cursor::CursorShared>,
/// The GDI cursor-shape poller (design §8): the overlay source while alive. `Some` when
/// `cursor_shared` is (both ride the negotiated cursor channel + successful delivery) — or
/// when `composite_forced` (no channel, but the target's sticky declare needs a blend source).
cursor_poll: Option<cursor_poll::CursorPoller>,
/// Retained delivery sender (`IOCTL_SET_CURSOR_CHANNEL`) for RE-delivery: a driver-side
/// monitor re-arrival (match-window re-arrival resize, a sibling session recreating the
/// shared slot) destroys the driver's cursor worker — the section here survives, so the
/// channel is re-delivered on ring recreates.
cursor_sender: Option<crate::CursorChannelSender>,
/// The cursor-render flip sender (`IOCTL_SET_CURSOR_FORWARD`) — the secure-desktop guard's
/// actuator. UAC/Winlogon render only through the OS's software-cursor path (its default on
/// every mode commit); with our hardware cursor declared (and re-declared on every
/// swap-chain assign) that path never comes back, and the secure desktop never presents —
/// the stream freezes on the last normal-desktop frame for the whole UAC/lock interaction.
/// [`Self::poll_secure_desktop`] flips the declare off/on at the secure-desktop edges.
cursor_forward: Option<crate::CursorForwardSender>,
/// The secure-desktop guard's edge state: `true` = the poller reports a secure input
/// desktop and the declare is currently stood down.
secure_active: bool,
/// The CAPTURE mouse model is active — the HOST composites the pointer into the frame
/// (see cursor_blend.rs for why DWM cannot: a declared IddCx hardware cursor is forever).
composite_cursor: bool,
/// This session never negotiated the cursor channel but its target carries an IRREVOCABLE
/// hardware-cursor declare from an earlier session (`WinCaptureTarget::cursor_excluded`,
/// §8.6): DWM delivers pointer-free frames and no client draws the cursor, so the ONLY path
/// to a visible pointer is compositing here. Pins `composite_cursor` on — nothing may turn
/// it off (there is no channel to hand the pointer to).
composite_forced: bool,
/// The cursor-quad blend pass (lazy; per capture device). `None` after a build failure —
/// composite mode then degrades to pointer-less frames (warned once).
cursor_blend: Option<cursor_blend::CursorBlendPass>,
cursor_blend_failed: bool,
/// The frame-sized blend scratch (slot copy + cursor quad): texture + SRV + (w, h, fmt)
/// it was built for — rebuilt when the ring geometry changes.
blend_scratch: Option<(
ID3D11Texture2D,
ID3D11ShaderResourceView,
u32,
u32,
DXGI_FORMAT,
)>,
/// The (serial, x, y, visible) of the LAST blended pointer — the composite-regen change
/// key: pointer-only motion produces no driver publish (the declared hardware cursor
/// doesn't dirty frames), so `try_consume` regenerates from the last slot when this moves.
last_blend_key: Option<(u64, i32, i32, bool)>,
/// The ring slot of the last FRESH publish — the regen source.
last_slot: Option<usize>,
/// The target's SDR-white scale (vs 80 nits) for HDR cursor compositing — refreshed on
/// each blend-scratch rebuild (first use + ring geometry changes). 2.5 ≈ the Windows
/// SDR-brightness default; without it the composited cursor renders visibly dark on HDR.
sdr_white_scale: f32,
width: u32, width: u32,
height: u32, height: u32,
slots: Vec<HostSlot>, slots: Vec<HostSlot>,
@@ -675,22 +611,11 @@ impl IddPushCapturer {
pyrowave: bool, pyrowave: bool,
keepalive: Box<dyn Send>, keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> { ) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so // The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime. // the stall log can correlate DWM holes with OS display events for the session's lifetime.
pf_win_display::display_events::spawn_once(); pf_win_display::display_events::spawn_once();
match Self::open_inner( match Self::open_inner(target, preferred, client_10bit, want_444, pyrowave, sender) {
target,
preferred,
client_10bit,
want_444,
pyrowave,
sender,
cursor_sender,
cursor_forward,
) {
Ok(mut me) => { Ok(mut me) => {
me._keepalive = keepalive; me._keepalive = keepalive;
Ok(me) Ok(me)
@@ -707,8 +632,6 @@ impl IddPushCapturer {
want_444: bool, want_444: bool,
pyrowave: bool, pyrowave: bool,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> { ) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the // The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor // selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -731,8 +654,6 @@ impl IddPushCapturer {
pyrowave, pyrowave,
luid, luid,
sender.clone(), sender.clone(),
cursor_sender.clone(),
cursor_forward.clone(),
) { ) {
Ok(me) => Ok(me), Ok(me) => Ok(me),
Err(e) => { Err(e) => {
@@ -766,8 +687,6 @@ impl IddPushCapturer {
pyrowave, pyrowave,
drv, drv,
sender, sender,
cursor_sender,
cursor_forward,
) )
.context("IDD-push rebind to the driver's reported render adapter") .context("IDD-push rebind to the driver's reported render adapter")
} }
@@ -783,8 +702,6 @@ impl IddPushCapturer {
pyrowave: bool, pyrowave: bool,
luid: LUID, luid: LUID,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> { ) -> Result<Self> {
let (pw, ph, _hz) = preferred let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?; .context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -1018,61 +935,6 @@ impl IddPushCapturer {
) )
.context("deliver IDD-push frame channel to the driver")?; .context("deliver IDD-push frame channel to the driver")?;
// v5 hardware-cursor channel (M2c): create + deliver the CursorShm section. Failure
// is NON-fatal — the driver never declares the hardware cursor without this delivery,
// so the session degrades to today's composited pointer (and the forwarder simply
// never sees a live overlay).
let cursor_shared = cursor_sender.as_ref().and_then(|send_cursor| {
match cursor::CursorShared::create(target.target_id) {
Ok(cs) => {
// Deliver via the shared helper (also used for RE-delivery after a
// driver-side monitor re-arrival destroyed the worker).
deliver_cursor_channel(&broker, target.target_id, &cs, send_cursor)
.then_some(cs)
}
Err(e) => {
tracing::warn!(
"cursor section creation failed (composited cursor stays): {e:#}"
);
None
}
}
});
// No channel this session, but the target's sticky declare (an EARLIER session's —
// irrevocable, §8.6) keeps DWM's frames pointer-free with no client drawing either:
// the only visible pointer is the one composited here, so force composite mode on.
let composite_forced = target.cursor_excluded && cursor_sender.is_none();
if composite_forced {
tracing::info!(
target_id = target.target_id,
"target carries an irrevocable hardware-cursor declare from an earlier \
desktop-mode session and this session has no cursor channel the host \
composites the pointer into frames (forced, for the session's life)"
);
}
// The GDI shape poller rides the SAME gate as the delivered channel: with the driver's
// hardware cursor keeping the frame cursor-free, the poller supplies the full-fidelity
// shape (masked/monochrome included — the IddCx query can't; see cursor_poll.rs).
// Forced-composite sessions need it too — it is their only shape/position source.
let cursor_poll = (cursor_shared.is_some() || composite_forced).then(|| {
// Safety of the CCD call: read-only QueryDisplayConfig over owned locals (same
// call CursorShared::create makes) — already inside open_on's unsafe region.
let rect = pf_win_display::win_display::source_desktop_rect(target.target_id)
.unwrap_or((0, 0, i32::MAX, i32::MAX));
cursor_poll::CursorPoller::spawn(target.target_id, rect)
});
// Heal the driver's persisted cursor-forward state: a session that died on the
// secure desktop (client drops at the lock screen — the common case) leaves the
// per-target desired state `false`, and the NEXT session's channel delivery would
// adopt UNdeclared (the exact cross-session composite trap of §8.6). A fresh
// session always starts declared; the secure-desktop guard re-disables if the
// secure desktop is (still) up, via its first `poll_secure_desktop` edge.
if let (Some(_), Some(fwd)) = (cursor_shared.as_ref(), cursor_forward.as_ref()) {
if let Err(e) = fwd(true) {
tracing::debug!("cursor-forward reset at open failed (pre-v6 driver?): {e:#}");
}
}
tracing::info!( tracing::info!(
target_id = target.target_id, target_id = target.target_id,
wudf_pid = target.wudf_pid, wudf_pid = target.wudf_pid,
@@ -1131,19 +993,6 @@ impl IddPushCapturer {
last_seq: 0, last_seq: 0,
last_present: None, last_present: None,
status_logged: false, status_logged: false,
cursor_shared,
cursor_poll,
cursor_sender,
cursor_forward,
secure_active: false,
composite_cursor: composite_forced,
composite_forced,
cursor_blend: None,
cursor_blend_failed: false,
blend_scratch: None,
last_blend_key: None,
last_slot: None,
sdr_white_scale: 1.0,
// Held from BEFORE the first-frame gate (the display must not idle off while we // Held from BEFORE the first-frame gate (the display must not idle off while we
// wait for the first compose) until the capturer drops with the session. // wait for the first compose) until the capturer drops with the session.
_display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(), _display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(),
@@ -1488,24 +1337,10 @@ impl IddPushCapturer {
"IDD push: frame-channel re-delivery failed after ring recreate" "IDD push: frame-channel re-delivery failed after ring recreate"
); );
} }
// Ring recreates ride display churn that can also have re-arrived the MONITOR driver-side
// (destroying its cursor worker with it) — re-deliver the surviving cursor section so the
// hardware-cursor declaration follows the CURRENT monitor generation.
if let (Some(cs), Some(send)) = (self.cursor_shared.as_ref(), self.cursor_sender.as_ref()) {
let _ = deliver_cursor_channel(&self.broker, self.target_id, cs, send);
}
self.blend_scratch = None; // ring geometry/format changed — rebuild at next blend
self.last_slot = None; // old-ring slot indices are meaningless now
self.last_seq = 0; self.last_seq = 0;
self.out_ring.clear(); // the output format changed → rebuild lazily at the new format self.out_ring.clear(); // the output format changed → rebuild lazily at the new format
self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode
self.hdr_p010_conv = None; self.hdr_p010_conv = None;
// The PyroWave CSC is mode-baked too (BgraToYuvPlanes picks different SDR vs HDR shaders
// and R8/R8G8 vs R16/R16G16 outputs). Without this, a display_hdr flip (Downgrade point D:
// client_10bit=true but HDR couldn't enable at open) reused the stale SDR converter against
// the freshly HDR-formatted pyro ring — every frame corrupted. `ensure_pyro_conv` only
// builds when None, so it must be reset here like its siblings.
self.pyro_conv = None;
self.pyro_ring.clear(); // PyroWave two-plane ring is sized → rebuild at the new mode self.pyro_ring.clear(); // PyroWave two-plane ring is sized → rebuild at the new mode
self.pyro_last = None; self.pyro_last = None;
self.out_idx = 0; self.out_idx = 0;
@@ -1781,201 +1616,8 @@ impl IddPushCapturer {
Ok(Some((self.pyro_fence_handle, value))) Ok(Some((self.pyro_fence_handle, value)))
} }
/// The (serial, x, y, visible) of the CURRENT polled cursor — the composite-regen change
/// key. `None` while the poller has no shape yet (or isn't running).
fn cursor_blend_key(&self) -> Option<(u64, i32, i32, bool)> {
self.cursor_poll
.as_ref()
.and_then(|p| p.read())
.map(|o| (o.serial, o.x, o.y, o.visible))
}
/// Composite the pointer for this convert: ensure the frame-sized blend scratch, copy the
/// slot into it, and alpha-blend the GDI poller's shape at its polled position. Returns the
/// scratch (texture + SRV) the conversion should read INSTEAD of the slot; `None` degrades
/// to the pointer-less slot (scratch/pass creation failed — warned once). A hidden pointer
/// blends nothing (the plain copy is the correct frame).
///
/// # Safety
/// D3D11 calls on the owning capture/encode thread's device + immediate context, called
/// while holding the slot's keyed mutex (the copy reads the slot).
unsafe fn prepare_blend_scratch(
&mut self,
slot_tex: &ID3D11Texture2D,
) -> Option<(ID3D11Texture2D, ID3D11ShaderResourceView)> {
let fmt = self.ring_format();
// (Re)build the scratch at the current ring geometry.
let stale = self
.blend_scratch
.as_ref()
.is_none_or(|(_, _, w, h, f)| (*w, *h, *f) != (self.width, self.height, fmt));
if stale {
self.blend_scratch = None;
let desc = D3D11_TEXTURE2D_DESC {
Width: self.width,
Height: self.height,
MipLevels: 1,
ArraySize: 1,
Format: fmt,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
..Default::default()
};
let mut tex: Option<ID3D11Texture2D> = None;
let built = self
.device
.CreateTexture2D(&desc, None, Some(&mut tex))
.ok()
.and(tex)
.and_then(|t| {
let mut srv: Option<ID3D11ShaderResourceView> = None;
self.device
.CreateShaderResourceView(&t, None, Some(&mut srv))
.ok()
.and(srv)
.map(|v| (t, v))
});
match built {
Some((t, v)) => {
self.blend_scratch = Some((t, v, self.width, self.height, fmt));
if self.display_hdr {
// Where DWM places SDR white on this HDR desktop — the composited
// cursor must match or it reads dark (~2.5x at the Windows default).
// Queried only here: scratch rebuilds are rare, and the CCD query
// contends on the display-config lock, which must stay OFF the
// per-frame path.
// Safety: read-only CCD query over owned locals (within unsafe fn).
let queried =
pf_win_display::win_display::sdr_white_level_scale(self.target_id);
self.sdr_white_scale = queried.unwrap_or(self.sdr_white_scale);
tracing::info!(
target_id = self.target_id,
queried = ?queried,
applied = self.sdr_white_scale,
"cursor composite: HDR SDR-white scale (1.0 = 80 nits; None = \
query failed keeping the prior value)"
);
}
}
None => {
if !self.cursor_blend_failed {
self.cursor_blend_failed = true;
tracing::warn!(
"cursor blend scratch creation failed — capture-model frames stay \
pointer-less this session"
);
}
return None;
}
}
}
let (tex, srv, ..) = self.blend_scratch.as_ref().expect("just ensured");
let (tex, srv) = (tex.clone(), srv.clone());
self.context.CopyResource(&tex, slot_tex);
// Blend the pointer (visible shapes only; hidden = the copy alone is the frame).
let overlay = self.cursor_poll.as_ref().and_then(|p| p.read());
self.last_blend_key = overlay.as_ref().map(|o| (o.serial, o.x, o.y, o.visible));
if let Some(ov) = overlay.filter(|o| o.visible) {
if self.cursor_blend.is_none() && !self.cursor_blend_failed {
match cursor_blend::CursorBlendPass::new(&self.device) {
Ok(p) => self.cursor_blend = Some(p),
Err(e) => {
self.cursor_blend_failed = true;
tracing::warn!(
"cursor blend pass build failed — capture-model frames stay \
pointer-less this session: {e:#}"
);
}
}
}
if let Some(pass) = self.cursor_blend.as_mut() {
// FP16 ring = scRGB linear composition (HDR): linearize the sRGB shape and
// scale it to the target's SDR white so it matches the desktop around it.
let scale = if self.display_hdr {
self.sdr_white_scale
} else {
0.0
};
if let Err(e) = pass.blend(&self.device, &self.context, &tex, &ov, scale) {
if !self.cursor_blend_failed {
self.cursor_blend_failed = true;
tracing::warn!("cursor blend draw failed — pointer-less frames: {e:#}");
}
}
}
}
Some((tex, srv))
}
/// The secure-desktop guard (the 0.18.0 UAC/Winlogon regression). UAC consent and Winlogon
/// live on the SECURE desktop, which the OS renders through the software-cursor path — its
/// per-mode-commit default. With this session's IddCx hardware cursor declared (and
/// re-declared by the driver on every swap-chain assign), that path never materialises, the
/// secure desktop never presents into our swap-chain, and the stream freezes on the last
/// normal-desktop frame for the entire UAC/lock interaction. On the poller's secure edge:
/// stand the declare down (`SET_CURSOR_FORWARD` off — the driver stops its per-assign
/// re-declare — plus the host facade's forced same-mode re-commit that actualises the
/// software cursor); on dismissal, re-declare. Runs on the capture/encode thread every tick
/// (it must keep running while frames are stalled — that is exactly the state it exits).
fn poll_secure_desktop(&mut self) {
let Some(fwd) = self.cursor_forward.as_ref() else {
return;
};
// Sessions with a declare possibly in play: the channel session that declared it, and
// the forced-composite session whose (reused) driver monitor may still run an earlier
// session's cursor worker. A plain session on a clean target has no poller — no guard.
if self.cursor_shared.is_none() && !self.composite_forced {
return;
}
let secure = self
.cursor_poll
.as_ref()
.is_some_and(|p| p.secure_desktop());
if secure == self.secure_active {
return;
}
self.secure_active = secure;
if secure {
tracing::info!(
target_id = self.target_id,
"secure desktop (UAC/Winlogon) active — standing the IddCx hardware-cursor \
declare down so the OS software-cursor path can render it"
);
if let Err(e) = fwd(false) {
tracing::warn!(
"secure-desktop cursor-forward stand-down failed (secure content may stay \
invisible this session): {e:#}"
);
}
} else {
tracing::info!(
target_id = self.target_id,
"secure desktop dismissed — restoring the cursor render model"
);
// Re-declare only for the session that RUNS the cursor channel; a forced-composite
// session never wanted the declare (leaving the driver's desired state off also
// stops a reused worker's per-assign re-declares for good — the next channel
// session's open-time reset re-arms it).
if self.cursor_shared.is_some() {
if let Err(e) = fwd(true) {
tracing::warn!(
"secure-desktop cursor-forward re-enable failed (client-drawn cursor \
may double with a composited one): {e:#}"
);
}
}
}
}
fn try_consume(&mut self) -> Result<Option<CapturedFrame>> { fn try_consume(&mut self) -> Result<Option<CapturedFrame>> {
self.log_driver_status_once(); self.log_driver_status_once();
// The secure-desktop guard first: while UAC/Winlogon is up there may be NO fresh frames
// at all — this edge is what brings them back.
self.poll_secure_desktop();
// Follow the display: a "Use HDR" flip recreates the ring at the matching format. // Follow the display: a "Use HDR" flip recreates the ring at the matching format.
self.poll_display_hdr(); self.poll_display_hdr();
// Recover-or-drop (GB1): if a descriptor change triggered a recreate but no fresh frame has resumed // Recover-or-drop (GB1): if a descriptor change triggered a recreate but no fresh frame has resumed
@@ -2034,26 +1676,10 @@ impl IddPushCapturer {
return Ok(None); return Ok(None);
} }
let seq = u64::from(tok.seq); let seq = u64::from(tok.seq);
let mut slot = tok.slot as usize; let slot = tok.slot as usize;
let fresh = seq != self.last_seq && slot < self.slots.len(); if seq == self.last_seq || slot >= self.slots.len() {
let mut regen = false;
if !fresh {
// Composite cursor model: pointer-only motion produces NO new publish (the declared
// hardware cursor never dirties the frame), so a static desktop would freeze the
// blended pointer. Regenerate from the LAST slot whenever the polled cursor state
// changed — the re-converted out-ring frame carries the pointer's new position.
let moved = self.composite_cursor
&& self.last_slot.is_some()
&& self.cursor_blend_key() != self.last_blend_key;
if !moved {
return Ok(None); return Ok(None);
} }
slot = self.last_slot.expect("checked above");
if slot >= self.slots.len() {
return Ok(None); // ring shrank across a recreate — wait for a fresh publish
}
regen = true;
}
// Build the ring + converter BEFORE acquiring the slot so nothing between Acquire and Release // Build the ring + converter BEFORE acquiring the slot so nothing between Acquire and Release
// can `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot). // can `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot).
// PyroWave uses its OWN two-plane ring (`pyro_ring`); everything else the single NV12/BGRA ring. // PyroWave uses its OWN two-plane ring (`pyro_ring`); everything else the single NV12/BGRA ring.
@@ -2086,31 +1712,18 @@ impl IddPushCapturer {
// Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the // Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the
// ~3 ms encode — NVENC reads the host out-ring slot, not the keyed-mutex slot), so the driver gets // ~3 ms encode — NVENC reads the host out-ring slot, not the keyed-mutex slot), so the driver gets
// the slot back immediately and the encode of the PREVIOUS frame overlaps this convert. // the slot back immediately and the encode of the PREVIOUS frame overlaps this convert.
// Clone the slot's COM interfaces (an AddRef each) so the guard borrows LOCALS, leaving
// `self` free for the composite blend prep inside the lock.
let (slot_tex, slot_srv, slot_mutex) = {
let s = &self.slots[slot]; let s = &self.slots[slot];
(s.tex.clone(), s.srv.clone(), s.mutex.clone())
};
// Acquire the slot's keyed mutex via a RAII guard, scoped to JUST the convert/copy below so it // Acquire the slot's keyed mutex via a RAII guard, scoped to JUST the convert/copy below so it
// releases at the same point as the old hand-written `ReleaseSync` (the driver gets the slot back // releases at the same point as the old hand-written `ReleaseSync` (the driver gets the slot back
// immediately, NOT held across the rest of `try_consume`) — but now leak-proof on any early return. // immediately, NOT held across the rest of `try_consume`) — but now leak-proof on any early return.
{ {
let Some(_lock) = KeyedMutexGuard::acquire(&slot_mutex, 0, 8) else { let Some(_lock) = KeyedMutexGuard::acquire(&s.mutex, 0, 8) else {
return Ok(None); return Ok(None);
}; };
// SAFETY: convert on the owning (encode) thread's immediate context, holding the slot lock. // SAFETY: convert on the owning (encode) thread's immediate context, holding the slot lock.
// A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing // A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing
// the slot back to the driver. // the slot back to the driver.
unsafe { unsafe {
// Composite cursor model: divert the convert input through the blend scratch —
// a slot copy with the pointer quad alpha-blended on top. `None` = compositing
// off or degraded (the conversion then reads the slot as always).
let blended = if self.composite_cursor {
self.prepare_blend_scratch(&slot_tex)
} else {
None
};
if self.pyrowave { if self.pyrowave {
// PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate // PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate
// plane textures via the mode-aware CSC; the shared fence signalled just after // plane textures via the mode-aware CSC; the shared fence signalled just after
@@ -2118,17 +1731,22 @@ impl IddPushCapturer {
// convert. The composition format is pinned to the negotiated depth. // convert. The composition format is pinned to the negotiated depth.
let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot"); let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot");
if let Some(conv) = self.pyro_conv.as_ref() { if let Some(conv) = self.pyro_conv.as_ref() {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv); conv.convert(
conv.convert(&self.context, src, y_rtv, cbcr_rtv, self.width, self.height)?; &self.context,
&s.srv,
y_rtv,
cbcr_rtv,
self.width,
self.height,
)?;
} }
} else if self.display_hdr { } else if self.display_hdr {
// HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010. // HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010.
if let Some(conv) = self.hdr_p010_conv.as_ref() { if let Some(conv) = self.hdr_p010_conv.as_ref() {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
conv.convert( conv.convert(
&self.device, &self.device,
&self.context, &self.context,
src, &s.srv,
out.as_ref().expect("out ring"), out.as_ref().expect("out ring"),
self.width, self.width,
self.height, self.height,
@@ -2138,14 +1756,12 @@ impl IddPushCapturer {
// SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma // SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma
// RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain // RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain
// copy-engine move; the slot releases back to the driver immediately. // copy-engine move; the slot releases back to the driver immediately.
let src = blended.as_ref().map(|(t, _)| t).unwrap_or(&slot_tex);
self.context self.context
.CopyResource(out.as_ref().expect("out ring"), src); .CopyResource(out.as_ref().expect("out ring"), &s.tex);
} else { } else {
// SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC. // SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC.
if let Some(conv) = self.video_conv.as_ref() { if let Some(conv) = self.video_conv.as_ref() {
let src = blended.as_ref().map(|(t, _)| t).unwrap_or(&slot_tex); conv.convert(&s.tex, out.as_ref().expect("out ring"))?;
conv.convert(src, out.as_ref().expect("out ring"))?;
} }
} }
} }
@@ -2153,20 +1769,13 @@ impl IddPushCapturer {
} }
self.out_idx = (i + 1) % ring_len; self.out_idx = (i + 1) % ring_len;
self.last_seq = seq; self.last_seq = seq;
if fresh {
self.last_slot = Some(slot);
}
if let Some((y, _, cbcr, _)) = pyro_slot.as_ref() { if let Some((y, _, cbcr, _)) = pyro_slot.as_ref() {
self.pyro_last = Some((y.clone(), cbcr.clone())); self.pyro_last = Some((y.clone(), cbcr.clone()));
} else { } else {
self.last_present = Some((out.as_ref().expect("out ring").clone(), pf)); self.last_present = Some((out.as_ref().expect("out ring").clone(), pf));
} }
let now = Instant::now(); let now = Instant::now();
if regen { if self.recovering_since.take().is_some() {
// A regen re-encodes OLD desktop content at a new pointer position — it is not a
// fresh driver frame; feeding the freshness/stall bookkeeping would mask a dead
// driver and pollute stall attribution.
} else if self.recovering_since.take().is_some() {
// A fresh frame resumed → recovered. The recovery gap is self-inflicted (ring // A fresh frame resumed → recovered. The recovery gap is self-inflicted (ring
// recreate, already logged by the recreate path) — reset the stall watch so it // recreate, already logged by the recreate path) — reset the stall watch so it
// doesn't read as a DWM stall. // doesn't read as a DWM stall.
@@ -2238,9 +1847,7 @@ impl IddPushCapturer {
} }
} }
} }
if !regen {
self.last_fresh = now; // feeds the driver-death watch self.last_fresh = now; // feeds the driver-death watch
}
// Build the frame. For PyroWave the encode input is the Y plane // Build the frame. For PyroWave the encode input is the Y plane
// (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence // (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence
// after the convert above. SAFETY: on the owning capture/encode thread. // after the convert above. SAFETY: on the owning capture/encode thread.
@@ -2254,7 +1861,6 @@ impl IddPushCapturer {
cbcr, cbcr,
fence_handle, fence_handle,
fence_value, fence_value,
ring_gen: self.generation,
}), }),
) )
} else { } else {
@@ -2313,7 +1919,6 @@ impl IddPushCapturer {
cbcr: dst_cbcr, cbcr: dst_cbcr,
fence_handle, fence_handle,
fence_value, fence_value,
ring_gen: self.generation,
}), }),
}), }),
cursor: None, cursor: None,
@@ -2368,82 +1973,7 @@ impl std::fmt::Display for AttachTexFail {
impl std::error::Error for AttachTexFail {} impl std::error::Error for AttachTexFail {}
/// Duplicate `cs`'s section into the driver's WUDFHost and send `IOCTL_SET_CURSOR_CHANNEL`.
/// `true` = the driver adopted it (worker declared per its `cursor_forward_on` state). Shared by
/// the open-time delivery and every RE-delivery (ring recreate / flip NOT_FOUND) — the request is
/// idempotent driver-side (a replaced worker is stopped + joined).
fn deliver_cursor_channel(
broker: &ChannelBroker,
target_id: u32,
cs: &cursor::CursorShared,
send_cursor: &crate::CursorChannelSender,
) -> bool {
// SAFETY: `cs.section_handle()` borrows the section mapping `cs` owns (live across this
// synchronous call); the broker's WUDFHost process handle is live for the broker's lifetime.
let value = match unsafe { broker.dup_into_public(cs.section_handle()) } {
Ok(v) => v,
Err(e) => {
tracing::warn!("cursor section duplication failed (composited cursor stays): {e:#}");
return false;
}
};
let req = pf_driver_proto::control::SetCursorChannelRequest {
target_id,
_pad: 0,
header_handle: value,
};
match send_cursor(&req) {
Ok(()) => {
tracing::info!(
target_id,
"IDD push(host): cursor channel delivered — driver declares the hardware cursor"
);
true
}
Err(e) => {
broker.close_remote_public(value);
tracing::warn!("cursor channel delivery failed (composited cursor stays): {e:#}");
false
}
}
}
impl Capturer for IddPushCapturer { impl Capturer for IddPushCapturer {
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
// A LIVE poller is the sole source — even while it still reports `None` (pre-first-shape):
// falling back to the shm mid-session would interleave two serial namespaces and poison
// the client's shape cache. The shm read only serves a poller that failed to start/died.
if let Some(p) = &self.cursor_poll {
if p.alive() {
return p.read();
}
}
self.cursor_shared.as_mut().and_then(|c| c.read())
}
fn set_cursor_forward(&mut self, on: bool) {
// The composite (capture) model is implemented HOST-side: the driver's hardware cursor
// stays declared for the session's whole life — the only dependable state (there is NO
// working un-declare; see cursor_blend.rs) — keeping every frame pointer-free, and the
// capturer blends the GDI poller's shape into the frame itself. No driver round-trip.
// `composite_forced` (a channel-less session on a sticky-declared target) is pinned ON:
// with no client drawing, un-compositing would erase the pointer entirely.
let composite = (!on && self.cursor_shared.is_some()) || self.composite_forced;
if self.composite_cursor != composite {
self.composite_cursor = composite;
self.last_blend_key = None; // regenerate immediately at the current pointer state
tracing::info!(
composite,
"cursor render model: host compositing {}",
if composite {
"ON (capture model — blending the pointer into frames)"
} else {
"OFF (client draws locally)"
}
);
}
}
fn next_frame(&mut self) -> Result<CapturedFrame> { fn next_frame(&mut self) -> Result<CapturedFrame> {
let deadline = Instant::now() + Duration::from_secs(20); let deadline = Instant::now() + Duration::from_secs(20);
loop { loop {
@@ -2577,16 +2107,6 @@ fn warn_444_hdr_downgrade_once() {
impl Drop for IddPushCapturer { impl Drop for IddPushCapturer {
fn drop(&mut self) { fn drop(&mut self) {
// A channel session ending while the secure-desktop guard is engaged must not leave the
// driver's per-target desired state off — the next session's channel delivery would
// adopt UNdeclared and silently run the composite model (§8.6's cross-session trap).
// The open-time reset also covers this (host-crash case); this is the orderly-teardown
// belt.
if self.secure_active && self.cursor_shared.is_some() {
if let Some(fwd) = self.cursor_forward.as_ref() {
let _ = fwd(true);
}
}
self.slots.clear(); self.slots.clear();
// The shared header section (`MappedSection`), the frame-ready `event` (`OwnedHandle`) and the // The shared header section (`MappedSection`), the frame-ready `event` (`OwnedHandle`) and the
// broker's WUDFHost process handle free themselves via RAII (unmap view, then close handle) — // broker's WUDFHost process handle free themselves via RAII (unmap view, then close handle) —
@@ -105,22 +105,6 @@ impl ChannelBroker {
Ok(out.0 as usize as u64) Ok(out.0 as usize as u64)
} }
/// Duplicate the cursor section into WUDFHost (v5 cursor channel) with the same
/// least-privilege section rights as the frame header. Thin `pub(super)` face over
/// [`dup_into`](Self::dup_into) for the cursor-delivery path in `open_on`.
///
/// # Safety
/// `h` must be a live handle of the current process.
pub(super) unsafe fn dup_into_public(&self, h: HANDLE) -> Result<u64> {
// SAFETY: forwarded contract — `h` is live per this fn's own contract.
unsafe { self.dup_into(h, Some(SECTION_MAP_RW)) }
}
/// [`close_remote`](Self::close_remote) for the cursor-delivery failure path.
pub(super) fn close_remote_public(&self, value: u64) {
self.close_remote(value);
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE` /// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result. /// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) { fn close_remote(&self, value: u64) {
@@ -1,194 +0,0 @@
//! Host side of the v5 hardware-cursor channel (remote-desktop-sweep M2c): the capturer creates
//! an unnamed [`CursorShm`] section, delivers it to the pf-vdisplay driver (which declares an
//! IddCx hardware cursor — DWM then EXCLUDES the pointer from the frames we consume), and reads
//! the driver's seqlock publishes here at encode-tick pace, converting them into the same
//! [`pf_frame::CursorOverlay`] the Linux portal path produces — everything downstream (the
//! cursor forwarder, the wire, the client renderer) is shared.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use pf_driver_proto::cursor::{
CursorShm, CURSOR_MAGIC, CURSOR_SHAPE_BYTES, CURSOR_SHAPE_MAX, CURSOR_SHAPE_OFFSET,
CURSOR_SHM_SIZE, CURSOR_TYPE_MASKED_COLOR,
};
use std::sync::atomic::AtomicU32;
/// The host end of one monitor's cursor channel: the section (we created it — the mapping stays
/// valid for the capturer's life) plus the reader's conversion cache.
pub(super) struct CursorShared {
section: MappedSection,
/// The monitor's desktop origin — IddCx reports positions in DESKTOP coordinates; the
/// overlay wants frame-relative. Fetched at attach (the virtual monitor's placement is
/// stable for the session; a topology change recreates the pipeline anyway).
origin: (i32, i32),
/// Conversion cache: the last `shape_id` whose pixels were converted, and the result.
/// Position-only updates (the common case) reuse it — a refcount bump, no pixel work.
cached_id: u32,
cached: Option<ConvertedShape>,
}
struct ConvertedShape {
rgba: std::sync::Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
}
impl CursorShared {
/// Create + initialize the section (magic stamped, seq even/zero). The returned handle is
/// the section itself (owned by `self`); the caller duplicates it into the WUDFHost.
pub(super) fn create(target_id: u32) -> Result<CursorShared> {
// SAFETY: plain FFI. Unnamed pagefile-backed section, host-lifetime owned; the view is
// mapped once and unmapped never (the capturer's life = the session's life).
let section = unsafe {
let map = CreateFileMappingW(
INVALID_HANDLE_VALUE,
None,
PAGE_READWRITE,
0,
CURSOR_SHM_SIZE as u32,
PCWSTR::null(),
)
.context("CreateFileMapping(cursor)")?;
let map = OwnedHandle::from_raw_handle(map.0 as _);
let view = MapViewOfFile(
HANDLE(map.as_raw_handle()),
FILE_MAP_ALL_ACCESS,
0,
0,
CURSOR_SHM_SIZE,
);
if view.Value.is_null() {
bail!("MapViewOfFile failed for the cursor section");
}
let shm = view.Value.cast::<CursorShm>();
std::ptr::write_bytes(view.Value.cast::<u8>(), 0, CURSOR_SHM_SIZE);
// Magic LAST-ish (the driver validates it at adopt; seq 0 = even = consistent).
std::sync::atomic::fence(Ordering::Release);
(*shm).magic = CURSOR_MAGIC;
MappedSection { handle: map, view }
};
// Desktop origin of this monitor's source — for the desktop→frame coordinate shift.
// SAFETY: `source_desktop_rect` only runs the CCD QueryDisplayConfig FFI over owned
// locals (same call the compose-kick path makes).
let rect = unsafe { pf_win_display::win_display::source_desktop_rect(target_id) };
let origin = rect.map(|(x, y, _w, _h)| (x, y)).unwrap_or((0, 0));
Ok(CursorShared {
section,
origin,
cached_id: 0,
cached: None,
})
}
/// The section handle for the broker's duplication into the WUDFHost.
pub(super) fn section_handle(&self) -> HANDLE {
HANDLE(self.section.handle.as_raw_handle())
}
/// Seqlock-read the driver's latest publish → a frame-relative [`pf_frame::CursorOverlay`].
/// `None` until the first publish lands (or while the pointer has never been seen). A hidden
/// pointer returns `Some` with `visible: false` — the forwarder turns that into the client's
/// relative-mode hint, exactly like the Linux path.
pub(super) fn read(&mut self) -> Option<pf_frame::CursorOverlay> {
let shm = self.section.ptr::<CursorShm>();
// SAFETY: the view spans CURSOR_SHM_SIZE for self's lifetime; seq is 4-aligned in the
// fixed layout (offset 4).
let seq = unsafe { &*std::ptr::addr_of!((*shm).seq).cast::<AtomicU32>() };
for _ in 0..64 {
let s1 = seq.load(Ordering::Acquire);
if s1 == 0 {
return None; // no publish yet
}
if s1 & 1 != 0 {
std::hint::spin_loop();
continue; // writer mid-update
}
// SAFETY: header reads within the mapped view; consistency is validated by the
// seq re-check below (a torn read is discarded and retried).
let hdr = unsafe { std::ptr::read_volatile(shm) };
// Shape pixels: convert only when the OS minted a new shape id.
if hdr.visible != 0 && hdr.shape_id != self.cached_id {
let rows = hdr.height.min(CURSOR_SHAPE_MAX) as usize;
let width = hdr.width.min(CURSOR_SHAPE_MAX) as usize;
let pitch = (hdr.pitch as usize).min(CURSOR_SHAPE_BYTES / rows.max(1));
let mut raw = vec![0u8; rows * pitch];
// SAFETY: the shape region spans CURSOR_SHAPE_BYTES from CURSOR_SHAPE_OFFSET
// inside the mapped view; `rows * pitch` is clamped to it above.
unsafe {
std::ptr::copy_nonoverlapping(
self.section.ptr::<u8>().add(CURSOR_SHAPE_OFFSET),
raw.as_mut_ptr(),
rows * pitch,
);
}
// Discard the copy if the writer raced us mid-shape (seq moved) — retry.
if seq.load(Ordering::Acquire) != s1 {
continue;
}
self.cached = Some(convert_shape(&hdr, &raw, width, rows, pitch));
self.cached_id = hdr.shape_id;
} else if seq.load(Ordering::Acquire) != s1 {
continue;
}
let shape = self.cached.as_ref()?;
return Some(pf_frame::CursorOverlay {
x: hdr.x - self.origin.0,
y: hdr.y - self.origin.1,
w: shape.w,
h: shape.h,
rgba: shape.rgba.clone(),
serial: u64::from(hdr.shape_id),
hot_x: shape.hot_x,
hot_y: shape.hot_y,
visible: hdr.visible != 0,
});
}
None // persistent tearing (writer wedged mid-seq) — skip this tick
}
}
/// Convert the OS's 32-bpp pitch-strided shape rows into the overlay's packed straight RGBA.
/// ALPHA cursors are BGRA with straight per-pixel alpha (swap R↔B). MASKED_COLOR approximates:
/// alpha 0x00 = opaque color pixel; 0xFF = an XOR pixel we cannot honor client-side — rendered
/// as a translucent mid-gray so inversion cursors stay visible instead of vanishing.
fn convert_shape(
hdr: &CursorShm,
raw: &[u8],
width: usize,
rows: usize,
pitch: usize,
) -> ConvertedShape {
let masked = hdr.cursor_type == CURSOR_TYPE_MASKED_COLOR;
let mut rgba = Vec::with_capacity(width * rows * 4);
for y in 0..rows {
let row = &raw[y * pitch..];
for x in 0..width {
let o = x * 4;
if o + 4 > row.len() {
rgba.extend_from_slice(&[0, 0, 0, 0]);
continue;
}
let (b, g, r, a) = (row[o], row[o + 1], row[o + 2], row[o + 3]);
if masked {
if a == 0 {
rgba.extend_from_slice(&[r, g, b, 0xFF]);
} else {
rgba.extend_from_slice(&[0x80, 0x80, 0x80, 0xB4]);
}
} else {
rgba.extend_from_slice(&[r, g, b, a]);
}
}
}
ConvertedShape {
rgba: std::sync::Arc::new(rgba),
w: width as u32,
h: rows as u32,
hot_x: hdr.hot_x.min(width.saturating_sub(1) as u32),
hot_y: hdr.hot_y.min(rows.saturating_sub(1) as u32),
}
}
@@ -1,216 +0,0 @@
//! Host-side cursor compositing for the CAPTURE mouse model (design/remote-desktop-sweep.md §8).
//!
//! Why the host draws it: once a monitor has ever declared an IddCx hardware cursor, DWM will
//! not composite the software cursor back into its frames — there is no un-declare DDI (the
//! empty-caps re-setup is rejected `STATUS_INVALID_PARAMETER`), and a successful same-mode
//! re-commit with the driver's re-declare provably suppressed still leaves the pointer excluded
//! (all observed on-glass, 26100). So the driver keeps its hardware cursor declared for the
//! session's whole life — the state that works — and when the client flips to the capture model
//! the HOST composites the pointer into the frame itself: a slot→scratch copy plus one
//! alpha-blended quad (the GDI poller's full-fidelity shape at its polled position), entirely
//! GPU-side on the capture device, before the normal conversion runs from the scratch.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use windows::core::s;
use windows::Win32::Graphics::Direct3D::D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST;
use windows::Win32::Graphics::Direct3D11::{
ID3D11BlendState, ID3D11Buffer, ID3D11PixelShader, ID3D11SamplerState, ID3D11VertexShader,
D3D11_BIND_CONSTANT_BUFFER, D3D11_BLEND_DESC, D3D11_BLEND_INV_SRC_ALPHA, D3D11_BLEND_ONE,
D3D11_BLEND_OP_ADD, D3D11_BLEND_SRC_ALPHA, D3D11_BUFFER_DESC, D3D11_COMPARISON_NEVER,
D3D11_CPU_ACCESS_WRITE, D3D11_FILTER_MIN_MAG_MIP_LINEAR, D3D11_MAPPED_SUBRESOURCE,
D3D11_MAP_WRITE_DISCARD, D3D11_RENDER_TARGET_BLEND_DESC, D3D11_SAMPLER_DESC,
D3D11_SUBRESOURCE_DATA, D3D11_TEXTURE_ADDRESS_CLAMP, D3D11_USAGE_DYNAMIC, D3D11_VIEWPORT,
};
use windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT_R8G8B8A8_UNORM;
/// Straight-alpha sample of the cursor bitmap. `linear_scale` = 0 passes sRGB through (SDR
/// ring); non-zero linearizes sRGB→scRGB AND multiplies by the target's SDR-white scale
/// (`sdr_white_level_scale` — 1.0 would put cursor-white at 80 nits, visibly DARKER than the
/// surrounding SDR desktop content DWM composes at the user's SDR-brightness setting).
const CURSOR_PS: &str = r"
Texture2D<float4> tx : register(t0);
SamplerState sm : register(s0);
cbuffer C : register(b0) { float linear_scale; float3 pad; };
float4 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_Target {
float4 c = tx.Sample(sm, uv);
if (linear_scale != 0.0) {
c.rgb = pow(abs(c.rgb), 2.2) * linear_scale;
}
return c;
}
";
/// The cursor-quad blend pass + its shape-texture cache. One per capturer (device-scoped).
pub(super) struct CursorBlendPass {
vs: ID3D11VertexShader,
ps: ID3D11PixelShader,
sampler: ID3D11SamplerState,
blend: ID3D11BlendState,
cbuf: ID3D11Buffer,
cbuf_scale: Option<f32>,
/// The uploaded shape (serial-keyed): SRV + dims in host pixels.
shape: Option<(u64, ID3D11ShaderResourceView, u32, u32)>,
}
impl CursorBlendPass {
pub(super) unsafe fn new(device: &ID3D11Device) -> Result<Self> {
let vsb = crate::dxgi::compile_shader(crate::dxgi::HDR_VS, s!("main"), s!("vs_5_0"))?;
let psb = crate::dxgi::compile_shader(CURSOR_PS, s!("main"), s!("ps_5_0"))?;
let mut vs = None;
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
let mut ps = None;
device.CreatePixelShader(&psb, None, Some(&mut ps))?;
let sd = D3D11_SAMPLER_DESC {
// LINEAR: the quad is drawn 1:1 in frame pixels, so this only matters at the
// half-texel edges; linear keeps them soft instead of ringing.
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
ComparisonFunc: D3D11_COMPARISON_NEVER,
MaxLOD: f32::MAX,
..Default::default()
};
let mut sampler = None;
device.CreateSamplerState(&sd, Some(&mut sampler))?;
// Straight-alpha over: dst.rgb = src.rgb*a + dst.rgb*(1-a); keep dst alpha.
let mut bd = D3D11_BLEND_DESC::default();
bd.RenderTarget[0] = D3D11_RENDER_TARGET_BLEND_DESC {
BlendEnable: true.into(),
SrcBlend: D3D11_BLEND_SRC_ALPHA,
DestBlend: D3D11_BLEND_INV_SRC_ALPHA,
BlendOp: D3D11_BLEND_OP_ADD,
SrcBlendAlpha: D3D11_BLEND_ONE,
DestBlendAlpha: D3D11_BLEND_ONE,
BlendOpAlpha: D3D11_BLEND_OP_ADD,
RenderTargetWriteMask: 0x0F,
};
let mut blend = None;
device.CreateBlendState(&bd, Some(&mut blend))?;
let cbd = D3D11_BUFFER_DESC {
ByteWidth: 16, // float to_linear + float3 pad
Usage: D3D11_USAGE_DYNAMIC,
BindFlags: D3D11_BIND_CONSTANT_BUFFER.0 as u32,
CPUAccessFlags: D3D11_CPU_ACCESS_WRITE.0 as u32,
..Default::default()
};
let mut cbuf = None;
device.CreateBuffer(&cbd, None, Some(&mut cbuf))?;
Ok(Self {
vs: vs.context("cursor blend vs")?,
ps: ps.context("cursor blend ps")?,
sampler: sampler.context("cursor blend sampler")?,
blend: blend.context("cursor blend state")?,
cbuf: cbuf.context("cursor blend cbuf")?,
cbuf_scale: None,
shape: None,
})
}
/// Upload `ov`'s bitmap if its serial is new; reuse the cached SRV otherwise.
unsafe fn ensure_shape(
&mut self,
device: &ID3D11Device,
ov: &pf_frame::CursorOverlay,
) -> Result<()> {
if self.shape.as_ref().is_some_and(|(s, ..)| *s == ov.serial) {
return Ok(());
}
if ov.rgba.len() < (ov.w as usize) * (ov.h as usize) * 4 || ov.w == 0 || ov.h == 0 {
bail!("malformed cursor overlay ({}x{})", ov.w, ov.h);
}
let desc = D3D11_TEXTURE2D_DESC {
Width: ov.w,
Height: ov.h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R8G8B8A8_UNORM,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: ov.rgba.as_ptr().cast(),
SysMemPitch: ov.w * 4,
SysMemSlicePitch: 0,
};
let mut tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&init), Some(&mut tex))
.context("CreateTexture2D(cursor shape)")?;
let tex = tex.context("null cursor shape texture")?;
let mut srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&tex, None, Some(&mut srv))
.context("CreateShaderResourceView(cursor shape)")?;
self.shape = Some((ov.serial, srv.context("null cursor shape srv")?, ov.w, ov.h));
Ok(())
}
/// Alpha-blend `ov` onto `dst` (frame-sized, RENDER_TARGET-capable — the blend scratch).
/// `linear_scale`: 0 = SDR passthrough; non-zero = the frame is FP16 scRGB (HDR
/// composition) — linearize and scale to the target's SDR white. The quad is placed purely
/// via the viewport (the fullscreen-triangle VS fills whatever viewport is set), clipped by
/// the target automatically.
pub(super) unsafe fn blend(
&mut self,
device: &ID3D11Device,
ctx: &ID3D11DeviceContext,
dst: &ID3D11Texture2D,
ov: &pf_frame::CursorOverlay,
linear_scale: f32,
) -> Result<()> {
self.ensure_shape(device, ov)?;
let (_, srv, w, h) = self.shape.as_ref().expect("shape just ensured");
if self.cbuf_scale != Some(linear_scale) {
let cb: [f32; 4] = [linear_scale, 0.0, 0.0, 0.0];
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
if ctx
.Map(&self.cbuf, 0, D3D11_MAP_WRITE_DISCARD, 0, Some(&mut mapped))
.is_ok()
{
std::ptr::copy_nonoverlapping(cb.as_ptr(), mapped.pData as *mut f32, cb.len());
ctx.Unmap(&self.cbuf, 0);
}
self.cbuf_scale = Some(linear_scale);
}
let mut rtv: Option<ID3D11RenderTargetView> = None;
device
.CreateRenderTargetView(dst, None, Some(&mut rtv))
.context("CreateRenderTargetView(cursor blend scratch)")?;
let rtv = rtv.context("null cursor blend rtv")?;
ctx.OMSetRenderTargets(Some(&[Some(rtv)]), None);
ctx.OMSetBlendState(&self.blend, None, 0xffff_ffff);
ctx.VSSetShader(&self.vs, None);
ctx.PSSetShader(&self.ps, None);
ctx.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
ctx.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
ctx.PSSetConstantBuffers(0, Some(&[Some(self.cbuf.clone())]));
ctx.IASetInputLayout(None);
ctx.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
// Placement IS the viewport: the VS fills it, the OS clips it to the target.
let vp = D3D11_VIEWPORT {
TopLeftX: ov.x as f32,
TopLeftY: ov.y as f32,
Width: *w as f32,
Height: *h as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
};
ctx.RSSetViewports(Some(&[vp]));
ctx.Draw(3, 0);
// Unbind so the scratch can be bound as a conversion INPUT without a hazard warning.
ctx.OMSetRenderTargets(None, None);
let none_srv: [Option<ID3D11ShaderResourceView>; 1] = [None];
ctx.PSSetShaderResources(0, Some(&none_srv));
Ok(())
}
}
@@ -1,508 +0,0 @@
//! GDI cursor poller — the Windows cursor-SHAPE source for the cursor-forward channel
//! (design/remote-desktop-sweep.md §8, the M2c redesign).
//!
//! Why not the IddCx hardware-cursor query (the v5 `CursorShm` path, now the fallback): it is
//! alpha-only BY DESIGN — `IDDCX_CURSOR_SHAPE_TYPE` has no monochrome value, the OS pre-converts
//! monochrome to masked-color (IDDCX_CURSOR_CAPS docs), and masked-color delivery is dead code on
//! modern builds (proven on-glass at every `ColorXorCursorSupport` level; no public evidence of a
//! MASKED_COLOR delivery anywhere). The driver keeps its hardware cursor declared at XOR FULL
//! purely so DWM EXCLUDES every cursor type from the IDD frame.
//!
//! Why not DXGI Desktop Duplication `GetFramePointerShape`: its `PointerPosition.Visible` goes
//! stale when the cursor moves only via injected input on current Win11 (Sunshine #5293 — exactly
//! a Punktfunk session's topology), it burns one of the session's four duplication slots, and its
//! per-output metadata on IDD monitors has conflicting field reports. The GDI path below is the
//! metadata-forwarding-remote-desktop pattern (RustDesk, WebRTC/Chrome Remote Desktop, OBS): the
//! cursor is per-session global state in win32k, readable cross-process, and `CURSOR_SHOWING` is
//! the logical visibility — immune to all of the above.
//!
//! Works because the capture host runs as SYSTEM *inside the interactive session* on
//! `winsta0\default` (the service supervisor retargets the token — `windows/service.rs`
//! `spawn_host`), so the poller thread sees the session's cursor directly; no helper process.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use windows::Win32::Graphics::Gdi::{
DeleteObject, GetDC, GetDIBits, GetObjectW, ReleaseDC, BITMAP, BITMAPINFO, BITMAPINFOHEADER,
BI_RGB, DIB_RGB_COLORS, HBITMAP, HDC,
};
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, HDESK, UOI_NAME,
};
use windows::Win32::UI::HiDpi::{
SetThreadDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
};
use windows::Win32::UI::WindowsAndMessaging::{
CopyIcon, DestroyIcon, GetCursorInfo, GetIconInfo, CURSORINFO, HICON, ICONINFO,
};
/// `CURSORINFO.flags` bits (WindowsAndMessaging): the pointer is logically shown /
/// touch-or-pen-suppressed. Named locally so the visibility rule below reads as the docs do.
const CURSOR_SHOWING: u32 = 0x1;
const CURSOR_SUPPRESSED: u32 = 0x2;
/// A converted shape: the cache the per-tick overlay is assembled from. `rgba` is `Arc` so the
/// slot publish (and every downstream frame attach) is a refcount bump.
struct Shape {
rgba: std::sync::Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
serial: u64,
}
/// Off-thread GDI cursor poller. Samples `GetCursorInfo` at ~60 Hz, rasterises the `HCURSOR` only
/// when its handle value changes, and publishes a ready [`pf_frame::CursorOverlay`] snapshot; the
/// capture thread's per-tick cost is one uncontended mutex read + an `Arc` clone
/// (same split as [`DescriptorPoller`], and for the same reason: user32/gdi32 calls have no place
/// on the capture/encode thread).
pub(super) struct CursorPoller {
slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>>,
stop: Arc<AtomicBool>,
/// The input desktop is a SECURE desktop (Winlogon — UAC consent / lock / logon). Classified
/// on every reattach; the capturer polls it to stand the IddCx hardware-cursor declare down
/// while the secure desktop needs the software-cursor path to render (see
/// `IddPushCapturer::poll_secure_desktop`).
secure: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl CursorPoller {
/// ~250 Hz: the polled position is ALSO the composite-blend position (capture model), so
/// it must out-pace the fastest session — at 16 ms a 240 fps stream re-used a stale
/// position for ~4 consecutive frames and the composited pointer visibly stuttered
/// against the video. A tick is one `GetCursorInfo` syscall (rasterisation only on shape
/// change), so 250 Hz is still negligible CPU.
const INTERVAL: Duration = Duration::from_millis(4);
/// Unconditional input-desktop reattach cadence — catches secure-desktop (UAC/lock) switches
/// without a failure signal (`GetCursorInfo` on a stale desktop *succeeds* with stale data).
/// 250 ms, not the original 2 s: the reattach now also feeds [`Self::secure_desktop`], which
/// gates when the secure desktop becomes VISIBLE in the stream (the hardware-cursor
/// stand-down) — a 2 s freeze at every UAC prompt is user-visible, ~4 `OpenInputDesktop`
/// syscalls/s are not.
const REATTACH: Duration = Duration::from_millis(250);
/// Spawn the poller for the virtual display `target_id`. `rect` = the target's desktop rect
/// (`source_desktop_rect` order: x, y, w, h) — cursor positions are desktop-global; the
/// overlay wants frame-relative, and a pointer outside the rect reports `visible: false`
/// (per-output semantics, matching the driver shm path and the Linux portal).
pub(super) fn spawn(target_id: u32, rect: (i32, i32, i32, i32)) -> Self {
let slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>> = Arc::new(Mutex::new(None));
let stop = Arc::new(AtomicBool::new(false));
let secure = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t, secure_t) = (slot.clone(), stop.clone(), secure.clone());
let thread = std::thread::Builder::new()
.name("pf-cursor-poll".into())
.spawn(move || run(target_id, rect, &slot_t, &stop_t, &secure_t))
.ok();
if thread.is_none() {
tracing::warn!("cursor poller thread spawn failed — cursor falls back to driver shm");
}
Self {
slot,
stop,
secure,
thread,
}
}
/// The latest overlay snapshot (`None` until the first successful shape rasterisation).
pub(super) fn read(&self) -> Option<pf_frame::CursorOverlay> {
self.slot.lock().unwrap_or_else(|p| p.into_inner()).clone()
}
/// Whether the input desktop is currently a SECURE desktop (UAC consent / Winlogon lock or
/// logon). Latched by the poll thread on its reattach cadence (≤ [`Self::REATTACH`] stale).
pub(super) fn secure_desktop(&self) -> bool {
self.secure.load(Ordering::Relaxed)
}
/// Whether the worker thread is (still) alive — `false` degrades the capturer to the shm read.
pub(super) fn alive(&self) -> bool {
self.thread.as_ref().is_some_and(|t| !t.is_finished())
}
}
impl Drop for CursorPoller {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(t) = self.thread.take() {
let _ = t.join(); // worker sleeps ≤ INTERVAL — a bounded join
}
}
}
/// The poll loop. Owns the thread's input-desktop binding and the shape cache.
fn run(
target_id: u32,
rect: (i32, i32, i32, i32),
slot: &Mutex<Option<pf_frame::CursorOverlay>>,
stop: &AtomicBool,
secure: &AtomicBool,
) {
// Physical-pixel coordinates on this thread regardless of the process's DPI awareness:
// `rect` comes from CCD (always physical), and a DPI-virtualized `GetCursorInfo` position
// would land in the wrong frame pixel on any scaled display. Thread-scoped, so the rest of
// the host is untouched.
// SAFETY: takes and returns only a by-value context handle; affects this thread only.
let _ = unsafe { SetThreadDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) };
let mut desktop = DesktopBinding::default();
// best-effort: already on winsta0\default if this fails
publish_secure(secure, desktop.reattach());
let mut last_attach = Instant::now();
let mut shape: Option<Shape> = None;
let mut cached_handle: isize = 0;
let mut failed_handle: isize = 0; // don't re-rasterise a failing handle every tick
let mut serial: u64 = 0;
let mut logged_live = false;
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(CursorPoller::INTERVAL);
if last_attach.elapsed() >= CursorPoller::REATTACH {
last_attach = Instant::now();
publish_secure(secure, desktop.reattach());
}
let mut ci = CURSORINFO {
cbSize: std::mem::size_of::<CURSORINFO>() as u32,
..Default::default()
};
// SAFETY: `ci` is a live, correctly-sized out-param for this synchronous call; no pointer
// escapes it.
if unsafe { GetCursorInfo(&mut ci) }.is_err() {
// Desktop went away under us (secure-desktop switch mid-call) — rebind and retry
// next tick; the slot keeps its last snapshot meanwhile.
publish_secure(secure, desktop.reattach());
last_attach = Instant::now();
continue;
}
let flags = ci.flags.0;
let showing = flags & CURSOR_SHOWING != 0 && flags & CURSOR_SUPPRESSED == 0;
// Rasterise on handle change only (position-only ticks are a header update). Hidden
// cursors keep the cached shape — the forwarder's hidden-but-known contract needs the
// bitmap to have been seen. v1: animated cursors publish their first frame (the OBS
// behavior); frame cycling via DrawIconEx istep is a known follow-up.
let handle = ci.hCursor.0 as isize;
if showing && handle != 0 && handle != cached_handle && handle != failed_handle {
match rasterize(ci.hCursor) {
Some((rgba, w, h, hot_x, hot_y)) => {
serial += 1;
shape = Some(Shape {
rgba: std::sync::Arc::new(rgba),
w,
h,
hot_x,
hot_y,
serial,
});
cached_handle = handle;
failed_handle = 0;
if !logged_live {
logged_live = true;
tracing::info!(
target_id,
"cursor poller live — GDI shape source publishing (serial 1: {w}x{h})"
);
}
}
None => {
// The owning app may have destroyed the cursor mid-read; keep the previous
// shape and don't hammer this handle again until it changes.
failed_handle = handle;
}
}
}
let overlay = shape.as_ref().map(|s| {
let (px, py) = (ci.ptScreenPos.x - rect.0, ci.ptScreenPos.y - rect.1);
let in_rect = px >= 0 && py >= 0 && px < rect.2 && py < rect.3;
pf_frame::CursorOverlay {
// Overlay x/y = bitmap top-left (reported position hotspot), frame pixels.
x: px - s.hot_x as i32,
y: py - s.hot_y as i32,
w: s.w,
h: s.h,
rgba: s.rgba.clone(),
serial: s.serial,
hot_x: s.hot_x,
hot_y: s.hot_y,
visible: showing && in_rect,
}
});
*slot.lock().unwrap_or_else(|p| p.into_inner()) = overlay;
}
}
/// Store a reattach's secure-desktop verdict (`None` = classification unavailable — keep the
/// previous state rather than flapping the capturer's hardware-cursor stand-down).
fn publish_secure(secure: &AtomicBool, verdict: Option<bool>) {
if let Some(s) = verdict {
secure.store(s, Ordering::Relaxed);
}
}
/// The thread's owned input-desktop handle — the [`SendInputInjector`] reattach model
/// (`pf-inject` sendinput.rs): keep the current binding, swap on demand, close exactly once.
#[derive(Default)]
struct DesktopBinding(Option<HDESK>);
impl DesktopBinding {
/// Rebind to the CURRENT input desktop. Returns whether that desktop is a SECURE one
/// (`UOI_NAME` != "Default": "Winlogon" during UAC consent / lock / logon) — `None` when the
/// input desktop could not be opened, in which case the binding (and the caller's secure
/// state) stays put.
fn reattach(&mut self) -> Option<bool> {
const GENERIC_ALL: u32 = 0x1000_0000;
// SAFETY: `OpenInputDesktop`/`SetThreadDesktop`/`CloseDesktop` take only by-value args.
// `OpenInputDesktop` yields an owned `HDESK` only on `Ok`; it is either installed (and the
// previously-owned handle closed exactly once) or closed on failure — no handle is leaked
// or used after close. `SetThreadDesktop` rebinds only this calling thread (which owns
// no windows/hooks, so the rebind cannot fail on that account).
unsafe {
match OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
) {
Ok(h) => {
let secure = desktop_is_secure(h);
if SetThreadDesktop(h).is_ok() {
if let Some(old) = self.0.replace(h) {
let _ = CloseDesktop(old);
}
} else {
let _ = CloseDesktop(h);
}
Some(secure)
}
Err(_) => None, // not privileged for this desktop; stay put
}
}
}
}
/// `UOI_NAME` of `h` != "Default" — i.e. the input desktop is Winlogon (UAC consent / lock /
/// logon) or a screen-saver desktop, both of which need the OS's software-cursor render path.
/// Unnameable desktops read as NOT secure: the only in-contract failure is a too-small buffer,
/// and misreading secure-as-normal merely keeps today's behavior for a beat.
fn desktop_is_secure(h: HDESK) -> bool {
let mut name = [0u16; 64]; // "Default"/"Winlogon"/"Screen-saver" all fit with room to spare
let mut needed = 0u32;
// SAFETY: `h` is the live desktop handle the caller just opened; `name`/`needed` are live
// out-params sized exactly as passed; the call writes at most `nlength` bytes.
let ok = unsafe {
GetUserObjectInformationW(
windows::Win32::Foundation::HANDLE(h.0),
UOI_NAME,
Some(name.as_mut_ptr().cast()),
(name.len() * 2) as u32,
Some(&mut needed),
)
};
if ok.is_err() {
return false;
}
let len = name.iter().position(|&c| c == 0).unwrap_or(name.len());
let name = String::from_utf16_lossy(&name[..len]);
!name.eq_ignore_ascii_case("Default")
}
impl Drop for DesktopBinding {
fn drop(&mut self) {
if let Some(h) = self.0.take() {
// SAFETY: `h` is our owned desktop handle, closed exactly once here.
let _ = unsafe { CloseDesktop(h) };
}
}
}
/// Rasterise `hcursor` to straight-alpha RGBA: `(rgba, w, h, hot_x, hot_y)`. `None` on any
/// failure (caller keeps the previous shape).
fn rasterize(hcursor: windows::Win32::UI::WindowsAndMessaging::HCURSOR) -> RasterOut {
// CopyIcon first: the owning process can destroy its HCURSOR between GetCursorInfo and the
// reads below; the copy is ours (the OBS/WebRTC guard).
// SAFETY: `HICON(hcursor.0)` reinterprets the cursor handle as an icon handle (cursors ARE
// icons in user32); CopyIcon yields an owned HICON we destroy below.
let Ok(icon) = (unsafe { CopyIcon(HICON(hcursor.0)) }) else {
return None;
};
let mut ii = ICONINFO::default();
// SAFETY: `ii` is a live out-param. On Ok it hands us COPIES of the mask/color bitmaps —
// both deleted below (GDI-handle leak otherwise).
let got = unsafe { GetIconInfo(icon, &mut ii) };
let out = if got.is_ok() { convert(&ii) } else { None };
// SAFETY: deleting the two bitmap copies GetIconInfo returned (null-safe: DeleteObject on a
// null HGDIOBJ fails harmlessly) and the icon copy — each exactly once.
unsafe {
let _ = DeleteObject(ii.hbmColor.into());
let _ = DeleteObject(ii.hbmMask.into());
let _ = DestroyIcon(icon);
}
out.map(|(rgba, w, h)| {
let hot_x = ii.xHotspot.min(w.saturating_sub(1));
let hot_y = ii.yHotspot.min(h.saturating_sub(1));
(rgba, w, h, hot_x, hot_y)
})
}
type RasterOut = Option<(Vec<u8>, u32, u32, u32, u32)>;
/// Convert the ICONINFO bitmaps to straight RGBA. Two families:
/// - color (`hbmColor` set): 32bpp BGRA; if the alpha channel is entirely empty (old-style
/// cursors) the AND mask supplies it (mask bit 1 = transparent).
/// - monochrome (`hbmColor` null): `hbmMask` is DOUBLE height — AND plane over XOR plane, the
/// WebRTC truth table: (0,0) black, (0,1) white, (1,0) transparent, (1,1) invert. Invert
/// pixels — unrepresentable in straight alpha — become opaque black with a white outline
/// grown into adjacent transparency (the WebRTC approximation; keeps the I-beam legible on
/// any background, which the old translucent-gray stand-in did not).
fn convert(ii: &ICONINFO) -> Option<(Vec<u8>, u32, u32)> {
// SAFETY: GetDC(None) yields the screen DC, released below on every path; it is only used
// as the GetDIBits reference DC.
let dc = unsafe { GetDC(None) };
let result = (|| {
if !ii.hbmColor.is_invalid() {
let color = read_bitmap_32(dc, ii.hbmColor)?;
let (w, h) = (color.w as u32, color.h as u32);
let mut rgba = bgra_to_rgba(&color.bgra);
if rgba.chunks_exact(4).all(|p| p[3] == 0) {
// Alpha-less color cursor: transparency lives in the AND mask.
let mask = read_bitmap_32(dc, ii.hbmMask)?;
if mask.w != color.w || mask.h < color.h {
return None;
}
for (px, m) in rgba.chunks_exact_mut(4).zip(mask.bgra.chunks_exact(4)) {
px[3] = if m[0] != 0 { 0 } else { 0xFF }; // mask white (AND=1) = transparent
}
}
Some((rgba, w, h))
} else {
let mask = read_bitmap_32(dc, ii.hbmMask)?;
if mask.h < 2 || mask.h % 2 != 0 {
return None;
}
let (w, h) = (mask.w as usize, (mask.h / 2) as usize);
let row = w * 4;
let (and_plane, xor_plane) = mask.bgra.split_at(h * row);
let mut rgba = vec![0u8; w * h * 4];
let mut invert = vec![false; w * h];
for i in 0..w * h {
let (a, x) = (and_plane[i * 4] != 0, xor_plane[i * 4] != 0);
let px = &mut rgba[i * 4..i * 4 + 4];
match (a, x) {
(false, false) => px.copy_from_slice(&[0, 0, 0, 0xFF]),
(false, true) => px.copy_from_slice(&[0xFF, 0xFF, 0xFF, 0xFF]),
(true, false) => {} // transparent (already zeroed)
(true, true) => {
px.copy_from_slice(&[0, 0, 0, 0xFF]);
invert[i] = true;
}
}
}
// White outline around invert regions so the (now black) shape survives dark
// backgrounds: any transparent 8-neighbor of an invert pixel turns opaque white.
for y in 0..h as i32 {
for x in 0..w as i32 {
if !invert[(y * w as i32 + x) as usize] {
continue;
}
for (dx, dy) in NEIGHBORS {
let (nx, ny) = (x + dx, y + dy);
if nx < 0 || ny < 0 || nx >= w as i32 || ny >= h as i32 {
continue;
}
let o = (ny * w as i32 + nx) as usize * 4;
if rgba[o + 3] == 0 {
rgba[o..o + 4].copy_from_slice(&[0xFF, 0xFF, 0xFF, 0xFF]);
}
}
}
}
Some((rgba, w as u32, h as u32))
}
})();
// SAFETY: releasing the screen DC obtained above, exactly once.
unsafe {
ReleaseDC(None, dc);
}
result
}
const NEIGHBORS: [(i32, i32); 8] = [
(-1, -1),
(0, -1),
(1, -1),
(-1, 0),
(1, 0),
(-1, 1),
(0, 1),
(1, 1),
];
struct RawBitmap {
w: i32,
h: i32,
/// 32bpp top-down BGRA rows, `w*h*4` (monochrome sources arrive expanded: 0x00/0xFF channels).
bgra: Vec<u8>,
}
/// Read any GDI bitmap as 32bpp top-down via `GetDIBits` (which performs the 1bpp→32bpp
/// expansion for the mask planes).
fn read_bitmap_32(dc: HDC, hbm: HBITMAP) -> Option<RawBitmap> {
let mut bm = BITMAP::default();
// SAFETY: `bm` is a live out-param sized exactly as passed; GetObjectW only writes into it.
let n = unsafe {
GetObjectW(
hbm.into(),
std::mem::size_of::<BITMAP>() as i32,
Some((&mut bm as *mut BITMAP).cast()),
)
};
if n == 0 || bm.bmWidth <= 0 || bm.bmHeight <= 0 || bm.bmWidth > 512 || bm.bmHeight > 1024 {
return None; // 512/1024: sanity caps (256² is the wire max; XL accessibility ≤ that)
}
let (w, h) = (bm.bmWidth, bm.bmHeight);
let mut info = BITMAPINFO {
bmiHeader: BITMAPINFOHEADER {
biSize: std::mem::size_of::<BITMAPINFOHEADER>() as u32,
biWidth: w,
biHeight: -h, // top-down
biPlanes: 1,
biBitCount: 32,
biCompression: BI_RGB.0,
..Default::default()
},
..Default::default()
};
let mut buf = vec![0u8; (w as usize) * (h as usize) * 4];
// SAFETY: `buf` spans exactly `h` rows of `w` 32bpp pixels as described by `info`; both are
// live locals for this synchronous call, `hbm` is a live bitmap not selected into any DC
// (fresh GetIconInfo copies).
let rows = unsafe {
GetDIBits(
dc,
hbm,
0,
h as u32,
Some(buf.as_mut_ptr().cast()),
&mut info,
DIB_RGB_COLORS,
)
};
(rows != 0).then_some(RawBitmap { w, h, bgra: buf })
}
fn bgra_to_rgba(bgra: &[u8]) -> Vec<u8> {
let mut out = bgra.to_vec();
for px in out.chunks_exact_mut(4) {
px.swap(0, 2);
}
out
}
+4 -10
View File
@@ -876,16 +876,10 @@ impl Worker {
); );
return; return;
}; };
let pref = match self.pad_info(id) { let pref = self
// Steam Input's virtual pad standing in front of the Deck's built-in controls (the .pad_info(id)
// only-pad-forwarded case, [`Self::forwarded_ids`]): declare the DECK kind, not the .map(|p| p.pref)
// wrapper's Xbox 360 identity. [`Self::auto_pref`] already resolves the SESSION .unwrap_or(GamepadPref::Xbox360);
// default this way, but a current host honors the per-pad arrival over the session
// default — so without this the host builds an X-Box 360 pad on a real Deck.
Some(p) if p.steam_virtual && is_steam_deck() => GamepadPref::SteamDeck,
Some(p) => p.pref,
None => GamepadPref::Xbox360,
};
match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) { match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) {
Ok(pad) => { Ok(pad) => {
let mut slot = Slot::new(id, index, pref, pad); let mut slot = Slot::new(id, index, pref, pad);
+2 -22
View File
@@ -44,13 +44,6 @@ pub struct SessionParams {
/// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The /// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The
/// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`. /// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`.
pub clipboard: bool, pub clipboard: bool,
/// Advertise `quic::CLIENT_CAP_CURSOR`: this embedder renders the host cursor locally
/// (the presenter's cursor channel, design/remote-desktop-sweep.md M2), so the host may
/// stop compositing the pointer into the video. Only set when the embedder actually
/// draws it (the SDL presenter in desktop mouse mode) — a session that advertises it
/// without rendering streams with NO visible cursor. The host answers `HOST_CAP_CURSOR`
/// when its capture can forward (Linux portal, not gamescope/Windows).
pub cursor_forward: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see /// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
/// `video::Decoder::new`). /// `video::Decoder::new`).
pub decoder: String, pub decoder: String,
@@ -262,11 +255,6 @@ fn pump(
// This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an // This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an
// A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600). // A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600).
punktfunk_core::client::display_hdr_env_override().or(params.display_hdr), punktfunk_core::client::display_hdr_env_override().or(params.display_hdr),
if params.cursor_forward {
punktfunk_core::quic::CLIENT_CAP_CURSOR
} else {
0
},
params.launch.clone(), params.launch.clone(),
params.pin, params.pin,
Some(params.identity), Some(params.identity),
@@ -459,16 +447,8 @@ fn pump(
// every ~816 ms at 60120 Hz anyway, so this rarely times out mid-stream). // every ~816 ms at 60120 Hz anyway, so this rarely times out mid-stream).
match connector.next_frame(Duration::from_millis(20)) { match connector.next_frame(Duration::from_millis(20)) {
Ok(frame) => { Ok(frame) => {
// The `received` point: reassembly COMPLETION, stamped by the core session as // The `received` point: AU fully reassembled, in hand, before decode.
// the AU crossed poll_frame (ABI v9). Stamping here at the hand-off pull instead let received_ns = now_ns();
// would fold the pre-decode queue wait into `host+network` — a client-side
// standing backlog masquerading as network latency (the 2026-07 two-pair
// investigation). 0 = a core predating the stamp; fall back to the pull instant.
let received_ns = if frame.received_ns > 0 {
frame.received_ns
} else {
now_ns()
};
// fps / goodput count every received AU (spec), decoded or not. // fps / goodput count every received AU (spec), decoded or not.
frames_n += 1; frames_n += 1;
bytes_n += frame.data.len() as u64; bytes_n += frame.data.len() as u64;
-96
View File
@@ -212,40 +212,6 @@ pub fn persist_host(name: &str, addr: &str, port: u16, fp_hex: &str, paired: boo
let _ = known.save(); let _ = known.save();
} }
/// This machine's name — the label a host files this client under in its paired-devices list.
/// `/etc/hostname` first (the answer on any Linux box, and the only one available in a minimal
/// build with no GTK to ask), then the usual environment fallbacks.
pub fn device_name() -> String {
#[cfg(target_os = "linux")]
if let Ok(s) = std::fs::read_to_string("/etc/hostname") {
let s = s.trim();
if !s.is_empty() {
return s.to_string();
}
}
std::env::var("COMPUTERNAME")
.or_else(|_| std::env::var("HOSTNAME"))
.ok()
.filter(|s| !s.trim().is_empty())
.unwrap_or_else(|| "This device".into())
}
/// Drop an fp-less placeholder entry for `addr:port`. A host added by address before any
/// ceremony (`--add-host` with no `--fp`) is stored keyed by address with an empty fingerprint;
/// once pairing yields the real one, [`persist_host`] writes a second, fp-keyed entry — so the
/// placeholder has to go or the host list shows the same box twice. No-op (and no disk write)
/// when there is none, which is the usual case.
pub fn forget_placeholder(addr: &str, port: u16) {
let mut known = KnownHosts::load();
let before = known.hosts.len();
known
.hosts
.retain(|h| !(h.fp_hex.is_empty() && h.addr == addr && h.port == port));
if known.hosts.len() != before {
let _ = known.save();
}
}
/// Learn/refresh a saved host's Wake-on-LAN MAC(s) from its live advert (called while the host /// Learn/refresh a saved host's Wake-on-LAN MAC(s) from its live advert (called while the host
/// is online, matched by fingerprint or address). No-op — and no disk write — when unchanged, so /// is online, matched by fingerprint or address). No-op — and no disk write — when unchanged, so
/// the hosts page can call it on every discovery tick without churning the store. /// the hosts page can call it on every discovery tick without churning the store.
@@ -359,11 +325,6 @@ pub fn connect_reject_message(reason: punktfunk_core::reject::RejectReason) -> S
"Client and host versions don't match — update both to the same release.".into() "Client and host versions don't match — update both to the same release.".into()
} }
R::Busy => "The host is busy with another session.".into(), R::Busy => "The host is busy with another session.".into(),
R::SetupFailed => {
"The host accepted the connection but couldn't start the stream — the host's log \
(web console Log) has the cause."
.into()
}
} }
} }
@@ -495,48 +456,6 @@ impl TouchMode {
} }
} }
/// How a physical mouse drives the host — the desktop-sweep mouse model
/// (design/remote-desktop-sweep.md M1). Stored stringly in [`Settings::mouse_mode`] so the
/// file stays readable; parsed with [`MouseMode::from_name`].
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum MouseMode {
/// Pointer lock (relative deltas, hidden cursor) — the game model, and the default:
/// the only cursor you see is the host's.
Capture,
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion goes on the wire as absolute positions through the letterbox. The remote
/// desktop model. Requires a host injector with absolute support (not gamescope).
Desktop,
}
impl MouseMode {
/// Cycle/picker order (also the settings pickers' option order).
pub const ALL: [MouseMode; 2] = [MouseMode::Capture, MouseMode::Desktop];
/// Parse the persisted name, defaulting to `Capture` for unset/unknown values.
pub fn from_name(s: &str) -> MouseMode {
match s {
"desktop" => MouseMode::Desktop,
_ => MouseMode::Capture,
}
}
/// The persisted name (the inverse of [`from_name`](Self::from_name)).
pub fn as_name(self) -> &'static str {
match self {
MouseMode::Capture => "capture",
MouseMode::Desktop => "desktop",
}
}
pub fn label(self) -> &'static str {
match self {
MouseMode::Capture => "Capture (games)",
MouseMode::Desktop => "Desktop (absolute)",
}
}
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file /// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time. /// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)] #[derive(Clone, Serialize, Deserialize)]
@@ -571,12 +490,6 @@ pub struct Settings {
/// stores load as trackpad. /// stores load as trackpad.
#[serde(default = "default_touch_mode")] #[serde(default = "default_touch_mode")]
pub touch_mode: String, pub touch_mode: String,
/// How a physical mouse drives the host: a [`MouseMode`] name — `"capture"` (default,
/// pointer lock + relative) or `"desktop"` (uncaptured absolute pointer). Read at
/// connect via [`Settings::mouse_mode`]. `default` so pre-existing stores load as
/// capture — today's behavior.
#[serde(default = "default_mouse_mode")]
pub mouse_mode: String,
/// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured. /// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured.
pub inhibit_shortcuts: bool, pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source. /// Stream the default microphone to the host's virtual mic source.
@@ -664,10 +577,6 @@ fn default_touch_mode() -> String {
"trackpad".into() "trackpad".into()
} }
fn default_mouse_mode() -> String {
"capture".into()
}
fn default_true() -> bool { fn default_true() -> bool {
true true
} }
@@ -695,10 +604,6 @@ impl Settings {
TouchMode::from_name(&self.touch_mode) TouchMode::from_name(&self.touch_mode)
} }
pub fn mouse_mode(&self) -> MouseMode {
MouseMode::from_name(&self.mouse_mode)
}
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto). /// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 { pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() { match self.codec.as_str() {
@@ -726,7 +631,6 @@ impl Default for Settings {
forward_pad: String::new(), forward_pad: String::new(),
compositor: "auto".into(), compositor: "auto".into(),
touch_mode: "trackpad".into(), touch_mode: "trackpad".into(),
mouse_mode: "capture".into(),
inhibit_shortcuts: true, inhibit_shortcuts: true,
mic_enabled: false, mic_enabled: false,
audio_channels: 2, audio_channels: 2,
+14 -37
View File
@@ -6,12 +6,11 @@
//! queue under the device's [`QueueLock`], fence-waited (sub-ms — Phase-0 measured //! queue under the device's [`QueueLock`], fence-waited (sub-ms — Phase-0 measured
//! 0.067 ms GPU at 1080p on the RTX 5070 Ti). //! 0.067 ms GPU at 1080p on the RTX 5070 Ti).
//! //!
//! Output: three separate single-component planes (Y full-res; Cb/Cr half-res, or //! Output: three separate R8 planes (Y full-res, Cb/Cr half-res) — the decode path
//! full-res on a 4:4:4 session) — R8, or R16 UNORM on a 10-bit session — the decode //! requires STORAGE usage and IDENTITY/R swizzles, so the encoder's two-component
//! path requires STORAGE usage and IDENTITY/R swizzles, so the encoder's two-component //! RG8 trick is not allowed here (pyrowave.h validation). The presenter samples them
//! RG trick is not allowed here (pyrowave.h validation). The presenter samples them //! with its planar CSC variant (BT.709 limited — the codec's fixed colour contract,
//! with its planar CSC variant (colour per the negotiated `ColorInfo` — the wavelet //! there is no VUI). A small ring of plane-sets keeps a decode from overwriting the set
//! bitstream carries no VUI). A small ring of plane-sets keeps a decode from overwriting the set
//! the presenter is still sampling; the synchronous fence bounds decode-side reuse and //! the presenter is still sampling; the synchronous fence bounds decode-side reuse and
//! the ring depth covers present-side latency (≤ 12 frames in this pipeline). //! the ring depth covers present-side latency (≤ 12 frames in this pipeline).
//! //!
@@ -223,10 +222,9 @@ unsafe extern "C" fn queue_unlock_cb(ud: *mut c_void) {
unsafe { (*(ud as *const crate::video::QueueLock)).unlock() } unsafe { (*(ud as *const crate::video::QueueLock)).unlock() }
} }
/// One decoded PyroWave frame: three single-component plane images (R8, or R16 on a /// One decoded PyroWave frame: three R8 plane images on the presenter's device, GENERAL
/// 10-bit session) on the presenter's device, GENERAL layout, decode-complete (the /// layout, decode-complete (the decoder fence-waits before handing it over). `slot`
/// decoder fence-waits before handing it over). `slot` identifies the ring entry; the /// identifies the ring entry; the images/views live as long as the decoder.
/// images/views live as long as the decoder.
pub struct PyroWavePlanarFrame { pub struct PyroWavePlanarFrame {
/// Raw `VkImageView`s (Y, Cb, Cr) for the presenter's planar CSC sampling. /// Raw `VkImageView`s (Y, Cb, Cr) for the presenter's planar CSC sampling.
pub views: [u64; 3], pub views: [u64; 3],
@@ -254,8 +252,7 @@ struct RetiredRing {
retired_at: Instant, retired_at: Instant,
} }
/// One decode-output plane (`fmt` = R8, or R16 on a 10-bit session): storage (decode /// One decode-output plane: R8, storage (decode writes) + sampled (presenter CSC).
/// writes) + sampled (presenter CSC).
unsafe fn make_plane( unsafe fn make_plane(
device: &ash::Device, device: &ash::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties, mem_props: &vk::PhysicalDeviceMemoryProperties,
@@ -542,8 +539,7 @@ impl PyroWaveDecoder {
return Err(e); return Err(e);
} }
// Plane-set ring: 3 single-component planes (R8; R16 on a 10-bit session), // Plane-set ring: 3 × R8, storage (decode writes) + sampled (presenter CSC).
// storage (decode writes) + sampled (presenter CSC).
let mem_props = instance.get_physical_device_memory_properties( let mem_props = instance.get_physical_device_memory_properties(
vk::PhysicalDevice::from_raw(vkd.physical_device as u64), vk::PhysicalDevice::from_raw(vkd.physical_device as u64),
); );
@@ -877,26 +873,12 @@ impl PyroWaveDecoder {
&vk::DependencyInfo::default().image_memory_barriers(&pre), &vk::DependencyInfo::default().image_memory_barriers(&pre),
); );
// The declared format/extent MUST equal the ring image's real ones: pyrowave wraps
// our VkImage under `image_format` and vkCreateImageView's its storage view with
// `view_format` (pyrowave_c.cpp `WrappedViewBuffers::wrap` — its own
// `pyrowave_image_get_image_view` helper fills both from the image itself).
// Declaring R8 over a 10-bit session's R16_UNORM planes is an invalid view the
// driver executes anyway: its addressing covers half the surface, so decoded 8-bit
// codes fuse pairwise into 16-bit texels (structured garbage) and the never-written
// remainder samples as all-plane zeros (saturated green) — the 2026-07 AMD-client
// field report. Same discipline for chroma extents: a 4:4:4 ring is full-res.
let fmt = if self.hdr16 {
pw::VkFormat_VK_FORMAT_R16_UNORM
} else {
pw::VkFormat_VK_FORMAT_R8_UNORM
};
let plane = |img: vk::Image, w: u32, h: u32| pw::pyrowave_image_view { let plane = |img: vk::Image, w: u32, h: u32| pw::pyrowave_image_view {
image: img.as_raw() as usize as pw::VkImage, image: img.as_raw() as usize as pw::VkImage,
width: w, width: w,
height: h, height: h,
image_format: fmt, image_format: pw::VkFormat_VK_FORMAT_R8_UNORM,
view_format: fmt, view_format: pw::VkFormat_VK_FORMAT_R8_UNORM,
mip_level: 0, mip_level: 0,
layer: 0, layer: 0,
aspect: pw::VkImageAspectFlagBits_VK_IMAGE_ASPECT_COLOR_BIT, aspect: pw::VkImageAspectFlagBits_VK_IMAGE_ASPECT_COLOR_BIT,
@@ -904,16 +886,11 @@ impl PyroWaveDecoder {
layout: pw::VkImageLayout_VK_IMAGE_LAYOUT_GENERAL, layout: pw::VkImageLayout_VK_IMAGE_LAYOUT_GENERAL,
}; };
let (w, h) = (self.width, self.height); let (w, h) = (self.width, self.height);
let (cw, ch) = if self.chroma444 {
(w, h)
} else {
(w / 2, h / 2)
};
let buffers = pw::pyrowave_gpu_buffers { let buffers = pw::pyrowave_gpu_buffers {
planes: [ planes: [
plane(self.ring[slot].imgs[0], w, h), plane(self.ring[slot].imgs[0], w, h),
plane(self.ring[slot].imgs[1], cw, ch), plane(self.ring[slot].imgs[1], w / 2, h / 2),
plane(self.ring[slot].imgs[2], cw, ch), plane(self.ring[slot].imgs[2], w / 2, h / 2),
], ],
}; };
pw::pyrowave_device_set_command_buffer( pw::pyrowave_device_set_command_buffer(
+2 -42
View File
@@ -10,7 +10,7 @@ use crate::screens::{Ctx, Outbox};
use crate::theme::{Fonts, DIM, W}; use crate::theme::{Fonts, DIM, W};
use crate::widgets::{ListMsg, MenuList, RowSpec}; use crate::widgets::{ListMsg, MenuList, RowSpec};
use pf_client_core::gamepad::{MenuEvent, MenuPulse}; use pf_client_core::gamepad::{MenuEvent, MenuPulse};
use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode}; use pf_client_core::trust::{StatsVerbosity, TouchMode};
use skia_safe::{Canvas, Rect}; use skia_safe::{Canvas, Rect};
/// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale /// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale
@@ -29,11 +29,10 @@ enum RowId {
Pad, Pad,
PadType, PadType,
Touch, Touch,
Mouse,
Stats, Stats,
} }
const ROWS: [RowId; 14] = [ const ROWS: [RowId; 13] = [
RowId::Resolution, RowId::Resolution,
RowId::Refresh, RowId::Refresh,
RowId::Bitrate, RowId::Bitrate,
@@ -46,7 +45,6 @@ const ROWS: [RowId; 14] = [
RowId::Pad, RowId::Pad,
RowId::PadType, RowId::PadType,
RowId::Touch, RowId::Touch,
RowId::Mouse,
RowId::Stats, RowId::Stats,
]; ];
@@ -253,7 +251,6 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
"Touch mode", "Touch mode",
s.touch_mode().label().into(), s.touch_mode().label().into(),
), ),
RowId::Mouse => (None, "Mouse mode", s.mouse_mode().label().into()),
RowId::Stats => ( RowId::Stats => (
Some("Interface"), Some("Interface"),
"Statistics overlay", "Statistics overlay",
@@ -295,11 +292,6 @@ fn detail(id: RowId) -> &'static str {
"How the touchscreen drives the host: Trackpad (relative cursor), \ "How the touchscreen drives the host: Trackpad (relative cursor), \
Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)." Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)."
} }
RowId::Mouse => {
"How a physical mouse drives the host: Capture locks the pointer (relative, \
for games), Desktop leaves it free and sends absolute positions. \
Ctrl+Alt+Shift+M switches live while streaming."
}
RowId::Stats => { RowId::Stats => {
"How much the overlay shows: Compact (one line) → Normal → Detailed. \ "How much the overlay shows: Compact (one line) → Normal → Detailed. \
Ctrl+Alt+Shift+S cycles it live while streaming." Ctrl+Alt+Shift+S cycles it live while streaming."
@@ -375,11 +367,6 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, TouchMode::ALL.len(), delta, wrap) step_option(cur, TouchMode::ALL.len(), delta, wrap)
.map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string()) .map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string())
} }
RowId::Mouse => {
let cur = MouseMode::ALL.iter().position(|m| *m == s.mouse_mode());
step_option(cur, MouseMode::ALL.len(), delta, wrap)
.map(|i| s.mouse_mode = MouseMode::ALL[i].as_name().to_string())
}
RowId::Stats => { RowId::Stats => {
let cur = StatsVerbosity::ALL let cur = StatsVerbosity::ALL
.iter() .iter()
@@ -523,33 +510,6 @@ mod tests {
assert_eq!(ctx.settings.touch_mode, "trackpad"); assert_eq!(ctx.settings.touch_mode, "trackpad");
} }
#[test]
fn mouse_mode_steps_and_wraps() {
let (mut settings, pads) = ctx_parts();
assert_eq!(settings.mouse_mode, "capture");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
// Capture → Desktop, then a step past the end is a boundary.
assert!(
!adjust(RowId::Mouse, -1, false, &mut ctx),
"already first = thud"
);
assert!(adjust(RowId::Mouse, 1, false, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "desktop");
assert!(!adjust(RowId::Mouse, 1, false, &mut ctx), "last = thud");
// A wraps back to the first.
assert!(adjust(RowId::Mouse, 1, true, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "capture");
}
#[test] #[test]
fn unknown_value_snaps_to_first() { fn unknown_value_snaps_to_first() {
let (mut settings, pads) = ctx_parts(); let (mut settings, pads) = ctx_parts();
+9 -213
View File
@@ -66,31 +66,7 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// too ([`MIN_DRIVER_PROTOCOL_VERSION`]) and simply falls back to the re-arrival resize against it; /// too ([`MIN_DRIVER_PROTOCOL_VERSION`]) and simply falls back to the re-arrival resize against it;
/// a v4 driver serving an older (v3-asserting) host fails that host's strict handshake — ship /// a v4 driver serving an older (v3-asserting) host fails that host's strict handshake — ship
/// driver+host together, as ever. /// driver+host together, as ever.
/// v5: ADDITIVE — the IddCx HARDWARE CURSOR channel (remote-desktop-sweep M2c): pub const PROTOCOL_VERSION: u32 = 4;
/// [`control::AddRequest::hw_cursor`] (the former tail `_reserved` — same size, same offsets)
/// asks the driver to declare a hardware cursor for this monitor (DWM then EXCLUDES the pointer
/// from the desktop frame and delivers shape/position out-of-band), and
/// [`control::IOCTL_SET_CURSOR_CHANNEL`] delivers the host-created [`cursor::CursorShm`] section
/// the driver's cursor thread seqlock-publishes into. Nothing existing changed; the host gates
/// the feature on the handshake-reported version (`>= 5`) and keeps composited-cursor behavior
/// against older drivers.
/// v6: ADDITIVE — [`control::IOCTL_SET_CURSOR_FORWARD`] (the mid-stream cursor-render flip,
/// remote-desktop-sweep §8): the client's mouse-model flip (un)declares the hardware cursor on a
/// LIVE monitor, so the capture mouse model gets DWM's composited pointer back (full fidelity)
/// and the desktop model gets exclusion + forwarding. Nothing existing changed; against a v5
/// driver the unknown IOCTL fails and the host logs + keeps the declared-at-ADD behavior.
/// v6 tail ext (no bump, the `AddRequest` luminance-tail discipline):
/// [`control::AddReply::cursor_excluded`] — the driver reports whether its ADAPTER already
/// carries a hardware-cursor declare from an earlier session. A declare is IRREVOCABLE
/// (remote-desktop-sweep §8.6, proven on-glass) and its exclusion reaches EVERY later monitor of
/// the adapter, not just the declaring target (on-glass 2026-07-23: a declare on one target left
/// a different client's fresh target cursor-less): DWM never composites the software cursor back
/// into any of the adapter's frames until the adapter resets. The host uses the flag to
/// self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// cursor channel — without it those sessions are silently cursor-less. Both skews degrade
/// cleanly: an old driver writes only the 20-byte reply prefix (host reads `0` = unknown/clean),
/// an old host retrieves a 20-byte buffer (driver writes just the prefix).
pub const PROTOCOL_VERSION: u32 = 6;
/// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on /// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on
/// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the /// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the
@@ -134,22 +110,6 @@ pub mod control {
/// identity (saved per-monitor DPI) and the driver's swap-chain/stash machinery survive. A v3 /// identity (saved per-monitor DPI) and the driver's swap-chain/stash machinery survive. A v3
/// driver fails this unknown IOCTL → the host falls back to the re-arrival resize. /// driver fails this unknown IOCTL → the host falls back to the re-arrival resize.
pub const IOCTL_UPDATE_MODES: u32 = ctl_code(0x907); pub const IOCTL_UPDATE_MODES: u32 = ctl_code(0x907);
/// Deliver a monitor's hardware-cursor channel (v5): the handle VALUE of the unnamed
/// [`cursor::CursorShm`](crate::cursor) file mapping the host duplicated into the WUDFHost
/// (same delivery model as [`IOCTL_SET_FRAME_CHANNEL`], no event — the host polls the
/// seqlock at its encode-tick pace). Sent once after ADD, only for a monitor whose
/// [`AddRequest::hw_cursor`] was set. The driver maps it, calls
/// `IddCxMonitorSetupHardwareCursor`, and starts its cursor-query thread. Input
/// [`SetCursorChannelRequest`].
pub const IOCTL_SET_CURSOR_CHANNEL: u32 = ctl_code(0x908);
/// Flip a LIVE monitor's hardware-cursor declaration (v6, the mid-stream cursor-render
/// flip): `enable = 1` re-declares (`IddCxMonitorSetupHardwareCursor` — DWM excludes the
/// pointer, the query/shape machinery resumes), `enable = 0` un-declares (the driver stops
/// re-declaring on mode commits and asks the OS to revert to the software cursor — DWM
/// composites the pointer into the frame, the pre-channel behavior the capture mouse
/// model wants). Only meaningful for a monitor whose cursor channel was delivered. Input
/// [`SetCursorForwardRequest`].
pub const IOCTL_SET_CURSOR_FORWARD: u32 = ctl_code(0x909);
/// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns /// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns
/// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this /// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this
@@ -187,13 +147,9 @@ pub mod control {
/// The client display's min luminance in MILLI-nits (0.001 cd/m² — the CTA min-luminance /// The client display's min luminance in MILLI-nits (0.001 cd/m² — the CTA min-luminance
/// range lives well below 1 nit) → Desired Content Min Luminance. `0` = unknown. /// range lives well below 1 nit) → Desired Content Min Luminance. `0` = unknown.
pub min_luminance_millinits: u32, pub min_luminance_millinits: u32,
/// Non-zero = declare an IddCx HARDWARE CURSOR for this monitor (v5, remote-desktop-sweep /// Pads the `u64`-aligned struct to a multiple of 8 (Pod forbids implicit tail padding);
/// M2c): DWM stops compositing the pointer into the frame and the driver publishes /// free expansion room for the next appended field.
/// shape/position into the [`cursor::CursorShm`](crate::cursor) section delivered by pub _reserved: u32,
/// [`IOCTL_SET_CURSOR_CHANNEL`]. Byte-compatible with the old tail `_reserved` (offset 36):
/// an un-upgraded driver ignores it (cursor stays composited — the host already gates on
/// the handshake version, this is defense in depth), an un-upgraded host sends `0` (off).
pub hw_cursor: u32,
} }
/// [`AddRequest`]'s size before the client-HDR luminance tail — the prefix an un-upgraded /// [`AddRequest`]'s size before the client-HDR luminance tail — the prefix an un-upgraded
@@ -218,22 +174,8 @@ pub mod control {
/// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a /// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a
/// WUDFHost restart between sessions can never leave the host duplicating into a dead process. /// WUDFHost restart between sessions can never leave the host duplicating into a dead process.
pub wudf_pid: u32, pub wudf_pid: u32,
/// Non-zero = the ADAPTER already carries an IRREVOCABLE hardware-cursor declare from an
/// earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23): DWM excludes the pointer from every frame on every monitor until
/// the adapter resets, and a session without the cursor channel must
/// self-composite (GDI poller + blend) or stream a cursor-less desktop. Appended after
/// [`ADD_REPLY_LEGACY_SIZE`] under the same dual-size discipline as the `AddRequest`
/// luminance tail: an un-upgraded driver writes only the legacy prefix (the host's
/// zero-initialized buffer then reads `0` = unknown/clean), an un-upgraded host retrieves a
/// legacy-size buffer (the driver writes just the prefix).
pub cursor_excluded: u32,
} }
/// [`AddReply`]'s size before the `cursor_excluded` tail — the prefix an un-upgraded driver
/// writes and an un-upgraded host retrieves (see the field docs).
pub const ADD_REPLY_LEGACY_SIZE: usize = 20;
/// `IOCTL_REMOVE` input. /// `IOCTL_REMOVE` input.
#[repr(C)] #[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
@@ -311,29 +253,6 @@ pub mod control {
/// at the compile-time maximum; `ring_len` says how many entries are live). /// at the compile-time maximum; `ring_len` says how many entries are live).
pub const RING_LEN_USIZE: usize = RING_LEN as usize; pub const RING_LEN_USIZE: usize = RING_LEN as usize;
/// `IOCTL_SET_CURSOR_CHANNEL` input (v5): the hardware-cursor section for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorChannelRequest {
/// The OS target id from [`AddReply`] — which monitor this channel belongs to.
pub target_id: u32,
pub _pad: u32,
/// The [`cursor::CursorShm`](crate::cursor) file-mapping handle VALUE, already duplicated
/// into the driver's WUDFHost process ([`AddReply::wudf_pid`]).
pub header_handle: u64,
}
/// `IOCTL_SET_CURSOR_FORWARD` input (v6): the mid-stream cursor-render flip for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorForwardRequest {
/// The OS target id from [`AddReply`] — which monitor to flip.
pub target_id: u32,
/// `1` = declare the hardware cursor (exclude + forward), `0` = un-declare (DWM
/// composites — the capture mouse model).
pub enable: u32,
}
// Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already // Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already
// rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!` // rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!`
// asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss. // asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss.
@@ -350,18 +269,13 @@ pub mod control {
assert!(offset_of!(AddRequest, max_luminance_nits) == ADD_REQUEST_LEGACY_SIZE); assert!(offset_of!(AddRequest, max_luminance_nits) == ADD_REQUEST_LEGACY_SIZE);
assert!(offset_of!(AddRequest, max_frame_avg_nits) == 28); assert!(offset_of!(AddRequest, max_frame_avg_nits) == 28);
assert!(offset_of!(AddRequest, min_luminance_millinits) == 32); assert!(offset_of!(AddRequest, min_luminance_millinits) == 32);
// v5: the former tail `_reserved` — same offset, same total size (rename-only).
assert!(offset_of!(AddRequest, hw_cursor) == 36);
assert!(size_of::<AddRequest>() == 40);
assert!(size_of::<AddReply>() == 24); assert!(size_of::<AddReply>() == 20);
assert!(offset_of!(AddReply, adapter_luid_low) == 0); assert!(offset_of!(AddReply, adapter_luid_low) == 0);
assert!(offset_of!(AddReply, adapter_luid_high) == 4); assert!(offset_of!(AddReply, adapter_luid_high) == 4);
assert!(offset_of!(AddReply, target_id) == 8); assert!(offset_of!(AddReply, target_id) == 8);
assert!(offset_of!(AddReply, resolved_monitor_id) == 12); assert!(offset_of!(AddReply, resolved_monitor_id) == 12);
assert!(offset_of!(AddReply, wudf_pid) == 16); assert!(offset_of!(AddReply, wudf_pid) == 16);
// The cursor-excluded tail starts exactly at the legacy boundary (prefix-compat).
assert!(offset_of!(AddReply, cursor_excluded) == ADD_REPLY_LEGACY_SIZE);
assert!(size_of::<SetFrameChannelRequest>() == 32 + 8 * RING_LEN_USIZE); assert!(size_of::<SetFrameChannelRequest>() == 32 + 8 * RING_LEN_USIZE);
assert!(offset_of!(SetFrameChannelRequest, target_id) == 0); assert!(offset_of!(SetFrameChannelRequest, target_id) == 0);
@@ -374,13 +288,6 @@ pub mod control {
assert!(size_of::<RemoveRequest>() == 8); assert!(size_of::<RemoveRequest>() == 8);
assert!(offset_of!(RemoveRequest, session_id) == 0); assert!(offset_of!(RemoveRequest, session_id) == 0);
assert!(size_of::<SetCursorChannelRequest>() == 16);
assert!(offset_of!(SetCursorChannelRequest, target_id) == 0);
assert!(offset_of!(SetCursorChannelRequest, header_handle) == 8);
assert!(size_of::<SetCursorForwardRequest>() == 8);
assert!(offset_of!(SetCursorForwardRequest, target_id) == 0);
assert!(offset_of!(SetCursorForwardRequest, enable) == 4);
assert!(size_of::<UpdateModesRequest>() == 24); assert!(size_of::<UpdateModesRequest>() == 24);
assert!(offset_of!(UpdateModesRequest, session_id) == 0); assert!(offset_of!(UpdateModesRequest, session_id) == 0);
assert!(offset_of!(UpdateModesRequest, width) == 8); assert!(offset_of!(UpdateModesRequest, width) == 8);
@@ -1053,84 +960,6 @@ pub mod mouse {
}; };
} }
/// The v5 hardware-cursor channel (remote-desktop-sweep M2c): one unnamed file mapping per
/// monitor, host-created, delivered by handle value ([`control::IOCTL_SET_CURSOR_CHANNEL`]).
/// The DRIVER's cursor thread (woken by its IddCx `hNewCursorDataAvailable` event) seqlock-writes
/// shape + position + visibility; the HOST reads at its encode-tick pace — no event crosses the
/// boundary. Writer: bump [`CursorShm::seq`] to ODD, write fields (+ shape bytes when the OS said
/// the shape changed), bump to EVEN. Reader: read seq (retry while odd), copy, re-read seq —
/// unchanged ⇒ consistent snapshot. Position-only updates never touch the shape bytes, so a
/// reader that skips unchanged `shape_id`s never copies torn pixels.
pub mod cursor {
use bytemuck::{Pod, Zeroable};
/// First field of [`CursorShm`] — `b"PFCU"` little-endian; anything else = not attached yet.
pub const CURSOR_MAGIC: u32 = u32::from_le_bytes(*b"PFCU");
/// Max cursor side (px) the driver declares to the OS (`IDDCX_CURSOR_CAPS::MaxX/MaxY`) and
/// the section's shape buffer is sized for. Windows XL accessibility cursors top out here;
/// the host's wire forwarder downscales to its own cap anyway.
pub const CURSOR_SHAPE_MAX: u32 = 256;
/// Shape-buffer bytes: 32-bpp at the declared max.
pub const CURSOR_SHAPE_BYTES: usize = (CURSOR_SHAPE_MAX * CURSOR_SHAPE_MAX * 4) as usize;
/// Byte offset of the shape pixels inside the section (one cache-line-ish header).
pub const CURSOR_SHAPE_OFFSET: usize = 64;
/// Total section size.
pub const CURSOR_SHM_SIZE: usize = CURSOR_SHAPE_OFFSET + CURSOR_SHAPE_BYTES;
/// `IDDCX_CURSOR_SHAPE_TYPE` values mirrored for the host (the driver writes the OS value
/// verbatim into [`CursorShm::cursor_type`]).
pub const CURSOR_TYPE_MASKED_COLOR: u32 = 1;
pub const CURSOR_TYPE_ALPHA: u32 = 2;
/// The section header (the shape pixels follow at [`CURSOR_SHAPE_OFFSET`]). `x`/`y` are the
/// shape's TOP-LEFT in desktop coordinates (the IddCx `IDARG_OUT_QUERY_HWCURSOR::X/Y`
/// convention — position hotspot, can be negative); `shape_id` is the OS's per-set counter
/// (bumps on every shape set, the overlay serial); pixels are the OS's 32-bpp rows at
/// `pitch` bytes (BGRA for ALPHA; color+mask for MASKED_COLOR — the host converts).
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct CursorShm {
pub magic: u32,
/// Seqlock: odd = writer mid-update.
pub seq: u32,
pub visible: u32,
pub cursor_type: u32,
pub x: i32,
pub y: i32,
pub shape_id: u32,
pub width: u32,
pub height: u32,
pub pitch: u32,
pub hot_x: u32,
pub hot_y: u32,
/// Reserved expansion room up to [`CURSOR_SHAPE_OFFSET`].
pub _reserved: [u32; 4],
}
// Layout is load-bearing across the process boundary — pin it.
const _: () = {
use core::mem::{offset_of, size_of};
assert!(size_of::<CursorShm>() == 64);
assert!(size_of::<CursorShm>() <= CURSOR_SHAPE_OFFSET);
assert!(offset_of!(CursorShm, magic) == 0);
assert!(offset_of!(CursorShm, seq) == 4);
assert!(offset_of!(CursorShm, visible) == 8);
assert!(offset_of!(CursorShm, cursor_type) == 12);
assert!(offset_of!(CursorShm, x) == 16);
assert!(offset_of!(CursorShm, y) == 20);
assert!(offset_of!(CursorShm, shape_id) == 24);
assert!(offset_of!(CursorShm, width) == 28);
assert!(offset_of!(CursorShm, height) == 32);
assert!(offset_of!(CursorShm, pitch) == 36);
assert!(offset_of!(CursorShm, hot_x) == 40);
assert!(offset_of!(CursorShm, hot_y) == 44);
};
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -1250,7 +1079,7 @@ mod tests {
max_luminance_nits: 800, max_luminance_nits: 800,
max_frame_avg_nits: 400, max_frame_avg_nits: 400,
min_luminance_millinits: 50, // 0.05 nits min_luminance_millinits: 50, // 0.05 nits
hw_cursor: 1, _reserved: 0,
}; };
let bytes = bytemuck::bytes_of(&req); let bytes = bytemuck::bytes_of(&req);
assert_eq!(bytes.len(), 40); assert_eq!(bytes.len(), 40);
@@ -1280,19 +1109,14 @@ mod tests {
target_id: 262, target_id: 262,
resolved_monitor_id: 7, resolved_monitor_id: 7,
wudf_pid: 4242, wudf_pid: 4242,
cursor_excluded: 1,
}; };
let rbytes = bytemuck::bytes_of(&reply); let rbytes = bytemuck::bytes_of(&reply);
assert_eq!(rbytes.len(), 24); assert_eq!(rbytes.len(), 20);
assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply); assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply);
// resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible. // resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible.
assert_eq!(rbytes[12..16], 7u32.to_le_bytes()); assert_eq!(rbytes[12..16], 7u32.to_le_bytes());
// The v2 duplication-target pid trails at offset 16. // The v2 duplication-target pid trails at offset 16.
assert_eq!(rbytes[16..20], 4242u32.to_le_bytes()); assert_eq!(rbytes[16..20], 4242u32.to_le_bytes());
// The cursor-excluded tail rides after the legacy boundary; an un-upgraded driver writes
// only the prefix, so a zero-filled tail reads as "unknown/clean" (see the field docs).
assert_eq!(rbytes[20..24], 1u32.to_le_bytes());
assert_eq!(control::ADD_REPLY_LEGACY_SIZE, 20);
} }
#[test] #[test]
@@ -1337,39 +1161,11 @@ mod tests {
req req
); );
assert_eq!(bytes[8..12], 2560u32.to_le_bytes()); assert_eq!(bytes[8..12], 2560u32.to_le_bytes());
// The compat window: v4v6 are additive over v3, so the host floor stays at 3. // The compat window: v4 is additive over v3, so the host floor stays one below.
assert_eq!(PROTOCOL_VERSION, 6); assert_eq!(PROTOCOL_VERSION, 4);
assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3); assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3);
} }
#[test]
fn cursor_shm_layout_is_pinned() {
use cursor::*;
// The header must leave the shape offset intact whatever grows inside `_reserved`.
assert_eq!(core::mem::size_of::<CursorShm>(), 64);
assert_eq!(CURSOR_SHM_SIZE, 64 + 256 * 256 * 4);
assert_eq!(CURSOR_MAGIC, u32::from_le_bytes(*b"PFCU"));
// Seqlock snapshot discipline survives a bytemuck roundtrip.
let hdr = CursorShm {
magic: CURSOR_MAGIC,
seq: 2,
visible: 1,
cursor_type: CURSOR_TYPE_ALPHA,
x: -3,
y: 7,
shape_id: 42,
width: 32,
height: 32,
pitch: 128,
hot_x: 4,
hot_y: 5,
_reserved: [0; 4],
};
let bytes = bytemuck::bytes_of(&hdr);
assert_eq!(*bytemuck::from_bytes::<CursorShm>(bytes), hdr);
assert_eq!(bytes[16..20], (-3i32).to_le_bytes());
}
#[test] #[test]
fn gamepad_names_and_magics_are_stable() { fn gamepad_names_and_magics_are_stable() {
assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0"); assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0");
-5
View File
@@ -25,11 +25,6 @@ tracing = "0.1"
# A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`). # A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`).
tracing-subscriber = { version = "0.3", features = ["env-filter"] } tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[target.'cfg(target_os = "windows")'.dev-dependencies]
# The QSV live e2e drives the REAL HdrP010Converter output (an RTV-written, ring-profile P010
# texture) into the encoder — the one seam the CPU-upload tests can't reach.
pf-capture = { path = "../pf-capture" }
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies] [target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
# Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms. # Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms.
openh264 = "0.9" openh264 = "0.9"
+1 -290
View File
@@ -7,45 +7,6 @@
use anyhow::Result; use anyhow::Result;
use pf_frame::CapturedFrame; use pf_frame::CapturedFrame;
/// Whether an encoder fed `format` must be built 10-bit — decided by **the pixels that actually
/// arrive**, never by the negotiated `bit_depth`.
///
/// The three Windows backends each derived this as `bit_depth >= 10 || matches!(format, P010 |
/// Rgb10a2)`, i.e. the *negotiated* depth could force a 10-bit encoder over an 8-bit capture. That
/// combination is not hypothetical: a client advertises 10-bit, the handshake negotiates
/// `bit_depth = 10`, and then enabling advanced colour on the IDD virtual display fails — at which
/// point the capturer says so and delivers 8-bit NV12 anyway (`pf-capture`'s idd_push logs "10-bit
/// HDR was negotiated but enabling advanced color on the virtual display FAILED — encoding 8-bit
/// SDR"). Every backend then lost the session, each in its own way: native AMF and native QSV
/// `bail!` at open because the format does not match the P010 they derived, and the libavcodec
/// path accepted the open and then failed EVERY submit forever (its per-frame depth check
/// recomputes from the frame, which never matches), where `reset()` could not help because the
/// rebuild re-derived the same wrong depth.
///
/// Following the pixels keeps the stream alive and, more importantly, keeps it HONEST: the depth
/// also selects the colour signalling (BT.2020 PQ vs BT.709) and the staging surface format, so an
/// 8-bit capture now yields an 8-bit stream that says it is SDR — which is what the capturer
/// already reported it is sending. The negotiated depth remains an upper bound; the session label
/// may still claim HDR, and that mismatch belongs to the negotiation, not to the encoder.
/// Windows-only: the three backends that derive an encoder depth from a capture live there
/// (native AMF, native QSV, libavcodec AMF/QSV). The Linux backends take the depth from the
/// negotiated `bit_depth` alone because their capture formats carry it unambiguously.
#[cfg(target_os = "windows")]
pub(crate) fn ten_bit_input(format: pf_frame::PixelFormat, negotiated_depth: u8) -> bool {
use pf_frame::PixelFormat;
let ten = matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2);
if negotiated_depth >= 10 && !ten {
tracing::warn!(
?format,
negotiated_depth,
"session negotiated 10-bit but the capturer delivers an 8-bit format — encoding 8-bit \
SDR (the stream's colour signalling follows the pixels; check whether advanced colour \
failed to enable on the virtual display)"
);
}
ten
}
/// An encoded access unit (one NAL/AU) to hand to `punktfunk_core` for FEC + packetization. /// An encoded access unit (one NAL/AU) to hand to `punktfunk_core` for FEC + packetization.
/// `data` is in-band Annex-B (the encoder is opened without a global header), so each /// `data` is in-band Annex-B (the encoder is opened without a global header), so each
/// keyframe carries its own VPS/SPS/PPS — the bytes are both a playable elementary /// keyframe carries its own VPS/SPS/PPS — the bytes are both a playable elementary
@@ -71,48 +32,8 @@ pub struct EncodedFrame {
pub chunk_aligned: bool, pub chunk_aligned: bool,
} }
/// One slice-boundary chunk of an encoded AU, emitted by a chunked-poll backend
/// ([`Encoder::poll_chunk`], latency plan §7 LN1): the encoder hands out completed slices while
/// the rest of the frame is still encoding, so packetize/FEC/pacing can overlap the encode tail.
/// The chunks of one AU concatenate to exactly the bytes [`Encoder::poll`] would have returned,
/// and every cut lands on an Annex-B NAL boundary (slice starts). AU-level metadata
/// (`pts_ns`/`keyframe`/`recovery_anchor`/`chunk_aligned`) is authoritative on the FIRST chunk
/// (`first`) — the host opens the wire frame from it; `last` closes the AU. `keyframe` on a
/// non-final chunk is the encoder's own prediction (exact under the P-only/infinite-GOP config —
/// the driver only ever emits an IDR we asked for); the final chunk re-checks it against the
/// driver's reported picture type.
pub struct AuChunk {
pub data: Vec<u8>,
pub pts_ns: u64,
pub keyframe: bool,
/// See [`EncodedFrame::recovery_anchor`].
pub recovery_anchor: bool,
/// See [`EncodedFrame::chunk_aligned`].
pub chunk_aligned: bool,
/// Opens the AU (carries the authoritative AU metadata).
pub first: bool,
/// Closes the AU (the concatenation is complete; the encoder's in-flight slot is released).
pub last: bool,
}
impl AuChunk {
/// A whole AU as a single self-closing chunk — what every non-chunked backend's
/// [`Encoder::poll_chunk`] default emits, so a chunk consumer needs no per-backend fork.
pub fn whole(f: EncodedFrame) -> Self {
AuChunk {
data: f.data,
pts_ns: f.pts_ns,
keyframe: f.keyframe,
recovery_anchor: f.recovery_anchor,
chunk_aligned: f.chunk_aligned,
first: true,
last: true,
}
}
}
/// Codec selection negotiated with the client. /// Codec selection negotiated with the client.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] #[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Codec { pub enum Codec {
H264, H264,
H265, H265,
@@ -294,28 +215,10 @@ pub struct EncoderCaps {
/// `USER_FLAG_RECOVERY_POINT` on every Nth emitted AU, re-phased at each IDR). 0 when intra-refresh /// `USER_FLAG_RECOVERY_POINT` on every Nth emitted AU, re-phased at each IDR). 0 when intra-refresh
/// is off. Only consulted when [`intra_refresh_recovery`](Self::intra_refresh_recovery) is set. /// is off. Only consulted when [`intra_refresh_recovery`](Self::intra_refresh_recovery) is set.
pub intra_refresh_period: u32, pub intra_refresh_period: u32,
/// The encoder composites [`CapturedFrame::cursor`] into the picture it encodes.
///
/// `open_video`'s `cursor_blend` argument is a REQUEST, and for most of this crate's life it was
/// nothing else: `lib.rs` literally did `let _ = cursor_blend;` and only three backends ever read
/// `frame.cursor`. So a session could ask for a composited pointer, get a backend that silently
/// discards it, and stream with no mouse cursor at all — the confirmed symptom on the VAAPI
/// dmabuf path and the libav-NVENC CUDA path.
///
/// This makes the answer queryable instead of assumed. It is deliberately a plain fact about the
/// encoder, not a policy: what to DO when a session wants blending and the backend cannot is the
/// host's call, since only the host can re-plan capture (fall back to capturer-side compositing).
/// `open_video` can only warn, which it does.
pub blends_cursor: bool,
} }
/// A hardware encoder. One per session; runs on the encode thread. /// A hardware encoder. One per session; runs on the encode thread.
pub trait Encoder: Send { pub trait Encoder: Send {
/// Submit one captured frame for encoding. Lifetime contract: the caller must keep `frame`
/// (and its GPU payload) alive until this frame's AU has been returned by
/// [`poll`](Self::poll) — a stream-ordered backend (Linux direct-NVENC's IO-stream binding)
/// may still be reading the payload asynchronously after `submit` returns. Both host encode
/// loops already hold the frame across their poll drain; new callers must do the same.
fn submit(&mut self, frame: &CapturedFrame) -> Result<()>; fn submit(&mut self, frame: &CapturedFrame) -> Result<()>;
/// [`submit`](Self::submit) with the **wire frame index** this frame's AU will carry — the /// [`submit`](Self::submit) with the **wire frame index** this frame's AU will carry — the
/// number the packetizer stamps on it and the client's loss reports/RFI requests name. The /// number the packetizer stamps on it and the client's loss reports/RFI requests name. The
@@ -361,44 +264,8 @@ pub trait Encoder: Send {
fn invalidate_ref_frames(&mut self, _first_frame: i64, _last_frame: i64) -> bool { fn invalidate_ref_frames(&mut self, _first_frame: i64, _last_frame: i64) -> bool {
false false
} }
/// Escalate into a pipelined (two-thread) retrieve mode under sustained GPU contention — the
/// encoder analog of the capturer depth escalation: AUs ride ~one loop tick behind (`poll`
/// may return `None` while an encode is in flight) in exchange for capture/submit no longer
/// serializing on the encode wait. Returns whether pipelined retrieve is (now) active; the
/// switch may be deferred to the next safe point internally. `set_pipelined(true)` returning
/// `false` (the default impl) = unsupported — the session loop stops asking.
///
/// `set_pipelined(false)` requests the wind-back (de-escalation, latency recovery): the
/// backend restores its sync-retrieve mode — and the latency features that mode carries
/// (IO-stream binding, sub-frame chunking) — at its next safe point, usually via a session
/// rebuild whose first frame is an IDR. The return is still "is pipelined retrieve active":
/// the caller polls until it reads `false`. Backends that never escalate return `false`
/// trivially. An operator pin (`PUNKTFUNK_NVENC_ASYNC=1`) refuses the wind-back.
fn set_pipelined(&mut self, _on: bool) -> bool {
false
}
/// Pull the next encoded AU if one is ready. /// Pull the next encoded AU if one is ready.
fn poll(&mut self) -> Result<Option<EncodedFrame>>; fn poll(&mut self) -> Result<Option<EncodedFrame>>;
/// Whether [`poll_chunk`](Self::poll_chunk) currently emits sub-AU chunks — i.e. the LIVE
/// session has slice-level readback armed (Linux direct-NVENC with the
/// `PUNKTFUNK_NVENC_SLICES` and `PUNKTFUNK_NVENC_SUBFRAME` knobs on a sync depth-1
/// retrieve). Dynamic, not static: a pipelined-retrieve escalation or a session rebuild can
/// turn it off — re-query per AU, never cache across frames. `false` (the default) means
/// `poll_chunk` degrades to one whole-AU chunk per frame.
fn supports_chunked_poll(&self) -> bool {
false
}
/// Pull the next slice-boundary chunk of the oldest in-flight AU (latency plan §7 LN1).
/// Semantics when chunking is live: BLOCKS until the next chunk is readable, and the final
/// (`last`) chunk blocks exactly like [`poll`](Self::poll) does — the depth-1 pump treats
/// `None` as re-poll-next-tick, so a non-blocking tail would ride the AU one tick late (the
/// `6dc195f9` Vulkan bug class). `Ok(None)` only when no AU is in flight. Each AU must be
/// drained through ONE method: calling `poll` on a partially-chunked AU is a caller bug (the
/// backend errors rather than double-emit bytes). Default: delegates to `poll`, wrapping the
/// whole AU as a single `first && last` chunk.
fn poll_chunk(&mut self) -> Result<Option<AuChunk>> {
Ok(self.poll()?.map(AuChunk::whole))
}
/// Tear the underlying hardware encoder down and rebuild it in place, keeping the session's /// Tear the underlying hardware encoder down and rebuild it in place, keeping the session's
/// negotiated parameters — the encode-stall watchdog's recovery lever (a wedged AMF/QSV /// negotiated parameters — the encode-stall watchdog's recovery lever (a wedged AMF/QSV
/// driver stops emitting AUs or accepting frames without ever returning an error). Returns /// driver stops emitting AUs or accepting frames without ever returning an error). Returns
@@ -419,16 +286,6 @@ pub trait Encoder: Send {
fn reconfigure_bitrate(&mut self, _bps: u64) -> bool { fn reconfigure_bitrate(&mut self, _bps: u64) -> bool {
false false
} }
/// The bitrate (bps) the encoder is ACTUALLY running at (or will open at, for a lazily-opened
/// backend) — the encoder-side truth after any internal clamp, e.g. the direct-NVENC
/// codec-level ceiling search. The session loop reads this after every open/reconfigure and
/// stores IT, not the requested rate, as the live bitrate — so the send pacer, the console
/// and the client controller's ack all track what the ASIC really targets (a controller fed
/// the requested rate keeps climbing from a phantom base, §ABR overdrive). `None` (the
/// default) = the backend doesn't track an applied rate; the caller keeps the requested one.
fn applied_bitrate_bps(&self) -> Option<u64> {
None
}
/// Wire-chunk the encoder's AUs at the session's shard payload size (the PyroWave /// Wire-chunk the encoder's AUs at the session's shard payload size (the PyroWave
/// datagram-aligned mode, plan §4.4): every `shard_payload` window of the emitted AU /// datagram-aligned mode, plan §4.4): every `shard_payload` window of the emitted AU
/// starts a fresh self-delimiting codec packet, zero-padded to the window — so a lost /// starts a fresh self-delimiting codec packet, zero-padded to the window — so a lost
@@ -436,32 +293,8 @@ pub trait Encoder: Send {
/// flagged [`EncodedFrame::chunk_aligned`] and the session marks them on the wire. /// flagged [`EncodedFrame::chunk_aligned`] and the session marks them on the wire.
/// Default: no-op (the H.26x backends' bitstreams cannot be cut losslessly). /// Default: no-op (the H.26x backends' bitstreams cannot be cut losslessly).
fn set_wire_chunking(&mut self, _shard_payload: usize) {} fn set_wire_chunking(&mut self, _shard_payload: usize) {}
/// How many frames the CAPTURER guarantees the encoder may hold in flight before it starts
/// reusing an input texture (`Capturer::pipeline_depth`). Backends that encode the capturer's
/// textures IN PLACE — no `CopyResource` — must not pipeline deeper than this: the capturer
/// rotates its output ring per delivered frame with no regard for encode completion, so a
/// deeper pipeline lets it overwrite a texture mid-encode. That is visual corruption (torn or
/// mixed frames), not UB, so it fails silently and intermittently.
///
/// Called once by the session glue after the capturer is known; a backend that copies its
/// input, or is synchronous, ignores it. Default: no-op.
fn set_input_ring_depth(&mut self, _depth: usize) {}
/// Signal end-of-stream. After this, drain the remaining AUs with [`poll`](Self::poll) /// Signal end-of-stream. After this, drain the remaining AUs with [`poll`](Self::poll)
/// until it returns `None` — NVENC buffers frames internally even at `delay=0`. /// until it returns `None` — NVENC buffers frames internally even at `delay=0`.
///
/// **The two production encode loops deliberately do not call this**, and that is not an
/// oversight to be "fixed" by a later sweep. Both reach their exit only after the transport is
/// already gone (the client disconnected, or the session was stopped), so the AUs a flush would
/// recover have nowhere to go — while flushing is the one call on this trait that can BLOCK on a
/// wedged encoder, on precisely the teardown path a stopped session needs to complete promptly.
/// The Linux direct-SDK NVENC backend makes that concrete: its retrieve-thread join is untimed
/// (see the note in `enc/linux/nvenc_cuda.rs`), so a flush there could hang a session that is
/// already ending.
///
/// It is kept rather than deleted because it does have real consumers: the `spike` dev
/// subcommand, which encodes a FINITE clip and genuinely wants the tail, and the `#[ignore]`d
/// hardware smoke tests across the backends, which assert the drain contract on real GPUs.
/// Those are finite-stream users; a live session is not one.
fn flush(&mut self) -> Result<()>; fn flush(&mut self) -> Result<()>;
} }
@@ -496,17 +329,6 @@ impl Codec {
} }
} }
/// Pixel rate (luma samples/s) at or above which NVENC split-frame encoding is FORCED 2-way —
/// one number shared by the direct-SDK selector (`nvenc_core::resolve_split_mode`) and the libav
/// `split_encode_mode` option author (`linux::NvencEncoder`), so the two paths can never disagree
/// about which modes split. A single NVENC engine tops out ~1 Gpix/s on HEVC, and AUTO doesn't
/// engage below ~2112 px height, so the sessions that need the second engine must be forced. Set
/// BELOW 1 Gpix/s deliberately: 4K120 — the mode this threshold exists for — is 3840×2160×120 =
/// 995,328,000, which a `> 1_000_000_000` gate missed by 0.47% and left on AUTO (pinned ~107 fps
/// on a 4090). 950 M keeps margin for fractional refresh rates while leaving 1440p240 (884.7 M,
/// comfortably single-engine) on AUTO.
pub const SPLIT_FORCE_PIXEL_RATE: u64 = 950_000_000;
/// `PUNKTFUNK_VBV_FRAMES` — HRD/VBV size in frame intervals (default 1.0, the strict low-latency /// `PUNKTFUNK_VBV_FRAMES` — HRD/VBV size in frame intervals (default 1.0, the strict low-latency
/// shape every backend ships: each frame must fit its rate share, keeping frame sizes uniform for /// shape every backend ships: each frame must fit its rate share, keeping frame sizes uniform for
/// the pacer). The AMF/VAAPI/QSV paths parse the same variable locally; this helper brings the /// the pacer). The AMF/VAAPI/QSV paths parse the same variable locally; this helper brings the
@@ -520,35 +342,6 @@ pub(crate) fn vbv_frames_env() -> f64 {
.unwrap_or(1.0) .unwrap_or(1.0)
} }
/// The same HRD/VBV window as [`vbv_frames_env`], expressed the way the Vulkan Video encode API
/// wants it: `(virtualBufferSizeInMs, initialVirtualBufferSizeInMs)`.
///
/// Every other backend states the window in **bits** (`bitrate / fps × frames`); Vulkan states it
/// in **milliseconds**. `vulkan_video.rs` consumes this ONLY when the driver advertises VBR
/// (WP6.3): a tight window under CBR makes the driver stuff underspent frames with filler NALs up
/// to the exact rate share — measured 97 % filler on the 780M — because CBR must keep the CPB from
/// overflowing and Vulkan exposes no filler-suppression control. VBR permits the underspend, so
/// the tight window only ever *bounds* a complex frame.
///
/// The initial fill stays at half the window, preserving the RATIO the hardcoded (1000, 500)
/// pair had — the direct-NVENC house shape uses a FULL-window initial fill instead; measured on
/// RADV the difference is inert (the firmware showed no window sensitivity at all). Both
/// VUIDs on `VkVideoEncodeRateControlInfoKHR`'s window fields are satisfied by construction: the
/// window clamps to `>= 1` so it is non-zero, and `window / 2 <= window` always
/// (`VUID-...-08358` is `<=`, relaxed in Vulkan 1.3.299).
///
/// Carries its only caller's gate: `vulkan_video.rs` is the sole ms-form consumer, and with the
/// crate-wide `allow(dead_code)` gone (WP0.3) an item unused in ANY feature combination is a hard
/// error — this is dead on every Windows leg.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
pub(crate) fn vbv_window_ms(fps: u32) -> (u32, u32) {
let frames = vbv_frames_env();
let ms = (frames * 1000.0 / fps.max(1) as f64).round();
// `f64 as u32` saturates at the bounds in Rust, so an absurd `PUNKTFUNK_VBV_FRAMES` cannot wrap.
let window = (ms as u32).max(1);
(window, window / 2)
}
/// Validate a requested encode resolution before we allocate buffers or open NVENC. Rejects /// Validate a requested encode resolution before we allocate buffers or open NVENC. Rejects
/// zero/odd-sized and out-of-range modes with a clear error instead of letting buffer math /// zero/odd-sized and out-of-range modes with a clear error instead of letting buffer math
/// overflow or the encoder open fail with an opaque NVENC code. A client can request any /// overflow or the encoder open fail with an opaque NVENC code. A client can request any
@@ -577,26 +370,6 @@ pub fn validate_dimensions(codec: Codec, width: u32, height: u32) -> Result<()>
(use HEVC/AV1 above 4096, or lower the client resolution)" (use HEVC/AV1 above 4096, or lower the client resolution)"
); );
} }
// PyroWave's vendored rate controller packs the 32×32 block index into the low 16 bits of
// `RDOperation::block_offset_saving` (pyrowave-sys `patches/0002-rdo-saving-clamp.patch`).
// Past `u16::MAX` blocks the index collides with the `saving` field, the resolve over-credits,
// and the emitted payload can overshoot the buffer `pyrowave_encoder_packetize` writes into —
// whose only bounds check is an `assert` that the Release (NDEBUG) vendored build compiles out.
// So this is a hard cap, not a quality knob.
//
// Checked against 4:2:0, the *most permissive* chroma: a mode that cannot fit even there can
// fit no PyroWave session at all, so it belongs at this single chokepoint (which both the
// negotiator and `open_video_backend` run) rather than only in the per-backend opens. 4:4:4
// has twice the block count and is checked again at open, where the real chroma is known —
// and the negotiator's 4:4:4 → 4:2:0 downgrade means an oversized mode arrives at the encoder
// as 4:2:0, which is exactly the case the old open-time guard skipped.
#[cfg(feature = "pyrowave")]
if codec == Codec::PyroWave && !crate::pyrowave_mode_fits_rdo(width, height, false) {
anyhow::bail!(
"invalid PyroWave resolution {width}x{height}: exceeds the rate controller's 16-bit \
block index (pyrowave-sys patches/0002) lower the client resolution"
);
}
Ok(()) Ok(())
} }
@@ -604,31 +377,6 @@ pub fn validate_dimensions(codec: Codec, width: u32, height: u32) -> Result<()>
mod tests { mod tests {
use super::*; use super::*;
/// WP6.3. The window VUIDs on `VkVideoEncodeRateControlInfoKHR` are the whole contract of
/// this helper, and both are edge cases: the window must be non-zero (a high-refresh mode
/// rounds a sub-1 ms window down to nothing) and the initial fill must be at most the window
/// (`<=` — `VUID-...-08358` was relaxed in 1.3.299). Env-free so it pins the default shape —
/// the scaled cases belong to whoever sets `PUNKTFUNK_VBV_FRAMES`. Carries the helper's own
/// cfg gate (see its note), so it runs on the Linux `vulkan-encode` leg.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[test]
fn vbv_window_is_about_one_frame_and_always_legal() {
// The house default is ~1 frame interval, not the 1000 ms the Vulkan backend hardwired.
assert_eq!(vbv_window_ms(60).0, 17); // 16.67 ms
assert_eq!(vbv_window_ms(30).0, 33);
assert_eq!(vbv_window_ms(240).0, 4);
for fps in [1, 24, 30, 60, 120, 144, 240, 480, 1000, 4000, u32::MAX] {
let (window, initial) = vbv_window_ms(fps);
assert!(window > 0, "virtualBufferSizeInMs must be > 0 (fps {fps})");
assert!(
initial <= window,
"initialVirtualBufferSizeInMs must be <= virtualBufferSizeInMs (fps {fps})"
);
}
// fps 0 must not divide by zero — `open` clamps, but the helper is called directly too.
assert!(vbv_window_ms(0).0 > 0);
}
#[test] #[test]
fn rejects_zero_and_odd_dimensions() { fn rejects_zero_and_odd_dimensions() {
assert!(validate_dimensions(Codec::H265, 0, 1080).is_err()); assert!(validate_dimensions(Codec::H265, 0, 1080).is_err());
@@ -645,26 +393,6 @@ mod tests {
assert!(validate_dimensions(Codec::H264, 3840, 4098).is_err()); assert!(validate_dimensions(Codec::H264, 3840, 4098).is_err());
} }
/// PyroWave's hard cap is the rate controller's 16-bit block index, not just
/// `max_dimension()`. Checked at 4:2:0 (the most permissive chroma), because a mode that
/// cannot fit there cannot fit at any chroma — and because the negotiator's 4:4:4 → 4:2:0
/// downgrade delivers oversized modes to the encoder AS 4:2:0. HEVC/AV1 at the same
/// dimensions must stay unaffected.
#[cfg(feature = "pyrowave")]
#[test]
fn pyrowave_rejects_modes_past_the_rdo_block_index() {
// Fits: 8K 4:2:0 is 49125 blocks.
assert!(validate_dimensions(Codec::PyroWave, 7680, 4320).is_ok());
// Does not fit at 4:2:0 (73728 / 98304 blocks) — must be refused even though both are
// within `Codec::PyroWave.max_dimension()` (8192).
assert!(validate_dimensions(Codec::PyroWave, 8192, 6144).is_err());
assert!(validate_dimensions(Codec::PyroWave, 8192, 8192).is_err());
// The same modes remain legal for the H.26x/AV1 codecs, which have no such rate
// controller — the cap must not leak across codecs.
assert!(validate_dimensions(Codec::H265, 8192, 8192).is_ok());
assert!(validate_dimensions(Codec::Av1, 8192, 6144).is_ok());
}
#[test] #[test]
fn hevc_and_av1_allow_up_to_8192() { fn hevc_and_av1_allow_up_to_8192() {
for c in [Codec::H265, Codec::Av1] { for c in [Codec::H265, Codec::Av1] {
@@ -684,23 +412,6 @@ mod tests {
} }
} }
/// The whole-AU chunk (every non-chunked backend's `poll_chunk` shape) must carry the AU's
/// metadata verbatim and be self-closing (`first && last`).
#[test]
fn whole_au_chunk_is_self_closing() {
let c = AuChunk::whole(EncodedFrame {
data: vec![0, 0, 0, 1, 0x40],
pts_ns: 42,
keyframe: true,
recovery_anchor: true,
chunk_aligned: false,
});
assert_eq!(c.data, vec![0, 0, 0, 1, 0x40]);
assert_eq!(c.pts_ns, 42);
assert!(c.keyframe && c.recovery_anchor && !c.chunk_aligned);
assert!(c.first && c.last);
}
/// Wire round-trip and the stats label stay in lockstep with the `quic::CODEC_*` bits. /// Wire round-trip and the stats label stay in lockstep with the `quic::CODEC_*` bits.
#[test] #[test]
fn codec_wire_roundtrip_and_label() { fn codec_wire_roundtrip_and_label() {
@@ -0,0 +1,105 @@
// Cursor-overlay blend kernels for the CUDA/NVENC path (cursor-as-metadata). The cursor bitmap is
// straight-alpha RGBA, row-packed (stride = curW*4). Blended into the encoder-OWNED NVENC input
// surface — never the compositor's dmabuf. One thread per cursor pixel (ARGB / YUV444) or per 2x2
// chroma block (NV12). Coefficients are BT.709 limited, matching rgb2yuv.comp so the cursor colour
// matches the rest of the frame regardless of which zero-copy backend encodes it.
//
// Build (regenerate cursor_blend.ptx after editing):
// nvcc -ptx -arch=compute_75 cursor_blend.cu -o cursor_blend.ptx
// PTX is JIT'd by the driver forward to the actual GPU, so a compute_75 (Turing) baseline runs on
// every Turing-or-newer NVENC GPU. (CUDA 13's nvcc no longer targets pre-Turing archs.)
typedef unsigned char u8;
__device__ __forceinline__ u8 blend8(int dst, int src, int a) {
return (u8)((src * a + dst * (255 - a)) / 255);
}
// Packed 4-byte surface. NVENC's ARGB format stores bytes B,G,R,A in memory; the cursor is R,G,B,A.
extern "C" __global__ void blend_argb(
u8* surf, int pitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int cx = blockIdx.x * blockDim.x + threadIdx.x;
int cy = blockIdx.y * blockDim.y + threadIdx.y;
if (cx >= curW || cy >= curH) return;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) return;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) return;
u8* d = surf + (size_t)py * pitch + (size_t)px * 4;
d[0] = blend8(d[0], s[2], a); // B <- cursor B
d[1] = blend8(d[1], s[1], a); // G <- cursor G
d[2] = blend8(d[2], s[0], a); // R <- cursor R
}
// Planar YUV444: three full-res planes stacked at base, base+plane, base+2*plane (plane=pitch*surfH).
extern "C" __global__ void blend_yuv444(
u8* base, int pitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int cx = blockIdx.x * blockDim.x + threadIdx.x;
int cy = blockIdx.y * blockDim.y + threadIdx.y;
if (cx >= curW || cy >= curH) return;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) return;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) return;
float R = s[0], G = s[1], B = s[2];
int Y = (int)(16.0f + 0.1826f * R + 0.6142f * G + 0.0620f * B + 0.5f);
int U = (int)(128.0f - 0.1006f * R - 0.3386f * G + 0.4392f * B + 0.5f);
int V = (int)(128.0f + 0.4392f * R - 0.3989f * G - 0.0403f * B + 0.5f);
size_t plane = (size_t)pitch * surfH;
u8* yp = base + (size_t)py * pitch + px;
u8* up = base + plane + (size_t)py * pitch + px;
u8* vp = base + 2 * plane + (size_t)py * pitch + px;
*yp = blend8(*yp, Y, a);
*up = blend8(*up, U, a);
*vp = blend8(*vp, V, a);
}
// NV12: full-res Y plane + interleaved half-res UV plane. One thread per 2x2 luma block; each blends
// up to four Y samples and one (alpha-weighted) UV sample.
extern "C" __global__ void blend_nv12(
u8* yb, int yPitch, u8* uvb, int uvPitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int bx = blockIdx.x * blockDim.x + threadIdx.x;
int by = blockIdx.y * blockDim.y + threadIdx.y;
int base_cx = bx * 2, base_cy = by * 2;
if (base_cx >= curW || base_cy >= curH) return;
float ua = 0.0f, va = 0.0f, wa = 0.0f;
int cnt = 0;
for (int j = 0; j < 2; j++) {
for (int i = 0; i < 2; i++) {
int cx = base_cx + i, cy = base_cy + j;
if (cx >= curW || cy >= curH) continue;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) continue;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) continue;
float R = s[0], G = s[1], B = s[2];
int Y = (int)(16.0f + 0.1826f * R + 0.6142f * G + 0.0620f * B + 0.5f);
u8* yp = yb + (size_t)py * yPitch + px;
*yp = blend8(*yp, Y, a);
ua += (128.0f - 0.1006f * R - 0.3386f * G + 0.4392f * B) * a;
va += (128.0f + 0.4392f * R - 0.3989f * G - 0.0403f * B) * a;
wa += a;
cnt++;
}
}
if (wa <= 0.0f || cnt == 0) return;
// The chroma sample covering this block's top-left surface pixel.
int uvx = (ox + base_cx) / 2;
int uvy = (oy + base_cy) / 2;
if (uvx < 0 || uvy < 0 || uvx * 2 >= surfW || uvy * 2 >= surfH) return;
int U = (int)(ua / wa + 0.5f);
int V = (int)(va / wa + 0.5f);
int amean = (int)(wa / cnt + 0.5f);
u8* uv = uvb + (size_t)uvy * uvPitch + (size_t)uvx * 2;
uv[0] = blend8(uv[0], U, amean);
uv[1] = blend8(uv[1], V, amean);
}
@@ -0,0 +1,576 @@
//
// Generated by NVIDIA NVVM Compiler
//
// Compiler Build ID: CL-38244171
// Cuda compilation tools, release 13.3, V13.3.73
// Based on NVVM 7.0.1
//
.version 9.3
.target sm_75
.address_size 64
// .globl blend_argb
.visible .entry blend_argb(
.param .u64 blend_argb_param_0,
.param .u32 blend_argb_param_1,
.param .u32 blend_argb_param_2,
.param .u32 blend_argb_param_3,
.param .u64 blend_argb_param_4,
.param .u32 blend_argb_param_5,
.param .u32 blend_argb_param_6,
.param .u32 blend_argb_param_7,
.param .u32 blend_argb_param_8
)
{
.reg .pred %p<10>;
.reg .b16 %rs<2>;
.reg .b32 %r<34>;
.reg .b64 %rd<17>;
ld.param.u64 %rd2, [blend_argb_param_0];
ld.param.u32 %r5, [blend_argb_param_1];
ld.param.u32 %r6, [blend_argb_param_2];
ld.param.u32 %r7, [blend_argb_param_3];
ld.param.u64 %rd3, [blend_argb_param_4];
ld.param.u32 %r8, [blend_argb_param_5];
ld.param.u32 %r11, [blend_argb_param_6];
ld.param.u32 %r9, [blend_argb_param_7];
ld.param.u32 %r10, [blend_argb_param_8];
mov.u32 %r12, %ntid.x;
mov.u32 %r13, %ctaid.x;
mov.u32 %r14, %tid.x;
mad.lo.s32 %r1, %r13, %r12, %r14;
mov.u32 %r15, %ntid.y;
mov.u32 %r16, %ctaid.y;
mov.u32 %r17, %tid.y;
mad.lo.s32 %r2, %r16, %r15, %r17;
setp.ge.s32 %p1, %r1, %r8;
setp.ge.s32 %p2, %r2, %r11;
or.pred %p3, %p1, %p2;
@%p3 bra $L__BB0_4;
add.s32 %r3, %r1, %r9;
add.s32 %r4, %r2, %r10;
or.b32 %r18, %r4, %r3;
setp.lt.s32 %p4, %r18, 0;
setp.ge.s32 %p5, %r3, %r6;
or.pred %p6, %p5, %p4;
setp.ge.s32 %p7, %r4, %r7;
or.pred %p8, %p7, %p6;
@%p8 bra $L__BB0_4;
mad.lo.s32 %r19, %r2, %r8, %r1;
mul.wide.s32 %rd4, %r19, 4;
cvta.to.global.u64 %rd5, %rd3;
add.s64 %rd1, %rd5, %rd4;
ld.global.u8 %rs1, [%rd1+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB0_4;
cvt.u32.u16 %r20, %rs1;
mul.wide.s32 %rd6, %r4, %r5;
mul.wide.s32 %rd7, %r3, 4;
add.s64 %rd8, %rd6, %rd7;
cvta.to.global.u64 %rd9, %rd2;
add.s64 %rd10, %rd9, %rd8;
ld.global.u8 %r21, [%rd10];
ld.global.u8 %r22, [%rd1+2];
xor.b32 %r23, %r20, 255;
mul.lo.s32 %r24, %r23, %r21;
mad.lo.s32 %r25, %r22, %r20, %r24;
mul.wide.u32 %rd11, %r25, -2139062143;
shr.u64 %rd12, %rd11, 39;
st.global.u8 [%rd10], %rd12;
ld.global.u8 %r26, [%rd10+1];
ld.global.u8 %r27, [%rd1+1];
mul.lo.s32 %r28, %r23, %r26;
mad.lo.s32 %r29, %r27, %r20, %r28;
mul.wide.u32 %rd13, %r29, -2139062143;
shr.u64 %rd14, %rd13, 39;
st.global.u8 [%rd10+1], %rd14;
ld.global.u8 %r30, [%rd10+2];
ld.global.u8 %r31, [%rd1];
mul.lo.s32 %r32, %r23, %r30;
mad.lo.s32 %r33, %r31, %r20, %r32;
mul.wide.u32 %rd15, %r33, -2139062143;
shr.u64 %rd16, %rd15, 39;
st.global.u8 [%rd10+2], %rd16;
$L__BB0_4:
ret;
}
// .globl blend_yuv444
.visible .entry blend_yuv444(
.param .u64 blend_yuv444_param_0,
.param .u32 blend_yuv444_param_1,
.param .u32 blend_yuv444_param_2,
.param .u32 blend_yuv444_param_3,
.param .u64 blend_yuv444_param_4,
.param .u32 blend_yuv444_param_5,
.param .u32 blend_yuv444_param_6,
.param .u32 blend_yuv444_param_7,
.param .u32 blend_yuv444_param_8
)
{
.reg .pred %p<10>;
.reg .b16 %rs<5>;
.reg .f32 %f<16>;
.reg .b32 %r<49>;
.reg .b64 %rd<14>;
ld.param.u64 %rd2, [blend_yuv444_param_0];
ld.param.u32 %r5, [blend_yuv444_param_1];
ld.param.u32 %r6, [blend_yuv444_param_2];
ld.param.u32 %r7, [blend_yuv444_param_3];
ld.param.u64 %rd3, [blend_yuv444_param_4];
ld.param.u32 %r8, [blend_yuv444_param_5];
ld.param.u32 %r11, [blend_yuv444_param_6];
ld.param.u32 %r9, [blend_yuv444_param_7];
ld.param.u32 %r10, [blend_yuv444_param_8];
mov.u32 %r12, %ntid.x;
mov.u32 %r13, %ctaid.x;
mov.u32 %r14, %tid.x;
mad.lo.s32 %r1, %r13, %r12, %r14;
mov.u32 %r15, %ntid.y;
mov.u32 %r16, %ctaid.y;
mov.u32 %r17, %tid.y;
mad.lo.s32 %r2, %r16, %r15, %r17;
setp.ge.s32 %p1, %r1, %r8;
setp.ge.s32 %p2, %r2, %r11;
or.pred %p3, %p1, %p2;
@%p3 bra $L__BB1_4;
add.s32 %r3, %r1, %r9;
add.s32 %r4, %r2, %r10;
or.b32 %r18, %r4, %r3;
setp.lt.s32 %p4, %r18, 0;
setp.ge.s32 %p5, %r3, %r6;
or.pred %p6, %p5, %p4;
setp.ge.s32 %p7, %r4, %r7;
or.pred %p8, %p7, %p6;
@%p8 bra $L__BB1_4;
mad.lo.s32 %r19, %r2, %r8, %r1;
mul.wide.s32 %rd4, %r19, 4;
cvta.to.global.u64 %rd5, %rd3;
add.s64 %rd1, %rd5, %rd4;
ld.global.u8 %rs1, [%rd1+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB1_4;
cvt.u32.u16 %r20, %rs1;
ld.global.u8 %rs2, [%rd1];
cvt.rn.f32.u16 %f1, %rs2;
ld.global.u8 %rs3, [%rd1+1];
cvt.rn.f32.u16 %f2, %rs3;
ld.global.u8 %rs4, [%rd1+2];
cvt.rn.f32.u16 %f3, %rs4;
fma.rn.f32 %f4, %f1, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f5, %f2, 0f3F1D3C36, %f4;
fma.rn.f32 %f6, %f3, 0f3D7DF3B6, %f5;
add.f32 %f7, %f6, 0f3F000000;
cvt.rzi.s32.f32 %r21, %f7;
fma.rn.f32 %f8, %f1, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f9, %f2, 0fBEAD5CFB, %f8;
fma.rn.f32 %f10, %f3, 0f3EE0DED3, %f9;
add.f32 %f11, %f10, 0f3F000000;
cvt.rzi.s32.f32 %r22, %f11;
fma.rn.f32 %f12, %f1, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f13, %f2, 0fBECC3C9F, %f12;
fma.rn.f32 %f14, %f3, 0fBD25119D, %f13;
add.f32 %f15, %f14, 0f3F000000;
cvt.rzi.s32.f32 %r23, %f15;
mul.wide.s32 %rd6, %r4, %r5;
cvt.s64.s32 %rd7, %r3;
add.s64 %rd8, %rd6, %rd7;
cvta.to.global.u64 %rd9, %rd2;
add.s64 %rd10, %rd9, %rd8;
ld.global.u8 %r24, [%rd10];
mul.lo.s32 %r25, %r21, %r20;
xor.b32 %r26, %r20, 255;
mad.lo.s32 %r27, %r26, %r24, %r25;
mul.hi.s32 %r28, %r27, -2139062143;
add.s32 %r29, %r28, %r27;
shr.u32 %r30, %r29, 31;
shr.u32 %r31, %r29, 7;
add.s32 %r32, %r31, %r30;
st.global.u8 [%rd10], %r32;
mul.wide.s32 %rd11, %r7, %r5;
add.s64 %rd12, %rd10, %rd11;
ld.global.u8 %r33, [%rd12];
mul.lo.s32 %r34, %r22, %r20;
mad.lo.s32 %r35, %r26, %r33, %r34;
mul.hi.s32 %r36, %r35, -2139062143;
add.s32 %r37, %r36, %r35;
shr.u32 %r38, %r37, 31;
shr.u32 %r39, %r37, 7;
add.s32 %r40, %r39, %r38;
st.global.u8 [%rd12], %r40;
add.s64 %rd13, %rd12, %rd11;
ld.global.u8 %r41, [%rd13];
mul.lo.s32 %r42, %r23, %r20;
mad.lo.s32 %r43, %r26, %r41, %r42;
mul.hi.s32 %r44, %r43, -2139062143;
add.s32 %r45, %r44, %r43;
shr.u32 %r46, %r45, 31;
shr.u32 %r47, %r45, 7;
add.s32 %r48, %r47, %r46;
st.global.u8 [%rd13], %r48;
$L__BB1_4:
ret;
}
// .globl blend_nv12
.visible .entry blend_nv12(
.param .u64 blend_nv12_param_0,
.param .u32 blend_nv12_param_1,
.param .u64 blend_nv12_param_2,
.param .u32 blend_nv12_param_3,
.param .u32 blend_nv12_param_4,
.param .u32 blend_nv12_param_5,
.param .u64 blend_nv12_param_6,
.param .u32 blend_nv12_param_7,
.param .u32 blend_nv12_param_8,
.param .u32 blend_nv12_param_9,
.param .u32 blend_nv12_param_10
)
{
.reg .pred %p<43>;
.reg .b16 %rs<17>;
.reg .f32 %f<108>;
.reg .b32 %r<123>;
.reg .b64 %rd<35>;
ld.param.u64 %rd11, [blend_nv12_param_0];
ld.param.u32 %r21, [blend_nv12_param_1];
ld.param.u64 %rd10, [blend_nv12_param_2];
ld.param.u32 %r22, [blend_nv12_param_3];
ld.param.u32 %r23, [blend_nv12_param_4];
ld.param.u32 %r24, [blend_nv12_param_5];
ld.param.u64 %rd12, [blend_nv12_param_6];
ld.param.u32 %r25, [blend_nv12_param_7];
ld.param.u32 %r26, [blend_nv12_param_8];
ld.param.u32 %r27, [blend_nv12_param_9];
ld.param.u32 %r28, [blend_nv12_param_10];
cvta.to.global.u64 %rd1, %rd11;
cvta.to.global.u64 %rd2, %rd12;
mov.u32 %r29, %ntid.x;
mov.u32 %r30, %ctaid.x;
mov.u32 %r31, %tid.x;
mad.lo.s32 %r32, %r30, %r29, %r31;
mov.u32 %r33, %ntid.y;
mov.u32 %r34, %ctaid.y;
mov.u32 %r35, %tid.y;
mad.lo.s32 %r36, %r34, %r33, %r35;
shl.b32 %r1, %r32, 1;
shl.b32 %r2, %r36, 1;
setp.ge.s32 %p1, %r1, %r25;
setp.ge.s32 %p2, %r2, %r26;
or.pred %p3, %p1, %p2;
mov.f32 %f102, 0f00000000;
mov.f32 %f103, 0f00000000;
mov.f32 %f104, 0f00000000;
@%p3 bra $L__BB2_19;
cvt.s64.s32 %rd3, %r21;
add.s32 %r3, %r2, %r28;
setp.ge.s32 %p4, %r3, %r24;
mul.lo.s32 %r4, %r2, %r25;
mul.wide.s32 %rd4, %r3, %r21;
add.s32 %r5, %r1, %r27;
or.b32 %r38, %r5, %r3;
setp.lt.s32 %p5, %r38, 0;
mov.u32 %r121, 0;
setp.ge.s32 %p6, %r5, %r23;
or.pred %p7, %p6, %p5;
or.pred %p8, %p4, %p7;
@%p8 bra $L__BB2_4;
add.s32 %r40, %r1, %r4;
mul.wide.s32 %rd13, %r40, 4;
add.s64 %rd5, %rd2, %rd13;
ld.global.u8 %rs1, [%rd5+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB2_4;
cvt.u32.u16 %r42, %rs1;
ld.global.u8 %rs5, [%rd5];
cvt.rn.f32.u16 %f31, %rs5;
ld.global.u8 %rs6, [%rd5+1];
cvt.rn.f32.u16 %f32, %rs6;
ld.global.u8 %rs7, [%rd5+2];
cvt.rn.f32.u16 %f33, %rs7;
fma.rn.f32 %f34, %f31, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f35, %f32, 0f3F1D3C36, %f34;
fma.rn.f32 %f36, %f33, 0f3D7DF3B6, %f35;
add.f32 %f37, %f36, 0f3F000000;
cvt.rzi.s32.f32 %r43, %f37;
cvt.s64.s32 %rd14, %r5;
add.s64 %rd15, %rd4, %rd14;
add.s64 %rd16, %rd1, %rd15;
ld.global.u8 %r44, [%rd16];
mul.lo.s32 %r45, %r43, %r42;
xor.b32 %r46, %r42, 255;
mad.lo.s32 %r47, %r46, %r44, %r45;
mul.hi.s32 %r48, %r47, -2139062143;
add.s32 %r49, %r48, %r47;
shr.u32 %r50, %r49, 31;
shr.u32 %r51, %r49, 7;
add.s32 %r52, %r51, %r50;
st.global.u8 [%rd16], %r52;
fma.rn.f32 %f38, %f31, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f39, %f32, 0fBEAD5CFB, %f38;
fma.rn.f32 %f40, %f33, 0f3EE0DED3, %f39;
cvt.rn.f32.u16 %f104, %rs1;
fma.rn.f32 %f102, %f40, %f104, 0f00000000;
fma.rn.f32 %f41, %f31, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f42, %f32, 0fBECC3C9F, %f41;
fma.rn.f32 %f43, %f33, 0fBD25119D, %f42;
fma.rn.f32 %f103, %f43, %f104, 0f00000000;
mov.u32 %r121, 1;
$L__BB2_4:
add.s32 %r7, %r1, 1;
setp.ge.s32 %p10, %r7, %r25;
@%p10 bra $L__BB2_8;
add.s32 %r8, %r7, %r27;
or.b32 %r53, %r8, %r3;
setp.lt.s32 %p12, %r53, 0;
setp.ge.s32 %p13, %r8, %r23;
or.pred %p14, %p13, %p12;
or.pred %p15, %p4, %p14;
@%p15 bra $L__BB2_8;
add.s32 %r54, %r7, %r4;
mul.wide.s32 %rd17, %r54, 4;
add.s64 %rd6, %rd2, %rd17;
ld.global.u8 %rs2, [%rd6+3];
setp.eq.s16 %p16, %rs2, 0;
@%p16 bra $L__BB2_8;
cvt.u32.u16 %r55, %rs2;
ld.global.u8 %rs8, [%rd6];
cvt.rn.f32.u16 %f44, %rs8;
ld.global.u8 %rs9, [%rd6+1];
cvt.rn.f32.u16 %f45, %rs9;
ld.global.u8 %rs10, [%rd6+2];
cvt.rn.f32.u16 %f46, %rs10;
fma.rn.f32 %f47, %f44, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f48, %f45, 0f3F1D3C36, %f47;
fma.rn.f32 %f49, %f46, 0f3D7DF3B6, %f48;
add.f32 %f50, %f49, 0f3F000000;
cvt.rzi.s32.f32 %r56, %f50;
cvt.s64.s32 %rd18, %r8;
add.s64 %rd19, %rd4, %rd18;
add.s64 %rd20, %rd1, %rd19;
ld.global.u8 %r57, [%rd20];
mul.lo.s32 %r58, %r56, %r55;
xor.b32 %r59, %r55, 255;
mad.lo.s32 %r60, %r59, %r57, %r58;
mul.hi.s32 %r61, %r60, -2139062143;
add.s32 %r62, %r61, %r60;
shr.u32 %r63, %r62, 31;
shr.u32 %r64, %r62, 7;
add.s32 %r65, %r64, %r63;
st.global.u8 [%rd20], %r65;
fma.rn.f32 %f51, %f44, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f52, %f45, 0fBEAD5CFB, %f51;
fma.rn.f32 %f53, %f46, 0f3EE0DED3, %f52;
cvt.rn.f32.u16 %f54, %rs2;
fma.rn.f32 %f102, %f53, %f54, %f102;
fma.rn.f32 %f55, %f44, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f56, %f45, 0fBECC3C9F, %f55;
fma.rn.f32 %f57, %f46, 0fBD25119D, %f56;
fma.rn.f32 %f103, %f57, %f54, %f103;
add.f32 %f104, %f104, %f54;
add.s32 %r121, %r121, 1;
$L__BB2_8:
add.s32 %r11, %r2, 1;
setp.ge.s32 %p17, %r11, %r26;
add.s32 %r12, %r11, %r28;
add.s32 %r13, %r4, %r25;
cvt.s64.s32 %rd21, %r12;
mul.lo.s64 %rd7, %rd21, %rd3;
@%p17 bra $L__BB2_12;
setp.ge.s32 %p18, %r12, %r24;
or.b32 %r66, %r5, %r12;
setp.lt.s32 %p19, %r66, 0;
or.pred %p21, %p6, %p19;
or.pred %p22, %p18, %p21;
@%p22 bra $L__BB2_12;
add.s32 %r67, %r1, %r13;
mul.wide.s32 %rd22, %r67, 4;
add.s64 %rd8, %rd2, %rd22;
ld.global.u8 %rs3, [%rd8+3];
setp.eq.s16 %p23, %rs3, 0;
@%p23 bra $L__BB2_12;
cvt.u32.u16 %r68, %rs3;
ld.global.u8 %rs11, [%rd8];
cvt.rn.f32.u16 %f58, %rs11;
ld.global.u8 %rs12, [%rd8+1];
cvt.rn.f32.u16 %f59, %rs12;
ld.global.u8 %rs13, [%rd8+2];
cvt.rn.f32.u16 %f60, %rs13;
fma.rn.f32 %f61, %f58, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f62, %f59, 0f3F1D3C36, %f61;
fma.rn.f32 %f63, %f60, 0f3D7DF3B6, %f62;
add.f32 %f64, %f63, 0f3F000000;
cvt.rzi.s32.f32 %r69, %f64;
cvt.s64.s32 %rd23, %r5;
add.s64 %rd24, %rd7, %rd23;
add.s64 %rd25, %rd1, %rd24;
ld.global.u8 %r70, [%rd25];
mul.lo.s32 %r71, %r69, %r68;
xor.b32 %r72, %r68, 255;
mad.lo.s32 %r73, %r72, %r70, %r71;
mul.hi.s32 %r74, %r73, -2139062143;
add.s32 %r75, %r74, %r73;
shr.u32 %r76, %r75, 31;
shr.u32 %r77, %r75, 7;
add.s32 %r78, %r77, %r76;
st.global.u8 [%rd25], %r78;
fma.rn.f32 %f65, %f58, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f66, %f59, 0fBEAD5CFB, %f65;
fma.rn.f32 %f67, %f60, 0f3EE0DED3, %f66;
cvt.rn.f32.u16 %f68, %rs3;
fma.rn.f32 %f102, %f67, %f68, %f102;
fma.rn.f32 %f69, %f58, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f70, %f59, 0fBECC3C9F, %f69;
fma.rn.f32 %f71, %f60, 0fBD25119D, %f70;
fma.rn.f32 %f103, %f71, %f68, %f103;
add.f32 %f104, %f104, %f68;
add.s32 %r121, %r121, 1;
$L__BB2_12:
or.pred %p26, %p17, %p10;
@%p26 bra $L__BB2_16;
setp.ge.s32 %p27, %r12, %r24;
add.s32 %r16, %r7, %r27;
or.b32 %r79, %r16, %r12;
setp.lt.s32 %p28, %r79, 0;
setp.ge.s32 %p29, %r16, %r23;
or.pred %p30, %p29, %p28;
or.pred %p31, %p27, %p30;
@%p31 bra $L__BB2_16;
add.s32 %r80, %r7, %r13;
mul.wide.s32 %rd26, %r80, 4;
add.s64 %rd9, %rd2, %rd26;
ld.global.u8 %rs4, [%rd9+3];
setp.eq.s16 %p32, %rs4, 0;
@%p32 bra $L__BB2_16;
cvt.u32.u16 %r81, %rs4;
ld.global.u8 %rs14, [%rd9];
cvt.rn.f32.u16 %f72, %rs14;
ld.global.u8 %rs15, [%rd9+1];
cvt.rn.f32.u16 %f73, %rs15;
ld.global.u8 %rs16, [%rd9+2];
cvt.rn.f32.u16 %f74, %rs16;
fma.rn.f32 %f75, %f72, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f76, %f73, 0f3F1D3C36, %f75;
fma.rn.f32 %f77, %f74, 0f3D7DF3B6, %f76;
add.f32 %f78, %f77, 0f3F000000;
cvt.rzi.s32.f32 %r82, %f78;
cvt.s64.s32 %rd27, %r16;
add.s64 %rd28, %rd7, %rd27;
add.s64 %rd29, %rd1, %rd28;
ld.global.u8 %r83, [%rd29];
mul.lo.s32 %r84, %r82, %r81;
xor.b32 %r85, %r81, 255;
mad.lo.s32 %r86, %r85, %r83, %r84;
mul.hi.s32 %r87, %r86, -2139062143;
add.s32 %r88, %r87, %r86;
shr.u32 %r89, %r88, 31;
shr.u32 %r90, %r88, 7;
add.s32 %r91, %r90, %r89;
st.global.u8 [%rd29], %r91;
fma.rn.f32 %f79, %f72, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f80, %f73, 0fBEAD5CFB, %f79;
fma.rn.f32 %f81, %f74, 0f3EE0DED3, %f80;
cvt.rn.f32.u16 %f82, %rs4;
fma.rn.f32 %f102, %f81, %f82, %f102;
fma.rn.f32 %f83, %f72, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f84, %f73, 0fBECC3C9F, %f83;
fma.rn.f32 %f85, %f74, 0fBD25119D, %f84;
fma.rn.f32 %f103, %f85, %f82, %f103;
add.f32 %f104, %f104, %f82;
add.s32 %r121, %r121, 1;
$L__BB2_16:
setp.eq.s32 %p33, %r121, 0;
setp.le.f32 %p34, %f104, 0f00000000;
or.pred %p35, %p34, %p33;
@%p35 bra $L__BB2_19;
shr.u32 %r92, %r5, 31;
add.s32 %r93, %r5, %r92;
shr.s32 %r19, %r93, 1;
setp.lt.s32 %p36, %r3, -1;
setp.lt.s32 %p37, %r5, -1;
or.pred %p38, %p37, %p36;
and.b32 %r94, %r93, -2;
setp.ge.s32 %p39, %r94, %r23;
or.pred %p40, %p38, %p39;
shr.u32 %r95, %r3, 31;
add.s32 %r96, %r3, %r95;
shr.s32 %r20, %r96, 1;
and.b32 %r97, %r96, -2;
setp.ge.s32 %p41, %r97, %r24;
or.pred %p42, %p40, %p41;
@%p42 bra $L__BB2_19;
div.rn.f32 %f86, %f102, %f104;
add.f32 %f87, %f86, 0f3F000000;
cvt.rzi.s32.f32 %r98, %f87;
div.rn.f32 %f88, %f103, %f104;
add.f32 %f89, %f88, 0f3F000000;
cvt.rzi.s32.f32 %r99, %f89;
cvt.rn.f32.s32 %f90, %r121;
div.rn.f32 %f91, %f104, %f90;
add.f32 %f92, %f91, 0f3F000000;
cvt.rzi.s32.f32 %r100, %f92;
mul.wide.s32 %rd30, %r20, %r22;
mul.wide.s32 %rd31, %r19, 2;
add.s64 %rd32, %rd30, %rd31;
cvta.to.global.u64 %rd33, %rd10;
add.s64 %rd34, %rd33, %rd32;
ld.global.u8 %r101, [%rd34];
mul.lo.s32 %r102, %r100, %r98;
mov.u32 %r103, 255;
sub.s32 %r104, %r103, %r100;
mad.lo.s32 %r105, %r104, %r101, %r102;
mul.hi.s32 %r106, %r105, -2139062143;
add.s32 %r107, %r106, %r105;
shr.u32 %r108, %r107, 31;
shr.u32 %r109, %r107, 7;
add.s32 %r110, %r109, %r108;
st.global.u8 [%rd34], %r110;
ld.global.u8 %r111, [%rd34+1];
mul.lo.s32 %r112, %r100, %r99;
mad.lo.s32 %r113, %r104, %r111, %r112;
mul.hi.s32 %r114, %r113, -2139062143;
add.s32 %r115, %r114, %r113;
shr.u32 %r116, %r115, 31;
shr.u32 %r117, %r115, 7;
add.s32 %r118, %r117, %r116;
st.global.u8 [%rd34+1], %r118;
$L__BB2_19:
ret;
}
+104 -168
View File
@@ -180,6 +180,7 @@ pub struct NvencEncoder {
/// This session opened as full-chroma 4:4:4 (FREXT) — via either input path. /// This session opened as full-chroma 4:4:4 (FREXT) — via either input path.
want_444: bool, want_444: bool,
src_format: PixelFormat, src_format: PixelFormat,
expand: bool,
width: u32, width: u32,
height: u32, height: u32,
fps: u32, fps: u32,
@@ -412,6 +413,57 @@ impl NvencEncoder {
None None
}; };
// CPU CSC paths: build the packed-RGB → planar swscale (no rescale) into the encoder's
// input frame. Two users: 4:4:4 (RGB→YUV444P, BT.709, range per the flag) and HDR
// (X2RGB10/X2BGR10→P010, BT.2020 limited — the PQ transfer is per-channel and rides
// through the matrix untouched). Skipped on the zero-copy path (`cuda`): the worker's GPU
// convert already delivers ready CUDA frames — no CPU pixels exist to scale.
let sws_csc = if (want_444 || want_hdr10) && !cuda {
let src_av = pixel_to_av(sws_src_pixel(format)?);
let dst_av = pixel_to_av(nvenc_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated packed-RGB source), the dst is
// YUV444P (4:4:4) or P010LE (HDR). The trailing filter/param pointers are null = "use
// defaults" (documented as accepted). No Rust memory is borrowed; the returned pointer is
// null-checked below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
height as c_int,
src_av,
width as c_int,
height as c_int,
dst_av,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
ptr::null(),
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→{nvenc_pixel:?}) failed");
}
// SAFETY: `sws` is the non-null context from the call above (null-checked). The
// coefficient tables from `sws_getCoefficients` (ITU-709 for 4:4:4, BT.2020 NCL for HDR
// — matching the VUI written above) are process-lifetime libswscale statics, reused for
// src+dst matrices; `sws_setColorspaceDetails` only reads them and writes scalar CSC
// settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
// PUNKTFUNK_444_FULLRANGE experiment; HDR is always limited). No Rust memory is passed.
unsafe {
let cs = ffi::sws_getCoefficients(if want_hdr10 {
super::libav::SWS_CS_BT2020
} else {
SWS_CS_ITU709
});
let dst_range = i32::from(full_range_444);
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, dst_range, 0, 1 << 16, 1 << 16);
}
Some(sws)
} else {
None
};
// Low-latency NVENC tuning (plan §7 / linux-setup doc). // Low-latency NVENC tuning (plan §7 / linux-setup doc).
let mut opts = Dictionary::new(); let mut opts = Dictionary::new();
opts.set("preset", "p1"); // fastest opts.set("preset", "p1"); // fastest
@@ -439,19 +491,15 @@ impl NvencEncoder {
} }
// Split-frame encode across both NVENC engines (GB203 has 2) when the pixel rate exceeds // Split-frame encode across both NVENC engines (GB203 has 2) when the pixel rate exceeds
// a single engine's HEVC capacity; e.g. 5120x1440@240 = 1.77 Gpix/s needs it, @120 // a single engine's HEVC capacity (~1 Gpix/s); e.g. 5120x1440@240 = 1.77 Gpix/s needs it,
// (0.88 Gpix/s) does not. HEVC/AV1 only (not H.264). AUTO won't engage below ~2112px // @120 = 0.88 Gpix/s does not. HEVC/AV1 only (not H.264). AUTO won't engage below ~2112px
// height, so we force `2`; below the threshold we leave it AUTO (split costs ~2% BD-rate). // height, so we force `2`; below the threshold we leave it AUTO (split costs ~2% BD-rate).
// Threshold shared with the direct-SDK selector ([`super::SPLIT_FORCE_PIXEL_RATE`] — set // Output is standard HEVC — transparent to the client. Override with PUNKTFUNK_SPLIT_ENCODE.
// so 4K120 = 995.3 Mpix/s forces, which `> 1e9` famously missed by 0.47%). Output is
// standard HEVC — transparent to the client. Override with PUNKTFUNK_SPLIT_ENCODE.
let pix_rate = width as u64 * height as u64 * fps as u64; let pix_rate = width as u64 * height as u64 * fps as u64;
let split = std::env::var("PUNKTFUNK_SPLIT_ENCODE").ok(); let split = std::env::var("PUNKTFUNK_SPLIT_ENCODE").ok();
match split.as_deref() { match split.as_deref() {
Some(mode) => opts.set("split_encode_mode", mode), Some(mode) => opts.set("split_encode_mode", mode),
None if matches!(codec, Codec::H265 | Codec::Av1) None if matches!(codec, Codec::H265 | Codec::Av1) && pix_rate > 1_000_000_000 => {
&& pix_rate >= super::SPLIT_FORCE_PIXEL_RATE =>
{
opts.set("split_encode_mode", "2"); opts.set("split_encode_mode", "2");
tracing::info!( tracing::info!(
pix_rate, pix_rate,
@@ -501,84 +549,6 @@ impl NvencEncoder {
); );
} }
// Built HERE, below the fallible encoder open, NOT above it. `sws_getContext` returns a raw
// pointer whose only free is `Drop for NvencEncoder` — and `Drop` needs a CONSTRUCTED
// `Self`, which does not exist on `open`'s early returns (the intra-refresh-unsupported
// retry, which recurses into `Self::open`, and the plain error return). Creating the
// context above them leaked one per failed attempt, and `open_nvenc_probed`'s EINVAL
// bitrate ladder calls `open` up to ~10 times, so a host stepping its bitrate down leaked a
// context per step. Nothing between here and the `Ok(NvencEncoder { … })` below can return,
// so this placement makes the leak unrepresentable rather than merely unlikely.
// CPU CSC paths: build the packed-RGB → planar swscale (no rescale) into the encoder's
// input frame. THREE users: 4:4:4 (RGB→YUV444P, BT.709, range per the flag), HDR
// (X2RGB10/X2BGR10→P010, BT.2020 limited — the PQ transfer is per-channel and rides
// through the matrix untouched), and the packed 3-bpp expand (RGB24/BGR24→rgb0/bgr0).
//
// The expand used to be a hand-written per-pixel loop in `submit_cpu`: `w*h` iterations,
// each building two bounds-checked sub-slices for a 3-byte copy — a shape LLVM will not
// vectorise into the byte shuffle it is, on the COMMON CPU path (the portal and wlroots
// both commonly fixate packed 24-bit RGB, and pf-capture offers it first). swscale's
// packed-RGB expanders are SIMD, the sibling VAAPI backend already routes RGB24/BGR24
// through them, and this file already owned the context lifecycle — so the change is net
// subtractive. The three are mutually exclusive by construction: `expand` is only ever
// true on the packed-RGB 4:2:0 path (see `nvenc_pixel`/`expand` above), never with
// `want_444`, so one context serves whichever applies.
//
// Skipped on the zero-copy path (`cuda`): the worker's GPU convert already delivers ready
// CUDA frames — no CPU pixels exist to scale.
let sws_csc = if (want_444 || want_hdr10 || expand) && !cuda {
let src_av = pixel_to_av(sws_src_pixel(format)?);
let dst_av = pixel_to_av(nvenc_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated packed-RGB source), the dst is
// YUV444P (4:4:4) or P010LE (HDR). The trailing filter/param pointers are null = "use
// defaults" (documented as accepted). No Rust memory is borrowed; the returned pointer is
// null-checked below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
height as c_int,
src_av,
width as c_int,
height as c_int,
dst_av,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
ptr::null(),
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→{nvenc_pixel:?}) failed");
}
// Colour math applies to the CSC users ONLY. The expand is a pure byte shuffle —
// packed 3-bpp RGB/BGR to the same channels in 4 bytes, `nvenc_pixel` being `rgb0`/
// `bgr0` — and NVENC does the RGB→YUV itself downstream. Handing it a matrix + range
// here would silently range-convert every packed-RGB session, which is exactly what the
// module header promises does not happen ("no colour math").
if want_444 || want_hdr10 {
// SAFETY: `sws` is the non-null context from the call above (null-checked). The
// coefficient tables from `sws_getCoefficients` (ITU-709 for 4:4:4, BT.2020 NCL for
// HDR — matching the VUI written above) are process-lifetime libswscale statics,
// reused for src+dst matrices; `sws_setColorspaceDetails` only reads them and writes
// scalar CSC settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
// PUNKTFUNK_444_FULLRANGE experiment; HDR is always limited). No Rust memory is passed.
unsafe {
let cs = ffi::sws_getCoefficients(if want_hdr10 {
super::libav::SWS_CS_BT2020
} else {
SWS_CS_ITU709
});
let dst_range = i32::from(full_range_444);
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, dst_range, 0, 1 << 16, 1 << 16);
}
}
Some(sws)
} else {
None
};
let frame = if cuda { let frame = if cuda {
None None
} else { } else {
@@ -591,6 +561,7 @@ impl NvencEncoder {
sws_csc, sws_csc,
want_444, want_444,
src_format: format, src_format: format,
expand,
width, width,
height, height,
fps, fps,
@@ -620,9 +591,6 @@ impl NvencEncoder {
impl Encoder for NvencEncoder { impl Encoder for NvencEncoder {
fn caps(&self) -> super::EncoderCaps { fn caps(&self) -> super::EncoderCaps {
super::EncoderCaps { super::EncoderCaps {
// libav NVENC hands the frame straight to the encoder — `frame.cursor` is never read,
// so a cursor-as-metadata session loses its pointer on this backend (audit finding).
blends_cursor: false,
// 4:4:4 iff this session opened FREXT — the CPU swscale path or the zero-copy GPU // 4:4:4 iff this session opened FREXT — the CPU swscale path or the zero-copy GPU
// convert. RFI/HDR-SEI stay unsupported on libavcodec NVENC (the trait defaults). // convert. RFI/HDR-SEI stay unsupported on libavcodec NVENC (the trait defaults).
chroma_444: self.want_444, chroma_444: self.want_444,
@@ -727,10 +695,8 @@ impl NvencEncoder {
bytes.len(), bytes.len(),
src_row * h src_row * h
); );
// swscale the packed RGB straight into the encoder's input frame, then send it. Serves all // 4:4:4: swscale the packed RGB straight into the planar YUV444P input frame (BT.709 limited),
// three CSC users (see `open`): 4:4:4 → planar YUV444P, HDR → P010, and the packed 3-bpp // then send it — no byte-expand. The 4:2:0 RGB path (below) feeds NVENC packed RGB directly.
// expand → `rgb0`/`bgr0`. The remaining branch below is the 4-bpp source, which needs no
// conversion at all — just a row copy honouring the destination stride.
if let Some(sws) = self.sws_csc { if let Some(sws) = self.sws_csc {
let frame = self let frame = self
.frame .frame
@@ -739,13 +705,10 @@ impl NvencEncoder {
// SAFETY: `format == self.src_format` and `bytes.len() >= src_row * h` (the `ensure!`s // SAFETY: `format == self.src_format` and `bytes.len() >= src_row * h` (the `ensure!`s
// above), so `sws_scale` reads `h` rows of `src_row` bytes from `src_data[0] = bytes` // above), so `sws_scale` reads `h` rows of `src_row` bytes from `src_data[0] = bytes`
// (packed RGB is single-plane; the other src planes are null/0) — all in bounds. `sws` is // (packed RGB is single-plane; the other src planes are null/0) — all in bounds. `sws` is
// the non-null context built in `open`. The dst is `frame`'s underlying `AVFrame`, whose // the non-null context built in `open`. The dst is `frame`'s underlying `AVFrame`: its
// `data`/`linesize` in-struct arrays were sized by `VideoFrame::new` for the very // `data`/`linesize` in-struct arrays were sized for YUV444P by `VideoFrame::new`, and the
// `nvenc_pixel` this context was built to output — 3 planes of `width`×`height` for // 3 planes are each `width`×`height`. All pointers are live locals for this synchronous
// YUV444P, 2 for P010, 1 packed plane for `rgb0`/`bgr0` — so swscale writes exactly the // call; the encoder runs only on this thread (`unsafe impl Send`), so no aliasing/race.
// planes it allocated, at the strides it reports. All pointers are live locals for this
// synchronous call; the encoder runs only on this thread (`unsafe impl Send`), so no
// aliasing/race.
unsafe { unsafe {
let dst_av = frame.as_mut_ptr(); let dst_av = frame.as_mut_ptr();
let src_data: [*const u8; 4] = let src_data: [*const u8; 4] =
@@ -761,7 +724,7 @@ impl NvencEncoder {
(*dst_av).linesize.as_ptr(), (*dst_av).linesize.as_ptr(),
); );
if r < 0 { if r < 0 {
bail!("sws_scale(CPU CSC → encoder input) failed ({r})"); bail!("sws_scale(RGB→YUV444P) failed ({r})");
} }
} }
frame.set_pts(Some(pts)); frame.set_pts(Some(pts));
@@ -770,7 +733,7 @@ impl NvencEncoder {
} else { } else {
ffmpeg::picture::Type::None ffmpeg::picture::Type::None
}); });
self.enc.send_frame(frame).context("send_frame(swscale)")?; self.enc.send_frame(frame).context("send_frame(444)")?;
return Ok(()); return Ok(());
} }
let frame = self let frame = self
@@ -779,9 +742,18 @@ impl NvencEncoder {
.context("CPU frame missing (encoder opened in CUDA mode)")?; .context("CPU frame missing (encoder opened in CUDA mode)")?;
let stride = frame.stride(0); // dst is 4-bpp, aligned let stride = frame.stride(0); // dst is 4-bpp, aligned
let dst = frame.data_mut(0); let dst = frame.data_mut(0);
{ if self.expand {
// 4-bpp → 4-bpp, honoring the (possibly larger) dst stride. The 3-bpp expand that used // packed 3-bpp RGB/BGR → 4-bpp *0 (copy 3 bytes, zero the pad byte)
// to live here as a per-pixel loop is now swscale's job (see the branch above). for y in 0..h {
let s = &bytes[y * src_row..y * src_row + src_row];
let drow = &mut dst[y * stride..y * stride + w * 4];
for x in 0..w {
drow[x * 4..x * 4 + 3].copy_from_slice(&s[x * 3..x * 3 + 3]);
drow[x * 4 + 3] = 0;
}
}
} else {
// 4-bpp → 4-bpp, honoring the (possibly larger) dst stride
for y in 0..h { for y in 0..h {
dst[y * stride..y * stride + src_row] dst[y * stride..y * stride + src_row]
.copy_from_slice(&bytes[y * src_row..y * src_row + src_row]); .copy_from_slice(&bytes[y * src_row..y * src_row + src_row]);
@@ -854,7 +826,7 @@ impl NvencEncoder {
(*f).linesize[i] as usize, (*f).linesize[i] as usize,
) )
}); });
pf_zerocopy::cuda::copy_yuv444_to_device(buf, dsts, true) pf_zerocopy::cuda::copy_yuv444_to_device(buf, dsts)
} else if self.want_444 { } else if self.want_444 {
ffi::av_frame_free(&mut f); ffi::av_frame_free(&mut f);
bail!( bail!(
@@ -867,11 +839,11 @@ impl NvencEncoder {
let y_pitch = (*f).linesize[0] as usize; let y_pitch = (*f).linesize[0] as usize;
let uv_ptr = (*f).data[1] as pf_zerocopy::cuda::CUdeviceptr; let uv_ptr = (*f).data[1] as pf_zerocopy::cuda::CUdeviceptr;
let uv_pitch = (*f).linesize[1] as usize; let uv_pitch = (*f).linesize[1] as usize;
pf_zerocopy::cuda::copy_nv12_to_device(buf, y_ptr, y_pitch, uv_ptr, uv_pitch, true) pf_zerocopy::cuda::copy_nv12_to_device(buf, y_ptr, y_pitch, uv_ptr, uv_pitch)
} else { } else {
let dst_ptr = (*f).data[0] as pf_zerocopy::cuda::CUdeviceptr; let dst_ptr = (*f).data[0] as pf_zerocopy::cuda::CUdeviceptr;
let dst_pitch = (*f).linesize[0] as usize; let dst_pitch = (*f).linesize[0] as usize;
pf_zerocopy::cuda::copy_device_to_device(buf, dst_ptr, dst_pitch, true) pf_zerocopy::cuda::copy_device_to_device(buf, dst_ptr, dst_pitch)
}; };
if let Err(e) = copy_res { if let Err(e) = copy_res {
ffi::av_frame_free(&mut f); ffi::av_frame_free(&mut f);
@@ -904,58 +876,6 @@ impl Drop for NvencEncoder {
} }
} }
/// Serialises the save → `AV_LOG_FATAL` → restore window that every capability probe opens around
/// an encoder open it *expects* to fail.
///
/// libav's log level is one process-global `int`, and the probes race each other for real: the
/// NVENC and VAAPI 4:4:4/10-bit probes are reached from `/serverinfo` and from session bring-up.
/// Two overlapping save/restore pairs interleave as get(INFO) → get(FATAL) → set(INFO) →
/// set(FATAL), and the process is then pinned at `AV_LOG_FATAL` for good — every later libav
/// diagnostic silently dropped, which is precisely the logging you want when a stream later fails
/// to open. The probes run process-once and already cost a real encoder open, so serialising them
/// costs nothing measurable.
static LIBAV_LOG_LEVEL: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// RAII quiet-window over libav's global log level: drops it to `AV_LOG_FATAL` on construction and
/// restores the previous level on drop, holding [`LIBAV_LOG_LEVEL`] for the whole window.
///
/// Callers must have completed `ffmpeg::init()` first. Not re-entrant — no probe may construct a
/// second guard while holding one (none do; the probe bodies only reach encoder-open helpers).
/// `pub(crate)` so the VAAPI probes share the one lock: they race the NVENC probes on the same
/// global.
pub(crate) struct QuietLibavLog {
prev: c_int,
// Held for the lifetime of the guard. `Drop for QuietLibavLog` runs before the struct's fields
// are dropped, so the restore below still happens under the lock.
_lock: std::sync::MutexGuard<'static, ()>,
}
impl QuietLibavLog {
pub(crate) fn new() -> Self {
// Poison-tolerant: a probe that panicked mid-window already restored the level via `Drop`,
// and refusing the lock forever afterwards would be a worse outcome than proceeding.
let lock = LIBAV_LOG_LEVEL
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
// SAFETY: libav is initialized by the caller; `av_log_{get,set}_level` only read/write the
// global int level (no pointer args) and are always sound post-init.
let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
Self { prev, _lock: lock }
}
}
impl Drop for QuietLibavLog {
fn drop(&mut self) {
// SAFETY: restore the saved global level (scalar arg, no pointers); libav was initialized
// before this guard was constructed.
unsafe { ffi::av_log_set_level(self.prev) };
}
}
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode HEVC **4:4:4** (Range /// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode HEVC **4:4:4** (Range
/// Extensions). Opens a tiny real `hevc_nvenc` 4:4:4 session — the exact path [`NvencEncoder::open`] /// Extensions). Opens a tiny real `hevc_nvenc` 4:4:4 session — the exact path [`NvencEncoder::open`]
/// takes for a live 4:4:4 stream — and reports whether it succeeded. HEVC-only; the result is cached /// takes for a live 4:4:4 stream — and reports whether it succeeded. HEVC-only; the result is cached
@@ -969,9 +889,14 @@ pub fn probe_can_encode_444(codec: Codec) -> bool {
return false; return false;
} }
// Quiet ffmpeg's open error on a GPU that lacks 4:4:4 — the probe failing is an expected outcome. // Quiet ffmpeg's open error on a GPU that lacks 4:4:4 — the probe failing is an expected outcome.
// Held until the function returns, so the level is restored after the open either way. // SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
let _quiet = QuietLibavLog::new(); // (no pointer args) and are always sound post-init.
NvencEncoder::open( let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
let ok = NvencEncoder::open(
codec, codec,
PixelFormat::Bgra, PixelFormat::Bgra,
640, 640,
@@ -982,7 +907,10 @@ pub fn probe_can_encode_444(codec: Codec) -> bool {
8, 8,
ChromaFormat::Yuv444, ChromaFormat::Yuv444,
) )
.is_ok() .is_ok();
// SAFETY: restore the saved global log level (scalar arg, no pointers).
unsafe { ffi::av_log_set_level(prev) };
ok
} }
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode 10-bit (HEVC Main10 / /// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode 10-bit (HEVC Main10 /
@@ -998,9 +926,14 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
return false; return false;
} }
// Quiet ffmpeg's open error on a GPU that lacks 10-bit — the probe failing is an expected outcome. // Quiet ffmpeg's open error on a GPU that lacks 10-bit — the probe failing is an expected outcome.
// Held until the function returns, so the level is restored after the open either way. // SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
let _quiet = QuietLibavLog::new(); // (no pointer args) and are always sound post-init.
NvencEncoder::open( let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
let ok = NvencEncoder::open(
codec, codec,
PixelFormat::X2Rgb10, PixelFormat::X2Rgb10,
640, 640,
@@ -1011,7 +944,10 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
10, 10,
ChromaFormat::Yuv420, ChromaFormat::Yuv420,
) )
.is_ok() .is_ok();
// SAFETY: restore the saved global log level (scalar arg, no pointers).
unsafe { ffi::av_log_set_level(prev) };
ok
} }
#[cfg(test)] #[cfg(test)]
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+53 -92
View File
@@ -28,18 +28,22 @@ use ffmpeg::format::Pixel;
use ffmpeg::{codec, encoder, Dictionary}; use ffmpeg::{codec, encoder, Dictionary};
use ffmpeg_next as ffmpeg; use ffmpeg_next as ffmpeg;
use pf_frame::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat}; use pf_frame::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat};
use std::collections::HashMap;
use std::ffi::{CStr, CString}; use std::ffi::{CStr, CString};
use std::os::fd::AsRawFd; use std::os::fd::AsRawFd;
use std::os::raw::c_int; use std::os::raw::c_int;
use std::ptr; use std::ptr;
use std::sync::{Mutex, OnceLock}; use std::sync::atomic::{AtomicU8, Ordering};
use super::libav::{ use super::libav::{
apply_low_latency_rc, pixel_to_av, poll_encoder, PollOutcome, SWS_CS_ITU709, SWS_POINT, apply_low_latency_rc, pixel_to_av, poll_encoder, PollOutcome, SWS_CS_ITU709, SWS_POINT,
}; };
use ffmpeg::ffi; // = ffmpeg_sys_next use ffmpeg::ffi; // = ffmpeg_sys_next
/// `fourcc(a,b,c,d)` — DRM FourCC packing (`a | b<<8 | c<<16 | d<<24`).
const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
(a as u32) | ((b as u32) << 8) | ((c as u32) << 16) | ((d as u32) << 24)
}
/// The render node a VAAPI/DRM device should open, from [`pf_gpu::linux_render_node`]: a /// The render node a VAAPI/DRM device should open, from [`pf_gpu::linux_render_node`]: a
/// matched web-console GPU preference pins it, else `PUNKTFUNK_RENDER_NODE`, else the single-GPU /// matched web-console GPU preference pins it, else `PUNKTFUNK_RENDER_NODE`, else the single-GPU
/// default. /// default.
@@ -67,38 +71,21 @@ fn vaapi_sws_src(format: PixelFormat) -> Result<Pixel> {
}) })
} }
/// Which VAAPI entrypoint mode opened successfully: 1 = default (full-feature `EncSlice`), /// Which VAAPI entrypoint mode opened successfully, cached per codec (index = [`lp_idx`]):
/// 2 = low-power (`EncSliceLP`/VDEnc). Modern Intel (Gen12+/Arc) removed the full-feature encode /// 0 = unknown, 1 = default (full-feature `EncSlice`), 2 = low-power (`EncSliceLP`/VDEnc).
/// entrypoints, so the default open fails there and only `low_power=1` works; AMD (radeonsi) is the /// Modern Intel (Gen12+/Arc) removed the full-feature encode entrypoints, so the default open
/// reverse. Caching the resolved mode lets later sessions/probes skip the known-failing attempt /// fails there and only `low_power=1` works; AMD (radeonsi) is the reverse. Caching the resolved
/// (and its libav error spew). /// mode lets later sessions/probes skip the known-failing attempt (and its libav error spew).
/// static LP_MODE: [AtomicU8; 3] = [AtomicU8::new(0), AtomicU8::new(0), AtomicU8::new(0)];
/// Keyed on **(render node, codec, bit depth)**, and every part of that key is load-bearing:
///
/// * The render node, because the entrypoint is a property of the DEVICE libva opens — and
/// `render_node()` follows the web-console GPU preference. This used to be a process-global array
/// keyed by codec alone, which made it a session-killer rather than a staleness bug: once a mode
/// is latched the open tries exactly ONE mode and the `[false, true]` fallback is gone. Latch
/// low-power on an Intel Arc, switch the preference to an AMD dGPU, and every VAAPI open there
/// passes `low_power=1` — which radeonsi rejects — with no full-feature retry, for the process
/// lifetime: the probe reports all-false AND the session's own encoder open fails.
/// Keyed on the node rather than `pf_gpu::selection_key()` on purpose — the node is literally what
/// `render_node()` hands libva, so key and device cannot describe different GPUs.
/// * The bit depth, because Main10 and 8-bit can resolve to different entrypoints on the same
/// device; one shared slot let an 8-bit answer pin the 10-bit open (HDR under-advertisement).
static LP_MODE: OnceLock<Mutex<HashMap<LpKey, u8>>> = OnceLock::new();
/// (render-node path, codec label, 10-bit) — see [`LP_MODE`]. fn lp_idx(codec: Codec) -> usize {
type LpKey = (String, &'static str, bool); match codec {
Codec::H264 => 0,
/// The [`LP_MODE`] key for this device/codec/depth. `render_node()` is re-read rather than cached Codec::H265 => 1,
/// so a GPU-preference change is picked up on the next open. Codec::Av1 => 2,
fn lp_key(codec: Codec, ten_bit: bool) -> LpKey { // Guarded by the open_video dispatch: PyroWave never opens the VAAPI backend.
( Codec::PyroWave => unreachable!("PyroWave has no VAAPI encoder"),
render_node().to_string_lossy().into_owned(), }
codec.label(),
ten_bit,
)
} }
/// `PUNKTFUNK_VAAPI_LOW_POWER` pins the entrypoint mode (`1` = low-power only, `0` = full-feature /// `PUNKTFUNK_VAAPI_LOW_POWER` pins the entrypoint mode (`1` = low-power only, `0` = full-feature
@@ -129,16 +116,11 @@ unsafe fn open_vaapi_encoder(
frames_ref: *mut ffi::AVBufferRef, frames_ref: *mut ffi::AVBufferRef,
ten_bit: bool, ten_bit: bool,
) -> Result<encoder::video::Encoder> { ) -> Result<encoder::video::Encoder> {
let key = lp_key(codec, ten_bit); let idx = lp_idx(codec);
let cached = LP_MODE
.get_or_init(|| Mutex::new(HashMap::new()))
.lock()
.map(|m| m.get(&key).copied().unwrap_or(0))
.unwrap_or(0);
let modes: &[bool] = match low_power_override() { let modes: &[bool] = match low_power_override() {
Some(true) => &[true], Some(true) => &[true],
Some(false) => &[false], Some(false) => &[false],
None => match cached { None => match LP_MODE[idx].load(Ordering::Relaxed) {
1 => &[false], 1 => &[false],
2 => &[true], 2 => &[true],
_ => &[false, true], _ => &[false, true],
@@ -158,9 +140,7 @@ unsafe fn open_vaapi_encoder(
lp, lp,
) { ) {
Ok(enc) => { Ok(enc) => {
if let Ok(mut m) = LP_MODE.get_or_init(|| Mutex::new(HashMap::new())).lock() { LP_MODE[idx].store(if lp { 2 } else { 1 }, Ordering::Relaxed);
m.insert(key.clone(), if lp { 2 } else { 1 });
}
if lp { if lp {
tracing::info!( tracing::info!(
encoder = codec.vaapi_name(), encoder = codec.vaapi_name(),
@@ -273,20 +253,20 @@ pub fn probe_can_encode(codec: Codec) -> bool {
if ffmpeg::init().is_err() { if ffmpeg::init().is_err() {
return false; return false;
} }
// A missing VA device (non-VAAPI host, GPU-less CI) is an expected probe outcome — quiet // SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `av_log_get_level`/
// ffmpeg's "No VA display found" error for the probe. Held until the function returns, so the // `av_log_set_level` only read/write libav's global integer log level (no pointer args) and are
// level is restored after the open either way. Shares one lock with the NVENC probes, which // always sound to call post-init. `VaapiHw::new` (an `unsafe fn`) builds a VAAPI device + NV12
// race this one on the same libav global (see [`crate::linux::QuietLibavLog`]). // frames pool from the literal NV12/640x480/pool=2 args and hands back a RAII handle that unrefs
let _quiet = crate::linux::QuietLibavLog::new(); // both `AVBufferRef`s on drop. `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `VaapiHw::new` (an // `hw.frames_ref` — the two non-null refs `VaapiHw::new` just created — and `av_buffer_ref`s them
// `unsafe fn`) builds a VAAPI device + NV12 frames pool from the literal NV12/640x480/pool=2 // into the encoder; `hw` is a live local for the whole match arm, so the borrows outlive the
// args and hands back a RAII handle that unrefs both `AVBufferRef`s on drop. // synchronous call, and both `hw` and the probe encoder are dropped (RAII) when the arm ends.
// `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` — the two
// non-null refs `VaapiHw::new` just created — and `av_buffer_ref`s them into the encoder; `hw`
// is a live local for the whole match arm, so the borrows outlive the synchronous call, and
// both `hw` and the probe encoder are dropped (RAII) when the arm ends.
unsafe { unsafe {
match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, 640, 480, 2) { // A missing VA device (non-VAAPI host, GPU-less CI) is an expected probe outcome — quiet
// ffmpeg's "No VA display found" error for the probe, then restore the level.
let prev = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, 640, 480, 2) {
Ok(hw) => open_vaapi_encoder( Ok(hw) => open_vaapi_encoder(
codec, codec,
640, 640,
@@ -299,7 +279,9 @@ pub fn probe_can_encode(codec: Codec) -> bool {
) )
.is_ok(), .is_ok(),
Err(_) => false, Err(_) => false,
} };
ffi::av_log_set_level(prev);
ok
} }
} }
@@ -315,17 +297,18 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
if ffmpeg::init().is_err() { if ffmpeg::init().is_err() {
return false; return false;
} }
// A missing VA device / no Main10 entrypoint is an expected probe outcome — quiet ffmpeg's // SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `av_log_{get,set}_level`
// error for the probe. Held until the function returns, so the level is restored after the open // only read/write libav's global integer log level (no pointer args). `VaapiHw::new` (an
// either way, and shared with the other probes (see [`crate::linux::QuietLibavLog`]).
let _quiet = crate::linux::QuietLibavLog::new();
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `VaapiHw::new` (an
// `unsafe fn`) builds a VAAPI device + P010 frames pool from the literal args and hands back a // `unsafe fn`) builds a VAAPI device + P010 frames pool from the literal args and hands back a
// RAII handle; `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` — // RAII handle; `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` —
// the two non-null refs `VaapiHw::new` just created, live locals for the whole match arm — and // the two non-null refs `VaapiHw::new` just created, live locals for the whole match arm — and
// `av_buffer_ref`s them into the probe encoder. Both `hw` and the encoder drop (RAII) at arm end. // `av_buffer_ref`s them into the probe encoder. Both `hw` and the encoder drop (RAII) at arm end.
unsafe { unsafe {
match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_P010LE, 640, 480, 2) { // A missing VA device / no Main10 entrypoint is an expected probe outcome — quiet ffmpeg's
// error for the probe, then restore the level.
let prev = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_P010LE, 640, 480, 2) {
Ok(hw) => open_vaapi_encoder( Ok(hw) => open_vaapi_encoder(
codec, codec,
640, 640,
@@ -338,7 +321,9 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
) )
.is_ok(), .is_ok(),
Err(_) => false, Err(_) => false,
} };
ffi::av_log_set_level(prev);
ok
} }
} }
@@ -949,8 +934,7 @@ impl DmabufInner {
// `Box` puts it on the heap with a unique owner. // `Box` puts it on the heap with a unique owner.
// * `dmabuf.fd.as_raw_fd()` is the fd of the caller's `&DmabufFrame`, which owns it for the // * `dmabuf.fd.as_raw_fd()` is the fd of the caller's `&DmabufFrame`, which owns it for the
// whole synchronous `submit`; we describe one object/layer/plane from its // whole synchronous `submit`; we describe one object/layer/plane from its
// fourcc/modifier/offset/stride and its `lseek`-queried size. `libc::lseek` on that live // fourcc/modifier/offset/stride and pass `object.size = 0` (ffmpeg queries the real size).
// fd only reads the description's size and returns it (or -1); it touches no Rust memory.
// * `av_frame_alloc` → `drm` (null-checked); we set its scalar fields and // * `av_frame_alloc` → `drm` (null-checked); we set its scalar fields and
// `hw_frames_ctx = av_buffer_ref(self.drm_frames)` (new ref of the live owned ctx). // `hw_frames_ctx = av_buffer_ref(self.drm_frames)` (new ref of the live owned ctx).
// * `data[0] = Box::into_raw(desc)` transfers the box into the frame; `buf[0] = // * `data[0] = Box::into_raw(desc)` transfers the box into the frame; `buf[0] =
@@ -966,17 +950,7 @@ impl DmabufInner {
let mut desc: Box<ffi::AVDRMFrameDescriptor> = Box::new(std::mem::zeroed()); let mut desc: Box<ffi::AVDRMFrameDescriptor> = Box::new(std::mem::zeroed());
desc.nb_objects = 1; desc.nb_objects = 1;
desc.objects[0].fd = dmabuf.fd.as_raw_fd(); desc.objects[0].fd = dmabuf.fd.as_raw_fd();
// The object's REAL size, not 0. libav does not query it for us — both of its import desc.objects[0].size = 0;
// paths hand this value straight to libva, as `prime_desc.objects[i].size` on the
// PRIME_2 path and `buffer_desc.data_size` on the legacy fallback — so a 0 told every
// VA driver the backing object was empty and left it to work the real size out itself.
// The drivers this has run on (radeonsi, modern Intel iHD) do; a Gen9 Intel host
// answered `vaCreateSurfaces` with VA_STATUS_ERROR_ALLOCATION_FAILED on every single
// frame. `lseek(SEEK_END)` is the standard dma-buf size query — the same one the
// Vulkan bridge already uses on these fds (`pf_zerocopy::imp::vulkan`). If a kernel
// refuses it, keep the old 0 rather than drop a frame we could still have encoded.
let obj_size = libc::lseek(dmabuf.fd.as_raw_fd(), 0, libc::SEEK_END);
desc.objects[0].size = if obj_size > 0 { obj_size as _ } else { 0 };
desc.objects[0].format_modifier = dmabuf.modifier; desc.objects[0].format_modifier = dmabuf.modifier;
desc.nb_layers = 1; desc.nb_layers = 1;
desc.layers[0].format = self.fourcc; desc.layers[0].format = self.fourcc;
@@ -1025,18 +999,8 @@ impl DmabufInner {
ffi::AV_BUFFERSRC_FLAG_KEEP_REF as c_int, ffi::AV_BUFFERSRC_FLAG_KEEP_REF as c_int,
); );
ffi::av_frame_free(&mut drm); ffi::av_frame_free(&mut drm);
// These two stages ARE the import: the push hands libav our DRM-PRIME descriptor, and
// the pull is where `hwmap` actually maps it into a VA surface (and `scale_vaapi` runs
// the CSC). A failure here means this driver would not take this compositor's dmabuf —
// which no encoder rebuild can fix — so tell the process-wide latch, and capture
// negotiates CPU frames from the next session on. `avcodec_send_frame` below is
// deliberately NOT counted: that one is the encoder stalling, which the in-place
// rebuild above us exists to recover, and disabling zero-copy over it would be a
// permanent penalty for a transient fault.
if r < 0 { if r < 0 {
let e = format!("av_buffersrc_add_frame failed ({r})"); bail!("av_buffersrc_add_frame failed ({r})");
pf_zerocopy::note_raw_dmabuf_import_failure(&e);
bail!("{e}");
} }
t_push = t0.elapsed(); t_push = t0.elapsed();
let mut nv12 = ffi::av_frame_alloc(); let mut nv12 = ffi::av_frame_alloc();
@@ -1046,11 +1010,8 @@ impl DmabufInner {
let r = ffi::av_buffersink_get_frame(self.sink, nv12); let r = ffi::av_buffersink_get_frame(self.sink, nv12);
if r < 0 { if r < 0 {
ffi::av_frame_free(&mut nv12); ffi::av_frame_free(&mut nv12);
let e = format!("av_buffersink_get_frame failed ({r})"); bail!("av_buffersink_get_frame failed ({r})");
pf_zerocopy::note_raw_dmabuf_import_failure(&e);
bail!("{e}");
} }
pf_zerocopy::note_raw_dmabuf_import_ok();
t_pull = t0.elapsed() - t_push; t_pull = t0.elapsed() - t_push;
(*nv12).pts = pts; (*nv12).pts = pts;
(*nv12).pict_type = if idr { (*nv12).pict_type = if idr {
@@ -47,15 +47,6 @@ pub const PRIMARY_REF_NONE: u8 = 7;
/// `VK_VIDEO_ENCODE_AV1_SUPERBLOCK_SIZE_128_BIT_KHR` (bit 1 of the superblock-size flags). /// `VK_VIDEO_ENCODE_AV1_SUPERBLOCK_SIZE_128_BIT_KHR` (bit 1 of the superblock-size flags).
pub const SUPERBLOCK_SIZE_128: u32 = 0x2; pub const SUPERBLOCK_SIZE_128: u32 = 0x2;
// `VkVideoEncodeAV1CapabilityFlagBitsKHR` — the two that decide whether the encode source may be a
// different size from the declared frame. Both absent on RADV PHOENIX.
/// Without this, the source's `codedExtent` MUST equal the sequence header's
/// `max_frame_{width,height}_minus_1 + 1` (`VUID-vkCmdEncodeVideoKHR-flags-10324`).
pub const CAPABILITY_FRAME_SIZE_OVERRIDE: u32 = 0x0000_0008;
/// Without this, EVERY reference slot's `codedExtent` MUST equal the source's
/// (`VUID-vkCmdEncodeVideoKHR-flags-10325`).
pub const CAPABILITY_MOTION_VECTOR_SCALING: u32 = 0x0000_0010;
// `VkVideoEncodeAV1PredictionModeKHR` // `VkVideoEncodeAV1PredictionModeKHR`
pub const PREDICTION_MODE_INTRA_ONLY: i32 = 0; pub const PREDICTION_MODE_INTRA_ONLY: i32 = 0;
pub const PREDICTION_MODE_SINGLE_REFERENCE: i32 = 1; pub const PREDICTION_MODE_SINGLE_REFERENCE: i32 = 1;
@@ -507,366 +498,3 @@ pub struct StdVideoEncodeAV1OperatingPointInfo {
pub fn stype(raw: i32) -> vk::StructureType { pub fn stype(raw: i32) -> vk::StructureType {
vk::StructureType::from_raw(raw) vk::StructureType::from_raw(raw)
} }
// ---------- ABI layout guard ----------
//
// These structs are hand-copied and handed to the driver through raw `p_next` chains, so nothing in
// the type system relates them to the C definitions any more: an edit that inserts, drops, widens or
// re-pads a field is not a compile error, it is the driver reading our bytes at the wrong offsets.
// The assertions below are the missing compile error. They are `const` rather than `#[cfg(test)]`
// (the shape `amf.rs` uses) so they hold in every build, including the shipped one, and on any
// target this module compiles for.
//
// What they catch: a changed field width, an inserted or removed field, a changed array length, a
// padding assumption that only holds on one target. What they CANNOT catch: swapping two fields of
// the same type — offsets are unchanged. That case is only caught by reading the registry, so the
// field order here was diffed against `vulkan_core.h` and `vk_video/vulkan_video_codec_av1std_encode.h`
// (Vulkan-Headers `main`, 2026-07-25) when these assertions were written, along with every `ST_*`,
// flag-bit and enum value above; the bitfield member order is pinned by the test module below.
//
// Deliberately duplicated in `vk_valve_rgb.rs` rather than shared: both modules exist to be deleted
// wholesale once `ash` ships these bindings, and a shared helper would make deleting one break the
// other.
macro_rules! assert_abi_layout {
($t:ty { size: $size:expr, align: $align:expr $(, $field:ident @ $off:expr)* $(,)? }) => {
const _: () = {
assert!(
::core::mem::size_of::<$t>() == $size,
concat!(stringify!($t), ": size does not match the C ABI")
);
assert!(
::core::mem::align_of::<$t>() == $align,
concat!(stringify!($t), ": alignment does not match the C ABI")
);
$(assert!(
::core::mem::offset_of!($t, $field) == $off,
concat!(stringify!($t), ".", stringify!($field), ": offset does not match the C ABI")
);)*
};
};
}
// Std encode structs. The three `*Flags` types are a single C `uint32_t` of bitfields, so only their
// size and alignment are layout-checkable here; their member order is covered by `abi_tests`.
assert_abi_layout!(StdVideoEncodeAV1PictureInfoFlags { size: 4, align: 4 });
assert_abi_layout!(StdVideoEncodeAV1PictureInfo {
size: 152, align: 8,
flags @ 0,
frame_type @ 4,
frame_presentation_time @ 8,
current_frame_id @ 12,
order_hint @ 16,
primary_ref_frame @ 17,
refresh_frame_flags @ 18,
coded_denom @ 19,
render_width_minus_1 @ 20,
render_height_minus_1 @ 22,
interpolation_filter @ 24,
TxMode @ 28,
delta_q_res @ 32,
delta_lf_res @ 33,
ref_order_hint @ 34,
ref_frame_idx @ 42,
reserved1 @ 49,
delta_frame_id_minus_1 @ 52,
pTileInfo @ 80,
pQuantization @ 88,
pSegmentation @ 96,
pLoopFilter @ 104,
pCDEF @ 112,
pLoopRestoration @ 120,
pGlobalMotion @ 128,
pExtensionHeader @ 136,
pBufferRemovalTimes @ 144,
});
assert_abi_layout!(StdVideoEncodeAV1ReferenceInfoFlags { size: 4, align: 4 });
assert_abi_layout!(StdVideoEncodeAV1ReferenceInfo {
size: 24, align: 8,
flags @ 0,
RefFrameId @ 4,
frame_type @ 8,
OrderHint @ 12,
reserved1 @ 13,
pExtensionHeader @ 16,
});
assert_abi_layout!(StdVideoEncodeAV1ExtensionHeader {
size: 2, align: 1,
temporal_id @ 0,
spatial_id @ 1,
});
assert_abi_layout!(StdVideoEncodeAV1OperatingPointInfoFlags { size: 4, align: 4 });
assert_abi_layout!(StdVideoEncodeAV1OperatingPointInfo {
size: 20, align: 4,
flags @ 0,
operating_point_idc @ 4,
seq_level_idx @ 6,
seq_tier @ 7,
decoder_buffer_delay @ 8,
encoder_buffer_delay @ 12,
initial_display_delay_minus_1 @ 16,
});
// KHR extension structs.
assert_abi_layout!(VideoEncodeAV1ProfileInfoKHR {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
std_profile @ 16,
});
assert_abi_layout!(PhysicalDeviceVideoEncodeAV1FeaturesKHR {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
video_encode_av1 @ 16,
});
assert_abi_layout!(VideoEncodeAV1CapabilitiesKHR {
size: 128, align: 8,
s_type @ 0,
p_next @ 8,
flags @ 16,
max_level @ 20,
coded_picture_alignment @ 24,
max_tiles @ 32,
min_tile_size @ 40,
max_tile_size @ 48,
superblock_sizes @ 56,
max_single_reference_count @ 60,
single_reference_name_mask @ 64,
max_unidirectional_compound_reference_count @ 68,
max_unidirectional_compound_group1_reference_count @ 72,
unidirectional_compound_reference_name_mask @ 76,
max_bidirectional_compound_reference_count @ 80,
max_bidirectional_compound_group1_reference_count @ 84,
max_bidirectional_compound_group2_reference_count @ 88,
bidirectional_compound_reference_name_mask @ 92,
max_temporal_layer_count @ 96,
max_spatial_layer_count @ 100,
max_operating_points @ 104,
min_q_index @ 108,
max_q_index @ 112,
prefers_gop_remaining_frames @ 116,
requires_gop_remaining_frames @ 120,
std_syntax_flags @ 124,
});
assert_abi_layout!(VideoEncodeAV1SessionParametersCreateInfoKHR {
size: 48, align: 8,
s_type @ 0,
p_next @ 8,
p_std_sequence_header @ 16,
p_std_decoder_model_info @ 24,
std_operating_point_count @ 32,
p_std_operating_points @ 40,
});
assert_abi_layout!(VideoEncodeAV1PictureInfoKHR {
size: 80, align: 8,
s_type @ 0,
p_next @ 8,
prediction_mode @ 16,
rate_control_group @ 20,
constant_q_index @ 24,
p_std_picture_info @ 32,
reference_name_slot_indices @ 40,
primary_reference_cdf_only @ 68,
generate_obu_extension_header @ 72,
});
assert_abi_layout!(VideoEncodeAV1DpbSlotInfoKHR {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
p_std_reference_info @ 16,
});
assert_abi_layout!(VideoEncodeAV1RateControlInfoKHR {
size: 40, align: 8,
s_type @ 0,
p_next @ 8,
flags @ 16,
gop_frame_count @ 20,
key_frame_period @ 24,
consecutive_bipredictive_frame_count @ 28,
temporal_layer_count @ 32,
});
assert_abi_layout!(VideoEncodeAV1QIndexKHR {
size: 12, align: 4,
intra_q_index @ 0,
predictive_q_index @ 4,
bipredictive_q_index @ 8,
});
assert_abi_layout!(VideoEncodeAV1FrameSizeKHR {
size: 12, align: 4,
intra_frame_size @ 0,
predictive_frame_size @ 4,
bipredictive_frame_size @ 8,
});
assert_abi_layout!(VideoEncodeAV1RateControlLayerInfoKHR {
size: 64, align: 8,
s_type @ 0,
p_next @ 8,
use_min_q_index @ 16,
min_q_index @ 20,
use_max_q_index @ 32,
max_q_index @ 36,
use_max_frame_size @ 48,
max_frame_size @ 52,
});
assert_abi_layout!(VideoEncodeAV1GopRemainingFrameInfoKHR {
size: 32, align: 8,
s_type @ 0,
p_next @ 8,
use_gop_remaining_frames @ 16,
gop_remaining_intra @ 20,
gop_remaining_predictive @ 24,
gop_remaining_bipredictive @ 28,
});
assert_abi_layout!(VideoEncodeAV1SessionCreateInfoKHR {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
use_max_level @ 16,
max_level @ 20,
});
#[cfg(test)]
mod abi_tests {
use super::*;
/// Assert that `set` lights exactly one bit of the flags word, at `bit`.
fn sets_only_bit(
storage: __BindgenBitfieldUnit<[u8; 4]>,
name: &str,
bit: usize,
expected_width: usize,
) {
for probe in 0..32 {
let want = probe >= bit && probe < bit + expected_width;
assert_eq!(
storage.get_bit(probe),
want,
"`{name}` should occupy bit(s) {bit}..{}, but bit {probe} disagrees",
bit + expected_width
);
}
}
/// A C bitfield allocates its members from bit 0 upward in declaration order, so the setter for
/// the Nth member of `StdVideoEncodeAV1PictureInfoFlags` must write bit N. The array below is
/// the member list of `vulkan_video_codec_av1std_encode.h` **in declaration order** — so this
/// pins the hand-copied bit indices to the header rather than merely to themselves. A silent
/// renumbering here would make the driver read, say, `use_superres` where we meant
/// `render_and_frame_size_different`.
#[test]
fn picture_info_flag_setters_follow_the_c_declaration_order() {
#[allow(clippy::type_complexity)]
let members: [(&str, fn(&mut StdVideoEncodeAV1PictureInfoFlags)); 29] = [
("error_resilient_mode", |f| f.set_error_resilient_mode(1)),
("disable_cdf_update", |f| f.set_disable_cdf_update(1)),
("use_superres", |f| f.set_use_superres(1)),
("render_and_frame_size_different", |f| {
f.set_render_and_frame_size_different(1)
}),
("allow_screen_content_tools", |f| {
f.set_allow_screen_content_tools(1)
}),
("is_filter_switchable", |f| f.set_is_filter_switchable(1)),
("force_integer_mv", |f| f.set_force_integer_mv(1)),
("frame_size_override_flag", |f| {
f.set_frame_size_override_flag(1)
}),
("buffer_removal_time_present_flag", |f| {
f.set_buffer_removal_time_present_flag(1)
}),
("allow_intrabc", |f| f.set_allow_intrabc(1)),
("frame_refs_short_signaling", |f| {
f.set_frame_refs_short_signaling(1)
}),
("allow_high_precision_mv", |f| {
f.set_allow_high_precision_mv(1)
}),
("is_motion_mode_switchable", |f| {
f.set_is_motion_mode_switchable(1)
}),
("use_ref_frame_mvs", |f| f.set_use_ref_frame_mvs(1)),
("disable_frame_end_update_cdf", |f| {
f.set_disable_frame_end_update_cdf(1)
}),
("allow_warped_motion", |f| f.set_allow_warped_motion(1)),
("reduced_tx_set", |f| f.set_reduced_tx_set(1)),
("skip_mode_present", |f| f.set_skip_mode_present(1)),
("delta_q_present", |f| f.set_delta_q_present(1)),
("delta_lf_present", |f| f.set_delta_lf_present(1)),
("delta_lf_multi", |f| f.set_delta_lf_multi(1)),
("segmentation_enabled", |f| f.set_segmentation_enabled(1)),
("segmentation_update_map", |f| {
f.set_segmentation_update_map(1)
}),
("segmentation_temporal_update", |f| {
f.set_segmentation_temporal_update(1)
}),
("segmentation_update_data", |f| {
f.set_segmentation_update_data(1)
}),
("UsesLr", |f| f.set_UsesLr(1)),
("usesChromaLr", |f| f.set_usesChromaLr(1)),
("show_frame", |f| f.set_show_frame(1)),
("showable_frame", |f| f.set_showable_frame(1)),
];
for (bit, (name, set)) in members.into_iter().enumerate() {
let mut flags = StdVideoEncodeAV1PictureInfoFlags {
_bitfield_align_1: [],
_bitfield_1: __BindgenBitfieldUnit::new([0u8; 4]),
};
set(&mut flags);
sets_only_bit(flags._bitfield_1, name, bit, 1);
}
}
/// `reserved : 3` closes out the word — bits 29..32. Checking it is what proves the 29 members
/// above are the *whole* list: a dropped member would shift `reserved` down and fail here.
#[test]
fn picture_info_reserved_occupies_the_top_three_bits() {
let mut flags = StdVideoEncodeAV1PictureInfoFlags {
_bitfield_align_1: [],
_bitfield_1: __BindgenBitfieldUnit::new([0u8; 4]),
};
flags.set_reserved(0b111);
sets_only_bit(flags._bitfield_1, "reserved", 29, 3);
}
/// The same invariant for the two-member reference-info flags word.
#[test]
fn reference_info_flag_setters_follow_the_c_declaration_order() {
#[allow(clippy::type_complexity)]
let members: [(&str, fn(&mut StdVideoEncodeAV1ReferenceInfoFlags)); 2] = [
("disable_frame_end_update_cdf", |f| {
f.set_disable_frame_end_update_cdf(1)
}),
("segmentation_enabled", |f| f.set_segmentation_enabled(1)),
];
for (bit, (name, set)) in members.into_iter().enumerate() {
let mut flags = StdVideoEncodeAV1ReferenceInfoFlags {
_bitfield_align_1: [],
_bitfield_1: __BindgenBitfieldUnit::new([0u8; 4]),
};
set(&mut flags);
sets_only_bit(flags._bitfield_1, name, bit, 1);
}
}
}
-836
View File
@@ -1,836 +0,0 @@
//! Session/frame **construction** for the Vulkan Video encoder — the unsafe builders
//! (`make_frame*`, `make_video_image`, `probe_rgb_direct`) and the parameter-set bitstream
//! writers (`build_parameters_h265`/`_av1`, the AV1 sequence-header OBU). Split from
//! `vulkan_video.rs` (WP7.5) the way `amf_sys.rs` was split from `amf.rs`: a `#[path]` child
//! module, so this file sees the parent's private items (`Frame` and friends) with zero
//! visibility churn, and ~800 lines of construction `unsafe` get their own review surface.
//! Steady-state encode logic stays in the parent.
// The parent's whole item namespace (Frame, the consts, sibling helpers) — the point of the
// child-module shape. External imports are this file's own; `vk_util` is a crate-root sibling,
// so the path is `crate::`, not the parent-relative `super::` the parent uses.
use super::*;
use crate::vk_util::{find_mem, make_plain_image, make_view};
use anyhow::{bail, Result};
use ash::vk;
use std::ffi::c_void;
pub(super) fn align_up(v: u64, a: u64) -> u64 {
v.div_ceil(a) * a
}
/// Probe for the RGB-direct encode source (design/vulkan-rgb-direct-encode.md): can this device
/// take the captured RGB dmabuf directly, with the VCN EFC front-end doing the 709-narrow CSC,
/// via `VK_VALVE_video_encode_rgb_conversion` (RADV since Mesa 26.0, gated on EFC hardware)?
/// `Ok((x_offset, y_offset))` carries the chroma-siting bits a session must be created with
/// (the preferred available bit per axis); `Err` is the first missing requirement, logged as
/// the open-time verdict.
pub(super) unsafe fn probe_rgb_direct(
instance: &ash::Instance,
vq_inst: &ash::khr::video_queue::Instance,
pd: vk::PhysicalDevice,
codec_op: vk::VideoCodecOperationFlagsKHR,
av1: bool,
) -> Result<(u32, u32), &'static str> {
use crate::vk_av1_encode as av1b;
use crate::vk_valve_rgb as vrgb;
// 1. The device extension must exist (Mesa >= 26.0 AND the VCN has an EFC block).
let Ok(exts) = instance.enumerate_device_extension_properties(pd) else {
return Err("probe-failed(ext-enum)");
};
if !exts
.iter()
.any(|e| std::ffi::CStr::from_ptr(e.extension_name.as_ptr()) == vrgb::EXTENSION_NAME)
{
return Err("no-ext(mesa<26.0-or-no-efc)");
}
// 2. Feature bit.
let mut feat = vrgb::PhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE {
s_type: vrgb::stype(vrgb::ST_PHYSICAL_DEVICE_FEATURES),
p_next: std::ptr::null_mut(),
video_encode_rgb_conversion: vk::FALSE,
};
let mut f2 = vk::PhysicalDeviceFeatures2 {
p_next: &mut feat as *mut _ as *mut c_void,
..Default::default()
};
instance.get_physical_device_features2(pd, &mut f2);
if feat.video_encode_rgb_conversion == vk::FALSE {
return Err("no-feature");
}
// 3. Capabilities under the rgb-chained profile — the conversion must cover the compute
// CSC's colour math (rgb2yuv.comp: BT.709, narrow range; chroma siting is looser, see
// below). The profile chain is the same one every rgb-direct consumer presents.
let mut ps = RgbProfileStack::new(codec_op);
let profile = *ps.wire(av1);
let mut rgb_caps = vrgb::VideoEncodeRgbConversionCapabilitiesVALVE {
s_type: vrgb::stype(vrgb::ST_CAPABILITIES),
p_next: std::ptr::null_mut(),
rgb_models: 0,
rgb_ranges: 0,
x_chroma_offsets: 0,
y_chroma_offsets: 0,
};
let mut h265_caps = vk::VideoEncodeH265CapabilitiesKHR::default();
let mut av1_caps: av1b::VideoEncodeAV1CapabilitiesKHR = std::mem::zeroed();
av1_caps.s_type = av1b::stype(av1b::ST_CAPABILITIES);
let mut enc_caps = vk::VideoEncodeCapabilitiesKHR::default();
let mut caps = vk::VideoCapabilitiesKHR::default();
if av1 {
av1_caps.p_next = &mut rgb_caps as *mut _ as *mut c_void;
enc_caps.p_next = &mut av1_caps as *mut _ as *mut c_void;
} else {
h265_caps.p_next = &mut rgb_caps as *mut _ as *mut c_void;
enc_caps.p_next = &mut h265_caps as *mut _ as *mut c_void;
}
caps.p_next = &mut enc_caps as *mut _ as *mut c_void;
let r = (vq_inst.fp().get_physical_device_video_capabilities_khr)(pd, &profile, &mut caps);
if r != vk::Result::SUCCESS {
return Err("no-rgb-profile(caps)");
}
// Colour model + range must match the shader exactly (709 narrow). Chroma siting is looser
// BY ON-GLASS FINDING (RADV 26.0.4 / 780M): the VCN EFC advertises x=COSITED_EVEN only —
// the canonical H.26x left-cosited siting — while our 2x2-average shader is midpoint. The
// difference is a half-pel chroma-x phase, imperceptible (and EFC's is arguably the more
// correct one since nothing in our bitstream signals siting). Accept either bit per axis
// and choose the closest to the shader's math: midpoint if offered, else cosited-even.
let pick = |offered: u32| -> Option<u32> {
if offered & vrgb::CHROMA_OFFSET_MIDPOINT != 0 {
Some(vrgb::CHROMA_OFFSET_MIDPOINT)
} else if offered & vrgb::CHROMA_OFFSET_COSITED_EVEN != 0 {
Some(vrgb::CHROMA_OFFSET_COSITED_EVEN)
} else {
None
}
};
if rgb_caps.rgb_models & vrgb::MODEL_YCBCR_709 == 0
|| rgb_caps.rgb_ranges & vrgb::RANGE_NARROW == 0
{
return Err("no-709-narrow");
}
let (Some(x_offset), Some(y_offset)) = (
pick(rgb_caps.x_chroma_offsets),
pick(rgb_caps.y_chroma_offsets),
) else {
return Err("no-chroma-siting");
};
// 4. The encode-src format set under this profile must offer BGRA with DRM-modifier tiling —
// the capture hands LINEAR BGRx dmabufs (fourcc XR24), which import as B8G8R8A8_UNORM.
let profile_arr = [profile];
let plist = vk::VideoProfileListInfoKHR::default().profiles(&profile_arr);
let mut fmt_info = vk::PhysicalDeviceVideoFormatInfoKHR::default()
.image_usage(vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR);
fmt_info.p_next = &plist as *const _ as *const c_void;
let get_fmt = vq_inst.fp().get_physical_device_video_format_properties_khr;
let mut count = 0u32;
let r = get_fmt(pd, &fmt_info, &mut count, std::ptr::null_mut());
if r != vk::Result::SUCCESS || count == 0 {
return Err("no-rgb-format");
}
let mut props = vec![vk::VideoFormatPropertiesKHR::default(); count as usize];
let r = get_fmt(pd, &fmt_info, &mut count, props.as_mut_ptr());
if r != vk::Result::SUCCESS && r != vk::Result::INCOMPLETE {
return Err("no-rgb-format");
}
if !props[..count as usize].iter().any(|p| {
p.format == vk::Format::B8G8R8A8_UNORM
&& p.image_tiling == vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT
}) {
return Err("no-bgra-modifier-tiling");
}
Ok((x_offset, y_offset))
}
pub(super) unsafe fn make_video_image(
device: &ash::Device,
mp: &vk::PhysicalDeviceMemoryProperties,
fmt: vk::Format,
w: u32,
h: u32,
layers: u32,
usage: vk::ImageUsageFlags,
profile_list: &mut vk::VideoProfileListInfoKHR,
concurrent: &[u32],
) -> Result<(vk::Image, vk::DeviceMemory)> {
let mut ci = vk::ImageCreateInfo::default()
.image_type(vk::ImageType::TYPE_2D)
.format(fmt)
.extent(vk::Extent3D {
width: w,
height: h,
depth: 1,
})
.mip_levels(1)
.array_layers(layers)
.samples(vk::SampleCountFlags::TYPE_1)
.tiling(vk::ImageTiling::OPTIMAL)
.usage(usage)
.initial_layout(vk::ImageLayout::UNDEFINED)
.push_next(profile_list);
if concurrent.len() >= 2 {
ci = ci
.sharing_mode(vk::SharingMode::CONCURRENT)
.queue_family_indices(concurrent);
} else {
ci = ci.sharing_mode(vk::SharingMode::EXCLUSIVE);
}
let img = device.create_image(&ci, None)?;
let req = device.get_image_memory_requirements(img);
// Unwind on failure: callers (the open path) only ever see the completed pair.
let mem = match device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(find_mem(
mp,
req.memory_type_bits,
vk::MemoryPropertyFlags::DEVICE_LOCAL,
)),
None,
) {
Ok(m) => m,
Err(e) => {
device.destroy_image(img, None);
return Err(e.into());
}
};
if let Err(e) = device.bind_image_memory(img, mem, 0) {
device.destroy_image(img, None);
device.free_memory(mem, None);
return Err(e.into());
}
Ok((img, mem))
}
/// Build one in-flight frame's private resources: NV12 encode-src, Y/UV CSC scratch, its CSC
/// descriptor set (Y/UV bound now, RGB per use), the bitstream buffer + feedback query, and the
/// per-frame command buffers + sync. `profile_list`/`profile` are borrowed only during creation.
///
/// Builds in place into `f` — a [`Frame::default`] the caller has already parked in its
/// [`VkTeardown`] guard — so every handle is owned by the unwind the moment it exists and a
/// mid-build failure leaks nothing.
pub(super) unsafe fn make_frame(
device: &ash::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
w: u32,
h: u32,
fams: &[u32],
profile: &vk::VideoProfileInfoKHR,
profile_list: &mut vk::VideoProfileListInfoKHR,
csc_dsl: vk::DescriptorSetLayout,
csc_pool: vk::DescriptorPool,
cmd_pool: vk::CommandPool,
compute_pool: vk::CommandPool,
bs_size: u64,
sampler: vk::Sampler,
with_ts: bool,
csc: bool,
pad_fmt: Option<vk::Format>,
f: &mut Frame,
) -> Result<()> {
// "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`).
f.cursor_serial = u64::MAX;
// Padded-copy staging (unaligned-mode RGB-direct or native NV12): an aligned encode-src in
// the session's picture format, filled by a transfer blit each frame — concurrent compute
// (copy) + encode (source read). TRANSFER_SRC because the width-padding pass self-copies the
// staging image's own last visible column (see `record_pad_blit`).
if let Some(fmt) = pad_fmt {
(f.pad_img, f.pad_mem) = make_video_image(
device,
mem_props,
fmt,
w,
h,
1,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR
| vk::ImageUsageFlags::TRANSFER_DST
| vk::ImageUsageFlags::TRANSFER_SRC,
profile_list,
fams,
)?;
f.pad_view = make_view(device, f.pad_img, fmt, 0)?;
}
// RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no
// cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the
// CPU staging image, built lazily). Their Frame keeps the null handles (teardown-safe).
if csc {
make_frame_csc(
device,
mem_props,
w,
h,
fams,
profile_list,
csc_dsl,
csc_pool,
sampler,
f,
)?;
}
make_frame_common(
device,
mem_props,
profile,
profile_list,
cmd_pool,
compute_pool,
bs_size,
with_ts,
f,
)
}
/// The CSC-only half of [`make_frame`]: NV12 encode-src + Y/UV scratch + cursor + descriptors.
#[allow(clippy::too_many_arguments)]
unsafe fn make_frame_csc(
device: &ash::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
w: u32,
h: u32,
fams: &[u32],
profile_list: &mut vk::VideoProfileListInfoKHR,
csc_dsl: vk::DescriptorSetLayout,
csc_pool: vk::DescriptorPool,
sampler: vk::Sampler,
f: &mut Frame,
) -> Result<()> {
// NV12 encode-src (filled by the CSC copy) — concurrent compute+encode.
(f.nv12_src, f.nv12_mem) = make_video_image(
device,
mem_props,
NV12,
w,
h,
1,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR | vk::ImageUsageFlags::TRANSFER_DST,
profile_list,
fams,
)?;
f.nv12_view = make_view(device, f.nv12_src, NV12, 0)?;
// CSC scratch (Y R8 full-res, UV RG8 half-res).
(f.y_img, f.y_mem, f.y_view) = make_plain_image(
device,
mem_props,
vk::Format::R8_UNORM,
w,
h,
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::TRANSFER_SRC,
)?;
(f.uv_img, f.uv_mem, f.uv_view) = make_plain_image(
device,
mem_props,
vk::Format::R8G8_UNORM,
w / 2,
h / 2,
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::TRANSFER_SRC,
)?;
// Cursor overlay: fixed CURSOR_MAX² RGBA8 sampled image + host staging (cursor-as-metadata). The
// view/descriptor is static (bound at binding 3 below); only the image *content* changes, and
// only when the pointer bitmap does — see `prep_cursor`.
(f.cursor_img, f.cursor_mem, f.cursor_view) = make_plain_image(
device,
mem_props,
vk::Format::R8G8B8A8_UNORM,
CURSOR_MAX,
CURSOR_MAX,
vk::ImageUsageFlags::SAMPLED | vk::ImageUsageFlags::TRANSFER_DST,
)?;
f.cursor_stage = device.create_buffer(
&vk::BufferCreateInfo::default()
.size((CURSOR_MAX * CURSOR_MAX * 4) as u64)
.usage(vk::BufferUsageFlags::TRANSFER_SRC),
None,
)?;
let cs_req = device.get_buffer_memory_requirements(f.cursor_stage);
f.cursor_stage_mem = device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(cs_req.size)
.memory_type_index(find_mem(
mem_props,
cs_req.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
)?;
device.bind_buffer_memory(f.cursor_stage, f.cursor_stage_mem, 0)?;
// Descriptor set — Y/UV storage bindings fixed; binding 0 (RGB) rewritten per use; binding 3
// (cursor) points at the static cursor image (its layout is SHADER_READ_ONLY once prepped).
let dsls = [csc_dsl];
f.csc_set = device.allocate_descriptor_sets(
&vk::DescriptorSetAllocateInfo::default()
.descriptor_pool(csc_pool)
.set_layouts(&dsls),
)?[0];
let y_info = [vk::DescriptorImageInfo::default()
.image_view(f.y_view)
.image_layout(vk::ImageLayout::GENERAL)];
let uv_info = [vk::DescriptorImageInfo::default()
.image_view(f.uv_view)
.image_layout(vk::ImageLayout::GENERAL)];
let cur_info = [vk::DescriptorImageInfo::default()
.sampler(sampler)
.image_view(f.cursor_view)
.image_layout(vk::ImageLayout::SHADER_READ_ONLY_OPTIMAL)];
device.update_descriptor_sets(
&[
vk::WriteDescriptorSet::default()
.dst_set(f.csc_set)
.dst_binding(1)
.descriptor_type(vk::DescriptorType::STORAGE_IMAGE)
.image_info(&y_info),
vk::WriteDescriptorSet::default()
.dst_set(f.csc_set)
.dst_binding(2)
.descriptor_type(vk::DescriptorType::STORAGE_IMAGE)
.image_info(&uv_info),
vk::WriteDescriptorSet::default()
.dst_set(f.csc_set)
.dst_binding(3)
.descriptor_type(vk::DescriptorType::COMBINED_IMAGE_SAMPLER)
.image_info(&cur_info),
],
&[],
);
Ok(())
}
/// The mode-independent half of [`make_frame`]: bitstream buffer (+ persistent map), feedback
/// query, optional timestamp pool, command buffers and sync objects.
#[allow(clippy::too_many_arguments)]
unsafe fn make_frame_common(
device: &ash::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
profile: &vk::VideoProfileInfoKHR,
profile_list: &mut vk::VideoProfileListInfoKHR,
cmd_pool: vk::CommandPool,
compute_pool: vk::CommandPool,
bs_size: u64,
with_ts: bool,
f: &mut Frame,
) -> Result<()> {
// Bitstream buffer + feedback query.
f.bs_buf = device.create_buffer(
&vk::BufferCreateInfo::default()
.size(bs_size)
.usage(vk::BufferUsageFlags::VIDEO_ENCODE_DST_KHR)
.push_next(profile_list),
None,
)?;
let bs_req = device.get_buffer_memory_requirements(f.bs_buf);
f.bs_mem = device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(bs_req.size)
.memory_type_index(find_mem(
mem_props,
bs_req.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
)?;
device.bind_buffer_memory(f.bs_buf, f.bs_mem, 0)?;
// Map once for the slot's lifetime — read_slot copies AUs straight out of this (coherent
// memory, no per-frame map/unmap); vkFreeMemory implicitly unmaps at teardown.
f.bs_ptr = BsPtr(
device.map_memory(f.bs_mem, 0, vk::WHOLE_SIZE, vk::MemoryMapFlags::empty())? as *const u8,
);
// PUNKTFUNK_PERF: a 2-slot timestamp pool bracketing this slot's compute batch (CSC split).
if with_ts {
f.ts_pool = device.create_query_pool(
&vk::QueryPoolCreateInfo::default()
.query_type(vk::QueryType::TIMESTAMP)
.query_count(2),
None,
)?;
}
let mut fb_ci = vk::QueryPoolVideoEncodeFeedbackCreateInfoKHR::default().encode_feedback_flags(
vk::VideoEncodeFeedbackFlagsKHR::BITSTREAM_BUFFER_OFFSET
| vk::VideoEncodeFeedbackFlagsKHR::BITSTREAM_BYTES_WRITTEN,
);
fb_ci.p_next = profile as *const _ as *const c_void;
let mut query_ci = vk::QueryPoolCreateInfo::default()
.query_type(vk::QueryType::VIDEO_ENCODE_FEEDBACK_KHR)
.query_count(1);
query_ci.p_next = &fb_ci as *const _ as *const c_void;
f.query_pool = device.create_query_pool(&query_ci, None)?;
// Command buffers + per-frame sync.
f.cmd = device.allocate_command_buffers(
&vk::CommandBufferAllocateInfo::default()
.command_pool(cmd_pool)
.command_buffer_count(1),
)?[0];
f.compute_cmd = device.allocate_command_buffers(
&vk::CommandBufferAllocateInfo::default()
.command_pool(compute_pool)
.command_buffer_count(1),
)?[0];
f.csc_sem = device.create_semaphore(&vk::SemaphoreCreateInfo::default(), None)?;
f.fence = device.create_fence(&vk::FenceCreateInfo::default(), None)?;
Ok(())
}
/// Author VPS/SPS/PPS (Main, level 4.0, low-latency, conformance-window crop) and return the
/// session-parameters object + the encoded header bytes (VPS+SPS+PPS NALs) for keyframes.
pub(super) unsafe fn build_parameters_h265(
device: &ash::Device,
vq_dev: &ash::khr::video_queue::Device,
venc_dev: &ash::khr::video_encode_queue::Device,
session: vk::VideoSessionKHR,
w: u32,
h: u32,
rw: u32,
rh: u32,
quality_level: u32,
) -> Result<(vk::VideoSessionParametersKHR, Vec<u8>)> {
use ash::vk::native as hh;
let mut ptl: hh::StdVideoH265ProfileTierLevel = std::mem::zeroed();
ptl.flags.set_general_progressive_source_flag(1);
ptl.flags.set_general_frame_only_constraint_flag(1);
ptl.general_profile_idc = hh::StdVideoH265ProfileIdc_STD_VIDEO_H265_PROFILE_IDC_MAIN;
ptl.general_level_idc = hh::StdVideoH265LevelIdc_STD_VIDEO_H265_LEVEL_IDC_6_0;
let mut dpbm: hh::StdVideoH265DecPicBufMgr = std::mem::zeroed();
dpbm.max_dec_pic_buffering_minus1[0] = (DPB_SLOTS - 1) as u8;
dpbm.max_num_reorder_pics[0] = 0;
dpbm.max_latency_increase_plus1[0] = 0;
let mut vps: hh::StdVideoH265VideoParameterSet = std::mem::zeroed();
vps.flags.set_vps_temporal_id_nesting_flag(1);
vps.flags.set_vps_sub_layer_ordering_info_present_flag(1);
vps.pDecPicBufMgr = &dpbm;
vps.pProfileTierLevel = &ptl;
let mut sps: hh::StdVideoH265SequenceParameterSet = std::mem::zeroed();
sps.flags.set_sps_temporal_id_nesting_flag(1);
sps.flags.set_sps_sub_layer_ordering_info_present_flag(1);
sps.chroma_format_idc = hh::StdVideoH265ChromaFormatIdc_STD_VIDEO_H265_CHROMA_FORMAT_IDC_420;
sps.pic_width_in_luma_samples = w;
sps.pic_height_in_luma_samples = h;
sps.log2_max_pic_order_cnt_lsb_minus4 = 4;
sps.log2_diff_max_min_luma_coding_block_size = 3;
sps.log2_diff_max_min_luma_transform_block_size = 3;
sps.max_transform_hierarchy_depth_inter = 4;
sps.max_transform_hierarchy_depth_intra = 4;
sps.pProfileTierLevel = &ptl;
sps.pDecPicBufMgr = &dpbm;
if w != rw || h != rh {
sps.flags.set_conformance_window_flag(1);
sps.conf_win_right_offset = (w - rw) / 2; // 4:2:0 SubWidthC = 2
sps.conf_win_bottom_offset = (h - rh) / 2; // 4:2:0 SubHeightC = 2
}
let mut pps: hh::StdVideoH265PictureParameterSet = std::mem::zeroed();
pps.flags.set_cu_qp_delta_enabled_flag(1);
pps.flags.set_pps_loop_filter_across_slices_enabled_flag(1);
let vps_arr = [vps];
let sps_arr = [sps];
let pps_arr = [pps];
let add = vk::VideoEncodeH265SessionParametersAddInfoKHR::default()
.std_vp_ss(&vps_arr)
.std_sp_ss(&sps_arr)
.std_pp_ss(&pps_arr);
let mut h265_ci = vk::VideoEncodeH265SessionParametersCreateInfoKHR::default()
.max_std_vps_count(1)
.max_std_sps_count(1)
.max_std_pps_count(1)
.parameters_add_info(&add);
// Bake the session's quality level into the parameters object — the spec requires it to match
// the level the first frame's ENCODE_QUALITY_LEVEL control installs.
let mut q_info = vk::VideoEncodeQualityLevelInfoKHR::default().quality_level(quality_level);
let ci = vk::VideoSessionParametersCreateInfoKHR::default()
.video_session(session)
.push_next(&mut h265_ci)
.push_next(&mut q_info);
let mut params = vk::VideoSessionParametersKHR::null();
let r = (vq_dev.fp().create_video_session_parameters_khr)(
device.handle(),
&ci,
std::ptr::null(),
&mut params,
);
if r != vk::Result::SUCCESS {
bail!("create_video_session_parameters: {r:?}");
}
let mut get_h265 = vk::VideoEncodeH265SessionParametersGetInfoKHR::default()
.write_std_vps(true)
.write_std_sps(true)
.write_std_pps(true)
.std_vps_id(0)
.std_sps_id(0)
.std_pps_id(0);
let get = vk::VideoEncodeSessionParametersGetInfoKHR::default()
.video_session_parameters(params)
.push_next(&mut get_h265);
let get_fn = venc_dev.fp().get_encoded_video_session_parameters_khr;
let mut fb = vk::VideoEncodeSessionParametersFeedbackInfoKHR::default();
let mut size: usize = 0;
let r = get_fn(
device.handle(),
&get,
&mut fb,
&mut size,
std::ptr::null_mut(),
);
if r != vk::Result::SUCCESS {
// `params` is live but not yet the caller's guard's to unwind — destroy before bailing.
(vq_dev.fp().destroy_video_session_parameters_khr)(
device.handle(),
params,
std::ptr::null(),
);
bail!("get header size: {r:?}");
}
let mut buf = vec![0u8; size];
let r = get_fn(
device.handle(),
&get,
&mut fb,
&mut size,
buf.as_mut_ptr() as *mut c_void,
);
if r != vk::Result::SUCCESS {
(vq_dev.fp().destroy_video_session_parameters_khr)(
device.handle(),
params,
std::ptr::null(),
);
bail!("get header bytes: {r:?}");
}
buf.truncate(size);
Ok((params, buf))
}
/// AV1 low-overhead OBU bit-writer (MSB-first), used to hand-pack the sequence-header OBU that
/// Vulkan AV1 encode (unlike H26x) never emits itself.
struct Av1BitWriter {
buf: Vec<u8>,
cur: u8,
fill: u8,
}
impl Av1BitWriter {
fn new() -> Self {
Self {
buf: Vec::new(),
cur: 0,
fill: 0,
}
}
fn bit(&mut self, b: u32) {
self.cur = (self.cur << 1) | (b as u8 & 1);
self.fill += 1;
if self.fill == 8 {
self.buf.push(self.cur);
self.cur = 0;
self.fill = 0;
}
}
fn put(&mut self, val: u32, bits: u32) {
for i in (0..bits).rev() {
self.bit((val >> i) & 1);
}
}
/// Flush, zero-padding the final partial byte (OBU size field delimits the payload).
fn finish(mut self) -> Vec<u8> {
if self.fill > 0 {
self.cur <<= 8 - self.fill;
self.buf.push(self.cur);
}
self.buf
}
}
/// AV1 leb128 (little-endian base-128) encoding of an OBU size.
fn leb128(mut v: u64) -> Vec<u8> {
let mut out = Vec::new();
loop {
let mut byte = (v & 0x7f) as u8;
v >>= 7;
if v != 0 {
byte |= 0x80;
}
out.push(byte);
if v == 0 {
break;
}
}
out
}
/// Bit-pack a `sequence_header_obu` (AV1 spec §5.5) into a size-delimited OBU. The field values here
/// MUST mirror the `StdVideoAV1SequenceHeader` handed to the driver in `build_parameters_av1` so the
/// driver-emitted frame OBUs parse against this header. Single operating point, 8-bit 4:2:0,
/// order-hint on, CDEF+restoration+filter-intra allowed, everything exotic (compound/warp/superres)
/// disabled — the profile our single-reference P-frame encoder actually uses.
fn av1_sequence_header_obu(
sb128: bool,
fwb: u32,
fhb: u32,
max_w_m1: u32,
max_h_m1: u32,
order_hint_bits_minus_1: u32,
seq_level_idx: u32,
) -> Vec<u8> {
let mut w = Av1BitWriter::new();
w.put(0, 3); // seq_profile = MAIN
w.bit(0); // still_picture
w.bit(0); // reduced_still_picture_header
w.bit(0); // timing_info_present_flag
w.bit(0); // initial_display_delay_present_flag
w.put(0, 5); // operating_points_cnt_minus_1 = 0
w.put(0, 12); // operating_point_idc[0]
w.put(seq_level_idx, 5); // seq_level_idx[0]
if seq_level_idx > 7 {
w.bit(0); // seq_tier[0] = 0
}
w.put(fwb, 4); // frame_width_bits_minus_1
w.put(fhb, 4); // frame_height_bits_minus_1
w.put(max_w_m1, fwb + 1); // max_frame_width_minus_1
w.put(max_h_m1, fhb + 1); // max_frame_height_minus_1
w.bit(0); // frame_id_numbers_present_flag
w.bit(sb128 as u32); // use_128x128_superblock
w.bit(0); // enable_filter_intra
w.bit(0); // enable_intra_edge_filter
w.bit(0); // enable_interintra_compound
w.bit(0); // enable_masked_compound
w.bit(0); // enable_warped_motion
w.bit(0); // enable_dual_filter
w.bit(1); // enable_order_hint
w.bit(0); // enable_jnt_comp
w.bit(0); // enable_ref_frame_mvs
w.bit(1); // seq_choose_screen_content_tools -> seq_force_screen_content_tools = SELECT
w.bit(1); // seq_choose_integer_mv -> seq_force_integer_mv = SELECT
w.put(order_hint_bits_minus_1, 3); // order_hint_bits_minus_1
w.bit(0); // enable_superres
w.bit(0); // enable_cdef
w.bit(0); // enable_restoration
// color_config(): 8-bit 4:2:0, unspecified primaries/transfer/matrix, limited range
w.bit(0); // high_bitdepth
w.bit(0); // mono_chrome
w.bit(0); // color_description_present_flag
w.bit(0); // color_range (studio/limited)
w.put(0, 2); // chroma_sample_position = CSP_UNKNOWN (subsampling_x==subsampling_y==1 for profile 0)
w.bit(0); // separate_uv_delta_q
w.bit(0); // film_grain_params_present
// trailing_bits(): a stop `1` bit then zero-pad to a byte (the size field delimits the OBU, but
// the parser still requires the trailing_one_bit — dav1d/cbs reject a plain zero pad).
w.bit(1);
let payload = w.finish();
let mut obu = vec![0x0au8]; // obu_header: type=OBU_SEQUENCE_HEADER(1), has_size_field=1
obu.extend_from_slice(&leb128(payload.len() as u64));
obu.extend_from_slice(&payload);
obu
}
/// AV1 session parameters + header framing. Vulkan AV1 encode emits only the per-frame OBU, so we
/// return the app-owned prefixes: a temporal-delimiter OBU that opens every temporal unit
/// (`frame_prefix`), and TD + the bit-packed sequence-header OBU for keyframes (`header`).
#[allow(clippy::too_many_arguments)]
pub(super) unsafe fn build_parameters_av1(
device: &ash::Device,
vq_dev: &ash::khr::video_queue::Device,
session: vk::VideoSessionKHR,
w: u32,
h: u32,
_rw: u32,
_rh: u32,
max_level: ash::vk::native::StdVideoAV1Level,
sb128: bool,
quality_level: u32,
) -> Result<(vk::VideoSessionParametersKHR, Vec<u8>, Vec<u8>)> {
use crate::vk_av1_encode as av1;
use ash::vk::native as hh;
let fwb = 31 - w.leading_zeros(); // av_log2(w): enough bits for max_frame_width_minus_1 = w-1
let fhb = 31 - h.leading_zeros();
let order_hint_bits_minus_1: u32 = 7; // OrderHintBits = 8
let seq_level_idx = max_level; // StdVideoAV1Level's numeric value IS the AV1 seq_level_idx
// ---- Std sequence header (must match the OBU packed below) ----
let mut cc_flags: hh::StdVideoAV1ColorConfigFlags = std::mem::zeroed();
let _ = &mut cc_flags; // all zero: mono_chrome/color_range/description/separate_uv_delta_q = 0
let mut cc: hh::StdVideoAV1ColorConfig = std::mem::zeroed();
cc.flags = cc_flags;
cc.BitDepth = 8;
cc.subsampling_x = 1;
cc.subsampling_y = 1;
cc.color_primaries = hh::StdVideoAV1ColorPrimaries_STD_VIDEO_AV1_COLOR_PRIMARIES_BT_UNSPECIFIED;
cc.transfer_characteristics =
hh::StdVideoAV1TransferCharacteristics_STD_VIDEO_AV1_TRANSFER_CHARACTERISTICS_UNSPECIFIED;
cc.matrix_coefficients =
hh::StdVideoAV1MatrixCoefficients_STD_VIDEO_AV1_MATRIX_COEFFICIENTS_UNSPECIFIED;
cc.chroma_sample_position =
hh::StdVideoAV1ChromaSamplePosition_STD_VIDEO_AV1_CHROMA_SAMPLE_POSITION_UNKNOWN;
// Match FFmpeg's Vulkan AV1 encoder (proven on this RADV/VCN path): the ONLY coding tools
// enabled are order-hint and (per caps) 128x128 superblocks. CDEF, loop restoration, filter-
// intra, warped/compound motion, superres all OFF — enabling them made VCN emit frame-header
// sections whose bit layout our sequence header didn't match, desyncing every inter frame.
let mut sh_flags: hh::StdVideoAV1SequenceHeaderFlags = std::mem::zeroed();
if sb128 {
sh_flags.set_use_128x128_superblock(1);
}
sh_flags.set_enable_order_hint(1);
let mut sh: hh::StdVideoAV1SequenceHeader = std::mem::zeroed();
sh.flags = sh_flags;
sh.seq_profile = hh::StdVideoAV1Profile_STD_VIDEO_AV1_PROFILE_MAIN;
sh.frame_width_bits_minus_1 = fwb as u8;
sh.frame_height_bits_minus_1 = fhb as u8;
sh.max_frame_width_minus_1 = (w - 1) as u16;
sh.max_frame_height_minus_1 = (h - 1) as u16;
sh.order_hint_bits_minus_1 = order_hint_bits_minus_1 as u8;
sh.seq_force_integer_mv = 2; // SELECT
sh.seq_force_screen_content_tools = 2; // SELECT
sh.pColorConfig = &cc;
// ---- single operating point conveying the level/tier the driver targets ----
let op = av1::StdVideoEncodeAV1OperatingPointInfo {
flags: std::mem::zeroed(),
operating_point_idc: 0,
seq_level_idx: seq_level_idx as u8,
seq_tier: 0,
decoder_buffer_delay: 0,
encoder_buffer_delay: 0,
initial_display_delay_minus_1: 0,
};
let ops = [op];
let av1_spci = av1::VideoEncodeAV1SessionParametersCreateInfoKHR {
s_type: av1::stype(av1::ST_SESSION_PARAMETERS_CREATE_INFO),
p_next: std::ptr::null(),
p_std_sequence_header: &sh,
p_std_decoder_model_info: std::ptr::null(),
std_operating_point_count: 1,
p_std_operating_points: ops.as_ptr() as *const c_void,
};
// Bake the session's quality level into the parameters object (must match the level the first
// frame's ENCODE_QUALITY_LEVEL control installs); chained raw ahead of the vendored AV1 struct.
let mut q_info = vk::VideoEncodeQualityLevelInfoKHR::default().quality_level(quality_level);
q_info.p_next = &av1_spci as *const _ as *const c_void;
let mut ci = vk::VideoSessionParametersCreateInfoKHR::default().video_session(session);
ci.p_next = &q_info as *const _ as *const c_void;
let mut params = vk::VideoSessionParametersKHR::null();
let r = (vq_dev.fp().create_video_session_parameters_khr)(
device.handle(),
&ci,
std::ptr::null(),
&mut params,
);
if r != vk::Result::SUCCESS {
bail!("create_video_session_parameters (av1): {r:?}");
}
// ---- header framing: TD every temporal unit; TD + seq-header OBU on keyframes ----
let td = vec![0x12u8, 0x00]; // temporal_delimiter OBU (type=2, size=0)
let seq_obu = av1_sequence_header_obu(
sb128,
fwb,
fhb,
w - 1,
h - 1,
order_hint_bits_minus_1,
seq_level_idx,
);
let mut keyframe_prefix = td.clone();
keyframe_prefix.extend_from_slice(&seq_obu);
Ok((params, keyframe_prefix, td))
}
+24 -299
View File
@@ -37,11 +37,9 @@ pub(crate) fn fourcc_to_vk(fourcc: u32) -> Option<vk::Format> {
const AR24: u32 = 0x3432_5241; // ARGB8888 const AR24: u32 = 0x3432_5241; // ARGB8888
const XB24: u32 = 0x3432_4258; // XBGR8888 const XB24: u32 = 0x3432_4258; // XBGR8888
const AB24: u32 = 0x3432_4241; // ABGR8888 const AB24: u32 = 0x3432_4241; // ABGR8888
const NV12: u32 = 0x3231_564e; // DRM_FORMAT_NV12
match fourcc { match fourcc {
XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM), XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM),
XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM), XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM),
NV12 => Some(vk::Format::G8_B8R8_2PLANE_420_UNORM),
_ => None, _ => None,
} }
} }
@@ -54,59 +52,6 @@ pub(crate) fn pixel_to_vk(fmt: PixelFormat) -> Option<vk::Format> {
} }
} }
/// Normalize a CPU RGB payload for Vulkan upload. The packed 24-bpp `Rgb`/`Bgr` the PipeWire
/// capturer can negotiate are expanded 3→4 into `scratch` (kept by the caller across frames — no
/// per-frame allocation) with the pad byte = 0xFF; refusing them instead used to kill a session
/// at its first frame (WP5.4). No packed 24-bpp VkFormat is reliably uploadable/sampleable on
/// target GPUs, and this path is CPU-sourced by definition, so one cheap expand pass serves it
/// (the same call NVENC answers with its swscale 3→4 expand, WP1.4).
///
/// `bgra_target = false` (the CSC paths): channel order is preserved — the sampler reads through
/// the matching view format, so any 4-bpp order works and 4-bpp inputs pass through borrowed.
/// `bgra_target = true` (the RGB-direct encode source): the output byte order is forced to
/// B,G,R,X, because the video session's `pictureFormat` is `B8G8R8A8_UNORM` and
/// VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 requires the source image to match it — an
/// R-first source (`Rgbx`/`Rgba`/`Rgb`) is channel-swapped during the same pass. (Caught live on
/// RADV by `vulkan_smoke_rgb_cpu24`; the mismatch predates the 24-bpp support for `Rgbx` CPU
/// sources.)
///
/// Payloads are tightly packed with no row padding (`FramePayload::Cpu`'s contract), so the
/// conversion is row-agnostic; a truncated source yields a truncated output, which the upload
/// paths already bound-check exactly as they did the raw bytes.
pub(crate) fn normalize_cpu_rgb<'a>(
fmt: PixelFormat,
bytes: &'a [u8],
scratch: &'a mut Vec<u8>,
bgra_target: bool,
) -> (PixelFormat, &'a [u8]) {
// Per-pixel source layout: bytes-per-pixel + where R, G, B sit in each pixel.
let (bpp, r, g, b) = match fmt {
PixelFormat::Rgb => (3usize, 0usize, 1usize, 2usize),
PixelFormat::Bgr => (3, 2, 1, 0),
PixelFormat::Rgbx | PixelFormat::Rgba => (4, 0, 1, 2),
PixelFormat::Bgrx | PixelFormat::Bgra => (4, 2, 1, 0),
_ => return (fmt, bytes),
};
if bpp == 4 && (!bgra_target || b == 0) {
return (fmt, bytes); // 4-bpp in an acceptable order: borrow untouched
}
let px = bytes.len() / bpp;
scratch.clear();
scratch.resize(px * 4, 0xFF);
let (dr, dg, db) = if bgra_target { (2, 1, 0) } else { (r, g, b) };
for (dst, src) in scratch.chunks_exact_mut(4).zip(bytes.chunks_exact(bpp)) {
dst[dr] = src[r];
dst[dg] = src[g];
dst[db] = src[b];
}
let out_fmt = if bgra_target || b == 0 {
PixelFormat::Bgrx
} else {
PixelFormat::Rgbx
};
(out_fmt, scratch.as_slice())
}
pub(crate) unsafe fn make_view( pub(crate) unsafe fn make_view(
device: &ash::Device, device: &ash::Device,
image: vk::Image, image: vk::Image,
@@ -123,21 +68,6 @@ pub(crate) unsafe fn make_view(
)?) )?)
} }
/// Whether a failed dmabuf import should count toward pf-zerocopy's raw-dmabuf degrade latch
/// (`note_raw_dmabuf_import_failure` — 3 consecutive failures flip capture to CPU delivery for
/// the process). Deterministic refusals (unsupported fourcc, the driver rejecting the buffer)
/// must count — they repeat identically forever and the latch is their only recovery. Transient
/// VRAM pressure must NOT: three tight allocation OOMs would otherwise permanently downgrade a
/// working host to CPU capture.
pub(crate) fn import_failure_feeds_latch(e: &anyhow::Error) -> bool {
match e.downcast_ref::<vk::Result>() {
Some(&r) => {
r != vk::Result::ERROR_OUT_OF_DEVICE_MEMORY && r != vk::Result::ERROR_OUT_OF_HOST_MEMORY
}
None => true,
}
}
/// Import a packed-RGB dmabuf as a SAMPLED VkImage (explicit DRM modifier). Caller destroys all /// Import a packed-RGB dmabuf as a SAMPLED VkImage (explicit DRM modifier). Caller destroys all
/// three returned handles. Extracted verbatim from `vulkan_video.rs`'s import path. /// three returned handles. Extracted verbatim from `vulkan_video.rs`'s import path.
pub(crate) unsafe fn import_rgb_dmabuf( pub(crate) unsafe fn import_rgb_dmabuf(
@@ -147,68 +77,21 @@ pub(crate) unsafe fn import_rgb_dmabuf(
d: &pf_frame::DmabufFrame, d: &pf_frame::DmabufFrame,
cw: u32, cw: u32,
ch: u32, ch: u32,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
import_rgb_dmabuf_as(
device,
ext_fd,
mem_props,
d,
cw,
ch,
vk::ImageUsageFlags::SAMPLED,
None,
)
}
/// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list.
/// Despite the historical name, this also imports gamescope's one-fd LINEAR NV12: the UV
/// subresource layout comes from the producer's plane-1 chunk when it reported one, falling
/// back to the shared-stride contiguous-plane contract.
#[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn import_rgb_dmabuf_as(
device: &ash::Device,
ext_fd: &ash::khr::external_memory_fd::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
d: &pf_frame::DmabufFrame,
cw: u32,
ch: u32,
usage: vk::ImageUsageFlags,
profile_list: Option<&mut vk::VideoProfileListInfoKHR>,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> { ) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
use anyhow::Context; use anyhow::Context;
use std::os::fd::{AsRawFd, IntoRawFd}; use std::os::fd::IntoRawFd;
let fmt = fourcc_to_vk(d.fourcc) let fmt = fourcc_to_vk(d.fourcc)
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?; .with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
// Dup the fd FIRST, and keep it OWNED: ownership transfers to Vulkan only on a SUCCESSFUL let plane = [vk::SubresourceLayout::default()
// `allocate_memory` (VK_KHR_external_memory_fd — from then on `vkFreeMemory` closes it), so
// the release below sits in exactly that arm. Every earlier failure drops the `OwnedFd` for
// a single clean close. An explicit `close` after a successful import would be a double
// close — and a recycled fd number then clobbers an unrelated descriptor in this process.
let dup = d.fd.try_clone().context("dup dmabuf fd")?;
let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
d.offset as u64 + d.stride as u64 * ch as u64,
d.stride as u64,
));
vec![
vk::SubresourceLayout::default()
.offset(d.offset as u64) .offset(d.offset as u64)
.row_pitch(d.stride as u64), .row_pitch(d.stride as u64)];
vk::SubresourceLayout::default()
.offset(uv_offset)
.row_pitch(uv_stride),
]
} else {
vec![vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64)]
};
let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default() let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default()
.drm_format_modifier(d.modifier) .drm_format_modifier(d.modifier)
.plane_layouts(&planes); .plane_layouts(&plane);
let mut ext = vk::ExternalMemoryImageCreateInfo::default() let mut ext = vk::ExternalMemoryImageCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT); .handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT);
let mut ci = vk::ImageCreateInfo::default() let img = device.create_image(
&vk::ImageCreateInfo::default()
.image_type(vk::ImageType::TYPE_2D) .image_type(vk::ImageType::TYPE_2D)
.format(fmt) .format(fmt)
.extent(vk::Extent3D { .extent(vk::Extent3D {
@@ -220,25 +103,21 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
.array_layers(1) .array_layers(1)
.samples(vk::SampleCountFlags::TYPE_1) .samples(vk::SampleCountFlags::TYPE_1)
.tiling(vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT) .tiling(vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT)
.usage(usage) .usage(vk::ImageUsageFlags::SAMPLED)
.sharing_mode(vk::SharingMode::EXCLUSIVE) .sharing_mode(vk::SharingMode::EXCLUSIVE)
.initial_layout(vk::ImageLayout::UNDEFINED) .initial_layout(vk::ImageLayout::UNDEFINED)
.push_next(&mut ext) .push_next(&mut ext)
.push_next(&mut drm); .push_next(&mut drm),
if let Some(pl) = profile_list { None,
ci = ci.push_next(pl); )?;
} // dup the fd; Vulkan takes ownership of the dup on a successful import.
let img = device.create_image(&ci, None)?; let dup = d.fd.try_clone().context("dup dmabuf fd")?.into_raw_fd();
// Unwind discipline below mirrors `make_plain_image`: every failure destroys what this call
// created (and ONLY that — the caller's `DmabufFrame` fd stays theirs).
let fd_props = { let fd_props = {
let mut p = vk::MemoryFdPropertiesKHR::default(); let mut p = vk::MemoryFdPropertiesKHR::default();
// Borrow-only query (no ownership transfer); an error leaves memory_type_bits = 0 and
// the fallback below uses the image requirements alone.
let _ = (ext_fd.fp().get_memory_fd_properties_khr)( let _ = (ext_fd.fp().get_memory_fd_properties_khr)(
device.handle(), device.handle(),
vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT, vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT,
dup.as_raw_fd(), dup,
&mut p, &mut p,
); );
p.memory_type_bits p.memory_type_bits
@@ -257,85 +136,25 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
let mut ded = vk::MemoryDedicatedAllocateInfo::default().image(img); let mut ded = vk::MemoryDedicatedAllocateInfo::default().image(img);
let mut import = vk::ImportMemoryFdInfoKHR::default() let mut import = vk::ImportMemoryFdInfoKHR::default()
.handle_type(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT) .handle_type(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT)
.fd(dup.as_raw_fd()); .fd(dup);
let mem = match device.allocate_memory( let mem = device.allocate_memory(
&vk::MemoryAllocateInfo::default() &vk::MemoryAllocateInfo::default()
.allocation_size(req.size) .allocation_size(req.size)
.memory_type_index(ti) .memory_type_index(ti)
.push_next(&mut ded) .push_next(&mut ded)
.push_next(&mut import), .push_next(&mut import),
None, None,
) { )?;
Ok(mem) => { device.bind_image_memory(img, mem, 0)?;
// Success transferred fd ownership to the memory object — release, don't close. let view = device.create_image_view(
let _ = dup.into_raw_fd();
mem
}
Err(e) => {
device.destroy_image(img, None);
return Err(e.into()); // `dup` drops here: the one close of the failed import's fd
}
};
if let Err(e) = device.bind_image_memory(img, mem, 0) {
device.destroy_image(img, None);
device.free_memory(mem, None); // closes the imported fd
return Err(e.into());
}
let view = match device.create_image_view(
&vk::ImageViewCreateInfo::default() &vk::ImageViewCreateInfo::default()
.image(img) .image(img)
.view_type(vk::ImageViewType::TYPE_2D) .view_type(vk::ImageViewType::TYPE_2D)
.format(fmt) .format(fmt)
.subresource_range(color_range(0)), .subresource_range(color_range(0)),
None, None,
) {
Ok(v) => v,
Err(e) => {
device.destroy_image(img, None);
device.free_memory(mem, None);
return Err(e.into());
}
};
Ok((img, mem, view))
}
/// Create + allocate + bind a host-visible/coherent buffer with `make_plain_image`'s unwind
/// discipline: on any failure everything this call created is destroyed before returning, so
/// callers can `?` freely. Both `ensure_cpu_rgb` staging twins open-coded this sequence and
/// leaked the buffer (and then buffer+memory) on the allocate/bind failure arms.
pub(crate) unsafe fn make_host_buffer(
device: &ash::Device,
mp: &vk::PhysicalDeviceMemoryProperties,
size: u64,
usage: vk::BufferUsageFlags,
) -> Result<(vk::Buffer, vk::DeviceMemory)> {
let buf = device.create_buffer(
&vk::BufferCreateInfo::default().size(size).usage(usage),
None,
)?; )?;
let req = device.get_buffer_memory_requirements(buf); Ok((img, mem, view))
let mem = match device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(find_mem(
mp,
req.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
) {
Ok(m) => m,
Err(e) => {
device.destroy_buffer(buf, None);
return Err(e.into());
}
};
if let Err(e) = device.bind_buffer_memory(buf, mem, 0) {
device.destroy_buffer(buf, None);
device.free_memory(mem, None);
return Err(e.into());
}
Ok((buf, mem))
} }
pub(crate) unsafe fn make_plain_image( pub(crate) unsafe fn make_plain_image(
@@ -364,8 +183,7 @@ pub(crate) unsafe fn make_plain_image(
None, None,
)?; )?;
let req = device.get_image_memory_requirements(img); let req = device.get_image_memory_requirements(img);
// Unwind on failure: callers (the encoders' open paths) only ever see the completed triple. let mem = device.allocate_memory(
let mem = match device.allocate_memory(
&vk::MemoryAllocateInfo::default() &vk::MemoryAllocateInfo::default()
.allocation_size(req.size) .allocation_size(req.size)
.memory_type_index(find_mem( .memory_type_index(find_mem(
@@ -374,101 +192,8 @@ pub(crate) unsafe fn make_plain_image(
vk::MemoryPropertyFlags::DEVICE_LOCAL, vk::MemoryPropertyFlags::DEVICE_LOCAL,
)), )),
None, None,
) { )?;
Ok(m) => m, device.bind_image_memory(img, mem, 0)?;
Err(e) => { let view = make_view(device, img, fmt, 0)?;
device.destroy_image(img, None); Ok((img, mem, view))
return Err(e.into());
}
};
if let Err(e) = device.bind_image_memory(img, mem, 0) {
device.destroy_image(img, None);
device.free_memory(mem, None);
return Err(e.into());
}
match make_view(device, img, fmt, 0) {
Ok(view) => Ok((img, mem, view)),
Err(e) => {
device.destroy_image(img, None);
device.free_memory(mem, None);
Err(e)
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// CSC mode (`bgra_target = false`): the 3→4 expand is a pure byte shuffle — no channel
/// reorder, pad byte 0xFF, truncated tail pixels dropped (never overrun) — and 4-bpp inputs
/// pass through borrowed untouched.
#[test]
fn normalize_cpu_rgb_expands_24bpp_and_borrows_4bpp() {
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3, 4, 5, 6], &mut scratch, false);
assert_eq!(f, PixelFormat::Rgbx);
assert_eq!(b, &[1, 2, 3, 0xFF, 4, 5, 6, 0xFF]);
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgr, &[9, 8, 7], &mut scratch, false);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[9, 8, 7, 0xFF]);
// Truncated tail: 5 bytes = one whole pixel + a 2-byte remainder that must be dropped.
let mut scratch = Vec::new();
let (_, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3, 4, 5], &mut scratch, false);
assert_eq!(b, &[1, 2, 3, 0xFF]);
// 4-bpp passthrough: borrowed, scratch untouched.
let src = [10u8, 20, 30, 40];
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgrx, &src, &mut scratch, false);
assert_eq!(f, PixelFormat::Bgrx);
assert!(std::ptr::eq(b.as_ptr(), src.as_ptr()));
assert!(scratch.is_empty());
// The 4-bpp mapping the expand lands on matches pixel_to_vk's existing table.
assert_eq!(
pixel_to_vk(PixelFormat::Rgbx),
Some(vk::Format::R8G8B8A8_UNORM)
);
assert_eq!(
pixel_to_vk(PixelFormat::Bgrx),
Some(vk::Format::B8G8R8A8_UNORM)
);
}
/// RGB-direct mode (`bgra_target = true`): everything lands in B,G,R,X order because the
/// video session's `pictureFormat` is `B8G8R8A8_UNORM` and the encode source must match it
/// (VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 — caught live on RADV). B-first inputs pass
/// through borrowed; R-first inputs are channel-swapped, 3-bpp and 4-bpp alike.
#[test]
fn normalize_cpu_rgb_forces_bgra_for_the_encode_source() {
// Rgb (R,G,B) → B,G,R,X with the swap folded into the expand.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[3, 2, 1, 0xFF]);
// Bgr (B,G,R) → same order, expanded.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgr, &[9, 8, 7], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[9, 8, 7, 0xFF]);
// Rgbx: 4-bpp but R-first — swapped, alpha replaced by the 0xFF pad.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgbx, &[1, 2, 3, 4], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[3, 2, 1, 0xFF]);
// Bgrx/Bgra already match the session order: borrowed untouched.
let src = [10u8, 20, 30, 40];
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgra, &src, &mut scratch, true);
assert_eq!(f, PixelFormat::Bgra);
assert!(std::ptr::eq(b.as_ptr(), src.as_ptr()));
assert!(scratch.is_empty());
}
} }
@@ -1,153 +0,0 @@
//! Vendored `VK_VALVE_video_encode_rgb_conversion` bindings — the RGB→YCbCr encode-source
//! extension (Vulkan 1.4.327; RADV since Mesa 26.0, hardware-gated on the VCN EFC front-end
//! conversion block). Our pinned `ash 0.38.0+1.3.281` predates it entirely; same vendoring
//! rationale as [`vk_av1_encode`](super::vk_av1_encode) — definitions copied from the registry
//! so the layouts are correct-by-construction, chained via raw `p_next`. Consumed by
//! `vulkan_video.rs`: B0 probes + logs availability (design/vulkan-rgb-direct-encode.md);
//! B1 makes the captured BGRx dmabuf the direct encode source with EFC doing the 709-narrow CSC.
#![allow(dead_code)]
use ash::vk;
use std::ffi::{c_void, CStr};
pub const EXTENSION_NAME: &CStr = c"VK_VALVE_video_encode_rgb_conversion";
// ---------- struct-type (VkStructureType) values — construct via `stype` ----------
pub const ST_PHYSICAL_DEVICE_FEATURES: i32 = 1_000_390_000;
pub const ST_CAPABILITIES: i32 = 1_000_390_001;
pub const ST_PROFILE_INFO: i32 = 1_000_390_002;
pub const ST_SESSION_CREATE_INFO: i32 = 1_000_390_003;
// `VkVideoEncodeRgbModelConversionFlagBitsVALVE`
pub const MODEL_RGB_IDENTITY: u32 = 0x01;
pub const MODEL_YCBCR_IDENTITY: u32 = 0x02;
pub const MODEL_YCBCR_709: u32 = 0x04;
pub const MODEL_YCBCR_601: u32 = 0x08;
pub const MODEL_YCBCR_2020: u32 = 0x10;
// `VkVideoEncodeRgbRangeCompressionFlagBitsVALVE`
pub const RANGE_FULL: u32 = 0x01;
pub const RANGE_NARROW: u32 = 0x02;
// `VkVideoEncodeRgbChromaOffsetFlagBitsVALVE`
pub const CHROMA_OFFSET_COSITED_EVEN: u32 = 0x01;
pub const CHROMA_OFFSET_MIDPOINT: u32 = 0x02;
/// `VkPhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE` — chain into
/// `VkPhysicalDeviceFeatures2` (query) / `VkDeviceCreateInfo` (enable).
#[repr(C)]
pub struct PhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE {
pub s_type: vk::StructureType,
pub p_next: *mut c_void,
pub video_encode_rgb_conversion: vk::Bool32,
}
/// `VkVideoEncodeRgbConversionCapabilitiesVALVE` — chain into the
/// `vkGetPhysicalDeviceVideoCapabilitiesKHR` output when the queried profile carries
/// [`VideoEncodeProfileRgbConversionInfoVALVE`]; reports which conversions the HW does.
#[repr(C)]
pub struct VideoEncodeRgbConversionCapabilitiesVALVE {
pub s_type: vk::StructureType,
pub p_next: *mut c_void,
pub rgb_models: u32,
pub rgb_ranges: u32,
pub x_chroma_offsets: u32,
pub y_chroma_offsets: u32,
}
/// `VkVideoEncodeProfileRgbConversionInfoVALVE` — part of the video-profile *identity*: every
/// consumer of the profile (caps query, format query, session, image profile lists) must carry
/// the same chain.
#[repr(C)]
pub struct VideoEncodeProfileRgbConversionInfoVALVE {
pub s_type: vk::StructureType,
pub p_next: *const c_void,
pub perform_encode_rgb_conversion: vk::Bool32,
}
/// `VkVideoEncodeSessionRgbConversionCreateInfoVALVE` — chain into
/// `VkVideoSessionCreateInfoKHR`; single-bit selections of the conversion actually performed.
#[repr(C)]
pub struct VideoEncodeSessionRgbConversionCreateInfoVALVE {
pub s_type: vk::StructureType,
pub p_next: *const c_void,
pub rgb_model: u32,
pub rgb_range: u32,
pub x_chroma_offset: u32,
pub y_chroma_offset: u32,
}
/// `vk::StructureType` for a raw `ST_*` constant above.
#[inline]
pub fn stype(raw: i32) -> vk::StructureType {
vk::StructureType::from_raw(raw)
}
// ---------- ABI layout guard ----------
//
// These structs are hand-copied from the registry and handed to the driver through raw `p_next`
// chains, so nothing in the type system relates them to the C definitions any more: an edit that
// inserts, drops, widens or re-pads a field is not a compile error, it is the driver reading our
// bytes at the wrong offsets. The assertions below are the missing compile error. They are `const`
// rather than `#[cfg(test)]` (the shape `amf.rs` uses) so they hold in every build, including the
// shipped one, and on any target this module compiles for.
//
// What they catch: a changed field width, an inserted or removed field, a changed array length, a
// padding assumption that only holds on one target. What they CANNOT catch: swapping two fields of
// the same type — offsets are unchanged. That case is only caught by reading the registry, so the
// field order here was diffed against `vulkan_core.h` (Vulkan-Headers `main`, 2026-07-25) when
// these assertions were written, along with every `ST_*` and flag-bit value above.
//
// Deliberately duplicated in `vk_av1_encode.rs` rather than shared: both modules exist to be
// deleted wholesale once `ash` ships these bindings, and a shared helper would make deleting one
// break the other.
macro_rules! assert_abi_layout {
($t:ty { size: $size:expr, align: $align:expr $(, $field:ident @ $off:expr)* $(,)? }) => {
const _: () = {
assert!(
::core::mem::size_of::<$t>() == $size,
concat!(stringify!($t), ": size does not match the C ABI")
);
assert!(
::core::mem::align_of::<$t>() == $align,
concat!(stringify!($t), ": alignment does not match the C ABI")
);
$(assert!(
::core::mem::offset_of!($t, $field) == $off,
concat!(stringify!($t), ".", stringify!($field), ": offset does not match the C ABI")
);)*
};
};
}
assert_abi_layout!(PhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
video_encode_rgb_conversion @ 16,
});
assert_abi_layout!(VideoEncodeRgbConversionCapabilitiesVALVE {
size: 32, align: 8,
s_type @ 0,
p_next @ 8,
rgb_models @ 16,
rgb_ranges @ 20,
x_chroma_offsets @ 24,
y_chroma_offsets @ 28,
});
assert_abi_layout!(VideoEncodeProfileRgbConversionInfoVALVE {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
perform_encode_rgb_conversion @ 16,
});
assert_abi_layout!(VideoEncodeSessionRgbConversionCreateInfoVALVE {
size: 32, align: 8,
s_type @ 0,
p_next @ 8,
rgb_model @ 16,
rgb_range @ 20,
x_chroma_offset @ 24,
y_chroma_offset @ 28,
});
File diff suppressed because it is too large Load Diff
+3 -465
View File
@@ -35,417 +35,12 @@ pub(super) fn codec_guid(codec: Codec) -> nv::GUID {
} }
} }
/// Resolved per-frame slice count for a session (latency plan §7 LN1, Phase 3): the
/// `PUNKTFUNK_NVENC_SLICES` env override wins (1..=32; **1 = the explicit single-slice
/// escape**, needed now that a backend can default higher), else the backend's
/// `default_slices` — 4 on Linux direct-NVENC since the Phase-3 default-on, 1 everywhere else
/// (the Windows async path is deliberately untouched). H.264/HEVC only (AV1 partitions via
/// tiles). ONE parse shared by the config author ([`apply_low_latency_config`] via
/// [`LowLatencyConfig::slices`]) and the Linux backend's chunked-poll arming, so the two can
/// never disagree about whether a session is multi-slice.
pub(super) fn resolve_slices(codec: Codec, default_slices: u32) -> u32 {
if !matches!(codec, Codec::H264 | Codec::H265) {
return 1;
}
std::env::var("PUNKTFUNK_NVENC_SLICES")
.ok()
.and_then(|s| s.parse::<u32>().ok())
.filter(|n| (1..=32).contains(n))
.unwrap_or(default_slices)
}
/// Resolved sub-frame readback (`enableSubFrameWrite` + `reportSliceOffsets`; sync sessions
/// only, see [`build_init_params`]): `PUNKTFUNK_NVENC_SUBFRAME` tri-state — `0` = never (the
/// default-on escape), `1` = force (even where the caps probe says unsupported — an operator
/// explicitly testing), unset = the backend's `default_on` (Linux direct-NVENC passes its
/// SUBFRAME_READBACK caps-probe result since Phase 3; Windows passes `false`).
pub(super) fn resolve_subframe(default_on: bool) -> bool {
match std::env::var("PUNKTFUNK_NVENC_SUBFRAME").as_deref() {
Ok("0") => false,
Ok("1") => true,
_ => default_on,
}
}
/// Resolved NVENC split-frame encode mode for a session — ONE selector shared by the Windows and
/// Linux direct-SDK backends (they had drifted into byte-identical duplicates, one of which
/// logged and one didn't). Precedence:
/// 1. `PUNKTFUNK_SPLIT_ENCODE` = `0`/`disable` | `1`/`auto` (AUTO_FORCED) | `2` | `3` — operator
/// override, always wins.
/// 2. 10-bit → DISABLE: 2-way split is measurably SLOWER on Ada for Main10 — at 5120×1440@240
/// forced-2 took 7.6 ms/frame (~131 fps) vs 2.8 ms (~357 fps) single-engine (the split/merge
/// overhead dominates), and a single engine handles 5K@240 Main10 well under budget. This was
/// the "broken animations in HDR" cap at ~131 fps.
/// 3. Pixel rate ≥ [`super::SPLIT_FORCE_PIXEL_RATE`] → force 2-way (AUTO never engages below
/// ~2112 px height, so 4K120 must be forced onto the second engine).
/// 4. Else AUTO (the ~2% BD-rate split cost isn't worth it at low pixel rates).
///
/// The caller still owns the rejection fallback (retry split-disabled) — a codec/config that
/// rejects the chosen mode downgrades at open, not here.
pub(super) fn resolve_split_mode(bit_depth: u8, pixel_rate: u64) -> u32 {
use nv::NV_ENC_SPLIT_ENCODE_MODE as M;
let mode = match std::env::var("PUNKTFUNK_SPLIT_ENCODE").ok().as_deref() {
Some("0") | Some("disable") => M::NV_ENC_SPLIT_DISABLE_MODE as u32,
Some("1") | Some("auto") => M::NV_ENC_SPLIT_AUTO_FORCED_MODE as u32,
Some("3") => M::NV_ENC_SPLIT_THREE_FORCED_MODE as u32,
Some("2") => M::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
_ if bit_depth >= 10 => M::NV_ENC_SPLIT_DISABLE_MODE as u32,
_ if pixel_rate >= super::SPLIT_FORCE_PIXEL_RATE => M::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
_ => M::NV_ENC_SPLIT_AUTO_MODE as u32,
};
tracing::debug!(
split_mode = mode,
bit_depth,
pixel_rate,
"NVENC split-encode mode selected"
);
mode
}
/// One session config's identity for the process-lifetime bitrate-ceiling cache
/// ([`cached_ceiling`]/[`store_ceiling`]). Everything the driver's codec-level validation keys
/// off: the GPU (different NVENC generations have different level ceilings), dims/fps (the luma
/// rate selects the level), depth/chroma (they select the profile) and the split mode the
/// sessions ACTUALLY opened with (a split session budgets per engine).
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
pub(super) struct CeilingKey {
/// GPU identity — Linux: the process-global shared `CUcontext` pointer; Windows: the render
/// adapter LUID (0 when unresolved). Best effort: the cache is advisory (see
/// [`cached_ceiling`]), so a colliding identity costs one failed open + re-search, never a
/// wrong session.
pub gpu: u64,
pub codec: Codec,
pub width: u32,
pub height: u32,
pub fps: u32,
pub bit_depth: u8,
pub chroma_444: bool,
pub split_mode: u32,
}
fn ceilings() -> &'static std::sync::Mutex<std::collections::HashMap<CeilingKey, u64>> {
static CEILINGS: std::sync::OnceLock<
std::sync::Mutex<std::collections::HashMap<CeilingKey, u64>>,
> = std::sync::OnceLock::new();
CEILINGS.get_or_init(Default::default)
}
/// The codec-level bitrate ceiling (bps) a previous clamp search discovered for `key` this
/// process lifetime, if any. ADVISORY: the consumer must treat a failed open at the cached value
/// as a stale entry (fall back to the full search, which rewrites it via [`store_ceiling`]) —
/// that self-healing is what lets the key's GPU identity be best-effort. What this buys: an ABR
/// overshoot on a config whose ceiling is already known opens (or in-place reconfigures) straight
/// AT the ceiling instead of re-running the ~6-open binary search and its ~half-second of session
/// churn per rebuild.
pub(super) fn cached_ceiling(key: &CeilingKey) -> Option<u64> {
ceilings().lock().unwrap().get(key).copied()
}
/// Record the clamp search's discovered max accepted bitrate (bps) for `key`.
pub(super) fn store_ceiling(key: CeilingKey, bps: u64) {
ceilings().lock().unwrap().insert(key, bps);
}
#[cfg(test)]
mod tests {
use super::*;
use nv::NV_ENC_SPLIT_ENCODE_MODE as M;
// These assume PUNKTFUNK_SPLIT_ENCODE is unset (CI); an operator override deliberately wins.
/// `encodeCodecConfig` is a C union, so the HEVC 4:4:4 arm must be codec-gated or it stamps
/// `hevcConfig` bytes onto another codec's config. Before the gate this branch was reached on
/// ANY codec with `chroma_444 && full_chroma_input` and stayed non-UB only because `lib.rs`
/// degrades 4:4:4 for non-HEVC — a two-file invariant with nothing asserting it.
///
/// It also had to stop swallowing the per-codec bit-depth arm: this is an `if`/`else if`, so a
/// non-HEVC 4:4:4 session used to take the HEVC branch and get NEITHER 4:4:4 nor its own 10-bit
/// setup. AV1 asserts the depth it actually needs.
fn low_latency_cfg(codec: Codec, chroma_444: bool, bit_depth: u8) -> LowLatencyConfig {
LowLatencyConfig {
codec,
bitrate: 20_000_000,
fps: 60,
custom_vbv: false,
chroma_444,
full_chroma_input: true,
bit_depth,
av1_input_depth_minus8: 0,
hdr: false,
rfi_supported: false,
slices: 0,
}
}
#[test]
fn hevc_444_still_takes_the_frext_path() {
// `NV_ENC_CONFIG` must NOT be `mem::zeroed` — `frameFieldMode`/`mvPrecision` are C enums
// whose discriminants start at 1, so all-zero is not a valid value and Rust's own
// zero-init check aborts the process. Production seeds it the same way, from `Default`
// (then overwrites from the driver's preset).
// SAFETY: `apply_low_latency_config` only writes into the caller's config (union writes
// included) and makes no driver calls, so this is pure in-memory work.
let cfg = unsafe {
let mut cfg = nv::NV_ENC_CONFIG {
version: nv::NV_ENC_CONFIG_VER,
..Default::default()
};
apply_low_latency_config(&mut cfg, low_latency_cfg(Codec::H265, true, 10));
cfg
};
assert_eq!(cfg.profileGUID, nv::NV_ENC_HEVC_PROFILE_FREXT_GUID);
// SAFETY: an HEVC session's union arm is `hevcConfig` — the one this path wrote.
unsafe {
assert_eq!(cfg.encodeCodecConfig.hevcConfig.chromaFormatIDC(), 3);
assert_eq!(cfg.encodeCodecConfig.hevcConfig.pixelBitDepthMinus8(), 2);
}
}
#[test]
fn av1_never_takes_the_hevc_444_union_write() {
// SAFETY: as above — pure in-memory config authoring, no driver involvement.
let cfg = unsafe {
let mut cfg = nv::NV_ENC_CONFIG {
version: nv::NV_ENC_CONFIG_VER,
..Default::default()
};
apply_low_latency_config(&mut cfg, low_latency_cfg(Codec::Av1, true, 10));
cfg
};
// The HEVC FREXT profile GUID on an AV1 session is an INVALID_PARAM at open.
assert_ne!(
cfg.profileGUID,
nv::NV_ENC_HEVC_PROFILE_FREXT_GUID,
"4:4:4 on AV1 must not stamp the HEVC FREXT profile"
);
// ...and the AV1 arm must still have run, which the old if/else-if skipped entirely.
// SAFETY: an AV1 session's union arm is `av1Config`.
unsafe {
assert_eq!(
cfg.encodeCodecConfig.av1Config.pixelBitDepthMinus8(),
2,
"AV1 10-bit setup was swallowed by the HEVC 4:4:4 branch"
);
}
}
#[test]
fn split_forces_two_way_at_4k120() {
// The regression this threshold constant exists for: 3840×2160×120 = 995,328,000 sat
// 0.47% under the old `> 1_000_000_000` gate and stayed AUTO — pinned ~107 fps on a
// 4090 because AUTO never engages at 2160 px height.
let four_k_120 = 3840u64 * 2160 * 120;
assert_eq!(
resolve_split_mode(8, four_k_120),
M::NV_ENC_SPLIT_TWO_FORCED_MODE as u32
);
}
#[test]
fn split_leaves_1440p240_auto() {
// 884.7 Mpix/s is comfortably single-engine — the threshold move must not drag it in.
let qhd_240 = 2560u64 * 1440 * 240;
assert_eq!(
resolve_split_mode(8, qhd_240),
M::NV_ENC_SPLIT_AUTO_MODE as u32
);
}
#[test]
fn split_disabled_for_10bit_even_at_high_pixel_rate() {
// The measured Main10 rule: split/merge overhead dominates 10-bit on Ada (7.6 ms forced-2
// vs 2.8 ms single-engine at 5K240) — 10-bit precedes the pixel-rate arm.
let five_k_240 = 5120u64 * 1440 * 240;
assert_eq!(
resolve_split_mode(10, five_k_240),
M::NV_ENC_SPLIT_DISABLE_MODE as u32
);
}
#[test]
fn ceiling_cache_round_trips_and_keys_precisely() {
let key = CeilingKey {
gpu: 0xB0B0,
codec: Codec::H265,
width: 3840,
height: 2160,
fps: 120,
bit_depth: 8,
chroma_444: false,
split_mode: M::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
};
assert_eq!(cached_ceiling(&key), None);
store_ceiling(key, 794_000_000);
assert_eq!(cached_ceiling(&key), Some(794_000_000));
// Any config-identity change is a different ceiling — a miss, never a wrong clamp.
assert_eq!(cached_ceiling(&CeilingKey { fps: 60, ..key }), None);
assert_eq!(
cached_ceiling(&CeilingKey {
split_mode: M::NV_ENC_SPLIT_DISABLE_MODE as u32,
..key
}),
None
);
// A re-search overwrites (the advisory-cache stale-entry path).
store_ceiling(key, 620_000_000);
assert_eq!(cached_ceiling(&key), Some(620_000_000));
}
}
/// Reference-frame DPB depth when RFI is supported (Apollo uses 5). A deeper DPB lets an invalidated /// Reference-frame DPB depth when RFI is supported (Apollo uses 5). A deeper DPB lets an invalidated
/// reference fall back to an older still-valid frame instead of a full IDR; `numRefL0 = 1` keeps each /// reference fall back to an older still-valid frame instead of a full IDR; `numRefL0 = 1` keeps each
/// P-frame single-reference for low latency. Also the window [`plan_range_recovery`] checks against /// P-frame single-reference for low latency. Also the window the backends' `invalidate_ref_frames`
/// (`next_ts - RFI_DPB` = the oldest frame still in the DPB). /// paths check against (`frame_idx - RFI_DPB` = the oldest frame still in the DPB).
pub(super) const RFI_DPB: u32 = 5; pub(super) const RFI_DPB: u32 = 5;
/// One loss event's recovery decision for the timestamp-range RFI both direct-NVENC backends run
/// (the range half of WP7.2's policy extraction; the slot half — AMF/QSV/Vulkan — is
/// `crate::rfi`). The mechanism (the per-timestamp `nvEncInvalidateRefFrames` loop, the
/// `last_rfi_range`/`pending_anchor` stores, the null-handle/`rfi_supported` gate) stays in each
/// backend.
pub(super) enum RangePlan {
/// The last successful invalidation already covers this range — no new driver calls, no IDR.
/// The caller must still RE-ARM its recovery anchor: the client re-asking means the previous
/// anchor AU may itself have been lost, and the next frame is just as clean a re-anchor.
Covered,
/// Invalidate `first..=last` (the CLAMPED range — this is also what the caller must record in
/// `last_rfi_range` on success, exactly as the inline code stored the post-clamp values).
Invalidate { first: i64, last: i64 },
/// Recovery without an IDR is impossible (nonsense range, loss older than the DPB, or a range
/// entirely in the future) — the caller returns `false` and its (coalesced) keyframe path
/// recovers. Deliberately NOT paired with any state clearing: neither twin touches
/// `pending_anchor` on decline (matching Vulkan's decline, opposite of AMF/QSV's
/// `pending_force` clear — see `crate::rfi`'s module doc before "harmonizing").
Decline,
}
/// The range-RFI policy, extracted verbatim from the two backends' `invalidate_ref_frames` (they
/// were hand-copied twins). Step order is load-bearing and pinned by tests:
///
/// 1. nonsense range (`first < 0 || first > last`) → [`RangePlan::Decline`];
/// 2. covering-range dedup — checked with the UNCLAMPED `last`, BEFORE the DPB window, so a
/// covered re-ask never touches the driver even when the range has since left the DPB;
/// 3. DPB window: `first < next_ts - RFI_DPB` → Decline (a lost frame older than the DPB cannot
/// be invalidated; the only correct recovery is an IDR);
/// 4. clamp `last` to `next_ts - 1` (never invalidate a timestamp never assigned); an inverted
/// range after the clamp (loss entirely in the future — a prediction desync) → Decline.
///
/// `next_ts` is the backend's `frame_idx`: the NEXT timestamp to assign, which `submit_indexed`
/// pins to the wire frame index — so the client's lost-frame range maps 1:1 onto the timestamps
/// the driver invalidates, across every rebuild/reset. Note `teardown()` clears `last_rfi_range`
/// but NOT `frame_idx`, so a post-reset call legitimately sees a stale-high `next_ts` with a
/// `None` range — the same view the inline code had.
pub(super) fn plan_range_recovery(
first: i64,
last: i64,
next_ts: i64,
last_rfi_range: Option<(i64, i64)>,
) -> RangePlan {
if first < 0 || first > last {
return RangePlan::Decline;
}
if let Some((pf, pl)) = last_rfi_range {
if first >= pf && last <= pl {
return RangePlan::Covered;
}
}
let oldest_in_dpb = next_ts - RFI_DPB as i64;
if first < oldest_in_dpb {
return RangePlan::Decline;
}
let last = last.min(next_ts - 1);
if first > last {
return RangePlan::Decline;
}
RangePlan::Invalidate { first, last }
}
#[cfg(test)]
mod range_policy_tests {
use super::{plan_range_recovery, RangePlan, RFI_DPB};
/// Convenience: the plan with no prior invalidation recorded.
fn plan(first: i64, last: i64, next_ts: i64) -> RangePlan {
plan_range_recovery(first, last, next_ts, None)
}
#[test]
fn nonsense_ranges_decline() {
assert!(matches!(plan(-1, 5, 100), RangePlan::Decline));
assert!(matches!(plan(7, 5, 100), RangePlan::Decline));
}
#[test]
fn covering_range_dedups_partial_overlap_does_not() {
let prior = Some((90i64, 95i64));
// Exact cover and sub-range → Covered. This pins EXISTING behavior, including that a
// covered range survives a forced IDR with zero driver calls (nothing clears
// `last_rfi_range` on a keyframe) — a recorded fact, not an endorsement.
assert!(matches!(
plan_range_recovery(90, 95, 100, prior),
RangePlan::Covered
));
assert!(matches!(
plan_range_recovery(92, 94, 100, prior),
RangePlan::Covered
));
// Partial overlap re-invalidates the FULL new range (next_ts = 98 keeps the window open:
// oldest_in_dpb = 93; at next_ts = 100 the same range would age out and Decline instead).
assert!(matches!(
plan_range_recovery(93, 97, 98, prior),
RangePlan::Invalidate {
first: 93,
last: 97
}
));
}
/// The covering check runs BEFORE the DPB window: a covered re-ask stays Covered (no driver
/// calls needed) even when the range has since aged out of the DPB.
#[test]
fn covered_is_checked_before_the_dpb_window() {
let prior = Some((10i64, 12i64));
assert!(matches!(
plan_range_recovery(10, 12, 100, prior),
RangePlan::Covered
));
// ...whereas the same range with no prior invalidation is outside the window → Decline.
assert!(matches!(plan(10, 12, 100), RangePlan::Decline));
}
#[test]
fn dpb_window_boundary() {
let next_ts = 100i64;
let oldest = next_ts - RFI_DPB as i64; // 95: the oldest timestamp still in the DPB
assert!(matches!(
plan(oldest, oldest, next_ts),
RangePlan::Invalidate { .. }
));
assert!(matches!(
plan(oldest - 1, oldest, next_ts),
RangePlan::Decline
));
}
/// `last` clamps to `next_ts - 1` (the newest encoded frame); the Invalidate carries the
/// CLAMPED value — which is also what the caller records in `last_rfi_range`.
#[test]
fn clamps_to_newest_encoded() {
assert!(matches!(
plan(98, 150, 100),
RangePlan::Invalidate {
first: 98,
last: 99
}
));
// A range entirely in the future inverts under the clamp → Decline (prediction desync).
assert!(matches!(plan(100, 150, 100), RangePlan::Decline));
// Fresh session (`frame_idx == 0`): window passes (oldest = -5) but the clamp gives
// last = -1 < first → Decline. The inline code behaved identically.
assert!(matches!(plan(0, 3, 0), RangePlan::Decline));
}
}
/// The per-session knobs both direct-NVENC backends feed [`apply_low_latency_config`]. `Copy` so the /// The per-session knobs both direct-NVENC backends feed [`apply_low_latency_config`]. `Copy` so the
/// backend fills it from `self` at the call. The two input-format fields bridge the only real /// backend fills it from `self` at the call. The two input-format fields bridge the only real
/// divergence between the CUDA and D3D11 paths (which surface formats can carry full chroma / 10-bit /// divergence between the CUDA and D3D11 paths (which surface formats can carry full chroma / 10-bit
@@ -471,9 +66,6 @@ pub(super) struct LowLatencyConfig {
pub hdr: bool, pub hdr: bool,
/// This GPU supports reference-frame invalidation (a deeper DPB for graceful loss recovery). /// This GPU supports reference-frame invalidation (a deeper DPB for graceful loss recovery).
pub rfi_supported: bool, pub rfi_supported: bool,
/// Resolved per-frame slice count ([`resolve_slices`] — env override, else the backend
/// default). ≤ 1 leaves the preset's single slice untouched.
pub slices: u32,
} }
/// Author the shared `NV_ENC_INITIALIZE_PARAMS` (P1/ULL preset, PTD, the session dimensions/rate) /// Author the shared `NV_ENC_INITIALIZE_PARAMS` (P1/ULL preset, PTD, the session dimensions/rate)
@@ -490,7 +82,6 @@ pub(super) fn build_init_params(
cfg: &mut nv::NV_ENC_CONFIG, cfg: &mut nv::NV_ENC_CONFIG,
split_mode: u32, split_mode: u32,
enable_async: bool, enable_async: bool,
subframe: bool,
) -> nv::NV_ENC_INITIALIZE_PARAMS { ) -> nv::NV_ENC_INITIALIZE_PARAMS {
let mut init = nv::NV_ENC_INITIALIZE_PARAMS { let mut init = nv::NV_ENC_INITIALIZE_PARAMS {
version: nv::NV_ENC_INITIALIZE_PARAMS_VER, version: nv::NV_ENC_INITIALIZE_PARAMS_VER,
@@ -510,16 +101,6 @@ pub(super) fn build_init_params(
}; };
// splitEncodeMode is a C bitfield — set via the generated accessor, not a struct field. // splitEncodeMode is a C bitfield — set via the generated accessor, not a struct field.
init.set_splitEncodeMode(split_mode); init.set_splitEncodeMode(split_mode);
// Sub-frame readback (latency plan §7 LN1; default-on for Linux direct-NVENC since Phase 3 —
// the caller resolves `subframe` via [`resolve_subframe`] + its caps probe): the driver
// writes each slice into the output buffer as it completes and reports per-slice offsets, so
// a sync-mode consumer can read slices out while the frame is still encoding. Pair with
// multi-slice (a single-slice frame yields nothing to read early). `reportSliceOffsets`
// requires `enableEncodeAsync = 0`, so async (Windows) sessions never arm.
if !enable_async && subframe {
init.set_enableSubFrameWrite(1);
init.set_reportSliceOffsets(1);
}
init init
} }
@@ -537,9 +118,6 @@ pub(super) unsafe fn apply_low_latency_config(cfg: &mut nv::NV_ENC_CONFIG, c: Lo
cfg.gopLength = nv::NVENC_INFINITE_GOPLENGTH; cfg.gopLength = nv::NVENC_INFINITE_GOPLENGTH;
cfg.frameIntervalP = 1; cfg.frameIntervalP = 1;
cfg.rcParams.rateControlMode = nv::NV_ENC_PARAMS_RC_MODE::NV_ENC_PARAMS_RC_CBR; cfg.rcParams.rateControlMode = nv::NV_ENC_PARAMS_RC_MODE::NV_ENC_PARAMS_RC_CBR;
// Explicit zero reorder delay: with P-only + no lookahead there is no reordering to buffer,
// but pin the bit so no preset/driver default can ever slip a frame of reorder delay in.
cfg.rcParams.set_zeroReorderDelay(1);
let bps = c.bitrate.min(u32::MAX as u64) as u32; let bps = c.bitrate.min(u32::MAX as u64) as u32;
cfg.rcParams.averageBitRate = bps; cfg.rcParams.averageBitRate = bps;
cfg.rcParams.maxBitRate = bps; cfg.rcParams.maxBitRate = bps;
@@ -568,51 +146,11 @@ pub(super) unsafe fn apply_low_latency_config(cfg: &mut nv::NV_ENC_CONFIG, c: Lo
Codec::PyroWave => unreachable!("PyroWave never opens the direct-NVENC backend"), Codec::PyroWave => unreachable!("PyroWave never opens the direct-NVENC backend"),
} }
// Multi-slice frames (latency plan §7 LN1): `c.slices` splits every frame into N slices
// (sliceMode 3 = "N slices per frame"), the unit sub-frame readback ships early and loss
// concealment can discard independently. Costs ~1-2 % bitrate in slice headers. H.264/HEVC
// only — AV1 partitions via tiles, not slices (the resolver already returns 1 there).
// Default 4 on Linux direct-NVENC (Phase 3), env-only elsewhere; ≤ 1 keeps the preset's
// single slice.
if let Some(n) = Some(c.slices).filter(|n| *n >= 2) {
match c.codec {
Codec::H264 => {
cfg.encodeCodecConfig.h264Config.sliceMode = 3;
cfg.encodeCodecConfig.h264Config.sliceModeData = n;
}
Codec::H265 => {
cfg.encodeCodecConfig.hevcConfig.sliceMode = 3;
cfg.encodeCodecConfig.hevcConfig.sliceModeData = n;
}
Codec::Av1 | Codec::PyroWave => {}
}
}
// Chroma + bit depth. Full-chroma 4:4:4 (HEVC Range Extensions, chromaFormatIDC=3 under the FREXT // Chroma + bit depth. Full-chroma 4:4:4 (HEVC Range Extensions, chromaFormatIDC=3 under the FREXT
// profile) takes precedence and composes with 10-bit (Main 4:4:4 10); it needs a full-chroma- // profile) takes precedence and composes with 10-bit (Main 4:4:4 10); it needs a full-chroma-
// capable input. Otherwise 10-bit selects Main10 (HEVC) or the AV1 output depth — stamping the // capable input. Otherwise 10-bit selects Main10 (HEVC) or the AV1 output depth — stamping the
// HEVC Main10 GUID onto an AV1 session is an INVALID_PARAM, so bit depth is set PER CODEC. // HEVC Main10 GUID onto an AV1 session is an INVALID_PARAM, so bit depth is set PER CODEC.
// `encodeCodecConfig` is a C UNION, so the `hevcConfig` writes below are only meaningful on an if c.chroma_444 && c.full_chroma_input {
// HEVC session — on an H.264 or AV1 one they reinterpret that codec's own config bytes. The
// codec test is therefore load-bearing, not defensive: without it this branch was gated purely
// on `chroma_444 && full_chroma_input` and stayed non-UB only because `lib.rs` degrades 4:4:4
// for non-HEVC codecs. That was a two-file invariant with nothing asserting it, on the path
// BOTH direct-NVENC backends take.
//
// Being a codec test also fixes a second, quieter bug in the same shape: this is an
// `if`/`else if`, so a non-HEVC session that somehow arrived with `chroma_444` set took this
// branch and skipped the per-codec bit-depth arm entirely — ending up with neither HEVC 4:4:4
// (wrong for it) nor its own 10-bit configuration (simply missing). Non-HEVC now falls through
// to the arm that knows what to do with it.
let want_444 = c.chroma_444 && c.full_chroma_input;
if want_444 && c.codec != Codec::H265 {
tracing::warn!(
codec = ?c.codec,
"4:4:4 requested on a non-HEVC NVENC session — ignoring it (Range Extensions are \
HEVC-only); the negotiator should have degraded this to 4:2:0 before the open"
);
}
if want_444 && c.codec == Codec::H265 {
cfg.profileGUID = nv::NV_ENC_HEVC_PROFILE_FREXT_GUID; cfg.profileGUID = nv::NV_ENC_HEVC_PROFILE_FREXT_GUID;
cfg.encodeCodecConfig.hevcConfig.set_chromaFormatIDC(3); cfg.encodeCodecConfig.hevcConfig.set_chromaFormatIDC(3);
if c.bit_depth == 10 { if c.bit_depth == 10 {
+16 -201
View File
@@ -8,77 +8,27 @@
//! means and what the operator should do, and folds that cause into the `anyhow::Error` at //! means and what the operator should do, and folds that cause into the `anyhow::Error` at
//! construction, so every downstream `{e:#}` log (the encode-recovery loop, session teardown) says //! construction, so every downstream `{e:#}` log (the encode-recovery loop, session teardown) says
//! the useful thing without extra plumbing. //! the useful thing without extra plumbing.
//!
//! One status needs process state to explain honestly: the driver reports BOTH "your headers are
//! newer than my kernel module" and "I can no longer hand this process a session" as
//! `NV_ENC_ERR_INVALID_VERSION`. [`note_session_opened`] latches the fact that a session already
//! opened here, which tells the two apart — see [`explain`].
use std::sync::atomic::{AtomicBool, Ordering};
use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv; use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv;
/// Latched the first time `nvEncOpenEncodeSessionEx` succeeds in this process (the caps probe, a
/// real session open, or the Windows availability probe — every one of them completes the
/// userspace↔kernel-module handshake).
///
/// The load-time gate (`NvEncodeAPIGetMaxSupportedVersion`, both backends' `load_api`) can NOT
/// serve this purpose: it is a pure userspace query, so the genuine "updated the driver, didn't
/// reboot" skew sails through it and fails later, at the open. Only a session that actually opened
/// proves the kernel module agreed.
static SESSION_OPENED: AtomicBool = AtomicBool::new(false);
/// Record that an NVENC session opened. Call right after every successful
/// `open_encode_session_ex`, so [`explain`] can rule a version skew out for the rest of the
/// process.
pub(super) fn note_session_opened() {
SESSION_OPENED.store(true, Ordering::Relaxed);
}
/// The two very different failures the driver reports as `NV_ENC_ERR_INVALID_VERSION`, split on
/// whether a session has already opened here (`session_opened`). Pure, so both halves are testable
/// without touching the process-wide latch.
fn invalid_version(session_opened: bool) -> String {
if session_opened {
// Same status, opposite cause: a session ALREADY opened in this process, so the driver's
// kernel module accepted this exact build's version word minutes ago. A version skew is
// static — it cannot come and go — so "update the driver / reboot" is the wrong advice
// here, and following it costs the operator a reboot per stream (2026-07 field report: one
// stream works, the next fails at the caps probe, forever, until the PROCESS restarts).
// What is left is per-process driver state: a resource the last session did not give back,
// or a wedged/lost device. Say that, and point at the cheap fix.
// Worded for ANY call (`explain` also serves `lock_bitstream`); `call_err` already names
// the entry point ahead of this text, so it must not assume the session open.
return "this process already opened an NVENC session successfully, so this is NOT a driver \
version mismatch that cannot come and go within a process, and a reboot is not \
the fix. The NVIDIA driver state in THIS process is exhausted or wedged: restart \
the Punktfunk host service to clear it, and please report this with the host log \
so it can be fixed properly"
.to_string();
}
// No session has ever opened here, so the version word really is in question. Either the
// driver is genuinely older than our headers, or (the sneaky case) the userspace
// `libnvidia-encode` reports a new-enough version to the pre-flight probe but the running
// kernel module is older and rejects the session — the classic "updated the driver, didn't
// reboot" skew. Both heal the same way.
format!(
"the NVIDIA driver is older than this build's NVENC headers (needs NVENC API {}.{} or \
newer), or the userspace and kernel-module driver versions are mismatched common right \
after a driver update without a reboot. Update the NVIDIA driver, or reboot if you just \
updated it (a host restart is the usual fix).",
nv::NVENCAPI_MAJOR_VERSION,
nv::NVENCAPI_MINOR_VERSION,
)
}
/// A one-line, operator-actionable cause for an NVENC status. Does not repeat the raw code — /// A one-line, operator-actionable cause for an NVENC status. Does not repeat the raw code —
/// callers print that alongside (see [`call_err`]). Public for the few sites that build a /// callers print that alongside (see [`call_err`]). Public for the few sites that build a
/// `String`/`format!` error instead of an `anyhow::Error`. /// `String`/`format!` error instead of an `anyhow::Error`.
pub(super) fn explain(status: nv::NVENCSTATUS) -> String { pub(super) fn explain(status: nv::NVENCSTATUS) -> String {
match status { match status {
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_VERSION => { // The one this whole module exists for: a version word the driver rejects. Either the
invalid_version(SESSION_OPENED.load(Ordering::Relaxed)) // driver is genuinely older than our headers, or (the sneaky case) the userspace
} // `libnvidia-encode` reports a new-enough version to the pre-flight probe but the running
// kernel module is older and rejects the session — the classic "updated the driver, didn't
// reboot" skew. Both heal the same way.
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_VERSION => format!(
"the NVIDIA driver is older than this build's NVENC headers (needs NVENC API {}.{} or \
newer), or the userspace and kernel-module driver versions are mismatched common \
right after a driver update without a reboot. Update the NVIDIA driver, or reboot if \
you just updated it (a host restart is the usual fix).",
nv::NVENCAPI_MAJOR_VERSION,
nv::NVENCAPI_MINOR_VERSION,
),
nv::NVENCSTATUS::NV_ENC_ERR_NO_ENCODE_DEVICE => { nv::NVENCSTATUS::NV_ENC_ERR_NO_ENCODE_DEVICE => {
"this GPU exposes no usable NVENC engine — it has no hardware video encoder, or NVENC is \ "this GPU exposes no usable NVENC engine — it has no hardware video encoder, or NVENC is \
disabled on this card" disabled on this card"
@@ -122,144 +72,9 @@ pub(super) fn explain(status: nv::NVENCSTATUS) -> String {
} }
} }
/// Typed root of a failed NVENC entry-point call: carries the raw status so callers can classify
/// the failure class, not just print it — the bitrate-clamp search must only read a
/// parameter/caps rejection as "above the codec-level ceiling"; a transient failure shrinking the
/// search would discover (and cache) a bogus ceiling. Recover it through an `anyhow` chain with
/// `err.downcast_ref::<NvCallError>()` (see [`is_param_rejection`]).
#[derive(Debug)]
pub(super) struct NvCallError(pub(super) nv::NVENCSTATUS);
impl std::fmt::Display for NvCallError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{:?} — {}", self.0, explain(self.0))
}
}
impl std::error::Error for NvCallError {}
/// Whether `err` is an NVENC parameter/capability rejection: the driver understood the request
/// and says THIS config is not encodable — the clamp search's "bitrate above the ceiling"
/// evidence. Everything else (busy engine, session limit, OOM, device loss, version skew) is
/// environmental and must propagate instead of steering the search.
pub(super) fn is_param_rejection(err: &anyhow::Error) -> bool {
matches!(
err.downcast_ref::<NvCallError>(),
Some(NvCallError(
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_PARAM
| nv::NVENCSTATUS::NV_ENC_ERR_UNSUPPORTED_PARAM
| nv::NVENCSTATUS::NV_ENC_ERR_UNIMPLEMENTED,
))
)
}
/// Build an actionable `anyhow::Error` for a failed NVENC entry-point call. `call` names the API /// Build an actionable `anyhow::Error` for a failed NVENC entry-point call. `call` names the API
/// (e.g. `"open_encode_session_ex"`); the chain carries both the raw status and its real-world /// (e.g. `"open_encode_session_ex"`); the message carries both the raw status and its real-world
/// cause, so triage never again reads a version mismatch as "(no NVIDIA GPU?)". The /// cause, so triage never again reads a version mismatch as "(no NVIDIA GPU?)".
/// [`NvCallError`] root keeps the status downcastable for failure-class checks.
pub(super) fn call_err(call: &str, status: nv::NVENCSTATUS) -> anyhow::Error { pub(super) fn call_err(call: &str, status: nv::NVENCSTATUS) -> anyhow::Error {
anyhow::Error::new(NvCallError(status)).context(format!("NVENC {call} failed")) anyhow::anyhow!("NVENC {call} failed: {status:?} — {}", explain(status))
}
/// Whether a FAILED `nvEncDestroyEncoder` status PROVES the driver holds no session for the
/// handle — i.e. the per-process concurrent-session slot is not consumed, so the session's budget
/// units can be refunded immediately. These are the statuses the driver returns when the session
/// or its device no longer exists on its side (a TDR/device removal reclaims every session with
/// the context). Everything else — `GENERIC`, `ENCODER_BUSY`, OOM, ... — is AMBIGUOUS: the slot
/// may genuinely still be held, so the caller must park the handle fail-closed (units stay
/// charged) and let a later retry-destroy produce the proof. Splitting on proof is what keeps the
/// session budget from drifting low on failures (over-admitting parallel displays) WITHOUT letting
/// one transient wedge episode permanently poison admission until a host restart.
///
/// Used by the Windows D3D11 backend's teardown accounting; the Linux CUDA backend has no session
/// budget (parallel-display admission is a Windows feature), so there this exists for the unit
/// tests only.
#[cfg_attr(not(target_os = "windows"), allow(dead_code))]
pub(super) fn destroy_proves_no_session(status: nv::NVENCSTATUS) -> bool {
matches!(
status,
nv::NVENCSTATUS::NV_ENC_ERR_DEVICE_NOT_EXIST
| nv::NVENCSTATUS::NV_ENC_ERR_INVALID_ENCODERDEVICE
| nv::NVENCSTATUS::NV_ENC_ERR_INVALID_PTR
| nv::NVENCSTATUS::NV_ENC_ERR_ENCODER_NOT_INITIALIZED
)
}
#[cfg(test)]
mod tests {
use super::*;
/// Before any session has opened, the version word IS in question — keep the skew advice.
#[test]
fn invalid_version_before_any_session_blames_the_driver_version() {
let msg = invalid_version(false);
assert!(
msg.contains("older than this build's NVENC headers"),
"{msg}"
);
assert!(msg.contains("reboot if you just updated it"), "{msg}");
}
/// Once a session has opened here, a skew is impossible — the message must stop sending
/// operators to reboot (2026-07 field report: one stream per boot, forever).
#[test]
fn invalid_version_after_a_session_blames_process_state_not_the_driver() {
let msg = invalid_version(true);
assert!(msg.contains("NOT a driver version mismatch"), "{msg}");
assert!(msg.contains("restart the Punktfunk host service"), "{msg}");
assert!(
!msg.contains("older than this build's NVENC headers"),
"must not repeat the version-skew advice: {msg}"
);
assert!(
!msg.contains("Update the NVIDIA driver"),
"must not tell the operator to update a driver that just worked: {msg}"
);
}
/// The latch is one-way and only touches this status.
#[test]
fn note_session_opened_latches() {
note_session_opened();
assert!(SESSION_OPENED.load(Ordering::Relaxed));
note_session_opened();
assert!(SESSION_OPENED.load(Ordering::Relaxed));
assert_eq!(
explain(nv::NVENCSTATUS::NV_ENC_ERR_OUT_OF_MEMORY),
"the GPU is out of memory"
);
}
/// Destroy-failure classification: session-gone statuses refund; everything ambiguous parks.
/// The split is the load-bearing part of the session-budget accounting — a wrong `true`
/// under-counts (over-admits parallel displays), a wrong `false` merely defers the refund to
/// a reap retry.
#[test]
fn destroy_classification_refunds_only_on_proof() {
for gone in [
nv::NVENCSTATUS::NV_ENC_ERR_DEVICE_NOT_EXIST,
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_ENCODERDEVICE,
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_PTR,
nv::NVENCSTATUS::NV_ENC_ERR_ENCODER_NOT_INITIALIZED,
] {
assert!(
destroy_proves_no_session(gone),
"{gone:?} proves no session"
);
}
for ambiguous in [
nv::NVENCSTATUS::NV_ENC_ERR_GENERIC,
nv::NVENCSTATUS::NV_ENC_ERR_ENCODER_BUSY,
nv::NVENCSTATUS::NV_ENC_ERR_OUT_OF_MEMORY,
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_PARAM,
// INVALID_DEVICE sounds like the gone class but is also what a transiently-confused
// driver returns — deliberately fail-closed (park + retry), not refunded.
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_DEVICE,
] {
assert!(
!destroy_proves_no_session(ambiguous),
"{ambiguous:?} must park fail-closed"
);
}
}
} }
-11
View File
@@ -246,17 +246,6 @@ mod tests {
assert!(block_count_32x32(3840, 2160, true) <= u16::MAX as u32); assert!(block_count_32x32(3840, 2160, true) <= u16::MAX as u32);
assert!(block_count_32x32(7680, 4320, true) > u16::MAX as u32); assert!(block_count_32x32(7680, 4320, true) > u16::MAX as u32);
assert!(block_count_32x32(7680, 4320, false) <= u16::MAX as u32); assert!(block_count_32x32(7680, 4320, false) <= u16::MAX as u32);
// …and 4:2:0 wraps it too, just later — the hole the old 4:4:4-only open guard left.
// `Codec::max_dimension()` allows PyroWave 8192px per axis, so these modes were
// reachable from a client-requested `mode=WxHxFPS`, and the negotiator's 4:4:4 → 4:2:0
// downgrade routed oversized modes straight into the unguarded branch.
// `validate_dimensions` now rejects them against this 4:2:0 count.
assert_eq!(block_count_32x32(8192, 6144, false), 73728);
assert_eq!(block_count_32x32(8192, 8192, false), 98304);
assert!(block_count_32x32(8192, 6144, false) > u16::MAX as u32);
assert!(block_count_32x32(8192, 8192, false) > u16::MAX as u32);
// The largest 4:2:0 mode that still fits, for the boundary the validator enforces.
assert!(block_count_32x32(7680, 4320, false) <= u16::MAX as u32);
} }
#[test] #[test]
-178
View File
@@ -1,178 +0,0 @@
//! The slot-family RFI (reference-frame invalidation) recovery **policy**, shared by the three
//! backends that answer a loss with a re-reference to a known-good older frame instead of an IDR:
//! native AMF (user-LTR bitfield), native QSV (`mfxExtRefListCtrl` LTR) and Vulkan Video (the
//! app-owned DPB slot table). One decision, three mechanisms — the policy lived as three
//! hand-copies and had already diverged once (the fecbec2d taint sweep reached AMF/QSV a commit
//! before the Vulkan backend was carved out, and Vulkan shipped without it until a later fix).
//! The NVENC twins' *range* policy is the other half of WP7.2, in
//! [`super::nvenc_core::plan_range_recovery`].
//!
//! Policy only. Every mechanism — how a force is applied, how distrust is *persisted* (AMF clears
//! its mirror slot to `None`, QSV sets a separate `ltr_tainted` flag because its mirror must keep
//! naming the slot for the RejectedRefList, Vulkan blanks `slot_wire` to `-1` while `slot_poc`
//! keeps the picture resident for the RPS) — stays in its backend. Callers feed their
//! **currently-trusted** references and apply the returned taints through their own marker; that
//! caller-side filter is exactly what makes the three persistence schemes equivalent under one
//! pure function.
//!
//! The decline arm is also the backend's: AMF/QSV clear an un-consumed `pending_force` (the sweep
//! may have emptied the slot it points at), while Vulkan deliberately leaves `pending_loss` armed
//! (a stale arm is re-resolved at frame-build, where a failed re-pick forces the IDR that heals
//! the stream). Do not harmonize them here.
/// One loss event's recovery decision over a slot table: which trusted references become
/// untrustworthy, and which one anchors the recovery.
pub(super) struct SlotPlan {
/// Bitmask of slots whose reference was encoded **at or after** the loss start — inside the
/// client's corrupt window, so the client either never received it or decoded it against a
/// broken chain. Serving one as "known-good" on a LATER loss ships corruption tagged as the
/// recovery anchor; the backend must record the distrust in its own persistent marker, because
/// these wires would otherwise look like valid pre-loss anchors to the next loss event.
pub(super) tainted: u32,
/// The recovery anchor: the newest trusted reference **strictly older** than the loss —
/// `(slot, wire)` — i.e. the most recent picture the client still holds intact, so
/// re-referencing it costs the smallest residual. `None`: every candidate is inside or after
/// the corrupt window — the caller declines and its (coalesced) keyframe path recovers.
pub(super) anchor: Option<(usize, i64)>,
}
/// Plan the recovery for a loss starting at wire frame `loss_first`, over the backend's
/// currently-trusted references (`(slot, wire)`; previously-distrusted entries must already be
/// filtered out by the caller — see the module doc). Sweep and pick are one call so the anchor is
/// chosen from the same snapshot the taints are computed from, by construction: the anchor
/// delegates to [`pick_anchor`], and `wire >= loss_first` (taint) and `wire < loss_first`
/// (anchor) are disjoint, so a slot tainted by this call can never be this call's anchor.
pub(super) fn plan_slot_recovery(refs: &[(usize, i64)], loss_first: i64) -> SlotPlan {
// The callers' validity gate (`first < 0 → decline`) is what makes their sentinel filters
// (-1 / `None`) exact views of "trusted"; this assert keeps the contract visible from inside
// the extracted code. Plain assert: the lint legs run --release, and a compiled-out check
// here would silently drop taints instead of failing loudly.
assert!(
loss_first >= 0,
"loss_first must be validity-gated by the caller"
);
let mut tainted = 0u32;
for &(slot, wire) in refs {
if wire >= loss_first {
assert!(slot < 32, "slot table exceeds the u32 taint mask");
tainted |= 1 << slot;
}
}
SlotPlan {
tainted,
anchor: pick_anchor(refs, loss_first),
}
}
/// The pick half alone: newest trusted reference strictly older than the loss. Ties break to the
/// first entry in `refs` (every caller feeds ascending slot order, so the lowest slot wins —
/// the strict `>` all three backends used). Standalone because Vulkan re-picks at frame-build
/// time: its arm carries the loss start, not the slot, so the slot is resolved against the table
/// as it stands when the recovery frame is actually encoded.
pub(super) fn pick_anchor(refs: &[(usize, i64)], loss_first: i64) -> Option<(usize, i64)> {
let mut best: Option<(usize, i64)> = None;
for &(slot, wire) in refs {
if wire < loss_first && best.is_none_or(|(_, b)| wire > b) {
best = Some((slot, wire));
}
}
best
}
#[cfg(test)]
mod tests {
use super::{pick_anchor, plan_slot_recovery};
/// Adapt a raw slot table (the Vulkan `slot_wire` shape: `-1` = empty) into the trusted view
/// the policy takes — the same filter the backend adapters apply.
fn view(wires: &[i64]) -> Vec<(usize, i64)> {
wires
.iter()
.enumerate()
.filter_map(|(s, &w)| (w >= 0).then_some((s, w)))
.collect()
}
/// Apply a plan's taints the way the Vulkan adapter does (blank the wire) — the persistence
/// half the pure fn hands back to the caller.
fn apply(wires: &mut [i64], tainted: u32) {
for (s, w) in wires.iter_mut().enumerate() {
if tainted & (1 << s) != 0 {
*w = -1;
}
}
}
/// The RFI anchor picker: newest resident wire strictly older than the loss; empty/newer
/// slots never qualify. (Migrated 1:1 from `vulkan_video.rs`'s `pick_recovery_slot` tests —
/// same vectors, now `(slot, wire)`-valued.)
#[test]
fn picks_newest_pre_loss() {
// slots hold wires 5..12 (ring position arbitrary); loss starts at 9 → anchor = wire 8.
let wires = [8i64, 9, 10, 11, 12, 5, 6, 7];
assert_eq!(pick_anchor(&view(&wires), 9), Some((0, 8)));
// loss older than everything resident → no anchor (caller keyframes).
assert_eq!(pick_anchor(&view(&wires), 5), None);
// empty slots (-1) are skipped by the view filter and never anchor.
assert_eq!(pick_anchor(&view(&[-1, 3, -1, 4]), 5), Some((3, 4)));
assert_eq!(pick_anchor(&view(&[-1; 8]), 5), None);
// wire == loss_first is INSIDE the corrupt window: strictly-older only.
assert_eq!(pick_anchor(&view(&[9, 8]), 9), Some((1, 8)));
// tie on the wire → first entry (= lowest slot) wins, the strict `>` all three backends
// used.
assert_eq!(pick_anchor(&[(2, 7), (5, 7)], 9), Some((2, 7)));
// empty view.
assert_eq!(pick_anchor(&[], 9), None);
}
/// The taint sweep (fecbec2d's fix): a slot encoded inside an EARLIER, still unrepaired loss
/// window must not become the "known-good" anchor of a LATER loss. Without persisted
/// distrust the picker accepts it — it is resident and its wire is below the second loss
/// start — and the frame ships tagged `recovery_anchor`, lifting the client's freeze onto a
/// reference it never decoded. (Migrated 1:1 from `vulkan_video.rs`; the hand-replicated
/// sweep is now `plan_slot_recovery` itself.)
#[test]
fn taint_sweep_excludes_slots_from_an_earlier_loss() {
// Slots hold wires 0..7. Loss 1 starts at wire 4, so wires 4..7 are undecodable at the
// client. A second loss report arrives at wire 6 while they are all still resident.
let tainted_wires = [4i64, 5, 6, 7];
// WITHOUT the sweep this is the bug: the newest wire below 6 is wire 5 — squarely inside
// loss 1's unrepaired window — and it would be served as the "known-good" anchor.
let unswept = [0i64, 1, 2, 3, 4, 5, 6, 7];
let (_, picked_wire) = pick_anchor(&view(&unswept), 6).expect("unswept picks something");
assert!(
tainted_wires.contains(&picked_wire),
"precondition: without the sweep the anchor comes from the earlier loss window"
);
// WITH the plan, loss 1 taints 4..7 (and anchors on wire 3 — never a tainted wire, by
// the disjoint predicates), so loss 2 can only reach genuinely clean wires.
let mut wires = unswept;
let plan = plan_slot_recovery(&view(&wires), 4);
assert_eq!(plan.tainted, 0b1111_0000);
assert_eq!(plan.anchor, Some((3, 3)));
apply(&mut wires, plan.tainted);
assert_eq!(wires, [0, 1, 2, 3, -1, -1, -1, -1]);
let (slot, wire) = pick_anchor(&view(&wires), 6).expect("clean wires remain");
assert_eq!((slot, wire), (3, 3), "newest clean survivor is wire 3");
// Encoding resumes after recovery; wires 8..11 refill the swept slots and are clean. A
// later loss at wire 10 legitimately anchors on wire 9 — the sweep must not over-reject.
wires[4] = 8;
wires[5] = 9;
wires[6] = 10;
wires[7] = 11;
let plan = plan_slot_recovery(&view(&wires), 10);
assert_eq!(plan.anchor, Some((5, 9)), "wire 9 is post-recovery, clean");
apply(&mut wires, plan.tainted);
// A loss covering every live wire leaves nothing clean → decline, caller serves an IDR.
let mut all = [5i64, 6, 7, 8, 9, 10, 11, 12];
let plan = plan_slot_recovery(&view(&all), 5);
assert_eq!(plan.tainted, 0b1111_1111);
assert_eq!(plan.anchor, None);
apply(&mut all, plan.tainted);
assert_eq!(pick_anchor(&view(&all), 5), None);
}
}
+1 -64
View File
@@ -51,24 +51,6 @@ pub struct OpenH264Encoder {
// whole value to that thread is therefore sound — there is no concurrent access to the handle. // whole value to that thread is therefore sound — there is no concurrent access to the handle.
unsafe impl Send for OpenH264Encoder {} unsafe impl Send for OpenH264Encoder {}
/// openh264's own ceiling: level 5.2, so 3840x2160 landscape or 2160x3840 portrait.
///
/// The long edge may reach 3840 and the short edge 2160 — the rule is orientation-aware, not a
/// per-axis `w <= 3840 && h <= 2160`, so a portrait 2160x3840 session is legal.
const OPENH264_MAX_LONG_EDGE: u32 = 3840;
const OPENH264_MAX_SHORT_EDGE: u32 = 2160;
/// Whether the bundled openh264 can encode this resolution at all.
///
/// Mirrors the check inside the crate we ship (openh264 0.9.3, `encoder.rs` `reinit`). That check
/// runs on the FIRST ENCODE, not at encoder construction — so without this gate a too-large mode
/// opens perfectly and then fails *every* submit, and the session connects and never delivers a
/// frame. `Codec::max_dimension` does not cover it: it is keyed on the codec, and H.264 legitimately
/// reaches 4096 on every hardware backend — this ceiling belongs to the software backend alone.
fn openh264_supports_dimensions(width: u32, height: u32) -> bool {
width.max(height) <= OPENH264_MAX_LONG_EDGE && width.min(height) <= OPENH264_MAX_SHORT_EDGE
}
impl OpenH264Encoder { impl OpenH264Encoder {
pub fn open( pub fn open(
format: PixelFormat, format: PixelFormat,
@@ -77,16 +59,7 @@ impl OpenH264Encoder {
fps: u32, fps: u32,
bitrate_bps: u64, bitrate_bps: u64,
) -> Result<Self> { ) -> Result<Self> {
// validate_dimensions() ran in open_video: even, non-zero, <= 4096. That leaves modes this // validate_dimensions() ran in open_video: even, non-zero, <= 4096.
// encoder cannot serve (e.g. a legal 4096-wide H.264 mode), so refuse them here — at the
// open, where the caller still gets a real error — rather than at every submit.
ensure!(
openh264_supports_dimensions(width, height),
"openh264 cannot encode {width}x{height}: the software encoder tops out at \
{OPENH264_MAX_LONG_EDGE}x{OPENH264_MAX_SHORT_EDGE} (or \
{OPENH264_MAX_SHORT_EDGE}x{OPENH264_MAX_LONG_EDGE} portrait) lower the client \
resolution, or use a host with a hardware encoder"
);
let bps: u32 = bitrate_bps.try_into().unwrap_or(u32::MAX); let bps: u32 = bitrate_bps.try_into().unwrap_or(u32::MAX);
let cfg = EncoderConfig::new() let cfg = EncoderConfig::new()
.usage_type(UsageType::ScreenContentRealTime) .usage_type(UsageType::ScreenContentRealTime)
@@ -363,40 +336,4 @@ mod tests {
.any(|w| w[0] == 0 && w[1] == 0 && w[2] == 0 && w[3] == 1 && (w[4] & 0x1f) == 7); .any(|w| w[0] == 0 && w[1] == 0 && w[2] == 0 && w[3] == 1 && (w[4] & 0x1f) == 7);
assert!(has_sps, "IDR must carry an SPS NAL (type 7)"); assert!(has_sps, "IDR must carry an SPS NAL (type 7)");
} }
/// The modes the software encoder can actually serve — including the portrait orientation,
/// which a naive per-axis `w <= 3840 && h <= 2160` would wrongly reject.
#[test]
fn openh264_accepts_up_to_4k_in_either_orientation() {
assert!(openh264_supports_dimensions(1920, 1080));
assert!(openh264_supports_dimensions(3840, 2160));
assert!(openh264_supports_dimensions(2160, 3840));
assert!(openh264_supports_dimensions(1080, 1920));
}
/// Modes `validate_dimensions` lets through (H.264 legitimately reaches 4096 on hardware) but
/// openh264 rejects on the first encode. Catching them at open is the whole point of the gate:
/// otherwise the session opens and then fails every single submit.
#[test]
fn openh264_rejects_modes_that_would_fail_on_first_submit() {
// 4096-wide is legal H.264 and passes `Codec::max_dimension`, but exceeds the long edge.
assert!(!openh264_supports_dimensions(4096, 2160));
assert!(!openh264_supports_dimensions(2160, 4096));
// Long edge is fine, short edge is not (e.g. an ultrawide-tall composite desktop).
assert!(!openh264_supports_dimensions(3840, 2400));
assert!(!openh264_supports_dimensions(2400, 3840));
}
/// A too-large mode must fail at `open`, not silently at every `submit`.
#[test]
fn open_refuses_a_mode_openh264_cannot_encode() {
// Matched rather than `expect_err`: `OpenH264Encoder` is not `Debug` (it wraps a raw C
// handle), so `expect_err` would not compile.
let err = match OpenH264Encoder::open(PixelFormat::Bgra, 4096, 2160, 60, 20_000_000) {
Ok(_) => panic!("4096x2160 exceeds openh264's long-edge ceiling and must be refused"),
Err(e) => e,
};
let msg = format!("{err:#}");
assert!(msg.contains("openh264 cannot encode 4096x2160"), "{msg}");
}
} }
+442 -36
View File
@@ -66,11 +66,425 @@ use windows::Win32::Storage::FileSystem::{
}; };
use windows::Win32::System::LibraryLoader::GetModuleFileNameW; use windows::Win32::System::LibraryLoader::GetModuleFileNameW;
// The hand-mirrored AMF C ABI lives in `amf_sys.rs` (WP7.4): pure FFI surface, no policy — // ---------------------------------------------------------------------------------------------
// isolating the unsafe vtable mirror from the encoder logic is the crate's stated review // Mirrored AMF C ABI (written against GPUOpen header release v1.4.36 — amf/public/include; every
// goal, and the `#[path]` keeps the module name (`sys::`) and every call site unchanged. // slot below is a base-interface slot whose layout is stable since <= v1.4.34, the loader's
#[path = "amf_sys.rs"] // accepted ABI floor, so the mirror is valid on every runtime the loader admits).
mod sys; //
// Layout rules this mirror relies on: every AMF interface is a struct whose sole member is a
// pointer to a C vtable; derived interfaces PREPEND their base's slots in order (AMFInterface →
// AMFPropertyStorage → AMFData → AMFBuffer/AMFSurface), so a derived pointer is usable through a
// base vtable mirror. Slots we never call are declared as bare `*const c_void` placeholders —
// same size/alignment as the function pointer they stand in for. `AMF_STD_CALL` is `__stdcall`
// (Rust `extern "system"`); the two DLL entry points are `__cdecl` (`extern "C"`); on x86_64
// both collapse to the one Windows calling convention.
// ---------------------------------------------------------------------------------------------
mod sys {
use std::ffi::c_void;
/// `AMF_RESULT` (core/Result.h) — a plain C enum, sequential from 0. Only the codes this
/// module branches on are named; everything else is reported numerically via [`result_name`].
pub type AmfResult = i32;
pub const AMF_OK: AmfResult = 0;
pub const AMF_EOF: AmfResult = 23;
pub const AMF_REPEAT: AmfResult = 24;
pub const AMF_INPUT_FULL: AmfResult = 25;
pub const AMF_NEED_MORE_INPUT: AmfResult = 44;
/// Human-readable name for an `AMF_RESULT` (diagnostics only — the numeric value rides along
/// so an unnamed code is still identifiable against Result.h).
pub fn result_name(r: AmfResult) -> &'static str {
match r {
0 => "AMF_OK",
1 => "AMF_FAIL",
2 => "AMF_UNEXPECTED",
3 => "AMF_ACCESS_DENIED",
4 => "AMF_INVALID_ARG",
5 => "AMF_OUT_OF_RANGE",
6 => "AMF_OUT_OF_MEMORY",
7 => "AMF_INVALID_POINTER",
8 => "AMF_NO_INTERFACE",
9 => "AMF_NOT_IMPLEMENTED",
10 => "AMF_NOT_SUPPORTED",
11 => "AMF_NOT_FOUND",
12 => "AMF_ALREADY_INITIALIZED",
13 => "AMF_NOT_INITIALIZED",
14 => "AMF_INVALID_FORMAT",
15 => "AMF_WRONG_STATE",
17 => "AMF_NO_DEVICE",
18 => "AMF_DIRECTX_FAILED",
23 => "AMF_EOF",
24 => "AMF_REPEAT",
25 => "AMF_INPUT_FULL",
26 => "AMF_RESOLUTION_CHANGED",
28 => "AMF_INVALID_DATA_TYPE",
29 => "AMF_INVALID_RESOLUTION",
30 => "AMF_CODEC_NOT_SUPPORTED",
31 => "AMF_SURFACE_FORMAT_NOT_SUPPORTED",
32 => "AMF_SURFACE_MUST_BE_SHARED",
36 => "AMF_ENCODER_NOT_PRESENT",
44 => "AMF_NEED_MORE_INPUT",
_ => "AMF_<unnamed>",
}
}
/// The AMF header release this FFI mirror was written against: `AMF_FULL_VERSION` for 1.4.36.0
/// (core/Version.h `AMF_MAKE_FULL_VERSION`). This is the version claimed to `AMFInit` — but
/// capped at the runtime's own reported version (see `load_factory`), so an older-but-accepted
/// runtime is asked only for the ABI it actually provides.
pub const AMF_HEADER_VERSION: u64 = (1u64 << 48) | (4u64 << 32) | (36u64 << 16);
/// The oldest AMF runtime the loader accepts (`AMF_FULL_VERSION` 1.4.34.0 — AMD Adrenalin
/// 24.6.1). This is an **ABI floor, not a feature floor**: every vtable slot mirrored in this
/// module belongs to a base interface (`AMFFactory`/`AMFContext`/`AMFComponent`/`AMFData`/
/// `AMFBuffer`) whose layout has been stable — append-only, no mid-vtable insertions — since
/// well before 1.4.34, so a 1.4.34 runtime is guaranteed to expose every mirrored slot at its
/// mirrored offset. Everything 1.4.35/1.4.36 added that this path can touch (new HQ presets,
/// AV1 B-frame / picture management) is a *string-keyed encoder property*, applied through
/// [`set_prop`] with `required=false` — a runtime that lacks it rejects the property (logged)
/// and the feature degrades, rather than shifting any vtable offset. Below this floor the
/// mirror is not guaranteed to match, so the loader declines cleanly (an old-driver decline,
/// never UB).
pub const AMF_MIN_VERSION: u64 = (1u64 << 48) | (4u64 << 32) | (34u64 << 16);
/// `AMF_SURFACE_FORMAT` (core/Surface.h).
pub const AMF_SURFACE_NV12: i32 = 1;
pub const AMF_SURFACE_P010: i32 = 10;
/// `AMF_DX_VERSION::AMF_DX11_1` (core/Data.h) — the `InitDX11` version argument.
pub const AMF_DX11_1: i32 = 111;
/// `AMF_MEMORY_TYPE::AMF_MEMORY_HOST` (core/Data.h) — the `AllocBuffer` memory type for the
/// CPU-filled HDR-metadata buffer.
pub const AMF_MEMORY_HOST: i32 = 1;
/// `AMFHDRMetadata` (components/ColorSpace.h) — the payload of the `*InHDRMetadata` encoder
/// property (an `AMFBuffer` holding exactly this struct). Same units as the HEVC ST.2086 SEI
/// and [`punktfunk_core::quic::HdrMeta`]: chromaticities in 1/50000, mastering luminance in
/// 0.0001 cd/m², CLL/FALL in nits. 28 bytes, no padding.
#[repr(C)]
pub struct AmfHdrMetadata {
pub red_primary: [u16; 2],
pub green_primary: [u16; 2],
pub blue_primary: [u16; 2],
pub white_point: [u16; 2],
pub max_mastering_luminance: u32,
pub min_mastering_luminance: u32,
pub max_content_light_level: u16,
pub max_frame_average_light_level: u16,
}
/// `AMFGuid` (core/Platform.h) — data41..data48 flattened into an array (identical layout).
#[repr(C)]
pub struct AmfGuid {
pub data1: u32,
pub data2: u16,
pub data3: u16,
pub data4: [u8; 8],
}
/// `IID_AMFBuffer` (core/Buffer.h `AMF_DECLARE_IID`) — for `QueryInterface` on the encoder's
/// output `AMFData` to reach `GetNative`/`GetSize`.
pub const IID_AMF_BUFFER: AmfGuid = AmfGuid {
data1: 0xb04b_7248,
data2: 0xb6f0,
data3: 0x4321,
data4: [0xb6, 0x91, 0xba, 0xa4, 0x74, 0x0f, 0x9f, 0xcb],
};
// `AMF_VARIANT_TYPE` (core/Variant.h) — the tags this module writes/reads.
pub const AMF_VARIANT_BOOL: i32 = 1;
pub const AMF_VARIANT_INT64: i32 = 2;
pub const AMF_VARIANT_RATE: i32 = 7;
pub const AMF_VARIANT_INTERFACE: i32 = 12;
/// `AMFVariantStruct` (core/Variant.h): a 4-byte C-enum tag + a 16-byte union whose largest
/// members are pointer/`amf_int64`/`AMFFloatVector4D` (align 8) → 24 bytes total, tag at 0,
/// payload at 8. Passed BY VALUE to `SetProperty` (Win64 passes >8-byte aggregates by hidden
/// reference on both sides, so declaring it by value matches the C compiler). The payload is
/// stored as two fully-initialised `u64`s — little-endian packing puts a bool in byte 0, an
/// `amf_int64` in word 0, and an `AMFRate{num,den}` as `num | den << 32`, exactly the union's
/// in-memory layout — so no partially-initialised union bytes ever cross the FFI.
#[repr(C)]
pub struct AmfVariant {
pub vtype: i32,
pub payload: [u64; 2],
}
impl AmfVariant {
pub fn zeroed() -> Self {
AmfVariant {
vtype: 0, // AMF_VARIANT_EMPTY
payload: [0, 0],
}
}
pub fn from_i64(v: i64) -> Self {
AmfVariant {
vtype: AMF_VARIANT_INT64,
payload: [v as u64, 0],
}
}
pub fn from_bool(v: bool) -> Self {
AmfVariant {
vtype: AMF_VARIANT_BOOL,
payload: [v as u64, 0],
}
}
/// `AMFRate { num, den }` — two little-endian `amf_uint32`s in the union's first 8 bytes.
pub fn from_rate(num: u32, den: u32) -> Self {
AmfVariant {
vtype: AMF_VARIANT_RATE,
payload: [num as u64 | ((den as u64) << 32), 0],
}
}
/// An `AMFInterface*` payload (`pInterface` in the union's first 8 bytes). The property
/// storage AddRefs the interface when it copies the variant in (the C++ template
/// `SetProperty(name, AMFVariant(value))` passes a temporary whose destructor releases,
/// so `SetProperty` must take its own reference) — the caller keeps sole ownership of the
/// reference it already holds.
pub fn from_interface(p: *mut c_void) -> Self {
AmfVariant {
vtype: AMF_VARIANT_INTERFACE,
payload: [p as usize as u64, 0],
}
}
/// Read back an `amf_int64` payload (only valid when `vtype == AMF_VARIANT_INT64`).
pub fn as_i64(&self) -> Option<i64> {
(self.vtype == AMF_VARIANT_INT64).then_some(self.payload[0] as i64)
}
}
/// Placeholder for a vtable slot this module never calls — same size/align as the function
/// pointer it stands in for, present only to keep the following slots at their C offsets.
pub type Slot = *const c_void;
// -- AMFFactory (core/Factory.h; NOT refcounted — a process singleton) ----------------------
#[repr(C)]
pub struct AmfFactory {
pub vtbl: *const AmfFactoryVtbl,
}
#[repr(C)]
pub struct AmfFactoryVtbl {
pub create_context:
unsafe extern "system" fn(*mut AmfFactory, *mut *mut AmfContext) -> AmfResult,
pub create_component: unsafe extern "system" fn(
*mut AmfFactory,
*mut AmfContext,
*const u16,
*mut *mut AmfComponent,
) -> AmfResult,
pub set_cache_folder: Slot,
pub get_cache_folder: Slot,
pub get_debug: Slot,
pub get_trace: Slot,
pub get_programs: Slot,
}
// -- AMFContext (core/Context.h) ------------------------------------------------------------
#[repr(C)]
pub struct AmfContext {
pub vtbl: *const AmfContextVtbl,
}
#[repr(C)]
pub struct AmfContextVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfContext) -> i32,
pub query_interface: Slot,
// AMFPropertyStorage
pub set_property: Slot,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFContext
pub terminate: unsafe extern "system" fn(*mut AmfContext) -> AmfResult,
pub init_dx9: Slot,
pub get_dx9_device: Slot,
pub lock_dx9: Slot,
pub unlock_dx9: Slot,
pub init_dx11: unsafe extern "system" fn(*mut AmfContext, *mut c_void, i32) -> AmfResult,
pub get_dx11_device: Slot,
pub lock_dx11: Slot,
pub unlock_dx11: Slot,
pub init_opencl: Slot,
pub get_opencl_context: Slot,
pub get_opencl_command_queue: Slot,
pub get_opencl_device_id: Slot,
pub get_opencl_compute_factory: Slot,
pub init_opencl_ex: Slot,
pub lock_opencl: Slot,
pub unlock_opencl: Slot,
pub init_opengl: Slot,
pub get_opengl_context: Slot,
pub get_opengl_drawable: Slot,
pub lock_opengl: Slot,
pub unlock_opengl: Slot,
pub init_xv: Slot,
pub get_xv_device: Slot,
pub lock_xv: Slot,
pub unlock_xv: Slot,
pub init_gralloc: Slot,
pub get_gralloc_device: Slot,
pub lock_gralloc: Slot,
pub unlock_gralloc: Slot,
pub alloc_buffer: unsafe extern "system" fn(
*mut AmfContext,
i32, // AMF_MEMORY_TYPE
usize,
*mut *mut AmfBuffer,
) -> AmfResult,
pub alloc_surface: Slot,
pub alloc_audio_buffer: Slot,
pub create_buffer_from_host_native: Slot,
pub create_surface_from_host_native: Slot,
pub create_surface_from_dx9_native: Slot,
/// Out-param is `AMFSurface**` in the header; declared as the `AmfData` base here because
/// every surface call this module makes (`SetPts`, `SetProperty`, `Release`,
/// `SubmitInput`) lives in the `AMFData` vtable prefix, which `AMFSurfaceVtbl` reproduces
/// slot-for-slot (single inheritance, same object pointer).
pub create_surface_from_dx11_native: unsafe extern "system" fn(
*mut AmfContext,
*mut c_void,
*mut *mut AmfData,
*mut c_void,
) -> AmfResult,
pub create_surface_from_opengl_native: Slot,
pub create_surface_from_gralloc_native: Slot,
pub create_surface_from_opencl_native: Slot,
pub create_buffer_from_opencl_native: Slot,
pub get_compute: Slot,
}
// -- AMFComponent (components/Component.h) --------------------------------------------------
#[repr(C)]
pub struct AmfComponent {
pub vtbl: *const AmfComponentVtbl,
}
#[repr(C)]
pub struct AmfComponentVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfComponent) -> i32,
pub query_interface: Slot,
// AMFPropertyStorage
pub set_property:
unsafe extern "system" fn(*mut AmfComponent, *const u16, AmfVariant) -> AmfResult,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFPropertyStorageEx
pub get_properties_info_count: Slot,
pub get_property_info_at: Slot,
pub get_property_info: Slot,
pub validate_property: Slot,
// AMFComponent
pub init: unsafe extern "system" fn(*mut AmfComponent, i32, i32, i32) -> AmfResult,
pub reinit: Slot,
pub terminate: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub drain: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub flush: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub submit_input: unsafe extern "system" fn(*mut AmfComponent, *mut AmfData) -> AmfResult,
pub query_output:
unsafe extern "system" fn(*mut AmfComponent, *mut *mut AmfData) -> AmfResult,
pub get_context: Slot,
pub set_output_data_allocator_cb: Slot,
pub get_caps: Slot,
pub optimize: Slot,
}
// -- AMFData (core/Data.h) — also the usable prefix of AMFSurface --------------------------
#[repr(C)]
pub struct AmfData {
pub vtbl: *const AmfDataVtbl,
}
#[repr(C)]
pub struct AmfDataVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfData) -> i32,
pub query_interface:
unsafe extern "system" fn(*mut AmfData, *const AmfGuid, *mut *mut c_void) -> AmfResult,
// AMFPropertyStorage
pub set_property:
unsafe extern "system" fn(*mut AmfData, *const u16, AmfVariant) -> AmfResult,
pub get_property:
unsafe extern "system" fn(*mut AmfData, *const u16, *mut AmfVariant) -> AmfResult,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFData
pub get_memory_type: Slot,
pub duplicate: Slot,
pub convert: Slot,
pub interop: Slot,
pub get_data_type: Slot,
pub is_reusable: Slot,
pub set_pts: unsafe extern "system" fn(*mut AmfData, i64),
pub get_pts: Slot,
pub set_duration: Slot,
pub get_duration: Slot,
}
// -- AMFBuffer (core/Buffer.h) — the encoder's output object -------------------------------
#[repr(C)]
pub struct AmfBuffer {
pub vtbl: *const AmfBufferVtbl,
}
#[repr(C)]
pub struct AmfBufferVtbl {
// AMFInterface + AMFPropertyStorage + AMFData prefix (identical order to AmfDataVtbl).
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfBuffer) -> i32,
pub query_interface: Slot,
pub set_property: Slot,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
pub get_memory_type: Slot,
pub duplicate: Slot,
pub convert: Slot,
pub interop: Slot,
pub get_data_type: Slot,
pub is_reusable: Slot,
pub set_pts: Slot,
pub get_pts: Slot,
pub set_duration: Slot,
pub get_duration: Slot,
// AMFBuffer
pub set_size: Slot,
pub get_size: unsafe extern "system" fn(*mut AmfBuffer) -> usize,
pub get_native: unsafe extern "system" fn(*mut AmfBuffer) -> *mut c_void,
pub add_observer_buffer: Slot,
pub remove_observer_buffer: Slot,
}
// -- DLL entry points (core/Factory.h; AMF_CDECL_CALL) --------------------------------------
pub type AmfQueryVersionFn = unsafe extern "C" fn(*mut u64) -> AmfResult;
pub type AmfInitFn = unsafe extern "C" fn(u64, *mut *mut AmfFactory) -> AmfResult;
}
use sys::{result_name, AmfVariant}; use sys::{result_name, AmfVariant};
@@ -858,11 +1272,7 @@ impl AmfEncoder {
if codec == Codec::Av1 && !probe_can_encode(Codec::Av1) { if codec == Codec::Av1 && !probe_can_encode(Codec::Av1) {
bail!("this GPU/driver declined AV1 encode (RDNA3+ required) — native AMF probe"); bail!("this GPU/driver declined AV1 encode (RDNA3+ required) — native AMF probe");
} }
// Depth follows the delivered pixels, not the negotiated depth ([`crate::ten_bit_input`]). let ten_bit = bit_depth >= 10 || matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2);
// With the old `bit_depth >= 10 || …` shape a 10-bit-negotiated session over an 8-bit
// capture derived `expected = P010`, tripped the format check below and ended the session
// at open — on exactly the configuration the capturer had already downgraded on purpose.
let ten_bit = crate::ten_bit_input(format, bit_depth);
// Zero-copy by construction: the input ring is NV12/P010 fed by same-format // Zero-copy by construction: the input ring is NV12/P010 fed by same-format
// CopySubresourceRegion. Any other capture format (Bgra/Rgb10a2 video-processor fallback, // CopySubresourceRegion. Any other capture format (Bgra/Rgb10a2 video-processor fallback,
// CPU frames) has no native input path — and since Phase 3 no ffmpeg readback to degrade // CPU frames) has no native input path — and since Phase 3 no ffmpeg readback to degrade
@@ -1959,27 +2369,31 @@ impl Encoder for AmfEncoder {
if !self.ltr_active || first < 0 || first > last { if !self.ltr_active || first < 0 || first > last {
return false; return false;
} }
// The taint-sweep + anchor-pick POLICY lives in `rfi::plan_slot_recovery` (one decision // Taint sweep BEFORE picking the anchor: an LTR marked at-or-after the loss start was
// shared with QSV and Vulkan Video); this backend's mechanism is: distrust = clear the // encoded inside the client's corrupt window — the client either never received it or
// mirror slot (dropped slots stay dropped; the cadence re-marks a clean frame within // decoded it against a broken reference chain. Serving it as "known-good" on a LATER
// ~1/4 s). `ltr_slots` store the WIRE frame index of the marked frame (`submit_indexed` // loss ships corruption as the recovery anchor (and every subsequent mark re-samples
// pins `frame_idx` to it per submission), so they compare directly against the client's // it). Dropped slots stay dropped; the cadence re-marks a clean frame within ~1/4 s.
// `first` — and stay comparable across encoder rebuilds/resets, where an internal counter for marked in self.ltr_slots.iter_mut() {
// would make the pre-loss check vacuous and risk force-referencing an LTR marked INSIDE if marked.is_some_and(|idx| idx >= first) {
// the lost range.
let view: Vec<(usize, i64)> = self
.ltr_slots
.iter()
.enumerate()
.filter_map(|(s, m)| m.map(|w| (s, w)))
.collect();
let plan = super::rfi::plan_slot_recovery(&view, first);
for (slot, marked) in self.ltr_slots.iter_mut().enumerate() {
if plan.tainted & (1 << slot) != 0 {
*marked = None; *marked = None;
} }
} }
match plan.anchor { // Pick the newest LTR strictly OLDER than the loss: the most recent known-good reference the
// client still holds, so re-referencing it costs the least (smallest recovery-frame residual).
// `ltr_slots` store the WIRE frame index of the marked frame (`submit_indexed` pins
// `frame_idx` to it per submission), so they compare directly against the client's `first`
// — and stay comparable across encoder rebuilds/resets, where an internal counter would
// make this check vacuous and risk force-referencing an LTR marked INSIDE the lost range.
let mut best: Option<(usize, i64)> = None;
for (slot, marked) in self.ltr_slots.iter().enumerate() {
if let Some(idx) = *marked {
if idx < first && best.is_none_or(|(_, b)| idx > b) {
best = Some((slot, idx));
}
}
}
match best {
Some((slot, ltr_frame)) => { Some((slot, ltr_frame)) => {
// Queue the force for the next submit; that frame ships tagged `recovery_anchor`. // Queue the force for the next submit; that frame ships tagged `recovery_anchor`.
self.pending_force = Some(slot); self.pending_force = Some(slot);
@@ -2008,8 +2422,6 @@ impl Encoder for AmfEncoder {
fn caps(&self) -> EncoderCaps { fn caps(&self) -> EncoderCaps {
EncoderCaps { EncoderCaps {
// As Windows NVENC: the capturer composites; this backend never reads `frame.cursor`.
blends_cursor: false,
// LTR-RFI: AMD's reference invalidation is the user long-term-reference path (mark a // LTR-RFI: AMD's reference invalidation is the user long-term-reference path (mark a
// frame, force a later one to re-reference it). True only when the live driver accepted // frame, force a later one to re-reference it). True only when the live driver accepted
// the LTR slots at open — otherwise loss recovery falls back to a full IDR. // the LTR slots at open — otherwise loss recovery falls back to a full IDR.
@@ -2348,10 +2760,6 @@ mod tests {
} }
/// The `p`-quantile of `samples` (µs), sorting in place. `0` when empty. /// The `p`-quantile of `samples` (µs), sorting in place. `0` when empty.
/// Gated like its only caller, the `amf-qsv`-only §5.2 latency A/B below — otherwise a
/// `--features nvenc,qsv` build compiles this helper with the benchmark cfg'd out and trips
/// `dead_code` (which the crate root no longer blanket-allows).
#[cfg(feature = "amf-qsv")]
fn percentile(samples: &mut [u128], p: f64) -> u128 { fn percentile(samples: &mut [u128], p: f64) -> u128 {
if samples.is_empty() { if samples.is_empty() {
return 0; return 0;
@@ -2370,8 +2778,6 @@ mod tests {
/// so its submit→AU is the bare ASIC time. The last ~2 unflushed frames on the ffmpeg path /// so its submit→AU is the bare ASIC time. The last ~2 unflushed frames on the ffmpeg path
/// are left unmeasured (dropped with the encoder) so every recorded sample is a genuine paced /// are left unmeasured (dropped with the encoder) so every recorded sample is a genuine paced
/// submit→AU. /// submit→AU.
/// Gated like its only caller (see [`percentile`]).
#[cfg(feature = "amf-qsv")]
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
fn drive_and_measure( fn drive_and_measure(
enc: &mut dyn Encoder, enc: &mut dyn Encoder,
-414
View File
@@ -1,414 +0,0 @@
//! Mirrored AMF C ABI (written against GPUOpen header release v1.4.36 — amf/public/include; every
//! slot below is a base-interface slot whose layout is stable since <= v1.4.34, the loader's
//! accepted ABI floor, so the mirror is valid on every runtime the loader admits).
//!
//! Layout rules this mirror relies on: every AMF interface is a struct whose sole member is a
//! pointer to a C vtable; derived interfaces PREPEND their base's slots in order (AMFInterface →
//! AMFPropertyStorage → AMFData → AMFBuffer/AMFSurface), so a derived pointer is usable through a
//! base vtable mirror. Slots we never call are declared as bare `*const c_void` placeholders —
//! same size/alignment as the function pointer they stand in for. `AMF_STD_CALL` is `__stdcall`
//! (Rust `extern "system"`); the two DLL entry points are `__cdecl` (`extern "C"`); on x86_64
//! both collapse to the one Windows calling convention.
use std::ffi::c_void;
/// `AMF_RESULT` (core/Result.h) — a plain C enum, sequential from 0. Only the codes this
/// module branches on are named; everything else is reported numerically via [`result_name`].
pub type AmfResult = i32;
pub const AMF_OK: AmfResult = 0;
pub const AMF_EOF: AmfResult = 23;
pub const AMF_REPEAT: AmfResult = 24;
pub const AMF_INPUT_FULL: AmfResult = 25;
pub const AMF_NEED_MORE_INPUT: AmfResult = 44;
/// Human-readable name for an `AMF_RESULT` (diagnostics only — the numeric value rides along
/// so an unnamed code is still identifiable against Result.h).
pub fn result_name(r: AmfResult) -> &'static str {
match r {
0 => "AMF_OK",
1 => "AMF_FAIL",
2 => "AMF_UNEXPECTED",
3 => "AMF_ACCESS_DENIED",
4 => "AMF_INVALID_ARG",
5 => "AMF_OUT_OF_RANGE",
6 => "AMF_OUT_OF_MEMORY",
7 => "AMF_INVALID_POINTER",
8 => "AMF_NO_INTERFACE",
9 => "AMF_NOT_IMPLEMENTED",
10 => "AMF_NOT_SUPPORTED",
11 => "AMF_NOT_FOUND",
12 => "AMF_ALREADY_INITIALIZED",
13 => "AMF_NOT_INITIALIZED",
14 => "AMF_INVALID_FORMAT",
15 => "AMF_WRONG_STATE",
17 => "AMF_NO_DEVICE",
18 => "AMF_DIRECTX_FAILED",
23 => "AMF_EOF",
24 => "AMF_REPEAT",
25 => "AMF_INPUT_FULL",
26 => "AMF_RESOLUTION_CHANGED",
28 => "AMF_INVALID_DATA_TYPE",
29 => "AMF_INVALID_RESOLUTION",
30 => "AMF_CODEC_NOT_SUPPORTED",
31 => "AMF_SURFACE_FORMAT_NOT_SUPPORTED",
32 => "AMF_SURFACE_MUST_BE_SHARED",
36 => "AMF_ENCODER_NOT_PRESENT",
44 => "AMF_NEED_MORE_INPUT",
_ => "AMF_<unnamed>",
}
}
/// The AMF header release this FFI mirror was written against: `AMF_FULL_VERSION` for 1.4.36.0
/// (core/Version.h `AMF_MAKE_FULL_VERSION`). This is the version claimed to `AMFInit` — but
/// capped at the runtime's own reported version (see `load_factory`), so an older-but-accepted
/// runtime is asked only for the ABI it actually provides.
pub const AMF_HEADER_VERSION: u64 = (1u64 << 48) | (4u64 << 32) | (36u64 << 16);
/// The oldest AMF runtime the loader accepts (`AMF_FULL_VERSION` 1.4.34.0 — AMD Adrenalin
/// 24.6.1). This is an **ABI floor, not a feature floor**: every vtable slot mirrored in this
/// module belongs to a base interface (`AMFFactory`/`AMFContext`/`AMFComponent`/`AMFData`/
/// `AMFBuffer`) whose layout has been stable — append-only, no mid-vtable insertions — since
/// well before 1.4.34, so a 1.4.34 runtime is guaranteed to expose every mirrored slot at its
/// mirrored offset. Everything 1.4.35/1.4.36 added that this path can touch (new HQ presets,
/// AV1 B-frame / picture management) is a *string-keyed encoder property*, applied through
/// [`set_prop`] with `required=false` — a runtime that lacks it rejects the property (logged)
/// and the feature degrades, rather than shifting any vtable offset. Below this floor the
/// mirror is not guaranteed to match, so the loader declines cleanly (an old-driver decline,
/// never UB).
pub const AMF_MIN_VERSION: u64 = (1u64 << 48) | (4u64 << 32) | (34u64 << 16);
/// `AMF_SURFACE_FORMAT` (core/Surface.h).
pub const AMF_SURFACE_NV12: i32 = 1;
pub const AMF_SURFACE_P010: i32 = 10;
/// `AMF_DX_VERSION::AMF_DX11_1` (core/Data.h) — the `InitDX11` version argument.
pub const AMF_DX11_1: i32 = 111;
/// `AMF_MEMORY_TYPE::AMF_MEMORY_HOST` (core/Data.h) — the `AllocBuffer` memory type for the
/// CPU-filled HDR-metadata buffer.
pub const AMF_MEMORY_HOST: i32 = 1;
/// `AMFHDRMetadata` (components/ColorSpace.h) — the payload of the `*InHDRMetadata` encoder
/// property (an `AMFBuffer` holding exactly this struct). Same units as the HEVC ST.2086 SEI
/// and [`punktfunk_core::quic::HdrMeta`]: chromaticities in 1/50000, mastering luminance in
/// 0.0001 cd/m², CLL/FALL in nits. 28 bytes, no padding.
#[repr(C)]
pub struct AmfHdrMetadata {
pub red_primary: [u16; 2],
pub green_primary: [u16; 2],
pub blue_primary: [u16; 2],
pub white_point: [u16; 2],
pub max_mastering_luminance: u32,
pub min_mastering_luminance: u32,
pub max_content_light_level: u16,
pub max_frame_average_light_level: u16,
}
/// `AMFGuid` (core/Platform.h) — data41..data48 flattened into an array (identical layout).
#[repr(C)]
pub struct AmfGuid {
pub data1: u32,
pub data2: u16,
pub data3: u16,
pub data4: [u8; 8],
}
/// `IID_AMFBuffer` (core/Buffer.h `AMF_DECLARE_IID`) — for `QueryInterface` on the encoder's
/// output `AMFData` to reach `GetNative`/`GetSize`.
pub const IID_AMF_BUFFER: AmfGuid = AmfGuid {
data1: 0xb04b_7248,
data2: 0xb6f0,
data3: 0x4321,
data4: [0xb6, 0x91, 0xba, 0xa4, 0x74, 0x0f, 0x9f, 0xcb],
};
// `AMF_VARIANT_TYPE` (core/Variant.h) — the tags this module writes/reads.
pub const AMF_VARIANT_BOOL: i32 = 1;
pub const AMF_VARIANT_INT64: i32 = 2;
pub const AMF_VARIANT_RATE: i32 = 7;
pub const AMF_VARIANT_INTERFACE: i32 = 12;
/// `AMFVariantStruct` (core/Variant.h): a 4-byte C-enum tag + a 16-byte union whose largest
/// members are pointer/`amf_int64`/`AMFFloatVector4D` (align 8) → 24 bytes total, tag at 0,
/// payload at 8. Passed BY VALUE to `SetProperty` (Win64 passes >8-byte aggregates by hidden
/// reference on both sides, so declaring it by value matches the C compiler). The payload is
/// stored as two fully-initialised `u64`s — little-endian packing puts a bool in byte 0, an
/// `amf_int64` in word 0, and an `AMFRate{num,den}` as `num | den << 32`, exactly the union's
/// in-memory layout — so no partially-initialised union bytes ever cross the FFI.
#[repr(C)]
pub struct AmfVariant {
pub vtype: i32,
pub payload: [u64; 2],
}
impl AmfVariant {
pub fn zeroed() -> Self {
AmfVariant {
vtype: 0, // AMF_VARIANT_EMPTY
payload: [0, 0],
}
}
pub fn from_i64(v: i64) -> Self {
AmfVariant {
vtype: AMF_VARIANT_INT64,
payload: [v as u64, 0],
}
}
pub fn from_bool(v: bool) -> Self {
AmfVariant {
vtype: AMF_VARIANT_BOOL,
payload: [v as u64, 0],
}
}
/// `AMFRate { num, den }` — two little-endian `amf_uint32`s in the union's first 8 bytes.
pub fn from_rate(num: u32, den: u32) -> Self {
AmfVariant {
vtype: AMF_VARIANT_RATE,
payload: [num as u64 | ((den as u64) << 32), 0],
}
}
/// An `AMFInterface*` payload (`pInterface` in the union's first 8 bytes). The property
/// storage AddRefs the interface when it copies the variant in (the C++ template
/// `SetProperty(name, AMFVariant(value))` passes a temporary whose destructor releases,
/// so `SetProperty` must take its own reference) — the caller keeps sole ownership of the
/// reference it already holds.
pub fn from_interface(p: *mut c_void) -> Self {
AmfVariant {
vtype: AMF_VARIANT_INTERFACE,
payload: [p as usize as u64, 0],
}
}
/// Read back an `amf_int64` payload (only valid when `vtype == AMF_VARIANT_INT64`).
pub fn as_i64(&self) -> Option<i64> {
(self.vtype == AMF_VARIANT_INT64).then_some(self.payload[0] as i64)
}
}
/// Placeholder for a vtable slot this module never calls — same size/align as the function
/// pointer it stands in for, present only to keep the following slots at their C offsets.
pub type Slot = *const c_void;
// -- AMFFactory (core/Factory.h; NOT refcounted — a process singleton) ----------------------
#[repr(C)]
pub struct AmfFactory {
pub vtbl: *const AmfFactoryVtbl,
}
#[repr(C)]
pub struct AmfFactoryVtbl {
pub create_context:
unsafe extern "system" fn(*mut AmfFactory, *mut *mut AmfContext) -> AmfResult,
pub create_component: unsafe extern "system" fn(
*mut AmfFactory,
*mut AmfContext,
*const u16,
*mut *mut AmfComponent,
) -> AmfResult,
pub set_cache_folder: Slot,
pub get_cache_folder: Slot,
pub get_debug: Slot,
pub get_trace: Slot,
pub get_programs: Slot,
}
// -- AMFContext (core/Context.h) ------------------------------------------------------------
#[repr(C)]
pub struct AmfContext {
pub vtbl: *const AmfContextVtbl,
}
#[repr(C)]
pub struct AmfContextVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfContext) -> i32,
pub query_interface: Slot,
// AMFPropertyStorage
pub set_property: Slot,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFContext
pub terminate: unsafe extern "system" fn(*mut AmfContext) -> AmfResult,
pub init_dx9: Slot,
pub get_dx9_device: Slot,
pub lock_dx9: Slot,
pub unlock_dx9: Slot,
pub init_dx11: unsafe extern "system" fn(*mut AmfContext, *mut c_void, i32) -> AmfResult,
pub get_dx11_device: Slot,
pub lock_dx11: Slot,
pub unlock_dx11: Slot,
pub init_opencl: Slot,
pub get_opencl_context: Slot,
pub get_opencl_command_queue: Slot,
pub get_opencl_device_id: Slot,
pub get_opencl_compute_factory: Slot,
pub init_opencl_ex: Slot,
pub lock_opencl: Slot,
pub unlock_opencl: Slot,
pub init_opengl: Slot,
pub get_opengl_context: Slot,
pub get_opengl_drawable: Slot,
pub lock_opengl: Slot,
pub unlock_opengl: Slot,
pub init_xv: Slot,
pub get_xv_device: Slot,
pub lock_xv: Slot,
pub unlock_xv: Slot,
pub init_gralloc: Slot,
pub get_gralloc_device: Slot,
pub lock_gralloc: Slot,
pub unlock_gralloc: Slot,
pub alloc_buffer: unsafe extern "system" fn(
*mut AmfContext,
i32, // AMF_MEMORY_TYPE
usize,
*mut *mut AmfBuffer,
) -> AmfResult,
pub alloc_surface: Slot,
pub alloc_audio_buffer: Slot,
pub create_buffer_from_host_native: Slot,
pub create_surface_from_host_native: Slot,
pub create_surface_from_dx9_native: Slot,
/// Out-param is `AMFSurface**` in the header; declared as the `AmfData` base here because
/// every surface call this module makes (`SetPts`, `SetProperty`, `Release`,
/// `SubmitInput`) lives in the `AMFData` vtable prefix, which `AMFSurfaceVtbl` reproduces
/// slot-for-slot (single inheritance, same object pointer).
pub create_surface_from_dx11_native: unsafe extern "system" fn(
*mut AmfContext,
*mut c_void,
*mut *mut AmfData,
*mut c_void,
) -> AmfResult,
pub create_surface_from_opengl_native: Slot,
pub create_surface_from_gralloc_native: Slot,
pub create_surface_from_opencl_native: Slot,
pub create_buffer_from_opencl_native: Slot,
pub get_compute: Slot,
}
// -- AMFComponent (components/Component.h) --------------------------------------------------
#[repr(C)]
pub struct AmfComponent {
pub vtbl: *const AmfComponentVtbl,
}
#[repr(C)]
pub struct AmfComponentVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfComponent) -> i32,
pub query_interface: Slot,
// AMFPropertyStorage
pub set_property:
unsafe extern "system" fn(*mut AmfComponent, *const u16, AmfVariant) -> AmfResult,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFPropertyStorageEx
pub get_properties_info_count: Slot,
pub get_property_info_at: Slot,
pub get_property_info: Slot,
pub validate_property: Slot,
// AMFComponent
pub init: unsafe extern "system" fn(*mut AmfComponent, i32, i32, i32) -> AmfResult,
pub reinit: Slot,
pub terminate: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub drain: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub flush: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub submit_input: unsafe extern "system" fn(*mut AmfComponent, *mut AmfData) -> AmfResult,
pub query_output: unsafe extern "system" fn(*mut AmfComponent, *mut *mut AmfData) -> AmfResult,
pub get_context: Slot,
pub set_output_data_allocator_cb: Slot,
pub get_caps: Slot,
pub optimize: Slot,
}
// -- AMFData (core/Data.h) — also the usable prefix of AMFSurface --------------------------
#[repr(C)]
pub struct AmfData {
pub vtbl: *const AmfDataVtbl,
}
#[repr(C)]
pub struct AmfDataVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfData) -> i32,
pub query_interface:
unsafe extern "system" fn(*mut AmfData, *const AmfGuid, *mut *mut c_void) -> AmfResult,
// AMFPropertyStorage
pub set_property: unsafe extern "system" fn(*mut AmfData, *const u16, AmfVariant) -> AmfResult,
pub get_property:
unsafe extern "system" fn(*mut AmfData, *const u16, *mut AmfVariant) -> AmfResult,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFData
pub get_memory_type: Slot,
pub duplicate: Slot,
pub convert: Slot,
pub interop: Slot,
pub get_data_type: Slot,
pub is_reusable: Slot,
pub set_pts: unsafe extern "system" fn(*mut AmfData, i64),
pub get_pts: Slot,
pub set_duration: Slot,
pub get_duration: Slot,
}
// -- AMFBuffer (core/Buffer.h) — the encoder's output object -------------------------------
#[repr(C)]
pub struct AmfBuffer {
pub vtbl: *const AmfBufferVtbl,
}
#[repr(C)]
pub struct AmfBufferVtbl {
// AMFInterface + AMFPropertyStorage + AMFData prefix (identical order to AmfDataVtbl).
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfBuffer) -> i32,
pub query_interface: Slot,
pub set_property: Slot,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
pub get_memory_type: Slot,
pub duplicate: Slot,
pub convert: Slot,
pub interop: Slot,
pub get_data_type: Slot,
pub is_reusable: Slot,
pub set_pts: Slot,
pub get_pts: Slot,
pub set_duration: Slot,
pub get_duration: Slot,
// AMFBuffer
pub set_size: Slot,
pub get_size: unsafe extern "system" fn(*mut AmfBuffer) -> usize,
pub get_native: unsafe extern "system" fn(*mut AmfBuffer) -> *mut c_void,
pub add_observer_buffer: Slot,
pub remove_observer_buffer: Slot,
}
// -- DLL entry points (core/Factory.h; AMF_CDECL_CALL) --------------------------------------
pub type AmfQueryVersionFn = unsafe extern "C" fn(*mut u64) -> AmfResult;
pub type AmfInitFn = unsafe extern "C" fn(u64, *mut *mut AmfFactory) -> AmfResult;

Some files were not shown because too many files have changed in this diff Show More