Compare commits

..
Author SHA1 Message Date
enricobuehlerandClaude Opus 4.8 7cde80ce84 fix(encode): NVENC partial-init session leak + three backend-parity gaps
All four found in the pf-encode quality sweep and verified against source.

- NVENC partial-init leak (BOTH platforms, high): `init_session` publishes
  `self.encoder` — and on Windows charges LIVE_SESSION_UNITS — *before* its
  remaining fallible steps (bitstream buffers; on Linux also the input-surface
  alloc and `register_resource`). A failure there left a live session with
  `inited == false`, and every guard on the re-init path keys off `inited`, so
  the next submit skipped teardown and overwrote `self.encoder`: the session
  leaked permanently toward the driver's per-process cap, and its budget units
  never returned, progressively starving parallel-display admission. `teardown`
  already keys off `encoder.is_null()` rather than `inited`, so it cleans up
  exactly this half-built state — it just was never called. Now invoked on the
  `init_session` error path on both platforms.

- `can_encode_10bit` asked the wrong backend (medium): it resolved via
  `linux_auto_is_vaapi`, which ignores `encoder_pref`, while `can_encode_444`
  and `open_video` honour it. On a host that forces a backend (e.g.
  `encoder_pref = "vaapi"` on an NVIDIA box) the probe answered for NVENC while
  the session opened VAAPI, so the negotiated bit depth — and the HDR/SDR colour
  label derived from it — described a backend that never ran. Now uses the same
  `linux_zero_copy_is_vaapi` mirror, and `linux_auto_is_vaapi` carries a warning
  that it resolves the `auto` case only and is not a dispatch mirror.

- Linux software arm ignored SW_BITRATE_CEIL (low): the Windows arm clamped
  openh264 to 100 Mbps, the Linux arm passed the full negotiated rate. The
  constant is now module-scope so both arms share one value.

- QSV/AMF env-parity (low): `PUNKTFUNK_IR_PERIOD_FRAMES` was a no-op on QSV
  despite the comment claiming parity with AMF, and `PUNKTFUNK_NO_QSV_LTR` /
  `PUNKTFUNK_INTRA_REFRESH` had dropped AMF's `trim()` and `yes`/`on` spellings,
  so a value with stray whitespace silently did nothing on Intel while the same
  value worked on AMD.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 20:43:16 +02:00
enricobuehlerandClaude Opus 4.8 ab679a56e7 chore(release): bump workspace version to 0.15.0
MINOR, not patch: 44 commits since v0.14.0 split 21 features / 19 fixes, and
/api/v1 gained the per-scanner library-toggle endpoints (additive, already
regenerated into api/openapi.json by c2bba134). sdk-publish pushes this version
to consumers, so a patch bump would understate a new API surface to anyone
pinning ~0.14.

Headline work is the desktop clients drawing level with the Apple revamp: the
Windows client picked up settings parity, a findable console UI in the header,
the shared clipboard (with a per-host toggle), PyroWave decode in the codec
picker, and D3D11VA-first decode + HDR pass-through on Intel; the Linux GTK4
client got the same category-map settings rebuild. Apple landed the intent-based
presenter rebuild and the Dynamic Island redesign. Also: the punktfunk-host
plugins CLI, per-scanner library toggles in the console, and PyroWave raw-dmabuf
zero-copy capture on the Linux NVIDIA host.

Notable fixes: LTR-RFI loss recovery under sustained loss, two encode teardown
memory-safety holes, the audio first-open retry that was leaving sessions
silent, and the GameStream stream-marker announcement on the compat plane.

Every workspace crate is on version.workspace = true, so this stayed a one-line
bump plus the lock sync. (fec-rs, pf-driver-proto, usbip-sim, the Windows driver
crates and pf-vkhdr-layer are deliberately versioned independently and stay put.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 20:32:43 +02:00
304 changed files with 6594 additions and 35312 deletions
+2 -12
View File
@@ -46,23 +46,13 @@ jobs:
# SHA-pinned: this workflow's release job carries the signing keystore + Play service-account
# secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e.
uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3
with:
# Only platform-tools — NOT the action's default legacy `tools`, whose dependency chain
# drags in the ~250 MB emulator nobody here runs (instrumentation tests are deferred).
# That download was the single flakiest piece of this job: the shared runner fleet drops
# packets under parallel-job load and sdkmanager's streamed unzip turns a truncated
# stream into "Error on ZipFile unknown archive" (observed 2026-07-22, twice).
packages: platform-tools
- name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build)
# cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path
# kit/build.gradle.kts prepends to PATH for cargo-ndk's audiopus_sys (libopus) CMake build.
# Note: platforms;android-37 is sometimes missing from standard channels; AGP will
# Note: platforms;android-37 is sometimes missing from standard channels; AGP will
# auto-download it if needed during the build.
# retry.sh: sdkmanager is a single-shot multi-hundred-MB fetch, exactly the class the
# helper exists for (fleet-load packet drops truncate the stream mid-unzip); a failed
# attempt leaves no partial package behind, so a plain re-invoke is safe.
run: bash scripts/ci/retry.sh 4 sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0" "ndk;30.0.14904198" "cmake;3.22.1"
run: sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0" "ndk;30.0.14904198" "cmake;3.22.1"
- name: Caches (cargo + gradle)
uses: actions/cache@v4
+30 -24
View File
@@ -6,12 +6,17 @@
#
# The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT
# need the Decky CLI (which requires Docker + rust-nightly only to compile native backends).
# We build the frontend with pnpm and stage the store-layout tree with the SAME script local
# builds use (clients/decky/scripts/package.sh) — the plugin's file list lives in exactly ONE
# place, so a file added there (bin/, assets/, controller_config/, …) can never be silently
# missing from the published build. (Hand-assembling the zip here is how the shipped plugin
# lost the shortcut artwork + Steam Input layout for a while.) CI only adds `update.json` on
# top: the {channel, manifest} pointer the plugin's self-update check polls.
# We build the frontend with pnpm and assemble the store-layout zip by hand:
#
# punktfunk.zip
# punktfunk/ <- single top-level dir == plugin.json "name"
# plugin.json [required]
# package.json [required; CI stamps "version" — Decky reads the installed version here]
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
#
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
@@ -85,27 +90,28 @@ jobs:
- name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }}
run: |
# node:22-bookworm ships python3 (a package.sh dep) but not zip; install both anyway
# so an image change can't silently break the build.
apt-get update && apt-get install -y --no-install-recommends zip python3 >/dev/null
# Stage the canonical plugin tree (dist/, main.py, bin/, assets/, controller_config/,
# LICENSE, …) with the same script local/sideload builds use — see the header comment.
# Runs AFTER the version stamp, so the staged package.json carries $VERSION.
bash clients/decky/scripts/package.sh
DEST="clients/decky/out/$PLUGIN"
# CI-only addition: the self-update channel pointer the backend reads (main.py
# check_update). It points at THIS channel's manifest.json (published below); that
# manifest in turn points at the immutable per-version zip, so its sha256 stays valid
# across future alias re-uploads.
apt-get update && apt-get install -y --no-install-recommends zip >/dev/null
STAGE="$RUNNER_TEMP/decky"
DEST="$STAGE/$PLUGIN"
rm -rf "$STAGE"; mkdir -p "$DEST/dist" "$DEST/bin"
cp clients/decky/plugin.json "$DEST/"
cp clients/decky/package.json "$DEST/"
cp clients/decky/main.py "$DEST/"
cp clients/decky/dist/index.js "$DEST/dist/"
cp clients/decky/README.md "$DEST/"
# The stream-launch wrapper (target of the Steam shortcut); keep it executable
# (runner_info() also re-chmods at runtime in case the zip/extract drops the bit).
cp clients/decky/bin/punktfunkrun.sh "$DEST/bin/"
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd clients/decky/out && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip"
# Backstop against packaging drift: the runtime-loaded pieces MUST be in the zip.
for f in main.py dist/index.js bin/punktfunkrun.sh assets/grid.png \
controller_config/punktfunk.vdf update.json; do
unzip -l "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN/$f" >/dev/null || { echo "MISSING $f" >&2; exit 1; }
done
# The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias).
+2 -15
View File
@@ -73,21 +73,8 @@ jobs:
# sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM
# trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied
# there, right before the first `flatpak` network call.
- name: Fix container DNS (drop nss-resolve)
run: |
sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf
# History: this step used to ALSO force glibc onto TCP DNS (`options use-vc`) because
# the runner fleet's Docker embedded resolver dropped UDP lookups under parallel-job
# load (investigated 2026-07-11; v0.15.0/v0.16.0 each burned retry.sh's whole budget).
# That root cause is now fixed at the infra level (2026-07-22): the runner host runs a
# local dnsmasq cache on the docker bridge and daemon.json points every job container
# at it, so lookups terminate on-box instead of crossing the saturated uplink — the
# UDP path is reliable again. The TCP path through the same chain proved FLAKY under
# fleet concurrency (flatpak remote-add failed 10/10 with instant NXDOMAIN while dnf
# in the same container resolved fine), so `use-vc` flipped from mitigation to sole
# cause of this leg's failures — removed. retry.sh (10×) stays as the backstop for
# genuine upstream blips.
cat /etc/resolv.conf || true
- name: Fix container DNS (drop nss-resolve — no systemd-resolved in CI)
run: sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf
# fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step
# executes via the container shell (no node needed), so install node BEFORE checkout.
-69
View File
@@ -1,69 +0,0 @@
# Publish the plugin framework (@punktfunk/plugin-kit) to the Gitea npm registry
# (https://git.unom.io/api/packages/unom/npm/).
#
# Trigger: push a tag `plugin-kit-vX.Y.Z` (must equal plugin-kit/package.json "version"),
# or run manually. Versions independently of the app's `v*` and the SDK's `sdk-v*` tags.
#
# The kit's devDependency on @punktfunk/host is `file:../sdk`, so the SDK's dist must be
# built BEFORE the kit's `bun install` copies it.
#
# Auth: REGISTRY_TOKEN — the same repo Actions secret sdk-publish.yml uses.
name: plugin-kit-publish
on:
push:
tags: ['plugin-kit-v*']
workflow_dispatch:
jobs:
publish:
runs-on: ubuntu-24.04
container:
image: oven/bun:1
timeout-minutes: 15
steps:
# oven/bun's slim base ships neither git, a CA bundle, nor node — actions/checkout's HTTPS
# fetch needs git + ca-certificates, and the version-guard step below uses node.
- name: Install git + node + CA certs
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates git nodejs
- uses: actions/checkout@v4
- name: Build the SDK (file:../sdk dependency source)
working-directory: sdk
run: |
bun install --frozen-lockfile --ignore-scripts
bun run build
- name: Install dependencies
working-directory: plugin-kit
run: bun install --frozen-lockfile --ignore-scripts
- name: Typecheck
working-directory: plugin-kit
run: bun run typecheck
- name: Test
working-directory: plugin-kit
run: bun test
- name: Build (dist/ JS + .d.ts + theme.css)
working-directory: plugin-kit
run: bun run build
- name: Tag matches package version
if: startsWith(github.ref, 'refs/tags/')
working-directory: plugin-kit
run: |
TAG="${GITHUB_REF_NAME#plugin-kit-v}"
PKG="$(node -p "require('./package.json').version")"
test "$TAG" = "$PKG" || { echo "tag $GITHUB_REF_NAME does not match package version $PKG"; exit 1; }
- name: Publish to Gitea registry
working-directory: plugin-kit
env:
NODE_AUTH_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
test -n "$NODE_AUTH_TOKEN" || { echo "REGISTRY_TOKEN secret is empty"; exit 1; }
printf '//git.unom.io/api/packages/unom/npm/:_authToken=%s\n' "$NODE_AUTH_TOKEN" >> .npmrc
bun publish
-24
View File
@@ -149,26 +149,6 @@ jobs:
# inherits this from the env during the xcframework build).
echo "CMAKE_POLICY_VERSION_MINIMUM=3.5" >> "$GITHUB_ENV"
- name: Pin + prune Xcode DerivedData
# Without -derivedDataPath, xcodebuild derives its DerivedData directory name from the
# PROJECT'S ABSOLUTE PATH — and act_runner rotates its workspace
# (~/.cache/act/<hash>/hostexecutor), so each rotation minted a brand new ~760 MB tree
# under ~/Library that nothing ever collected. 31 of them piled up in three days
# (~32 GB with the shared ModuleCache), filled the runner's boot volume, and failed
# v0.16.0's xcframework build with "No space left on device". Pinning one path makes the
# tree REUSED instead of multiplied — it also keeps the module cache warm between runs.
run: |
DD="$HOME/ci/derived-data/release"
mkdir -p "$DD"
echo "DERIVED_DATA=$DD" >> "$GITHUB_ENV"
# Safety net for trees the pin does not own: the legacy per-path ones from before this
# change, and anything another job leaves in the default root. Untouched for a week ⇒ gone.
if [ -d "$HOME/Library/Developer/Xcode/DerivedData" ]; then
find "$HOME/Library/Developer/Xcode/DerivedData" -mindepth 1 -maxdepth 1 \
-mtime +7 -exec rm -rf {} + 2>/dev/null || true
fi
echo "disk after prune:"; df -h /System/Volumes/Data | tail -1
- name: Build PunktfunkCore.xcframework (mac + iOS + tvOS)
# tvOS is a tier-3 target (nightly -Zbuild-std): slow on the first build, then cached on
# the self-hosted runner. Built on canary too so the tvOS archive/upload below runs on the
@@ -196,7 +176,6 @@ jobs:
-project "$PROJECT" -scheme Punktfunk \
-destination 'generic/platform=macOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-macos.xcarchive" \
-derivedDataPath "$DERIVED_DATA" \
-skipMacroValidation -skipPackagePluginValidation \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGNING_ALLOWED=NO
@@ -294,7 +273,6 @@ jobs:
-project "$PROJECT" -scheme Punktfunk \
-destination 'generic/platform=macOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-macos-appstore.xcarchive" \
-derivedDataPath "$DERIVED_DATA" \
-skipMacroValidation -skipPackagePluginValidation \
-allowProvisioningUpdates \
-authenticationKeyPath "$RUNNER_TEMP/asc.p8" \
@@ -358,7 +336,6 @@ jobs:
-project "$PROJECT" -scheme Punktfunk-iOS \
-destination 'generic/platform=iOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-ios.xcarchive" \
-derivedDataPath "$DERIVED_DATA" \
-skipMacroValidation -skipPackagePluginValidation \
-allowProvisioningUpdates \
-authenticationKeyPath "$RUNNER_TEMP/asc.p8" \
@@ -417,7 +394,6 @@ jobs:
-project "$PROJECT" -scheme Punktfunk-tvOS \
-destination 'generic/platform=tvOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-tvos.xcarchive" \
-derivedDataPath "$DERIVED_DATA" \
-skipMacroValidation -skipPackagePluginValidation \
-allowProvisioningUpdates \
-authenticationKeyPath "$RUNNER_TEMP/asc.p8" \
Generated
+27 -95
View File
@@ -656,30 +656,6 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "chacha20poly1305"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35"
dependencies = [
"aead",
"chacha20",
"cipher",
"poly1305",
"zeroize",
]
[[package]]
name = "ciborium"
version = "0.2.2"
@@ -715,7 +691,6 @@ checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [
"crypto-common",
"inout",
"zeroize",
]
[[package]]
@@ -1459,16 +1434,6 @@ dependencies = [
"version_check",
]
[[package]]
name = "gethostname"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bd49230192a3797a9a4d6abe9b3eed6f7fa4c8a8a4947977c6f80025f92cbd8"
dependencies = [
"rustix",
"windows-link 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "getrandom"
version = "0.2.17"
@@ -2194,7 +2159,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.18.0"
version = "0.15.0"
[[package]]
name = "lazy_static"
@@ -2299,7 +2264,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"bindgen",
"cmake",
@@ -2334,7 +2299,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"punktfunk-core",
]
@@ -2823,7 +2788,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2839,12 +2804,11 @@ dependencies = [
"tokio",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"x11rb",
]
[[package]]
name = "pf-client-core"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ash",
@@ -2868,7 +2832,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2886,7 +2850,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ash",
@@ -2907,7 +2871,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ash",
@@ -2917,7 +2881,6 @@ dependencies = [
"libvpl-sys",
"nvidia-video-codec-sdk",
"openh264",
"pf-capture",
"pf-frame",
"pf-gpu",
"pf-host-config",
@@ -2931,7 +2894,7 @@ dependencies = [
[[package]]
name = "pf-ffvk"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"ash",
"bindgen",
@@ -2940,7 +2903,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"libc",
@@ -2952,7 +2915,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"pf-host-config",
@@ -2966,11 +2929,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.18.0"
version = "0.15.0"
[[package]]
name = "pf-inject"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2998,14 +2961,14 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ash",
@@ -3020,7 +2983,7 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3050,7 +3013,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"pf-paths",
@@ -3062,7 +3025,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ash",
@@ -3173,17 +3136,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "poly1305"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf"
dependencies = [
"cpufeatures",
"opaque-debug",
"universal-hash",
]
[[package]]
name = "polyval"
version = "0.6.2"
@@ -3269,7 +3221,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"android_logger",
"jni",
@@ -3285,7 +3237,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3301,7 +3253,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3316,7 +3268,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3335,12 +3287,11 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"aes-gcm",
"bytes",
"cbindgen",
"chacha20poly1305",
"criterion",
"fec-rs",
"hmac",
@@ -3367,7 +3318,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"aes",
"aes-gcm",
@@ -3412,13 +3363,11 @@ dependencies = [
"rand 0.8.6",
"rcgen",
"reis",
"ring",
"roxmltree",
"rsa",
"rusqlite",
"rustls",
"rusty_enet",
"semver",
"serde",
"serde_json",
"sha2",
@@ -3451,7 +3400,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3465,7 +3414,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"anyhow",
"ksni",
@@ -3488,7 +3437,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.18.0"
version = "0.15.0"
dependencies = [
"bindgen",
"cmake",
@@ -5897,23 +5846,6 @@ version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
[[package]]
name = "x11rb"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9993aa5be5a26815fe2c3eacfc1fde061fc1a1f094bf1ad2a18bf9c495dd7414"
dependencies = [
"gethostname",
"rustix",
"x11rb-protocol",
]
[[package]]
name = "x11rb-protocol"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea6fc2961e4ef194dcbfe56bb845534d0dc8098940c7e5c012a258bfec6701bd"
[[package]]
name = "x509-parser"
version = "0.16.0"
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.18.0"
version = "0.15.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+1 -1133
View File
File diff suppressed because it is too large Load Diff
@@ -1,107 +0,0 @@
package io.unom.punktfunk
import android.content.ClipData
import android.content.ClipboardManager
import android.content.Context
import android.os.Handler
import android.os.Looper
import io.unom.punktfunk.kit.NativeBridge
/**
* Text clipboard sync for the active session (the desktop-client model, text-only v1):
* * **Device host**: a local copy (the primary-clip listener, plus one probe at start) is
* announced as a lazy offer the text crosses only when the host actually pastes (a
* `fetch:` event, answered with the clipboard's current content).
* * **Host device**: a host copy arrives as an `offer:` event and is fetched eagerly into
* the system clipboard (Android apps can't lazily materialize a paste from the network
* without a content-provider round-trip that isn't worth it here).
*
* Loop guard: text set from a host fetch is remembered ([lastFromHost]) so the resulting
* primary-clip-changed callback doesn't bounce it straight back as a new offer. Clipboard reads
* happen while the stream is foreground (Android only allows focused-app reads). The native
* events are drained on a dedicated thread and applied on the main thread; [stop] joins it.
*/
class ClipboardSync(
private val context: Context,
private val handle: Long,
) {
private val main = Handler(Looper.getMainLooper())
private val cm = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
@Volatile private var running = true
private var seq = 0
private var lastOffered: String? = null
private var lastFromHost: String? = null
private var pendingFetch = -1
private var thread: Thread? = null
private val clipListener = ClipboardManager.OnPrimaryClipChangedListener { offerLocal() }
fun start() {
NativeBridge.nativeClipControl(handle, true)
cm.addPrimaryClipChangedListener(clipListener)
thread = Thread({ pollLoop() }, "pf-clipboard").also { it.start() }
offerLocal() // whatever is already on the clipboard is pasteable host-side right away
}
fun stop() {
running = false
cm.removePrimaryClipChangedListener(clipListener)
thread?.join(600) // one poll timeout (250 ms) + slack
thread = null
}
/** Announce the current local text (if it's new and not an echo of a host copy). */
private fun offerLocal() {
if (!running) return
val text = currentClipText() ?: return
if (text == lastOffered || text == lastFromHost) return
lastOffered = text
seq += 1
NativeBridge.nativeClipOfferText(handle, seq)
}
private fun currentClipText(): String? = runCatching {
cm.primaryClip?.takeIf { it.itemCount > 0 }?.getItemAt(0)
?.coerceToText(context)?.toString()?.takeIf { it.isNotEmpty() }
}.getOrNull()
private fun pollLoop() {
while (running) {
val ev = NativeBridge.nativeNextClip(handle) ?: continue
if (ev == "closed") return
main.post { handleEvent(ev) }
}
}
private fun handleEvent(ev: String) {
if (!running) return
val parts = ev.split(":", limit = 3)
when (parts[0]) {
"offer" -> {
val offerSeq = parts.getOrNull(1)?.toIntOrNull() ?: return
if (parts.getOrNull(2) == "1") {
pendingFetch = NativeBridge.nativeClipFetchText(handle, offerSeq)
}
}
"fetch" -> {
val req = parts.getOrNull(1)?.toIntOrNull() ?: return
val text = currentClipText()
if (text != null) {
NativeBridge.nativeClipServeText(handle, req, text)
} else {
NativeBridge.nativeClipCancel(handle, req)
}
}
"data" -> {
val xfer = parts.getOrNull(1)?.toIntOrNull() ?: return
if (xfer != pendingFetch) return // stale/unknown transfer
pendingFetch = -1
val text = parts.getOrNull(2)?.takeIf { it.isNotEmpty() } ?: return
lastFromHost = text
runCatching { cm.setPrimaryClip(ClipData.newPlainText("Punktfunk", text)) }
}
// "state"/"cancel"/"error": nothing to drive in the text-only v1.
}
}
}
@@ -54,21 +54,6 @@ class MainActivity : ComponentActivity() {
var padKeyProbe: ((KeyEvent) -> Boolean)? = null
var padMotionProbe: ((MotionEvent) -> Boolean)? = null
/**
* Physical-mouse forwarder for the active session (built/released by StreamScreen, like
* [gamepadRouter]): uncaptured hover/click/wheel forwards as absolute cursor input, captured
* ([android.view.View.requestPointerCapture]) raw deltas as relative mouse-look. The dispatch
* overrides below route every SOURCE_MOUSE event here while streaming. Null while not streaming.
*/
var mouseForwarder: MouseForwarder? = null
/**
* TV remote-as-pointer for the active session (StreamScreen builds it on TV devices only):
* hold SELECT to toggle, then the D-pad glides the host cursor. Consulted first for
* non-gamepad keys while streaming. Null while not streaming or not a TV.
*/
var remotePointer: RemotePointer? = null
/**
* Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a
* couch user with no keyboard/Back can always leave a stream.
@@ -339,29 +324,9 @@ class MainActivity : ComponentActivity() {
return true // consumed
}
}
// TV remote-as-pointer sees non-gamepad keys first (SELECT long-press toggles it;
// while active it owns the D-pad/SELECT/PLAY-PAUSE/BACK).
if (!event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
remotePointer?.let { if (it.onKey(event)) return true }
}
// Ctrl+Alt+Shift+Q — the cross-client pointer-capture toggle chord. Swallow both
// edges of the Q (the modifiers already went over the wire, exactly like desktop).
if (event.keyCode == KeyEvent.KEYCODE_Q &&
event.isCtrlPressed && event.isAltPressed && event.isShiftPressed
) {
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) {
mouseForwarder?.toggleCapture()
}
return true
}
when (event.keyCode) {
// A mouse back/forward button whose BUTTON_* press went unconsumed makes the
// framework synthesize a FALLBACK BACK — the button already went over the wire
// as X1/X2, and it must never yank the user out of the stream.
KeyEvent.KEYCODE_BACK ->
if (event.flags and KeyEvent.FLAG_FALLBACK != 0) return true
// Leave these to the system even while streaming.
// (BACK above → BackHandler leaves the stream.)
KeyEvent.KEYCODE_BACK, // → BackHandler leaves the stream
KeyEvent.KEYCODE_VOLUME_UP,
KeyEvent.KEYCODE_VOLUME_DOWN,
KeyEvent.KEYCODE_VOLUME_MUTE,
@@ -429,10 +394,6 @@ class MainActivity : ComponentActivity() {
override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
if (streamHandle != 0L) {
if (gamepadRouter?.onMotion(event) == true) return true
// Physical mouse (uncaptured): hover motion, wheel, button edges.
if (event.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onGenericMotion(event)) return true }
}
return super.dispatchGenericMotionEvent(event)
}
// The Controllers debug screen sees pad motion before the stick→D-pad synthesis below.
@@ -470,24 +431,6 @@ class MainActivity : ComponentActivity() {
return super.dispatchGenericMotionEvent(event)
}
/**
* Mouse clicks/drags ride the TOUCH stream (the pointer is "down"). While streaming they
* belong to the mouse forwarder, never to the Compose touch-gesture layer a physical
* mouse click must be a real click at the cursor, not a synthesized trackpad tap.
*/
override fun dispatchTouchEvent(ev: MotionEvent): Boolean {
if (streamHandle != 0L && ev.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onTouchEvent(ev)) return true }
}
return super.dispatchTouchEvent(ev)
}
/** The OS is the source of truth for pointer capture (it releases on focus loss). */
override fun onPointerCaptureChanged(hasCapture: Boolean) {
super.onPointerCaptureChanged(hasCapture)
mouseForwarder?.onCaptureChanged(hasCapture)
}
/** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */
private fun isConsoleNavKey(kc: Int): Boolean = when (kc) {
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT,
@@ -1,206 +0,0 @@
package io.unom.punktfunk
import android.view.InputDevice
import android.view.MotionEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.roundToInt
/** True when any connected input device is a pointer (USB/BT mouse, or a touchpad driving one). */
fun hasPhysicalMouse(): Boolean = InputDevice.getDeviceIds().any { id ->
InputDevice.getDevice(id)?.supportsSource(InputDevice.SOURCE_MOUSE) == true
}
/**
* Physical mouse wire, in two modes (the iPadOS/desktop model):
* * **uncaptured** (default): hover/drag positions forward as absolute cursor moves
* (`MouseMoveAbs`, host-normalized against the window size) desktop-style pointing. The
* local cursor is hidden over the stream (StreamScreen sets a TYPE_NULL pointer icon); the
* host's own cursor, composited into the video, is the one you see.
* * **captured**: the OS pointer is grabbed ([android.view.View.requestPointerCapture]) and raw
* relative deltas forward as `MouseMove` FPS mouse-look. Engaged at stream start / by
* clicking into the stream when the "Capture pointer for games" setting is on, and toggled
* any time by Ctrl+Alt+Shift+Q (the cross-client chord). Focus loss releases it (the OS
* guarantees that); a click re-engages.
*
* Buttons ride [MotionEvent.ACTION_BUTTON_PRESS]/RELEASE edges (left/middle/right/back/forward
* wire 1/2/3/4/5), the wheel rides [MotionEvent.ACTION_SCROLL] with fractional accumulation so
* high-resolution wheels don't lose sub-notch travel. Held buttons are tracked and flushed on
* capture loss / stream exit so nothing sticks on the host. Events reach this class from
* MainActivity's dispatch overrides (uncaptured) and the capture view's captured-pointer listener.
*/
class MouseForwarder(
private val handle: Long,
private val invertScroll: Boolean,
private val captureWanted: Boolean,
private val surfaceSize: () -> Pair<Int, Int>,
) {
/** Capture plumbing, owned by StreamScreen (the focusable capture view). */
var onRequestCapture: (() -> Unit)? = null
var onReleaseCapture: (() -> Unit)? = null
/** Live capture state, updated from [android.app.Activity.onPointerCaptureChanged]. */
var captured = false
private set
/** Chord-released: no auto re-engage (start / click) until the user opts back in. */
private var userReleased = false
private val heldButtons = mutableSetOf<Int>()
private var scrollAccV = 0f
private var scrollAccH = 0f
private var moveAccX = 0f
private var moveAccY = 0f
/** Uncaptured mouse events on the TOUCH stream (position while a button is down). */
fun onTouchEvent(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_DOWN -> {
if (captureWanted && !captured && !userReleased) {
// The engaging click: grab the pointer and swallow the click (desktop
// parity — the click that captures never reaches the host). The paired
// BUTTON_RELEASE is dropped by the held-set guard in [button].
onRequestCapture?.invoke()
return true
}
sendAbs(ev)
}
MotionEvent.ACTION_MOVE -> sendAbs(ev)
// Button edges are documented on the generic stream, but be robust to either.
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
}
return true
}
/** Uncaptured mouse events on the GENERIC stream (hover motion, wheel, button edges). */
fun onGenericMotion(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_HOVER_MOVE -> sendAbs(ev)
MotionEvent.ACTION_SCROLL -> wheel(ev)
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_EXIT -> {}
else -> return false
}
return true
}
/**
* Captured-pointer events (the view holds [android.view.View.requestPointerCapture]): x/y ARE
* the relative deltas ([InputDevice.SOURCE_MOUSE_RELATIVE]), batched samples included. A
* captured touchpad reports absolute finger coordinates instead not handled (the touch
* gesture layer is the touchpad story); returning false leaves those to the framework.
*/
fun onCapturedPointer(ev: MotionEvent): Boolean {
if (!ev.isFromSource(InputDevice.SOURCE_MOUSE_RELATIVE)) return false
when (ev.actionMasked) {
MotionEvent.ACTION_MOVE -> {
var dx = 0f
var dy = 0f
for (i in 0 until ev.historySize) {
dx += ev.getHistoricalX(i)
dy += ev.getHistoricalY(i)
}
dx += ev.x
dy += ev.y
moveAccX += dx
moveAccY += dy
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept w/ sign
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_SCROLL -> wheel(ev)
}
return true
}
/** Ctrl+Alt+Shift+Q: release the grab, or (re-)engage it — works even when auto-capture is off. */
fun toggleCapture() {
if (captured) {
userReleased = true
onReleaseCapture?.invoke()
} else {
userReleased = false
onRequestCapture?.invoke()
}
}
/** Auto-engage at stream start (setting on + a mouse actually present). */
fun engageFromStart() {
if (captureWanted && !captured && !userReleased && hasPhysicalMouse()) {
onRequestCapture?.invoke()
}
}
/** From [android.app.Activity.onPointerCaptureChanged] — the OS is the source of truth. */
fun onCaptureChanged(has: Boolean) {
captured = has
// Losing the grab (focus loss, chord) must not leave buttons held on the host.
if (!has) flushButtons()
}
/** Stream teardown: lift anything held and let the grab go. */
fun release() {
flushButtons()
if (captured) onReleaseCapture?.invoke()
}
private fun sendAbs(ev: MotionEvent) {
val (w, h) = surfaceSize()
if (w <= 0 || h <= 0) return
NativeBridge.nativeSendPointerAbs(
handle,
ev.x.roundToInt().coerceIn(0, w - 1),
ev.y.roundToInt().coerceIn(0, h - 1),
w,
h,
)
}
private fun wheel(ev: MotionEvent) {
val dir = if (invertScroll) -1f else 1f
// Android: AXIS_VSCROLL + = up/away, AXIS_HSCROLL + = right — the wire's convention too.
scrollAccV += ev.getAxisValue(MotionEvent.AXIS_VSCROLL) * 120f * dir
scrollAccH += ev.getAxisValue(MotionEvent.AXIS_HSCROLL) * 120f * dir
val v = scrollAccV.toInt()
if (v != 0) {
NativeBridge.nativeSendScroll(handle, 0, v)
scrollAccV -= v
}
val h = scrollAccH.toInt()
if (h != 0) {
NativeBridge.nativeSendScroll(handle, 1, h)
scrollAccH -= h
}
}
private fun button(actionButton: Int, down: Boolean) {
val b = when (actionButton) {
MotionEvent.BUTTON_PRIMARY -> 1
MotionEvent.BUTTON_TERTIARY -> 2
MotionEvent.BUTTON_SECONDARY -> 3
MotionEvent.BUTTON_BACK -> 4
MotionEvent.BUTTON_FORWARD -> 5
else -> return
}
if (down) {
heldButtons.add(b)
NativeBridge.nativeSendPointerButton(handle, b, true)
} else if (heldButtons.remove(b)) {
// Only release what we pressed — drops the release of a swallowed engaging click
// and anything that raced a capture transition.
NativeBridge.nativeSendPointerButton(handle, b, false)
}
}
private fun flushButtons() {
heldButtons.forEach { NativeBridge.nativeSendPointerButton(handle, it, false) }
heldButtons.clear()
}
}
@@ -1,193 +0,0 @@
package io.unom.punktfunk
import android.os.Handler
import android.os.Looper
import android.view.Choreographer
import android.view.KeyEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.hypot
// Hold this long on SELECT (pointer-mode toggle) / PLAY-PAUSE (keyboard toggle) for the long-press
// action instead of the tap action.
private const val LONG_PRESS_MS = 800L
// D-pad glide ballistics, in screen-widths per second: start slow enough to hit a close button,
// ramp over RAMP_S seconds of continuous hold so crossing the desktop doesn't take all day.
private const val SPEED_MIN = 0.14f
private const val SPEED_MAX = 0.70f
private const val RAMP_S = 1.2f
/**
* Android TV remote as a pointer the Android analogue of the Apple client's Siri-remote pointer,
* adapted for D-pad-only remotes (most Android TV remotes have no touch surface). For the
* "TV as a desktop client" use case, where a plain remote is often the only thing in hand.
*
* While streaming on a TV, **hold SELECT 0.8 s** to toggle pointer mode. While active:
* * D-pad (held) glides the host cursor with ramping acceleration (relative `MouseMove`,
* Choreographer-paced, diagonal-normalized);
* * SELECT tap = left click; PLAY/PAUSE tap = right click (Siri-remote parity);
* * PLAY/PAUSE held = toggle the on-screen keyboard; BACK = leave pointer mode
* (a second BACK then leaves the stream as usual).
* While inactive, everything except the SELECT long-press passes through untouched (D-pad =
* arrow keys, SELECT tap = Enter synthesized on release, since the down was held back to
* disambiguate the long-press).
*
* Only consulted for non-gamepad key events on TV devices (MainActivity gates the calls); all
* state lives on the main thread.
*/
class RemotePointer(
private val handle: Long,
private val surfaceWidth: () -> Int,
private val onActiveChanged: (Boolean) -> Unit,
private val onKeyboardToggle: () -> Unit,
) {
var active = false
private set
private val handler = Handler(Looper.getMainLooper())
private val held = mutableSetOf<Int>() // D-pad keycodes currently down
private var moveAccX = 0f
private var moveAccY = 0f
private var lastFrameNs = 0L
private var rampSec = 0f
private var tickerRunning = false
private var centerLongFired = false
private var playLongFired = false
private val centerLong = Runnable {
centerLongFired = true
toggle()
}
private val playLong = Runnable {
playLongFired = true
onKeyboardToggle()
}
private val frame = object : Choreographer.FrameCallback {
override fun doFrame(nowNs: Long) {
if (!tickerRunning) return
if (held.isEmpty() || !active) {
tickerRunning = false
return
}
val dt = if (lastFrameNs == 0L) {
1f / 60f
} else {
((nowNs - lastFrameNs) / 1e9f).coerceIn(0.001f, 0.1f)
}
lastFrameNs = nowNs
rampSec += dt
var vx = 0f
var vy = 0f
if (KeyEvent.KEYCODE_DPAD_LEFT in held) vx -= 1f
if (KeyEvent.KEYCODE_DPAD_RIGHT in held) vx += 1f
if (KeyEvent.KEYCODE_DPAD_UP in held) vy -= 1f
if (KeyEvent.KEYCODE_DPAD_DOWN in held) vy += 1f
val mag = hypot(vx, vy)
if (mag > 0f) {
val w = surfaceWidth().coerceAtLeast(640)
val speed = w * (SPEED_MIN + (SPEED_MAX - SPEED_MIN) * (rampSec / RAMP_S).coerceAtMost(1f))
moveAccX += vx / mag * speed * dt
moveAccY += vy / mag * speed * dt
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
Choreographer.getInstance().postFrameCallback(this)
}
}
/** One remote key event; true = consumed. Ignore key repeats — the ticker owns motion. */
fun onKey(event: KeyEvent): Boolean {
val down = event.action == KeyEvent.ACTION_DOWN
when (event.keyCode) {
KeyEvent.KEYCODE_DPAD_CENTER -> {
if (down) {
if (event.repeatCount == 0) {
centerLongFired = false
handler.postDelayed(centerLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(centerLong)
if (!centerLongFired) {
if (active) {
click(1)
} else {
// The down was held back to disambiguate the long-press, so the
// normal path never saw it — synthesize the Enter here instead.
NativeBridge.nativeSendKey(handle, 0x0D, true, 0)
NativeBridge.nativeSendKey(handle, 0x0D, false, 0)
}
}
}
return true
}
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN,
KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_RIGHT,
-> {
if (!active) return false
if (down) {
if (held.add(event.keyCode) && held.size == 1) startTicker()
} else {
held.remove(event.keyCode)
}
return true
}
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE -> {
if (!active) return false // inactive: the media-key VK path owns it
if (down) {
if (event.repeatCount == 0) {
playLongFired = false
handler.postDelayed(playLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(playLong)
if (!playLongFired) click(3)
}
return true
}
KeyEvent.KEYCODE_BACK -> {
if (!active) return false
if (!down) toggle() // leave pointer mode; the next BACK leaves the stream
return true
}
else -> return false
}
}
/** Stream teardown: stop timers/ticker; nothing wire-held to flush (clicks are edges). */
fun release() {
handler.removeCallbacks(centerLong)
handler.removeCallbacks(playLong)
active = false
held.clear()
tickerRunning = false
}
private fun toggle() {
active = !active
if (!active) {
held.clear()
tickerRunning = false
}
onActiveChanged(active)
}
private fun startTicker() {
rampSec = 0f
lastFrameNs = 0L
if (!tickerRunning) {
tickerRunning = true
Choreographer.getInstance().postFrameCallback(frame)
}
}
private fun click(button: Int) {
NativeBridge.nativeSendPointerButton(handle, button, true)
NativeBridge.nativeSendPointerButton(handle, button, false)
}
}
@@ -109,27 +109,6 @@ data class Settings(
* setup where the OS-level pad (lizard mode) is preferred.
*/
val sc2Capture: Boolean = true,
/**
* Lock a physical mouse to the stream ([android.view.View.requestPointerCapture]) and forward
* raw relative motion FPS mouse-look, the iPad "Capture pointer for games" twin. Engages at
* stream start and on a click into the stream; Ctrl+Alt+Shift+Q toggles it live (the chord
* works even with this off). Off (default): a mouse points absolutely, desktop-style.
*/
val pointerCapture: Boolean = false,
/**
* Flip scroll direction the mouse wheel and the two-finger touch scroll both. Parity with
* the Apple/GTK clients' "Invert scroll direction".
*/
val invertScroll: Boolean = false,
/**
* Sync text copied on this device to the host and vice versa while streaming (the desktop
* clients' shared clipboard, text-only here). Only effective when the host advertises the
* clipboard capability; the protocol is opt-in per session either way.
*/
val clipboardSync: Boolean = true,
)
/** [Settings.touchMode] values; persisted by name. */
@@ -193,9 +172,6 @@ class SettingsStore(context: Context) {
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
pointerCapture = prefs.getBoolean(K_POINTER_CAPTURE, false),
invertScroll = prefs.getBoolean(K_INVERT_SCROLL, false),
clipboardSync = prefs.getBoolean(K_CLIPBOARD_SYNC, true),
)
fun save(s: Settings) {
@@ -219,9 +195,6 @@ class SettingsStore(context: Context) {
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.putBoolean(K_POINTER_CAPTURE, s.pointerCapture)
.putBoolean(K_INVERT_SCROLL, s.invertScroll)
.putBoolean(K_CLIPBOARD_SYNC, s.clipboardSync)
.apply()
}
@@ -260,9 +233,6 @@ class SettingsStore(context: Context) {
const val K_AUTO_WAKE = "auto_wake_enabled"
const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_SC2_CAPTURE = "sc2_capture"
const val K_POINTER_CAPTURE = "pointer_capture"
const val K_INVERT_SCROLL = "invert_scroll"
const val K_CLIPBOARD_SYNC = "clipboard_sync"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode"
@@ -412,27 +412,6 @@ private fun ControlsSettings(s: Settings, update: (Settings) -> Unit, onOpenCont
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
ToggleRow(
title = "Capture pointer for games",
subtitle = "Lock a connected mouse to the stream and send raw relative motion " +
"(mouse-look). Ctrl+Alt+Shift+Q toggles it live; click the stream to re-capture. " +
"Off: the mouse points at the desktop directly",
checked = s.pointerCapture,
onCheckedChange = { on -> update(s.copy(pointerCapture = on)) },
)
ToggleRow(
title = "Invert scroll direction",
subtitle = "Flip the mouse wheel and two-finger touch scrolling",
checked = s.invertScroll,
onCheckedChange = { on -> update(s.copy(invertScroll = on)) },
)
ToggleRow(
title = "Shared clipboard",
subtitle = "Text copied here pastes on the host and vice versa (hosts with " +
"clipboard sharing enabled)",
checked = s.clipboardSync,
onCheckedChange = { on -> update(s.copy(clipboardSync = on)) },
)
}
SettingsCard {
SettingDropdown(
@@ -176,14 +176,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// "hold to quit" hint overlay. Set from the router's onExitArmed (main thread).
var exitArming by remember { mutableStateOf(false) }
// True while the TV remote is acting as a pointer (hold SELECT toggles) — drives the mode hint.
var remotePointerOn by remember { mutableStateOf(false) }
// Focus anchor the soft keyboard is summoned onto AND the pointer-capture grab target (a grab
// needs a focusable view; captured-pointer events land on it). Declared before the effect
// below so the capture callbacks can reach the view once it exists.
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
wifiLocks.forEach { lock ->
@@ -229,54 +221,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed }
// Physical mouse: uncaptured hover/click/wheel forwards as absolute pointing; captured
// (setting or the Ctrl+Alt+Shift+Q chord) raw deltas forward as relative mouse-look.
// The local cursor is hidden over the stream — the host's own cursor, composited into
// the video, is the one the user sees (twin of the desktop clients' hidden cursor).
val decor = window?.decorView
val priorPointerIcon = decor?.pointerIcon
decor?.pointerIcon = android.view.PointerIcon.getSystemIcon(
context,
android.view.PointerIcon.TYPE_NULL,
)
val mouse = MouseForwarder(
handle,
invertScroll = initialSettings.invertScroll,
captureWanted = initialSettings.pointerCapture,
surfaceSize = { (decor?.width ?: 0) to (decor?.height ?: 0) },
)
mouse.onRequestCapture = {
// The grab needs the (focusable) capture view: focus it, then ask. Posted so a
// request racing view attach/focus settles on the next frame.
keyCapture?.let { v ->
v.post {
v.requestFocus()
v.requestPointerCapture()
}
}
}
mouse.onReleaseCapture = { keyCapture?.releasePointerCapture() }
activity?.mouseForwarder = mouse
// TV remote-as-pointer: hold SELECT ≈ 0.8 s to toggle; the D-pad then glides the host
// cursor (see RemotePointer). TV only — a phone's remote-less keys stay on the VK path.
val remote = if (isTv) {
RemotePointer(
handle,
surfaceWidth = { decor?.width ?: 1920 },
onActiveChanged = { on -> remotePointerOn = on },
onKeyboardToggle = { keyCapture?.let { it.setImeVisible(!it.imeShown) } },
)
} else {
null
}
activity?.remotePointer = remote
// Shared clipboard (text v1): only when the user setting is on AND the host has a
// working clipboard service. Protocol-level opt-in + the poll thread live in the sync.
val clip = if (initialSettings.clipboardSync && NativeBridge.nativeClipSupported(handle)) {
ClipboardSync(context, handle).also { it.start() }
} else {
null
}
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad
// index via the router; poll threads stopped + joined before the router is released and the
@@ -342,7 +286,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
}
onDispose {
closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it
clip?.stop() // stop + join the clipboard poll thread BEFORE the handle is freed
feedback.onHidRaw = null
feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed
sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
@@ -350,12 +293,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
mouse.release()
activity?.mouseForwarder = null
remote?.release()
activity?.remotePointer = null
decor?.pointerIcon = priorPointerIcon
activity?.streamHandle = 0L
activity?.requestStreamExit = null
// Back in the menus: the SC2 (if present) resumes driving the console UI.
@@ -383,12 +320,8 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Auto-engage pointer capture at stream start (setting on + a mouse actually present).
// Delayed a beat: the grab needs window focus and the capture view attached.
LaunchedEffect(handle) {
delay(400)
activity?.mouseForwarder?.engageFromStart()
}
// Focus anchor the three-finger keyboard swipe summons the IME onto (see KeyCaptureView).
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
Box(modifier = Modifier.fillMaxSize()) {
AndroidView(
@@ -446,26 +379,11 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (exitArming) {
ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Remote-pointer mode hint — the remote's keys are remapped while it's on, so say so.
if (remotePointerOn) {
RemotePointerHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe up
// in the mouse modes) AND the pointer-capture grab target — it never draws or takes
// touches, it just owns IME focus and receives captured-pointer events.
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe
// up in the mouse modes) — it never draws or takes touches, it just owns IME focus.
AndroidView(
modifier = Modifier.size(1.dp),
factory = { ctx ->
KeyCaptureView(ctx).also { v ->
keyCapture = v
// Real IME text path when the host types committed text (see KeyCaptureView).
v.textHandle =
if (NativeBridge.nativeTextInputSupported(handle)) handle else 0L
v.setOnCapturedPointerListener { _, ev ->
(ctx as? MainActivity)?.mouseForwarder?.onCapturedPointer(ev) ?: false
}
}
},
factory = { ctx -> KeyCaptureView(ctx).also { keyCapture = it } },
)
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
@@ -478,7 +396,6 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
else -> streamTouchInput(
handle,
trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() },
onKeyboard = { show -> keyCapture?.setImeVisible(show) },
)
@@ -506,35 +423,14 @@ private fun ExitChordHint(modifier: Modifier = Modifier) {
)
}
/**
* The remote-pointer mode cue: while active the remote's keys are remapped (D-pad glides the host
* cursor, SELECT clicks), so the overlay both confirms the toggle and teaches the vocabulary.
*/
@Composable
private fun RemotePointerHint(modifier: Modifier = Modifier) {
Text(
"Remote pointer — SELECT click · play/pause right-click · hold SELECT to exit",
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/**
* Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME
* onto this view. Two IME models, picked by the host's capabilities:
* * **Text path** ([textHandle] set the host advertised `HOST_CAP_TEXT_INPUT`): a real
* editable [HostTextConnection], so the IME gives autocorrect, gesture typing, non-Latin
* composition and emoji, all mirrored to the host as committed text + diffs.
* * **Fallback** (older host): `TYPE_NULL` puts the IME in "dumb keyboard" mode raw
* [KeyEvent]s flow through `MainActivity.dispatchKeyEvent` `Keymap.toVk` the host, the
* exact path a hardware keyboard takes (with the IME-shift wrap documented there).
*
* Doubles as the pointer-capture grab target: a grab needs a focusable view, and captured-pointer
* events are delivered to it (routed to [MouseForwarder.onCapturedPointer] via the listener the
* stream screen installs).
* onto this view. `TYPE_NULL` puts the IME in "dumb keyboard" mode it delivers raw [KeyEvent]s
* (no composing text, no autocorrect), which flow through `MainActivity.dispatchKeyEvent`
* `Keymap.toVk` the host, the exact path a hardware keyboard takes. Text an IME insists on
* committing instead still arrives: the non-editable [BaseInputConnection] synthesizes KeyEvents
* for it via `KeyCharacterMap` (with Shift carried as meta state see the IME-shift wrap in
* `MainActivity.dispatchKeyEvent`).
*/
private class KeyCaptureView(context: Context) : View(context) {
init {
@@ -542,32 +438,17 @@ private class KeyCaptureView(context: Context) : View(context) {
isFocusableInTouchMode = true
}
/** The session handle when the host types committed text; `0` = VK-only fallback. */
var textHandle: Long = 0L
/** Whether [setImeVisible] last showed the IME — for toggle-style callers (remote pointer). */
var imeShown = false
private set
override fun onCheckIsTextEditor(): Boolean = true
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return if (textHandle != 0L) {
outAttrs.inputType = InputType.TYPE_CLASS_TEXT or
InputType.TYPE_TEXT_FLAG_AUTO_CORRECT or InputType.TYPE_TEXT_FLAG_MULTI_LINE
HostTextConnection(this, textHandle)
} else {
outAttrs.inputType = InputType.TYPE_NULL
BaseInputConnection(this, false)
}
outAttrs.inputType = InputType.TYPE_NULL
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN
return BaseInputConnection(this, false)
}
fun setImeVisible(show: Boolean) {
val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager
?: return
imeShown = show
if (show) {
requestFocus()
imm.showSoftInput(this, 0)
@@ -576,113 +457,3 @@ private class KeyCaptureView(context: Context) : View(context) {
}
}
}
/**
* IME host text bridge (the `HOST_CAP_TEXT_INPUT` path): a real **editable** connection, so
* the IME runs its full machinery (autocorrect, gesture typing, non-Latin composition), mirrored
* to the host as it happens. The one piece of host-side state tracked is *what the host currently
* shows of the active composition* ([sentComposition]): composing updates send a common-prefix
* diff (backspaces + the new suffix) so corrections materialize live on the host; a commit
* settles it. [setComposingRegion] adopts already-committed text as the active composition
* (autocorrect-revert / backspace-into-word flows), so the next update diffs against it instead
* of retyping. Newlines become Enter taps; [deleteSurroundingText] becomes Backspace/Delete taps.
*
* Known approximation: diff lengths are counted in Unicode scalars, assuming one host Backspace
* deletes one scalar true for the composition text IMEs actually produce (emoji and other
* multi-unit graphemes commit directly rather than composing).
*/
private class HostTextConnection(
view: KeyCaptureView,
private val handle: Long,
) : BaseInputConnection(view, true) {
/** What the host currently shows of the active composition ("" = none). */
private var sentComposition = ""
override fun commitText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
sentComposition = ""
val ok = super.commitText(text, newCursorPosition)
trimEditable()
return ok
}
override fun setComposingText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
return super.setComposingText(text, newCursorPosition)
}
override fun finishComposingText(): Boolean {
// The composition text stands as committed — the host already shows it verbatim.
sentComposition = ""
return super.finishComposingText()
}
override fun setComposingRegion(start: Int, end: Int): Boolean {
val e = editable
if (e != null) {
val a = start.coerceIn(0, e.length)
val b = end.coerceIn(0, e.length)
sentComposition = e.subSequence(minOf(a, b), maxOf(a, b)).toString()
}
return super.setComposingRegion(start, end)
}
override fun deleteSurroundingText(beforeLength: Int, afterLength: Int): Boolean {
repeat(beforeLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_BACK) }
repeat(afterLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_DELETE) }
return super.deleteSurroundingText(beforeLength, afterLength)
}
override fun performEditorAction(actionCode: Int): Boolean {
tapVk(VK_RETURN)
return true
}
/** Replace the host's view of the composition with [text] via a common-prefix diff. */
private fun retype(text: String) {
var common = sentComposition.commonPrefixWith(text)
// Never split a surrogate pair mid-diff — back off to the pair boundary.
if (common.isNotEmpty() && common.last().isHighSurrogate()) {
common = common.dropLast(1)
}
val stale = sentComposition.substring(common.length)
repeat(stale.codePointCount(0, stale.length).coerceAtMost(MAX_TAPS)) { tapVk(VK_BACK) }
sendText(text.substring(common.length))
sentComposition = text
}
/** Forward literal text, turning newlines into Enter taps (control chars never ride text). */
private fun sendText(s: String) {
var chunk = StringBuilder()
for (ch in s) {
if (ch == '\n') {
if (chunk.isNotEmpty()) {
NativeBridge.nativeSendText(handle, chunk.toString())
chunk = StringBuilder()
}
tapVk(VK_RETURN)
} else {
chunk.append(ch)
}
}
if (chunk.isNotEmpty()) NativeBridge.nativeSendText(handle, chunk.toString())
}
private fun tapVk(vk: Int) {
NativeBridge.nativeSendKey(handle, vk, true, 0)
NativeBridge.nativeSendKey(handle, vk, false, 0)
}
/** Bound the mirror buffer: once nothing is composing, old text serves no purpose. */
private fun trimEditable() {
val e = editable ?: return
if (getComposingSpanStart(e) == -1 && e.length > 4000) e.clear()
}
private companion object {
const val VK_BACK = 0x08
const val VK_RETURN = 0x0D
const val VK_DELETE = 0x2E
const val MAX_TAPS = 256
}
}
@@ -99,11 +99,9 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput(
handle: Long,
trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit,
onKeyboard: (show: Boolean) -> Unit,
) {
val scrollDir = if (invertScroll) -1 else 1
var lastTapUp = 0L
var lastTapX = 0f
var lastTapY = 0f
@@ -186,12 +184,12 @@ internal suspend fun PointerInputScope.streamTouchInput(
val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up
val sx = ((cx - prevCx) / SCROLL_DIV).toInt()
if (sy != 0) {
NativeBridge.nativeSendScroll(handle, 0, sy * 120 * scrollDir)
NativeBridge.nativeSendScroll(handle, 0, sy * 120)
prevCy = cy
moved = true
}
if (sx != 0) {
NativeBridge.nativeSendScroll(handle, 1, sx * 120 * scrollDir)
NativeBridge.nativeSendScroll(handle, 1, sx * 120)
prevCx = cx
moved = true
}
@@ -106,17 +106,6 @@ object Keymap {
KeyEvent.KEYCODE_DPAD_UP -> 0x26
KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27
KeyEvent.KEYCODE_DPAD_DOWN -> 0x28
// TV-remote SELECT = Enter (a gamepad's press routes via SOURCE_GAMEPAD before this).
KeyEvent.KEYCODE_DPAD_CENTER -> 0x0D
// Consumer/media keys — forwarded to the host while streaming (volume stays local:
// MainActivity's pass-through list wins before the map is consulted).
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE,
KeyEvent.KEYCODE_MEDIA_PLAY,
KeyEvent.KEYCODE_MEDIA_PAUSE -> 0xB3 // VK_MEDIA_PLAY_PAUSE
KeyEvent.KEYCODE_MEDIA_NEXT -> 0xB0 // VK_MEDIA_NEXT_TRACK
KeyEvent.KEYCODE_MEDIA_PREVIOUS -> 0xB1 // VK_MEDIA_PREV_TRACK
KeyEvent.KEYCODE_MEDIA_STOP -> 0xB2 // VK_MEDIA_STOP
// Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant)
KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0
@@ -287,50 +287,6 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
* gesture typing, non-Latin scripts) over the TYPE_NULL raw-key fallback. False on `0`.
*/
external fun nativeTextInputSupported(handle: Long): Boolean
/**
* Committed IME text one `TextInput` wire event per Unicode scalar, in order. Control
* characters are skipped natively (Enter/Backspace ride [nativeSendKey]). Only meaningful
* when [nativeTextInputSupported] returned true older hosts ignore the events.
*/
external fun nativeSendText(handle: Long, text: String)
// ---- Shared clipboard (text v1): Kotlin drives ClipboardManager, Rust the protocol ----
// Opt-in per session (nativeClipControl). Local copies are announced as lazy offers; bytes
// cross only when the host pastes (a "fetch:" event answered by nativeClipServeText). Host
// copies arrive as "offer:" events, fetched eagerly into the system clipboard.
/** Whether the host advertised a working shared-clipboard service (HOST_CAP_CLIPBOARD). */
external fun nativeClipSupported(handle: Long): Boolean
/** Session-level clipboard opt-in/out; nothing happens until enabled=true crosses. */
external fun nativeClipControl(handle: Long, enabled: Boolean)
/** Announce "this device's clipboard now holds text". [seq]: monotonic, newest wins. */
external fun nativeClipOfferText(handle: Long, seq: Int)
/** Pull the text of the host's offer [seq] → transfer id echoed on "data:"/"error:", or -1. */
external fun nativeClipFetchText(handle: Long, seq: Int): Int
/** Answer a "fetch:" event with the clipboard's current text (the host is pasting). */
external fun nativeClipServeText(handle: Long, reqId: Int, text: String)
/** Abort a clipboard transfer by id (either direction). */
external fun nativeClipCancel(handle: Long, id: Int)
/**
* Block 250 ms for the next clipboard event, as a compact string: `state:<0|1>` ·
* `offer:<seq>:<hasText>` · `fetch:<reqId>` · `data:<xferId>:<text>` · `cancel:<id>` ·
* `error:<id>:<code>` · `closed` (session gone) null on timeout. Dedicated poll thread.
*/
external fun nativeNextClip(handle: Long): String?
// ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ----
// The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0,
// so its wire is byte-identical to the old single-pad path. The core folds the per-transition
@@ -404,14 +404,7 @@ fn feeder_loop(
// stage is consumed: the HUD, or the ABR decode signal (`measure_decode`). The
// HUD-only `received` point + host/network split stay gated on the overlay.
if stats.enabled() || measure_decode {
// Core reassembly-completion stamp (ABI v9), NOT the pull instant: stamping
// here would fold the hand-off queue wait into the network latency figure
// (a client-side standing backlog masquerading as network). 0 = older core.
let received_ns = if frame.received_ns > 0 {
frame.received_ns as i128
} else {
now_realtime_ns()
};
let received_ns = now_realtime_ns();
{
let mut g = in_flight
.lock()
@@ -221,13 +221,7 @@ pub(super) fn run_sync(
// samplers (`received` point, host/network split) stay gated on the overlay so
// the hidden steady state adds only a wall-clock read + the receipt push.
if stats.enabled() || measure_decode {
// Core reassembly-completion stamp (ABI v9), not the pull instant — see
// async_loop: a pull stamp folds hand-off queue wait into "network".
let received_ns = if frame.received_ns > 0 {
frame.received_ns as i128
} else {
now_realtime_ns()
};
let received_ns = now_realtime_ns();
in_flight.push_back((frame.pts_ns / 1000, received_ns));
if in_flight.len() > IN_FLIGHT_CAP {
in_flight.pop_front(); // stale — codec never echoed it back
@@ -1,182 +0,0 @@
//! Shared-clipboard plane (text-only v1): Kotlin drives the Android `ClipboardManager`, these
//! shims drive [`punktfunk_core::client::NativeClient`]'s clipboard surface.
//!
//! Model (mirrors the desktop clients): opt-in via `nativeClipControl(true)`; local copies are
//! announced lazily as format-list offers (`nativeClipOfferText`) and the bytes cross only when
//! the host pastes (a `fetch` event answered by `nativeClipServeText`); a host copy arrives as an
//! `offer` event, which the Kotlin side fetches eagerly (Android's clipboard has no lazy provider
//! path worth the complexity) and lands in the system clipboard on the `data` event.
//!
//! Events cross to Kotlin as compact strings from the blocking `nativeNextClip` poll (drained on
//! a dedicated thread, same pattern as `nativeNextRumble`):
//! `state:<0|1>` · `offer:<seq>:<has_text 0|1>` · `fetch:<req_id>` · `data:<xfer_id>:<text>` ·
//! `cancel:<id>` · `error:<id>:<code>` · `closed` — null on a poll timeout. Non-text fetch
//! requests are cancelled natively (only text is ever offered, so they shouldn't occur).
use std::time::Duration;
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint, jlong, jstring};
use jni::JNIEnv;
use punktfunk_core::clipboard::ClipEventCore;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::quic::{ClipKind, CLIP_FILE_INDEX_NONE, HOST_CAP_CLIPBOARD};
use super::SessionHandle;
/// The portable wire MIME both ends map to their platform text type.
const TEXT_MIME: &str = "text/plain;charset=utf-8";
/// Deref the opaque handle (`0` → `None`).
///
/// SAFETY: live handle per the nativeConnect/nativeClose contract; every method used is `&self`
/// on the `Sync` connector.
fn client(handle: jlong) -> Option<&'static SessionHandle> {
if handle == 0 {
return None;
}
// SAFETY: see the function docs — the Kotlin side guarantees the handle outlives the call.
Some(unsafe { &*(handle as *const SessionHandle) })
}
/// `NativeBridge.nativeClipSupported(handle)` — the host advertised `HOST_CAP_CLIPBOARD`.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
client(handle).map_or(0, |h| {
u8::from(h.client.host_caps() & HOST_CAP_CLIPBOARD != 0)
})
}
/// `NativeBridge.nativeClipControl(handle, enabled)` — session-level opt-in/out. Nothing
/// clipboard-related happens on either side until an `enabled: true` crosses.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipControl(
_env: JNIEnv,
_this: JObject,
handle: jlong,
enabled: jboolean,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_control(enabled != 0, 0);
}
}
/// `NativeBridge.nativeClipOfferText(handle, seq)` — announce "the Android clipboard now holds
/// text" (format list only; bytes cross when the host fetches). `seq` is Kotlin's monotonic
/// counter, newest wins.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipOfferText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_offer(
seq as u32,
vec![ClipKind {
mime: TEXT_MIME.into(),
size_hint: 0,
}],
);
}
}
/// `NativeBridge.nativeClipFetchText(handle, seq)` — pull the text of the host's offer `seq`.
/// Returns the transfer id echoed on the matching `data:`/`error:` event, or 1.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipFetchText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) -> jint {
client(handle)
.and_then(|h| {
h.client
.clip_fetch(seq as u32, TEXT_MIME.into(), CLIP_FILE_INDEX_NONE)
.ok()
})
.map_or(-1, |xfer| xfer as jint)
}
/// `NativeBridge.nativeClipServeText(handle, reqId, text)` — answer a `fetch:` event with the
/// clipboard's current text (the host is pasting our offer).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipServeText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
req_id: jint,
text: JString,
) {
let Some(h) = client(handle) else { return };
let Ok(s) = env.get_string(&text) else {
let _ = h.client.clip_cancel(req_id as u32);
return;
};
let _ = h
.client
.clip_serve(req_id as u32, String::from(s).into_bytes(), true);
}
/// `NativeBridge.nativeClipCancel(handle, id)` — abort a transfer (either direction).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipCancel(
_env: JNIEnv,
_this: JObject,
handle: jlong,
id: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_cancel(id as u32);
}
}
/// `NativeBridge.nativeNextClip(handle)` — block ≤250 ms for the next clipboard event, encoded
/// as a compact string (module docs); null on timeout, `"closed"` once the session is gone.
/// Call from a dedicated poll thread.
///
/// Text payloads ride `data:<xfer_id>:<text>` decoded lossily — safe because the phase-0
/// clipboard task delivers a whole payload in ONE event (`last = true`), so a chunk boundary
/// can never split a UTF-8 sequence.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextClip(
env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jstring {
let Some(h) = client(handle) else {
return std::ptr::null_mut();
};
let msg = match h.client.next_clip(Duration::from_millis(250)) {
Ok(ClipEventCore::State { enabled, .. }) => format!("state:{}", u8::from(enabled)),
Ok(ClipEventCore::RemoteOffer { seq, kinds }) => {
let has_text = kinds.iter().any(|k| k.mime.starts_with("text/plain"));
format!("offer:{seq}:{}", u8::from(has_text))
}
Ok(ClipEventCore::FetchRequest { req_id, mime, .. }) => {
if mime.starts_with("text/plain") {
format!("fetch:{req_id}")
} else {
// We only ever offer text; cancel anything else rather than stall the host.
let _ = h.client.clip_cancel(req_id);
return std::ptr::null_mut();
}
}
Ok(ClipEventCore::Data { xfer_id, bytes, .. }) => {
format!("data:{xfer_id}:{}", String::from_utf8_lossy(&bytes))
}
Ok(ClipEventCore::Cancelled { id }) => format!("cancel:{id}"),
Ok(ClipEventCore::Error { id, code }) => format!("error:{id}:{code}"),
Err(PunktfunkError::NoFrame) => return std::ptr::null_mut(),
Err(_) => "closed".into(),
};
env.new_string(msg)
.map(|s| s.into_raw())
.unwrap_or(std::ptr::null_mut())
}
@@ -201,9 +201,6 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// No display-volume forwarding from Android yet (the panel tone-maps PQ itself via the
// Surface dataspace + static metadata) — the host keeps its virtual-display EDID defaults.
None,
// No non-video caps: this client does not render the host cursor locally (no shape/state
// planes in the jni surface), so advertising CLIENT_CAP_CURSOR would stream cursor-less.
0,
launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous)
+2 -41
View File
@@ -6,11 +6,11 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject, JString};
use jni::objects::{JByteBuffer, JObject};
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX, HOST_CAP_TEXT_INPUT};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX};
use super::SessionHandle;
@@ -145,45 +145,6 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
send_event(handle, kind, vk as u32, 0, 0, mods as u32);
}
/// `NativeBridge.nativeTextInputSupported(handle)` — whether the host advertised
/// `HOST_CAP_TEXT_INPUT` (its inject backend types committed text), so the Kotlin side can pick
/// the real IME `InputConnection` over the TYPE_NULL raw-key fallback. `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
/// [`Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported`] returned true.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
text: JString,
) {
if handle == 0 {
return;
}
let Ok(s) = env.get_string(&text) else {
return;
};
for ch in String::from(s).chars().filter(|c| !c.is_control()) {
send_event(handle, InputKind::TextInput, ch as u32, 0, 0, 0);
}
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried
// in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a
@@ -17,7 +17,6 @@
//! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode
//! renegotiation. Port the remaining orchestration from `clients/linux`.
mod clipboard;
mod connect;
mod input;
mod planes;
@@ -579,21 +579,13 @@ struct ContentView: View {
model?.disconnect() // the captured-state D combo
},
onFrame: { [meter = model.meter, latency = model.latency,
split = model.latencySplit, queue = model.clientQueue,
offset = conn.clockOffsetNs] au in
split = model.latencySplit, offset = conn.clockOffsetNs] au in
meter.note(byteCount: au.data.count)
latency.record(ptsNs: au.ptsNs, offsetNs: offset)
// The same receipt, keyed by pts, awaiting its 0xCF host timing (the
// host/network split drained by the 1 s stats tick). receivedNs is
// the core's reassembly stamp (ABI v9), so the split's network term no
// longer contains the client-queue wait...
// host/network split drained by the 1 s stats tick).
split.recordReceipt(
ptsNs: au.ptsNs, receivedNs: au.receivedNs, offsetNs: offset)
// ...which is measured as its own term instead (receiptpull, both
// client-local).
queue.record(
ptsNs: UInt64(bitPattern: au.receivedNs), atNs: au.pulledNs,
offsetNs: 0)
},
onSessionEnd: { [weak model] in
Task { @MainActor in model?.sessionEnded() }
@@ -102,12 +102,6 @@ final class SessionModel: ObservableObject {
@Published var decodeValid = false
@Published var displayP50Ms = 0.0
@Published var displayValid = false
/// Client-queue wait: core reassembly receipt the pump's pull (`AccessUnit.pulledNs
/// receivedNs`, ABI v9 receipt split the 2026-07 two-pair investigation). ~0 on a healthy
/// stream; a persistent value is a client-side standing backlog that used to hide inside
/// "network". Shown in the detailed tier only when it says something ( ~2 ms).
@Published var clientQueueP50Ms = 0.0
@Published var clientQueueValid = false
/// The measured OS present floor (design/apple-presentation-rebuild.md): the deadline
/// engine's vendglass pipeline depth an OS property no client can pace under (~2 refresh
/// intervals composited; would read ~1 under direct-to-display). The HUD subtracts it from
@@ -153,9 +147,6 @@ final class SessionModel: ObservableObject {
let endToEnd = LatencyMeter()
let decodeStage = LatencyMeter()
let displayStage = LatencyMeter()
/// Client-queue sampler (see `clientQueueP50Ms`) fed per AU by the stream view's onFrame,
/// drained by the same 1 s tick as the stage meters.
let clientQueue = LatencyMeter()
/// The OS present floor sampler (see `osFloorP50Ms`) fed one sample per display-link
/// update by the deadline engine, drained by the same 1 s tick as the stage meters.
let presentFloor = LatencyMeter()
@@ -302,26 +293,13 @@ final class SessionModel: ObservableObject {
if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported {
videoCodecs |= PunktfunkConnection.codecPyroWave
}
// Cursor channel (remote-desktop-sweep M2, macOS): sessions STARTING in the desktop
// mouse model advertise local cursor rendering the host then stops compositing
// the pointer and forwards shape/state, which StreamView draws as the real
// NSCursor. Capture-mode sessions keep today's composited pointer.
#if os(macOS)
let clientCaps: UInt8 =
(MouseInputMode(
rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? "")
?? .capture) == .desktop ? 0x01 : 0
#else
let clientCaps: UInt8 = 0
#endif
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
pinSHA256: pin, identity: identity, compositor: compositor,
gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps,
audioChannels: audioChannels,
videoCodecs: videoCodecs, preferredCodec: preferredCodec,
clientCaps: clientCaps, launchID: launchID,
videoCodecs: videoCodecs, preferredCodec: preferredCodec, launchID: launchID,
// Delegated approval: the host holds this connect open until the operator approves
// it (~180 s) outwait that window so a slow approval still lands here. Normal
// connects keep the snappy default.
@@ -511,7 +489,6 @@ final class SessionModel: ObservableObject {
endToEndValid = false
decodeValid = false
displayValid = false
clientQueueValid = false
osFloorValid = false
lostFrames = 0
lostPct = 0
@@ -702,12 +679,6 @@ final class SessionModel: ObservableObject {
} else {
self.osFloorValid = false
}
if let q = self.clientQueue.drain() {
self.clientQueueP50Ms = q.p50Ms
self.clientQueueValid = true
} else {
self.clientQueueValid = false
}
// Mirror the window to the unified log (see statsLog) one line per second,
// stages in ms, only while frames actually flowed. `fps` counts RECEIVED AUs;
// `presents` counts frames that reached glass (the display meter's sample count)
@@ -718,12 +689,9 @@ final class SessionModel: ObservableObject {
// captured before the 2026-07 floor policy); the appended trio carries the
// measured OS present floor and the floor-shaved values the HUD displays.
let line = String(
// Swift Int is 64-bit %lld, NOT %d (which is a 32-bit C int); macOS 26's
// strict String(format:) validator rejects the %d/Int mismatch and drops
// the whole line (a cascade error that also mis-blames the float args).
format: "fps=%lld presents=%lld e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%lld "
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f",
format: "fps=%d presents=%d e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%d "
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f",
frames,
displayWindow?.count ?? 0,
self.endToEndValid ? self.endToEndP50Ms : -1,
@@ -734,8 +702,7 @@ final class SessionModel: ObservableObject {
lost,
self.osFloorValid ? self.osFloorP50Ms : -1,
self.displayValid ? self.displayAdjP50Ms : -1,
self.endToEndValid ? self.endToEndAdjP50Ms : -1,
self.clientQueueValid ? self.clientQueueP50Ms : -1)
self.endToEndValid ? self.endToEndAdjP50Ms : -1)
statsLog.info("\(line, privacy: .public)")
}
}
@@ -118,16 +118,6 @@ struct StreamHUDView: View {
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.tertiary)
}
// Client-queue wait (reassembly receipt decode pull, ABI v9 split): ~0 on
// a healthy stream and hidden as noise; shown from 2 ms a persistent value
// is a client-side standing backlog that pre-split builds displayed as
// "network" (the 2026-07 two-pair plateau). The core's standing-latency
// bleed logs alongside when it acts on the same state.
if model.clientQueueValid && model.clientQueueP50Ms >= 2 {
Text("client queue +\(model.clientQueueP50Ms, specifier: "%.1f") (receive backlog — standing if it persists)")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.tertiary)
}
}
} else if model.hostNetworkValid {
// Stage-1 fallback presenter: the layer decodes + presents internally with no
@@ -43,9 +43,6 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.presentPriority) private var presentPriority =
SettingsOptions.presentPriorityDefault
@AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0
#if os(macOS)
@AppStorage(DefaultsKey.windowedSafePresent) private var windowedSafePresent = true
#endif
#if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
#endif
@@ -348,22 +345,6 @@ struct GamepadSettingsView: View {
detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled),
]
#if os(macOS)
// The windowed safe-present toggle slots in after "Smoothness buffer" (staying inside
// the Video group) macOS only, mirroring the touch SettingsView's Presentation row
// (the DCP swapID-panic mitigation; see DefaultsKey.windowedSafePresent).
if let at = list.firstIndex(where: { $0.id == "smoothBuffer" }) {
list.insert(
toggleRow(
id: "windowedSafePresent", icon: "macwindow.badge.plus",
label: "Safe windowed presentation",
detail: "Windowed streams present in step with the compositor — avoids a "
+ "macOS display-driver crash on high-refresh displays, at a small "
+ "latency cost. Fullscreen always uses the fastest path.",
value: $windowedSafePresent),
at: at + 1)
}
#endif
#if os(iOS)
// The device-rumble mirror slots in after "Controller type" (staying inside the
// Controller group the next row carries the "Interface" header). iPhone only in
@@ -300,18 +300,6 @@ extension SettingsView {
+ "of added latency. Off shows frames as soon as they're ready.") {
Toggle("V-Sync", isOn: $vsync)
}
// The DCP swapID-panic mitigation's user handle (see DefaultsKey.windowedSafePresent
// for the saga). Default ON: turning it off re-arms a WHOLE-MACHINE kernel panic on
// affected setups, so the caption says so in plain words.
described(windowedSafePresent
? "Windowed streams present in step with the system compositor — avoids a macOS "
+ "display-driver crash seen on high-refresh displays, at a small latency "
+ "cost. Fullscreen always uses the fastest path."
: "Windowed streams use the fastest present path. On some high-refresh setups "
+ "this can crash macOS itself (kernel panic) — turn back on if your Mac "
+ "restarts during windowed streaming.") {
Toggle("Safe windowed presentation", isOn: $windowedSafePresent)
}
#endif
}
}
@@ -447,14 +435,6 @@ extension SettingsView {
/// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path).
@ViewBuilder var inputSection: some View {
Section("Keyboard & mouse") {
#if os(macOS)
described(mouseModeDescription) {
Picker("Mouse input", selection: $mouseMode) {
Text("Capture (games)").tag(MouseInputMode.capture.rawValue)
Text("Desktop (absolute)").tag(MouseInputMode.desktop.rawValue)
}
}
#endif
described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) {
Picker("Modifier keys", selection: $modifierLayout) {
ForEach(ModifierLayout.allCases, id: \.self) { layout in
@@ -467,20 +447,6 @@ extension SettingsView {
}
}
}
#if os(macOS)
/// The SELECTED mouse model explained dynamic, like the touch-mode caption.
private var mouseModeDescription: String {
switch MouseInputMode(rawValue: mouseMode) ?? .capture {
case .capture:
return "The pointer locks to the stream and sends relative motion — best for "
+ "games. ⌃⌥⇧M switches live; applies from the next capture otherwise."
case .desktop:
return "The pointer moves freely in and out of the stream and sends absolute "
+ "positions — best for remote desktop work. Unavailable on gamescope hosts."
}
}
#endif
#endif
// MARK: - Audio
@@ -40,7 +40,6 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0
#if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false
@AppStorage(DefaultsKey.windowedSafePresent) var windowedSafePresent = true
#endif
#if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true
@@ -89,7 +88,6 @@ struct SettingsView: View {
@State var customMode = false
#endif
#if os(macOS)
@AppStorage(DefaultsKey.mouseMode) var mouseMode = MouseInputMode.capture.rawValue
@AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0
@@ -35,31 +35,10 @@ public struct AccessUnit: Sendable {
public let ptsNs: UInt64
public let frameIndex: UInt32
public let flags: UInt32
/// Client `CLOCK_REALTIME` instant the AU finished reassembly in the core (post-FEC,
/// decrypted `PunktfunkFrame.received_ns`, ABI v9) the **received** measurement point of
/// design/stats-unification.md. NOT the pull instant: stamping at the pull folded the
/// pre-decode hand-off wait into the network term, which is how the 2026-07 two-pair
/// standing-latency plateau hid as "network". The decode stage is `decodedNs - receivedNs`,
/// both client-local (no skew offset applies).
/// Client `CLOCK_REALTIME` instant the AU was handed over by the core (post-FEC, decrypted)
/// the **received** measurement point of design/stats-unification.md. The decode stage is
/// `decodedNs - receivedNs`, both client-local (no skew offset applies).
public let receivedNs: Int64
/// Client `CLOCK_REALTIME` instant this pull returned. `pulledNs - receivedNs` is the
/// client-queue wait (kernel hand-off + FrameChannel dwell) the term the HUD splits out
/// so a client-side standing backlog can never masquerade as network latency again.
public let pulledNs: Int64
/// `pulledNs` defaults to `receivedNs` (zero queue wait) for callers with no pull instant
/// the synthetic probe AUs and decode tests, where the split is meaningless.
public init(
data: Data, ptsNs: UInt64, frameIndex: UInt32, flags: UInt32,
receivedNs: Int64, pulledNs: Int64? = nil
) {
self.data = data
self.ptsNs = ptsNs
self.frameIndex = frameIndex
self.flags = flags
self.receivedNs = receivedNs
self.pulledNs = pulledNs ?? receivedNs
}
}
/// One Opus audio packet (48 kHz stereo, 5 ms frames) decode with AVAudioConverter
@@ -221,9 +200,6 @@ public final class PunktfunkConnection {
/// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too:
/// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple.
private let clipboardLock = NSLock()
/// Serializes the (single) cursor pull thread against close() both cursor planes are
/// drained by ONE thread, so one lock covers them.
private let cursorLock = NSLock()
/// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0
@@ -384,98 +360,6 @@ public final class PunktfunkConnection {
public var hostSupportsClipboard: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0
}
/// The host answered `HOST_CAP_CURSOR`: it stopped compositing the pointer and forwards
/// shape/state on the cursor planes the client MUST draw the cursor locally.
public var hostSupportsCursor: Bool {
hostCaps & 0x04 != 0
}
/// One forwarded host-cursor shape (the cursor channel, ABI v11): straight-alpha RGBA,
/// `rgba.count == width * height * 4`, hotspot within the bitmap. Cache by `serial`
/// states reference shapes by it and a re-shown serial never resends pixels.
public struct CursorShapeEvent: Sendable {
public let serial: UInt32
public let width: Int
public let height: Int
public let hotX: Int
public let hotY: Int
public let rgba: Data
}
/// Per-host-tick cursor state: position (host video px, the pointer/hotspot point),
/// visibility, and the host-driven relative-mode hint (an app grabbed/hid the pointer
/// run captured relative; clear absolute, reappearing at `x`/`y`). Latest-wins.
public struct CursorStateEvent: Sendable {
public let serial: UInt32
public let visible: Bool
public let relativeHint: Bool
public let x: Int32
public let y: Int32
}
/// Pull the next forwarded cursor SHAPE (nil = timeout). Only a session connected with
/// `clientCaps` cursor bit against a `hostSupportsCursor` host receives any. Drain shape
/// AND state from ONE dedicated cursor thread (they share a lock).
public func nextCursorShape(timeoutMs: UInt32 = 0) throws -> CursorShapeEvent? {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkCursorShape()
let rc = punktfunk_connection_next_cursor_shape(h, &out, timeoutMs)
switch rc {
case statusOK:
// Copy out of the ABI borrow (valid until the next shape call) immediately.
let bytes = out.rgba.map { Data(bytes: $0, count: Int(out.len)) } ?? Data()
return CursorShapeEvent(
serial: out.serial, width: Int(out.w), height: Int(out.h),
hotX: Int(out.hot_x), hotY: Int(out.hot_y), rgba: bytes)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Pull the next cursor STATE (nil = timeout). Latest-wins drain the queue and apply
/// only the newest. Same thread + gate as [`nextCursorShape`].
public func nextCursorState(timeoutMs: UInt32 = 0) throws -> CursorStateEvent? {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkCursorState()
let rc = punktfunk_connection_next_cursor_state(h, &out, timeoutMs)
switch rc {
case statusOK:
return CursorStateEvent(
serial: out.serial,
visible: out.flags & 0x01 != 0,
relativeHint: out.flags & 0x02 != 0,
x: out.x, y: out.y)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Tell the host who renders the pointer (the §8 mid-stream mouse-model flip, ABI v12):
/// `clientDraws = true` this client draws it locally (the desktop mouse model; the host
/// excludes the pointer from the video and forwards shape/state); `false` the host
/// composites it into the video (the capture model, full fidelity). Idempotent,
/// latest-wins; harmless against hosts without the cursor cap. Fire-and-forget errors
/// are swallowed (a closed session is the only failure and it moots the flip).
public func setCursorRender(clientDraws: Bool) {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_set_cursor_render(h, clientDraws)
}
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
@@ -512,7 +396,6 @@ public final class PunktfunkConnection {
audioChannels: UInt8 = 2,
videoCodecs: UInt8 = 0x02, // PUNKTFUNK_CODEC_HEVC the codecs this client can decode
preferredCodec: UInt8 = 0, // 0 = auto; else PUNKTFUNK_CODEC_* soft preference
clientCaps: UInt8 = 0, // ABI v11: PUNKTFUNK_CLIENT_CAP_CURSOR = render the host cursor locally
launchID: String? = nil,
timeoutMs: UInt32 = 10_000
) throws {
@@ -532,18 +415,18 @@ public final class PunktfunkConnection {
withOptionalCString(launchID) { launch in
if let pin = pinSHA256 {
return pin.withUnsafeBytes { p in
punktfunk_connect_ex9(
punktfunk_connect_ex8(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, clientCaps, launch,
videoCodecs, preferredCodec, launch,
p.bindMemory(to: UInt8.self).baseAddress, &observed,
cert, key, timeoutMs, &connectStatus)
}
}
return punktfunk_connect_ex9(
return punktfunk_connect_ex8(
cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, clientCaps, launch,
videoCodecs, preferredCodec, launch,
nil, &observed, cert, key, timeoutMs, &connectStatus)
}
}
@@ -779,16 +662,11 @@ public final class PunktfunkConnection {
let data = Data(bytes: base, count: Int(frame.len)) // copy: ptr valid only until next call
var ts = timespec()
clock_gettime(CLOCK_REALTIME, &ts)
let pulledNs = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec)
// Receipt = the core's reassembly-completion stamp (ABI v9); the pull instant is
// kept separately so the client-queue wait is its own measured term. 0 would mean a
// pre-v9 core impossible here (core and Kit ship in one binary), but fall back to
// the pull instant rather than record a 1970 receipt.
let receivedNs = frame.received_ns > 0 ? Int64(frame.received_ns) : pulledNs
let receivedNs = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec)
return AccessUnit(
data: data, ptsNs: frame.pts_ns,
frameIndex: frame.frame_index, flags: frame.flags,
receivedNs: receivedNs, pulledNs: pulledNs)
receivedNs: receivedNs)
case statusNoFrame:
return nil
case statusClosed:
@@ -1155,12 +1033,10 @@ public final class PunktfunkConnection {
feedbackLock.lock()
statsLock.lock()
clipboardLock.lock()
cursorLock.lock()
abiLock.lock()
let h = handle
handle = nil
abiLock.unlock()
cursorLock.unlock()
clipboardLock.unlock()
statsLock.unlock()
feedbackLock.unlock()
@@ -110,11 +110,11 @@ public final class InputCapture {
/// event itself is swallowed). Main queue.
public var onToggleCapture: (() -> Void)?
/// Fired on M (the mouse-model flip, capture desktop cross-client parity with the
/// SDL clients' Ctrl+Alt+Shift+M; detected here, like , so it works regardless of the
/// current capture state and the event itself is swallowed). macOS only; the
/// absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleMouseMode: (() -> Void)?
/// Fired on C (the client-side-cursor toggle flips between the captured/disassociated
/// relative path and the visible-cursor absolute path; detected here, like , so it works
/// regardless of the current capture state and the event itself is swallowed). macOS only;
/// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleCursor: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
@@ -245,14 +245,13 @@ public final class InputCapture {
self.onToggleCapture?()
return nil
}
// M flips the mouse model (capture desktop the SDL clients' identical
// chord). Detected in both capture states, like , so the model can be set
// before engaging. keyCode 46 = kVK_ANSI_M; layout-independent. Suppress the M
// (latched like 's Esc) so it doesn't type into the host, and swallow the
// event so it doesn't beep.
if event.keyCode == 46 /* M */, flags == [.control, .option, .shift] {
self.suppressedVK = 0x4D // VK_M the same physical M is en route via GC
self.onToggleMouseMode?()
// C toggles the client-side cursor (visible-cursor absolute path vs the
// captured relative path). keyCode 8 = kVK_ANSI_C; layout-independent so it
// fires the same on any keyboard. Suppress the C (latched like 's Esc) so it
// doesn't type into the host, and swallow the event so it doesn't beep.
if event.keyCode == 8 /* C */, flags == [.command, .shift] {
self.suppressedVK = 0x43 // VK_C the same physical C is en route via GC
self.onToggleCursor?()
return nil
}
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
@@ -1,12 +0,0 @@
/// How a physical mouse drives the host the cross-client mouse model (the SDL clients'
/// `MouseMode` / `Settings::mouse_mode`, design/remote-desktop-sweep.md M1). Stored stringly
/// under `DefaultsKey.mouseMode`.
public enum MouseInputMode: String, CaseIterable, Sendable {
/// Pointer capture (disassociated, hidden cursor, relative deltas) the game model,
/// and the default: the only cursor you see is the host's.
case capture
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion is forwarded as absolute positions through the letterbox. The remote desktop
/// model. Requires a host injector with absolute support (not gamescope).
case desktop
}
@@ -23,34 +23,6 @@ import os
private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
#if os(macOS)
/// HOW a windowed (composited) macOS session pushes finished frames to glass the DCP
/// "mismatched swapID's" kernel-panic saga's mechanism picker. Fullscreen always presents
/// `async` (direct-scanout promotion, lowest latency, no panic reports there); the windowed
/// mechanism is resolved per session by SessionPresenter (user setting +
/// PUNKTFUNK_WINDOWED_PRESENT env override) and routed here via `setWindowedPresent`.
///
/// - `async`: the CAMetalLayer image queue (`commandBuffer.present`) the fastest composited
/// path and the PANIC TRIGGER on high-refresh displays (the out-of-band swaps race
/// WindowServer's compositor; it survived glass pacing and every codec).
/// - `transaction`: `CAMetalLayer.presentsWithTransaction` the swap commits WITH the layer
/// tree, in lockstep with the compositor (Apple's documented remedy; validated no-panic on
/// the 240 Hz repro machine). The present is committed from the RENDER thread inside an
/// explicit CATransaction + flush see `encodePresent` for why that beats the original
/// main-thread hop.
/// - `surface`: no image queue at all render into a pooled IOSurface and swap it into a plain
/// CALayer's `contents` (the f407f418 PyroWave mitigation, resurrected format-aware:
/// rgba16Float + PQ tagging keeps HDR). WindowServer treats it as ordinary layer damage on
/// its own composite cadence. PROTOTYPE: whether the compositor honors PQ/EDR for plain-layer
/// IOSurface contents still needs an on-glass eyeball the metal layer stays underneath with
/// `wantsExtendedDynamicRangeContent` as the EDR anchor.
enum WindowedPresentMode: String, Sendable {
case async
case transaction
case surface
}
#endif
/// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
@@ -226,8 +198,8 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
// The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
// EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff
// BT.709 OETF. Used by the tvOS biplanar tone-map and the tvOS planar (PyroWave) tone-map (the
// no-HDR-headroom fallback). macOS keeps real HDR windowed now see `WindowedPresentMode`.
// BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map the
// latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only.
static inline float3 pqToSdr(float3 pq) {
const float m1 = 2610.0/16384.0;
const float m2 = 78.84375;
@@ -258,7 +230,8 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
// fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave
// stream must land on an 8-bit surface: tvOS without HDR headroom. The passthrough planar
// stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions
// (the IOSurface present path the DCP-panic mitigation is BGRA8). The passthrough planar
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
@@ -286,51 +259,21 @@ public final class MetalVideoPresenter {
public let layer: CAMetalLayer
#if os(macOS)
/// WINDOWED-mode present coordination the macOS DCP KERNEL PANIC mitigation.
/// The WINDOWED-mode PyroWave present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it), fed IOSurfaces via `contents` inside ordinary CATransactions.
///
/// The panic ("mismatched swapID's" @UnifiedPipeline.cpp, WindowServer dies, machine reboots):
/// the CAMetalLayer's ASYNCHRONOUS image queue (`commandBuffer.present(drawable)` an
/// out-of-band flip, mandatory with `displaySyncEnabled=false`) diverges from WindowServer's
/// compositor on a high-refresh COMPOSITED (windowed) session the compositor's notion of the
/// current swap and the layer's queued swap disagree, and the DCP asserts. It survived glass
/// pacing: a fully serialized one-in-flight present stream still panicked a 240 Hz Mac Studio
/// (2026-07-18, PyroWave), and a windowed HEVC session panicked the same machine 2026-07-21
/// so it is the async image queue itself, at any pacing or codec, not a present rate.
///
/// The fix keeps the full render path (rgba16Float / PQ / EDR real HDR is preserved) and
/// only changes HOW the drawable is presented: `CAMetalLayer.presentsWithTransaction`. With it
/// set, we don't hand the drawable to the command buffer; we commit, wait until scheduled, then
/// call `drawable.present()` INSIDE a CATransaction the present is enrolled in Core
/// Animation's transaction and committed together with the layer tree, so the swap stays in
/// lockstep with the compositor instead of racing it (Apple's documented remedy for Metal
/// presentation drifting out of sync with CA). Fullscreen keeps the async path (direct-scanout
/// promotion, lowest latency, no compositor and no panic reports there).
///
/// 2026-07-21 latency rework: the mitigation MECHANISM is now a three-way pick
/// (`WindowedPresentMode`) and the transactional present commits from the RENDER thread
/// see `encodePresent`. Staged under `stagingLock` (main pushes it via
/// `setComposited``setWindowedPresent`); the render thread drains it and toggles the layer
/// property + present style. `Active` is the render-thread copy so the layer property flips
/// exactly once per mode change.
private var windowedPresentStaged: WindowedPresentMode = .async
private var windowedPresentActive: WindowedPresentMode = .async
/// PUNKTFUNK_TXN_PRESENT=main the ORIGINAL transactional present (commit
/// waitUntilScheduled hop to the MAIN thread and present inside its CATransaction), kept
/// as a field A/B lever. The default is the render-thread commit: the present harness
/// (2026-07-21, this saga) measured the main hop landing a runloop turn late on a busy main
/// thread, and an ACTIVE implicit transaction there NESTS the explicit one presents batch
/// at runloop-iteration rate (the field's presents=55 @ fps=240, display_p50 18.6 ms).
/// Off-main commits measured immune to main-thread churn (~10 ms glass p50 at 240 Hz
/// full-size vs 14+ ms under a churned main hop).
private let txnPresentOnMain =
ProcessInfo.processInfo.environment["PUNKTFUNK_TXN_PRESENT"] == "main"
/// The WINDOWED-mode `surface` present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it by SessionPresenter), fed IOSurfaces via `contents` inside explicit
/// CATransactions. Transparent (nil contents) whenever surface mode is off, so the metal
/// layer below shows through. See `WindowedPresentMode.surface`.
let surfaceLayer: CALayer = {
/// Why this exists the macOS DCP KERNEL PANIC ("mismatched swapID's" @UnifiedPipeline.cpp,
/// WindowServer dies, machine reboots): out-of-band CAMetalLayer image-queue swaps into a
/// COMPOSITED (windowed) session race WindowServer's own swap submissions on high-refresh
/// displays, and the race survives glass pacing a fully serialized one-in-flight present
/// stream still panicked a 240 Hz Mac Studio (2026-07-18, twice). So in windowed mode we stop
/// using the image queue entirely and present the way video players do: render the planar CSC
/// into an IOSurface pool and swap `contents` on main WindowServer treats it as ordinary
/// damage on its own composite cadence, coalescing faster-than-refresh updates instead of
/// latching queue swaps mid-cycle. Fullscreen keeps the CAMetalLayer path (direct-scanout
/// promotion, no compositing, no panic reports). Contents updates are transparent to the
/// layer below when nil, so flipping modes just covers/uncovers the metal layer.
public let surfaceLayer: CALayer = {
let l = CALayer()
l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout
l.isOpaque = true
@@ -338,8 +281,8 @@ public final class MetalVideoPresenter {
return l
}()
/// One IOSurface-backed render target of the windowed surface-present pool. All pool state
/// is RENDER-THREAD confined; only the immutable surface refs cross threads (contents swap).
/// One IOSurface-backed render target of the windowed present pool. All pool state is
/// RENDER-THREAD confined; only the immutable surface refs cross to main (contents swap).
private struct SurfaceSlot {
let surface: IOSurfaceRef
let texture: MTLTexture
@@ -349,52 +292,15 @@ public final class MetalVideoPresenter {
private var surfacePool: [SurfaceSlot] = []
private var surfacePoolSize: CGSize = .zero
private var surfacePoolHDR = false
private var surfaceSeq: UInt64 = 0
/// Index of the slot most recently handed to the layer never rewritten next, even if its
/// use count already dropped (the compositor may still be scanning out the previous frame).
private var lastHandedOff: Int?
/// Once-per-second decomposition of the ACTIVE windowed present path (the field-diagnosis
/// half of the DCP-latency work): scheduled/completed wait + commit/flush cost per present,
/// and how many presents/swaps were issued. The pf-present line shows the GLASS side
/// (latchMs / dropped); this shows the ISSUE side. Logged via `presenterLog` only while a
/// windowed mechanism is active (zero cost fullscreen). Lock-guarded: transaction mode
/// records from the render thread, surface mode from Metal completion threads.
private final class WindowedPresentDiag: @unchecked Sendable {
private let lock = NSLock()
private var presents = 0
private var schedMs: [Double] = []
private var commitMs: [Double] = []
private var last = CACurrentMediaTime()
func record(schedMs sched: Double, commitMs commit: Double, mode: WindowedPresentMode) {
lock.lock()
presents += 1
schedMs.append(sched)
commitMs.append(commit)
let now = CACurrentMediaTime()
guard now - last >= 1 else {
lock.unlock()
return
}
last = now
let sSched = schedMs.sorted()
let sCommit = commitMs.sorted()
let line = String(
format: "pf-windowed mode=%@ presents=%d schedMs p50=%.2f max=%.2f "
+ "commitMs p50=%.2f max=%.2f",
mode.rawValue, presents, sSched[sSched.count / 2], sSched.last ?? 0,
sCommit[sCommit.count / 2], sCommit.last ?? 0)
presents = 0
schedMs.removeAll(keepingCapacity: true)
commitMs.removeAll(keepingCapacity: true)
lock.unlock()
presenterLog.info("\(line, privacy: .public)")
}
}
private let windowedDiag = WindowedPresentDiag()
/// Staged (under `stagingLock`, like every cross-thread input): the hosting view's windowed
/// vs fullscreen state, pushed from main via `setSurfacePresents`. Drained in `renderPlanar`.
private var surfacePresentsStaged = false
/// Render-thread copy, so pool teardown happens exactly once on a mode flip.
private var surfacePresentsActive = false
#endif
private let device: MTLDevice
@@ -410,7 +316,7 @@ public final class MetalVideoPresenter {
private let pipelinePlanar: MTLRenderPipelineState
/// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour
/// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm
/// bgra8; tvOS without HDR headroom).
/// bgra8; tvOS without headroom + macOS windowed IOSurface presents).
private let pipelinePlanarHDR: MTLRenderPipelineState
private let pipelinePlanarToneMap: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache?
@@ -685,14 +591,13 @@ public final class MetalVideoPresenter {
}
#if os(macOS)
/// Park the windowed present mechanism (MAIN thread the hosting view pushes its window
/// state on every layout; SessionPresenter resolves the mechanism per session). `.async` =
/// FULLSCREEN (or the user opted out of the mitigation): the image queue. `.transaction` /
/// `.surface` = COMPOSITED (windowed) mitigation mechanisms see `WindowedPresentMode`.
/// Park the windowed-vs-fullscreen present routing (MAIN thread the hosting view pushes its
/// window state on every layout). true = PyroWave frames present via `surfaceLayer` contents
/// (the DCP swapID-panic mitigation see `surfaceLayer`); false = the CAMetalLayer path.
/// Applied by the render thread on the next frame, like every other staged value here.
func setWindowedPresent(_ mode: WindowedPresentMode) {
public func setSurfacePresents(_ on: Bool) {
stagingLock.lock()
windowedPresentStaged = mode
surfacePresentsStaged = on
stagingLock.unlock()
}
#endif
@@ -829,12 +734,36 @@ public final class MetalVideoPresenter {
) -> Bool {
stagingLock.lock()
let targetFromLayout = drawableTarget
#if os(macOS)
let surfaceMode = surfacePresentsStaged
#endif
stagingLock.unlock()
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path
// including macOS windowed sessions, which keep real HDR (the DCP mitigation is the
// transactional present in `encodePresent`, not a colour downgrade).
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path;
// macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader).
#if os(macOS)
configure(hdr: planes.pq && !surfaceMode)
#else
configure(hdr: planes.pq)
#endif
var csc = planes.csc
#if os(macOS)
if surfaceMode != surfacePresentsActive {
surfacePresentsActive = surfaceMode
presenterLog.info(
"stage2: windowed surface presents \(surfaceMode ? "ON" : "OFF", privacy: .public) (PyroWave DCP-panic mitigation)")
if !surfaceMode {
// Back to the metal path (fullscreen): drop the pool at 5K it holds >100 MB,
// and re-entering windowed mode rebuilds it in one frame.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
}
if surfaceMode {
return renderPlanarToSurface(
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
}
#endif
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
// 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps
// in-shader instead (the pipeline must match the drawable's pixel format).
@@ -863,6 +792,118 @@ public final class MetalVideoPresenter {
}
}
#if os(macOS)
/// The windowed-mode present tail (see `surfaceLayer` for why this path exists): render the
/// planar CSC into a pooled IOSurface and hand it to `surfaceLayer.contents` on MAIN inside a
/// plain CATransaction an ordinary damaged-layer update on WindowServer's own composite
/// cadence, no CAMetalLayer image-queue swap anywhere. `presentAtMediaTime` doesn't apply
/// (the compositor paces); `onPresented` fires after the contents swap is committed, stamped
/// with CLOCK_REALTIME then the closest observable analogue of "reached glass" here (the
/// composite follows within a refresh, so the meters' display stage reads slightly optimistic).
private func renderPlanarToSurface(
_ planes: WaveletPlanes, targetFromLayout: CGSize, csc: inout CscUniform,
onPresented: ((Int64?) -> Void)?
) -> Bool {
let decodedSize = CGSize(width: planes.width, height: planes.height)
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
ensureSurfacePool(size: targetSize)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let keepAlive: [Any] = [planes.y, planes.cb, planes.cr]
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // ring textures pinned until the GPU finished sampling
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
onPresented?(
Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size` 4 BGRA8 IOSurface render targets (one on glass, one queued
/// in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the pool empty;
/// the caller returns false and the ring's putBack + display-link retry take over.
private func ensureSurfacePool(size: CGSize) {
guard size != surfacePoolSize else { return }
surfacePool.removeAll()
surfacePoolSize = size
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// 256-byte row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerRow = ((w * 4) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: 4,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one
/// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule
/// the present and the on-glass callback.
@@ -895,36 +936,6 @@ public final class MetalVideoPresenter {
#if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
#endif
#if os(macOS)
// Windowed (composited) the DCP swapID-panic mitigation mechanism (see
// `WindowedPresentMode`). Toggle the layer property BEFORE vending a drawable so the
// vend matches how it will be presented; drained here on the render thread, flipped
// exactly once per mode change.
stagingLock.lock()
let windowedMode = windowedPresentStaged
stagingLock.unlock()
if windowedMode != windowedPresentActive {
windowedPresentActive = windowedMode
layer.presentsWithTransaction = windowedMode == .transaction
if windowedMode != .surface, !surfacePool.isEmpty {
// Leaving surface mode (fullscreen entry / mechanism A/B): drop the pool at 5K
// it holds >100 MB, and re-entering rebuilds it in one frame. SessionPresenter
// clears the surface layer's contents on main.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
presenterLog.info(
"stage2: windowed present mode \(windowedMode.rawValue, privacy: .public) (DCP swapID-panic mitigation)")
}
if windowedMode == .surface {
// No image queue at all: render into a pooled IOSurface and swap it into the
// sibling layer's contents. The drawable/queue tail below never runs.
return encodeToSurface(
targetSize: targetSize, pipeline: pipeline, onPresented: onPresented,
keepAlive: keepAlive, bind: bind)
}
#endif
if let providedDrawable,
providedDrawable.texture.pixelFormat != layer.pixelFormat {
return false // config outran the vend (HDR flip) next vend has the new format
@@ -963,61 +974,6 @@ public final class MetalVideoPresenter {
}
#endif
}
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
#if os(macOS)
if windowedPresentActive == .transaction {
// Windowed DCP mitigation: present the drawable THROUGH a Core Animation transaction
// (`presentsWithTransaction`, set above) instead of the async image queue, so the swap
// commits with the layer tree and stays in lockstep with the compositor (no out-of-band
// flip to race WindowServer's swaps). Wait until the GPU work is scheduled (contents
// will be ready p50 ~0.1 ms), then present inside an EXPLICIT CATransaction ON THIS
// RENDER THREAD and `flush()`. `presentAtMediaTime` does not apply the transaction
// paces.
//
// Threading history, because BOTH failure modes shipped or nearly shipped:
// A bare `present()` from this thread (no transaction) never flushes nothing
// commits a runloop-less thread's implicit transaction, so drawables are never
// released; after maximumDrawableCount vends `nextDrawable()` blocks forever and
// the stream FREEZES (the fullscreenwindowed switch did exactly this).
// The explicit begin/commit alone is NOT enough either: this thread has an ACTIVE
// implicit transaction (the layer mutations above drawableSize/colour created
// it), so the explicit transaction NESTS inside it and its commit defers to the
// implicit one that never comes. The harness reproduced the exact freeze: every
// present reported presentedTime=0, nothing reached glass. `CATransaction.flush()`
// pushes the implicit transaction (present included) to the render server NOW.
// The original fix hopped to MAIN and presented there correct, but slow in the
// field (presents=55 @ fps=240, display_p50 18.6 ms on the 240 Hz Studio): each
// present lands a runloop turn late, and main's own implicit transaction batches
// enrolled presents at runloop-iteration rate. Kept as PUNKTFUNK_TXN_PRESENT=main.
// The off-main commit measured immune to main-thread churn in the harness
// (2026-07-21: glass p50 ~10 ms at 240 Hz full-size, cadence a clean 4.17 ms).
commandBuffer.commit()
let schedStart = CACurrentMediaTime()
commandBuffer.waitUntilScheduled()
let schedMs = (CACurrentMediaTime() - schedStart) * 1000
let commitStart = CACurrentMediaTime()
if txnPresentOnMain {
let presentedDrawable = drawable
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
presentedDrawable.present()
CATransaction.commit()
}
} else {
CATransaction.begin()
CATransaction.setDisableActions(true)
drawable.present()
CATransaction.commit()
CATransaction.flush()
}
windowedDiag.record(
schedMs: schedMs, commitMs: (CACurrentMediaTime() - commitStart) * 1000,
mode: .transaction)
return true
}
#endif
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime {
@@ -1025,146 +981,12 @@ public final class MetalVideoPresenter {
} else {
commandBuffer.present(drawable)
}
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
commandBuffer.commit()
return true
}
#if os(macOS)
/// The WINDOWED `surface` present tail (see `WindowedPresentMode.surface`): render with the
/// same per-frame pipeline into a pooled IOSurface and hand it to `surfaceLayer.contents`
/// from the command buffer's COMPLETION handler, inside an explicit CATransaction + flush
/// (the same off-main commit discipline as the transactional present an ordinary
/// damaged-layer update on WindowServer's own composite cadence, no image queue anywhere).
/// RENDER THREAD. `onPresented` is stamped right after the contents swap commits the
/// closest observable analogue of "reached glass" here (the composite follows within a
/// refresh, so the display-stage meters read slightly OPTIMISTIC in this mode).
///
/// The pool tracks `hdrActive`: bgra8 for SDR, rgba16Float tagged BT.2100 PQ for HDR
/// `configure` already ran, so the caller's `pipeline` attachment format always matches.
/// HDR OPEN RISK (why this whole mode is a prototype): whether the compositor honors the
/// PQ tag + EDR for plain-CALayer IOSurface contents needs an on-glass eyeball; the metal
/// layer underneath keeps `wantsExtendedDynamicRangeContent` as the EDR anchor (the harness
/// measured the display's EDR headroom engaging with this arrangement).
private func encodeToSurface(
targetSize: CGSize, pipeline: MTLRenderPipelineState,
onPresented: ((Int64?) -> Void)?,
keepAlive: [Any], bind: (MTLRenderCommandEncoder) -> Void
) -> Bool {
ensureSurfacePool(size: targetSize, hdr: hdrActive)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(pipeline)
bind(encoder)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let diag = windowedDiag
let commitStamp = CACurrentMediaTime()
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // sources pinned until the GPU finished sampling
let completedAt = CACurrentMediaTime()
// Swap on THIS Metal completion thread: explicit transaction + flush, so the commit
// reaches the render server now, independent of main (completion handlers for one
// queue fire in execution order, so swaps can't reorder).
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
CATransaction.flush()
diag.record(
schedMs: (completedAt - commitStamp) * 1000,
commitMs: (CACurrentMediaTime() - completedAt) * 1000, mode: .surface)
onPresented?(Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size`/`hdr` 4 IOSurface render targets (one on glass, one
/// committed in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the
/// pool empty; the caller returns false and the ring's putBack + display-link retry take
/// over.
private func ensureSurfacePool(size: CGSize, hdr: Bool) {
guard size != surfacePoolSize || hdr != surfacePoolHDR else { return }
surfacePool.removeAll()
surfacePoolSize = size
surfacePoolHDR = hdr
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// rgba16Float (8 B/px) carries the PQ-encoded HDR samples; bgra8 the SDR ones. 256-byte
// row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerElement = hdr ? 8 : 4
let bytesPerRow = ((w * bytesPerElement) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: bytesPerElement,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: hdr
? kCVPixelFormatType_64RGBAHalf : kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: hdr ? .rgba16Float : .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
if hdr, let name = CGColorSpace(name: CGColorSpace.itur_2100_PQ)?.name {
// Tag the surface BT.2100 PQ so the compositor interprets the half-float
// samples as PQ-encoded HDR (the CALayer-contents analogue of the metal
// layer's colorspace).
IOSurfaceSetValue(surface, "IOSurfaceColorSpace" as CFString, name)
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
// The EDR request rides the SURFACE layer too (its contents are what composite); the
// metal layer underneath keeps its own from configureColor as the anchor. Layer flags
// are committed by the next swap's transaction flush.
surfaceLayer.wantsExtendedDynamicRangeContent = hdr
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture(
@@ -142,17 +142,20 @@ enum PresentPriority: Equatable {
final class SessionPresenter {
/// Present pacing for this session. Stage-3 always means glass gating; under the stage-2
/// default, macOS PyroWave sessions ALSO get glass gating for SMOOTHNESS, not as the panic
/// fix (that is the windowed transactional present see `setComposited`). PyroWave's wavelet
/// decode is near-instant Metal compute, so a network clump presents within the same
/// millisecond, and it is the codec that sustains stream rates above the panel's refresh; the
/// glass gate admits one presented-but-undisplayed swap at a time (serialized on the on-glass
/// callback, 100 ms stale backstop) so those bursts coalesce in the newest-wins ring instead
/// of flooding the queue. (Glass pacing was ALSO the original DCP-panic mitigation attempt
/// disproven: a fully serialized stream still panicked, which is why the real fix moved to the
/// present mechanism.) An explicit stage-2 pick (setting/env) still forces arrival pacing
/// that A/B lever must stay honest. VideoToolbox codecs keep arrival pacing: decode latency
/// spaces their presents.
/// default, macOS PyroWave sessions ALSO get glass gating a kernel-panic mitigation, not a
/// latency tweak. macOS's DCP panics ("mismatched swapID's" @UnifiedPipeline.cpp, the whole
/// machine dies) when WindowServer's swap submissions race, and the reliable trigger is
/// out-of-band CAMetalLayer presents (displaySyncEnabled=false mandatory for us, see
/// MetalVideoPresenter's init) arriving faster than the compositor latches them in a
/// COMPOSITED (windowed) session. Arrival pacing does exactly that with PyroWave: the wavelet
/// decode is near-instant Metal compute, so a network clump of frames presents within the
/// same millisecond, and PyroWave is the codec that sustains stream rates above the panel's
/// refresh. The glass gate admits one presented-but-undisplayed swap at a time (serialized on
/// the on-glass callback, 100 ms stale backstop), which removes the racing pattern outright;
/// frames the panel couldn't have shown anyway coalesce in the newest-wins ring. An explicit
/// stage-2 pick (setting/env) still forces arrival pacing that A/B lever must stay honest.
/// VideoToolbox codecs keep arrival pacing: decode latency spaces their presents, and years
/// of stage-2 defaults there predate any panic report.
static func pacing(
for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec
) -> PresentPacing {
@@ -175,25 +178,10 @@ final class SessionPresenter {
/// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing
/// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate.
///
#if os(macOS)
/// Resolve the windowed (composited) present MECHANISM for this session the DCP
/// swapID-panic mitigation picker (see `WindowedPresentMode`). The
/// `PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface` env lever wins (dev A/B);
/// otherwise the user's safe-present setting: ON/unset `.transaction` (the validated
/// mitigation), OFF `.async` (the fast pre-mitigation path the panic returns on
/// affected high-refresh setups; the Settings caption says so). `.surface` is currently
/// env-only (prototype HDR-composite verification owed). Fullscreen always presents
/// async regardless (`setComposited`). Internal (not private) for unit tests.
static func windowedPresentMode(setting: Bool?, env: String?) -> WindowedPresentMode {
if let env, let mode = WindowedPresentMode(rawValue: env) { return mode }
return (setting ?? true) ? .transaction : .async
}
#endif
/// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder
/// stays reproducible on-device; macOS is pinned to 1, env ignored a deeper gate only builds
/// a standing queue (see above), and macOS glass pacing exists for PyroWave smoothness
/// (see `pacing`), where depth 1 is the point. Internal (not private) for unit tests.
/// stays reproducible on-device; macOS is pinned to 1, env ignored glass pacing exists
/// there as the DCP swapID kernel-panic mitigation (see `pacing`), and STRICT present
/// serialization is its point. Internal (not private) for unit tests.
static func gateDepth(env: String?) -> Int {
#if os(macOS)
return 1
@@ -208,16 +196,10 @@ final class SessionPresenter {
private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer?
#if os(macOS)
/// The windowed present MECHANISM this session runs while composited (resolved once per
/// session in `start` the user's safe-present setting + the PUNKTFUNK_WINDOWED_PRESENT
/// dev override) and the routing last pushed to the pipeline see `setComposited` (the DCP
/// swapID-panic mitigation). Main-thread only, like all of this.
private var windowedMode: WindowedPresentMode = .transaction
private var windowedPresentApplied: WindowedPresentMode = .async
/// The windowed `surface` present target (sibling above `metalLayer`, transparent while
/// unused) installed whenever stage-2 runs so a mechanism flip never has to mutate the
/// layer tree mid-session.
/// The windowed-mode PyroWave present target (sibling above `metalLayer`) and the last
/// routing pushed to the pipeline see `setComposited`. Main-thread only, like all of this.
private var surfaceLayer: CALayer?
private var surfacePresentsActive = false
#endif
private var connection: PunktfunkConnection?
/// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's
@@ -301,17 +283,11 @@ final class SessionPresenter {
baseLayer.addSublayer(metal)
metalLayer = metal
#if os(macOS)
windowedPresentApplied = .async
// Resolve THIS session's windowed mechanism once (setting + dev env lever)
// `setComposited` routes between it and fullscreen-async from every layout.
windowedMode = Self.windowedPresentMode(
setting: UserDefaults.standard.object(
forKey: DefaultsKey.windowedSafePresent) as? Bool,
env: ProcessInfo.processInfo.environment["PUNKTFUNK_WINDOWED_PRESENT"])
// The surface present target sits ABOVE the metal layer: transparent (nil contents)
// unless the surface mechanism actually presents, covering it while it does.
// The windowed-PyroWave present target sits ABOVE the metal layer: transparent (nil
// contents) while the metal path presents, covering it while surface presents run.
baseLayer.addSublayer(pipeline.surfaceLayer)
surfaceLayer = pipeline.surfaceLayer
surfacePresentsActive = false
#endif
stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
@@ -456,23 +432,19 @@ final class SessionPresenter {
#if os(macOS)
/// Route presents for the window's composited state (MAIN thread the view pushes it on
/// every layout, which fullscreen transitions always trigger). A COMPOSITED (windowed)
/// session presents through this session's resolved mitigation mechanism (`windowedMode`
/// transactional by default, see `windowedPresentMode`) instead of the async image queue
/// the DCP "mismatched swapID's" kernel-panic mitigation (see `MetalVideoPresenter`; the
/// async-swap race survives glass pacing, so pacing alone was not enough). ALL codecs:
/// PyroWave hit it 2026-07-18 and windowed HEVC hit the same 240 Hz Mac Studio 2026-07-21
/// it is the async image queue itself, not any codec or present rate. Fullscreen keeps the
/// async path (direct scanout, lowest latency, no panic there). The full HDR/EDR render
/// path is preserved in every mechanism.
/// every layout, which fullscreen transitions always trigger). PyroWave sessions in a
/// COMPOSITED (windowed) session present via `surfaceLayer` contents instead of the
/// CAMetalLayer image queue the DCP "mismatched swapID's" kernel-panic mitigation (see
/// `MetalVideoPresenter.surfaceLayer`; the metal-swap race survives glass pacing, so pacing
/// alone was not enough). VT codecs keep the metal path: no panic reports there, and their
/// HDR/EDR presentation has no surface-contents equivalent wired.
func setComposited(_ composited: Bool) {
guard let stage2 else { return }
let mode: WindowedPresentMode = composited ? windowedMode : .async
guard mode != windowedPresentApplied else { return }
let wasSurface = windowedPresentApplied == .surface
windowedPresentApplied = mode
stage2.setWindowedPresent(mode)
if wasSurface {
guard let stage2, let connection else { return }
let wantsSurface = composited && connection.videoCodec == .pyrowave
guard wantsSurface != surfacePresentsActive else { return }
surfacePresentsActive = wantsSurface
stage2.setSurfacePresents(wantsSurface)
if !wantsSurface {
// Uncover the metal layer NOW (its last drawable is still attached, so fullscreen
// entry shows the previous frame until the next present no black flash).
CATransaction.begin()
@@ -499,7 +471,7 @@ final class SessionPresenter {
#if os(macOS)
surfaceLayer?.removeFromSuperlayer()
surfaceLayer = nil
windowedPresentApplied = .async
surfacePresentsActive = false
#endif
connection = nil
}
@@ -1114,15 +1114,15 @@ public final class Stage2Pipeline {
}
#if os(macOS)
/// Forward the windowed present mechanism (MAIN thread see
/// `MetalVideoPresenter.setWindowedPresent`, the DCP swapID-panic mitigation).
func setWindowedPresent(_ mode: WindowedPresentMode) {
presenter.setWindowedPresent(mode)
}
/// The windowed-mode PyroWave present target (see `MetalVideoPresenter.surfaceLayer` the
/// DCP swapID-panic mitigation). The hosting view installs it as a sibling above `layer`.
public var surfaceLayer: CALayer { presenter.surfaceLayer }
/// The windowed `surface` present target the hosting SessionPresenter installs as a sibling
/// ABOVE `layer` (transparent while unused see `MetalVideoPresenter.surfaceLayer`).
var surfaceLayer: CALayer { presenter.surfaceLayer }
/// Forward the windowed-vs-fullscreen present routing (MAIN thread see
/// `MetalVideoPresenter.setSurfacePresents`).
public func setSurfacePresents(_ on: Bool) {
presenter.setSurfacePresents(on)
}
#endif
/// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen`
@@ -1213,9 +1213,7 @@ public final class Stage2Pipeline {
let chunkAligned =
au.flags & PunktfunkConnection.userFlagChunkAligned != 0
let ptsNs = au.ptsNs
// Decode stage starts at the PULL (matching the VT path's FrameContext
// receiptpull is the HUD's separate client-queue term, ABI v9 split).
let receivedNs = au.pulledNs
let receivedNs = au.receivedNs
let flags = au.flags
let submitted = decoder.decode(
au: au.data, chunkAligned: chunkAligned, windowSize: windowSize
@@ -32,12 +32,9 @@ public enum ReadyImage: @unchecked Sendable {
public struct ReadyFrame: @unchecked Sendable {
/// Host capture clock (the AU's pts), in nanoseconds.
public let ptsNs: UInt64
/// Client `CLOCK_REALTIME` instant the AU left `nextAU` (`AccessUnit.pulledNs`, threaded
/// through the decode via the frame refcon), in nanoseconds the decode stage's start
/// point. (Named for its historical role; since the ABI v9 receipt split the true
/// reassembly receipt lives on `AccessUnit.receivedNs`, and receiptpull is the HUD's own
/// client-queue term.) 0 when unknown (a caller that didn't stamp) the decode-stage meter
/// then drops the sample via its sanity guard.
/// Client `CLOCK_REALTIME` instant the AU was received (`AccessUnit.receivedNs`, threaded
/// through the decode via the frame refcon), in nanoseconds. 0 when unknown (a caller that
/// didn't stamp receipt) the decode-stage meter then drops the sample via its sanity guard.
public let receivedNs: Int64
/// Client `CLOCK_REALTIME` instant decode completed, in nanoseconds.
public let decodedNs: Int64
@@ -170,11 +167,7 @@ public final class VideoDecoder: @unchecked Sendable {
var infoOut = VTDecodeInfoFlags()
// The AU's receipt instant + wire flags ride through as a retained context; the output
// callback reclaims it. Retain immediately before submit so no early return can leak it.
// The decode stage starts at the PULL (the AU leaving nextAU), not the reassembly
// receipt: both consumers the decode-stage meter and the ABR decode signal are
// specified from the pull, and the receiptpull wait is the HUD's separate client-queue
// term (see AccessUnit.pulledNs).
let ctx = FrameContext(receivedNs: au.pulledNs, flags: au.flags)
let ctx = FrameContext(receivedNs: au.receivedNs, flags: au.flags)
let refcon = Unmanaged.passRetained(ctx).toOpaque()
let status = VTDecompressionSessionDecodeFrame(
session,
@@ -38,11 +38,10 @@ private let streamInputDebug =
/// dragged deltas become the relative motion StreamLayerView forwards), and hide it.
/// hide/unhide and associate are balanced via `captured`.
///
/// In the DESKTOP mouse model (absolute pointer, remote-desktop-sweep M1) this is a no-op:
/// the pointer stays free (entering and leaving the stream at will) and StreamLayerView
/// forwards ABSOLUTE positions instead; the local cursor is hidden only while over the view
/// (cursor rects). `disassociate` selects between the two; `release()` only undoes what
/// `capture` actually did.
/// In CLIENT-SIDE-CURSOR mode (gamescope, whose capture carries no host cursor) this is a
/// no-op: the local cursor stays visible and free, and StreamLayerView forwards ABSOLUTE
/// positions instead the visible system cursor IS the on-screen cursor. `disassociate`
/// selects between the two; `release()` only undoes what `capture` actually did.
private final class CursorCapture {
private var captured = false
/// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode),
@@ -208,31 +207,14 @@ public final class StreamLayerView: NSView {
/// forwarded). Main-thread only.
public private(set) var captured = false
/// Desktop (absolute) mouse model remote-desktop-sweep M1: when true the pointer is
/// never disassociated (it enters and leaves the stream freely) and the mouse monitor
/// forwards ABSOLUTE positions through the letterbox; the local cursor is hidden only
/// while over this view (cursor rects the host's composited cursor, tracking our
/// sends, is the one you see) and reappears the moment it leaves. When false the
/// captured/disassociated relative path runs unchanged. Initialized at session start
/// from the `mouseMode` setting gated by the host's resolved compositor (gamescope's
/// EIS is relative-only absolute sends would be dropped, so it pins to capture);
/// flipped live by M. A live flip re-engages capture in the new model so
/// disassociation + the abs/rel choice swap atomically. Main-thread only.
private var desktopMouse = false
/// Cursor channel (M2): the host forwards shape/state and WE draw the pointer. Active
/// when the Welcome carried `HOST_CAP_CURSOR` (only sessions that advertised the client
/// cap get it). Shapes cache by serial; state is latest-wins. Main-thread only.
private var cursorChannelActive = false
private var hostCursors: [UInt32: NSCursor] = [:]
private var cursorState: PunktfunkConnection.CursorStateEvent?
/// Last `CursorRenderMode.clientDraws` told to the host (the §8 mid-stream render flip);
/// nil = nothing sent yet. Edge-detected by [`reconcileCursorRender`] from the live mouse
/// model, so the chord, engage/release, and session start all reconcile through one path.
private var sentClientDraws: Bool?
/// M3 hint tracking: edge-triggered so a manual M isn't fought the override latch
/// holds until the HOST's intent next changes.
private var lastHint: Bool?
private var hintOverride = false
/// Client-side-cursor mode: when true the local system cursor stays VISIBLE over the
/// stream and the mouse monitor forwards ABSOLUTE positions (the visible cursor is the
/// on-screen cursor gamescope draws none, so no double cursor); when false the existing
/// captured/disassociated relative path runs unchanged. Initialized at session start from
/// the `cursorMode` setting + the host's resolved compositor, toggled live by C. A live
/// flip re-engages capture in the new mode so disassociation + the abs/rel choice swap
/// atomically. Main-thread only.
private var cursorVisible = false
/// One-shot auto-engage request (stream start, trust confirmed) attempted as soon
/// as the view is in a window with real bounds, then dropped, so it can never fire
/// surprisingly later (e.g. on a resize).
@@ -458,9 +440,9 @@ public final class StreamLayerView: NSView {
// If the cursor grab is refused (e.g. the reactivating click arrives before the app is
// frontmost), stay released so the NEXT click retries never latch captured=true over
// a free cursor, which would make mouseDown's `!captured` guard reject every later click.
// In the desktop mouse model there is no grab (the pointer stays free) capture
// In client-side-cursor mode there is no grab (the cursor stays visible) capture
// always engages and the monitor forwards absolute positions instead.
guard cursorCapture.capture(in: self, disassociate: !desktopMouse) else { return }
guard cursorCapture.capture(in: self, disassociate: !cursorVisible) else { return }
inputCapture?.setForwarding(true, suppressClick: fromClick)
// Install AFTER the warp + setForwarding: the engage warp generates no forwarded
// delta (the monitor isn't up yet), and the engage click's suppression latch is
@@ -468,9 +450,7 @@ public final class StreamLayerView: NSView {
installMouseMonitor()
captured = true
window?.makeFirstResponder(self)
window?.invalidateCursorRects(for: self) // desktop model: hide-over-view engages
notifyCaptureChange(true)
reconcileCursorRender()
}
private func releaseCapture() {
@@ -479,162 +459,7 @@ public final class StreamLayerView: NSView {
cursorCapture.release()
inputCapture?.setForwarding(false)
captured = false
window?.invalidateCursorRects(for: self)
notifyCaptureChange(false)
reconcileCursorRender() // released the host composites the pointer again
}
/// A fully transparent cursor for the desktop mouse model's hide-over-view rect
/// an empty 1×1 image draws nothing.
private static let invisibleCursor = NSCursor(
image: NSImage(size: NSSize(width: 1, height: 1)), hotSpot: .zero)
/// Desktop mouse model: the local cursor is hidden while over the stream (the host's
/// composited cursor, tracking our absolute sends, is the one you see) and reappears
/// the moment it leaves the view AppKit applies/removes the rect's cursor for us,
/// so there is no hide/unhide balancing to get wrong. Capture model instead hides
/// globally via `CursorCapture` (the pointer can't leave the view there).
override public func resetCursorRects() {
if captured && desktopMouse {
// Cursor channel active: wear the HOST's pointer shape (it is no longer in the
// video); hidden host pointer (or no shape yet) = invisible. Without the channel,
// M1 behavior: invisible local cursor, the composited host cursor is the visible one.
if cursorChannelActive, let st = cursorState, st.visible,
let host = hostCursors[st.serial] {
addCursorRect(bounds, cursor: host)
} else {
addCursorRect(bounds, cursor: Self.invisibleCursor)
}
} else {
super.resetCursorRects()
}
}
/// Tell the host who renders the pointer (the §8 mid-stream render flip): we draw it only
/// while the DESKTOP model is engaged (the local OS cursor wears the host shape); under
/// the capture model and while released the host composites it into the video (full
/// fidelity, the pre-channel look). One edge-detected reconciler, called from every
/// transition (chord, engage/release, session start).
private func reconcileCursorRender() {
guard cursorChannelActive, let connection else { return }
let clientDraws = captured && desktopMouse
guard sentClientDraws != clientDraws else { return }
sentClientDraws = clientDraws
connection.setCursorRender(clientDraws: clientDraws)
}
/// Flip the mouse model with the atomic release/re-engage swap; `reappearAt` (host video
/// px the M3 hand-back position) warps the local pointer so leaving relative lands the
/// cursor exactly where the host last had it.
private func setDesktopMouse(_ on: Bool, reappearAt: (x: Int32, y: Int32)?) {
guard desktopMouse != on else { return }
let wasCaptured = captured
if wasCaptured { releaseCapture() }
desktopMouse = on
if wasCaptured { engageCapture(fromClick: false) }
window?.invalidateCursorRects(for: self)
if on, let p = reappearAt, let sp = cgScreenPoint(forHostX: p.x, p.y) {
CGWarpMouseCursorPosition(sp)
}
reconcileCursorRender()
}
/// The single cursor pull thread (both planes share the connection's cursor lock):
/// latest-wins state at a short timeout + a non-blocking shape poll per iteration.
/// Exits when the connection closes; events hop to main where all cursor state lives.
private func startCursorPump(_ connection: PunktfunkConnection) {
let thread = Thread { [weak self] in
while true {
do {
var newest: PunktfunkConnection.CursorStateEvent?
if let st = try connection.nextCursorState(timeoutMs: 100) {
newest = st
while let more = try connection.nextCursorState(timeoutMs: 0) {
newest = more // drain latest wins
}
}
while let shape = try connection.nextCursorShape(timeoutMs: 0) {
DispatchQueue.main.async { self?.applyCursorShape(shape) }
}
if let st = newest {
DispatchQueue.main.async { self?.applyCursorState(st) }
}
} catch {
return // connection closed the session is over
}
if self == nil { return }
}
}
thread.name = "pf-cursor-pump"
thread.start()
}
private func applyCursorShape(_ ev: PunktfunkConnection.CursorShapeEvent) {
guard let cursor = Self.makeCursor(ev) else {
streamInputLog.warning("cursor shape rejected (\(ev.width)x\(ev.height)) — keeping the previous cursor")
return
}
if hostCursors.count >= 64 { hostCursors.removeAll() } // degenerate host: reset
hostCursors[ev.serial] = cursor
if cursorState?.serial == ev.serial {
window?.invalidateCursorRects(for: self)
}
}
private func applyCursorState(_ ev: PunktfunkConnection.CursorStateEvent) {
let prev = cursorState
cursorState = ev
if prev?.visible != ev.visible || prev?.serial != ev.serial {
window?.invalidateCursorRects(for: self)
}
// M3 host-driven auto-flip is DISABLED: `relative_hint` is derived from host cursor
// VISIBILITY, and Windows hides the pointer for ordinary desktop activity (clicking,
// typing) not just when a game grabs it. Acting on those transients flipped
// desktopcapturedesktop, which warped the cursor to view-centre and flushed held
// buttons (a spurious button-up ~200 ms into every press broke window drags). Until
// the host exposes a real pointer-LOCK signal (ClipCursor/raw-input, not visibility),
// the mouse model is user-driven only (M). The hint still rides the wire, unused.
_ = (lastHint, hintOverride)
}
/// Build an `NSCursor` from a forwarded straight-alpha RGBA shape.
private static func makeCursor(_ ev: PunktfunkConnection.CursorShapeEvent) -> NSCursor? {
let (w, h) = (ev.width, ev.height)
guard w > 0, h > 0, ev.rgba.count >= w * h * 4,
let provider = CGDataProvider(data: ev.rgba as CFData),
let cg = CGImage(
width: w, height: h, bitsPerComponent: 8, bitsPerPixel: 32,
bytesPerRow: w * 4, space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue),
provider: provider, decode: nil, shouldInterpolate: false,
intent: .defaultIntent)
else { return nil }
let image = NSImage(cgImage: cg, size: NSSize(width: w, height: h))
return NSCursor(
image: image,
hotSpot: NSPoint(x: min(ev.hotX, w - 1), y: min(ev.hotY, h - 1)))
}
/// Host video px CG GLOBAL screen coordinates (top-left origin, the
/// `CGWarpMouseCursorPosition` convention `CursorCapture` established) through the
/// aspect-fit letterbox the inverse direction of `hostPoint(from:)`.
private func cgScreenPoint(forHostX hx: Int32, _ hy: Int32) -> CGPoint? {
guard let connection, let window else { return nil }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0 else { return nil }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
guard fit.width > 0, fit.height > 0 else { return nil }
let u = (CGFloat(hx) / CGFloat(mode.width)).clamped(to: 0...1)
let v = (CGFloat(hy) / CGFloat(mode.height)).clamped(to: 0...1)
let videoMinYTop = bounds.height - fit.maxY
let pTop = CGPoint(x: fit.minX + u * fit.width, y: videoMinYTop + v * fit.height)
let inView = CGPoint(x: pTop.x, y: bounds.height - pTop.y)
let inWindow = convert(inView, to: nil)
let onScreen = window.convertPoint(toScreen: inWindow)
let primaryHeight = NSScreen.screens.first?.frame.height ?? 0
return CGPoint(x: onScreen.x, y: primaryHeight - onScreen.y)
}
/// A single local monitor for motion + buttons, installed only while captured. A local
@@ -648,12 +473,12 @@ public final class StreamLayerView: NSView {
/// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're
/// inert locally.
///
/// In the desktop mouse model the cursor is NOT frozen, so bare `.mouseMoved` events are
/// In client-side-cursor mode the cursor is NOT frozen, so bare `.mouseMoved` events are
/// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and
/// restore it on removal so absolute hover-motion keeps flowing without a click held.
private func installMouseMonitor() {
guard mouseEventMonitor == nil else { return }
if desktopMouse {
if cursorVisible {
savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents
window?.acceptsMouseMovedEvents = true
}
@@ -665,8 +490,8 @@ public final class StreamLayerView: NSView {
guard let self, self.captured, let ic = self.inputCapture else { return event }
switch event.type {
case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged:
if self.desktopMouse {
// Desktop mouse model: forward the ABSOLUTE position (mapped through the
if self.cursorVisible {
// Client-side cursor: forward the ABSOLUTE position (mapped through the
// aspect-fit letterbox into host pixels), the same path the iPad pointer
// fallback uses. Events in the letterbox bars are dropped (nil host point).
if let p = self.hostPoint(from: event) {
@@ -784,25 +609,14 @@ public final class StreamLayerView: NSView {
// be a cursor trap with dead input.
self?.releaseCapture()
}
// M flips the mouse model (capture desktop) live the SDL clients' identical
// chord. Only the key window's stream owns it (same guard as the capture toggle).
// Re-engage capture in the new model so disassociation and the absolute/relative
// forwarding choice swap atomically releaseCapture restores the old model's grab
// (if any), engageCapture installs the new one. On a gamescope host the chord is a
// no-op: its EIS grants only a relative pointer, so the desktop model's absolute
// sends would be silently dropped (pointer stuck = "all input dead").
capture.onToggleMouseMode = { [weak self] in
guard let self, self.window?.isKeyWindow == true,
let conn = self.connection else { return }
guard conn.resolvedCompositor != .gamescope else {
streamInputLog.info("mouse-mode chord ignored: gamescope host is relative-only")
return
}
// A manual flip outranks the standing host hint until the hint next CHANGES.
self.hintOverride = true
self.setDesktopMouse(!self.desktopMouse, reappearAt: nil)
streamInputLog.info("chord: mouse mode \(self.desktopMouse ? "desktop" : "capture", privacy: .public)")
}
// C flips the client-side cursor live. Only the key window's stream owns it (same
// guard as the capture toggle). Re-engage capture in the new mode so disassociation
// and the absolute/relative forwarding choice swap atomically releaseCapture restores
// the old mode's grab (if any), engageCapture installs the new one.
// C would flip the client-side cursor live NEUTERED while the feature is disabled
// (see the cursorVisible resolution below): toggling it on under gamescope's relative-only
// input traps the pointer. Restore this body when absolute/synthetic-cursor support lands.
capture.onToggleCursor = {}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in
@@ -829,26 +643,15 @@ public final class StreamLayerView: NSView {
capture.start()
inputCapture = capture
// Desktop (absolute) mouse model resolved at session start from the mouseMode
// setting, gated by the host's compositor: gamescope's input socket (EIS) grants
// only a relative pointer, so absolute sends would be silently dropped there
// (pointer stuck = "all input dead") pinned to capture. M flips it live.
let mode = MouseInputMode(
rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? ""
) ?? .capture
let absOK = connection.resolvedCompositor != .gamescope
desktopMouse = mode == .desktop && absOK
if mode == .desktop && !absOK {
streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture")
}
// Cursor channel (M2): the host stopped compositing the pointer drain its shape/
// state planes and draw the pointer as the real NSCursor (plus the M3 auto-flip).
if connection.hostSupportsCursor {
cursorChannelActive = true
streamInputLog.info("cursor channel negotiated — host cursor renders locally")
startCursorPump(connection)
reconcileCursorRender() // initial render mode (a capture-model start composites)
}
// Client-side cursor is TEMPORARILY DISABLED. It positions the host cursor with ABSOLUTE
// events, but gamescope's input socket (EIS) grants only a relative pointer, so those are
// silently dropped the pointer never moves and clicks/scroll land on the stuck position
// (looks like "all input dead"). gamescope is exactly the compositor Auto enabled it for.
// Forced off until per-compositor gating (KWin/GNOME/Sway have absolute) or a synthetic-
// cursor-over-relative path lands; the resolution logic below is kept for that. See the
// C handler (also neutered) and the cursorMode setting (hidden).
cursorVisible = false
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent)
// Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
@@ -897,9 +700,9 @@ public final class StreamLayerView: NSView {
private func layoutPresenter() {
presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1)
// Present routing tracks the window's composited state (fullscreen transitions always
// re-layout, so this stays current): a windowed session presents through a Core Animation
// transaction the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited).
// A view not yet in a window counts as composited (the safe default).
// re-layout, so this stays current): windowed PyroWave presents via surface contents
// the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited). A view
// not yet in a window counts as composited (the safe default).
presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false))
// Feed the follower only once in a window (backing scale is real then) and with real
// bounds a pre-window layout would report point-sized dimensions.
@@ -938,14 +741,6 @@ public final class StreamLayerView: NSView {
matchFollower = nil
lastDecodedContentSize = nil // the next session re-derives it from its first frame
connection = nil
// Cursor-channel state is per-session: without this reset a next session against a
// host WITHOUT the cap would wear this session's stale shapes (`cursorChannelActive`
// stayed latched true across sessions).
cursorChannelActive = false
cursorState = nil
hostCursors.removeAll()
sentClientDraws = nil
window?.invalidateCursorRects(for: self)
}
deinit {
@@ -70,16 +70,6 @@ public enum DefaultsKey {
/// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync"
/// macOS: present WINDOWED sessions in lockstep with the system compositor (the DCP
/// "mismatched swapID's" kernel-panic mitigation see SessionPresenter.windowedPresentMode
/// and the MetalVideoPresenter saga notes). ON/unset (the default): windowed presents ride
/// a Core Animation transaction validated panic-free on the 240 Hz repro machine, at a
/// small display-latency cost vs the raw path. OFF: windowed sessions keep the fast async
/// image queue ON AFFECTED SETUPS (high-refresh displays) THAT PATH KERNEL-PANICS THE
/// WHOLE MAC, which is why the default is ON. Fullscreen always presents async (fast path)
/// regardless. Resolved once per session; PUNKTFUNK_WINDOWED_PRESENT=async|transaction|
/// surface overrides it for dev A/B.
public static let windowedSafePresent = "punktfunk.windowedSafePresent"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
@@ -94,11 +84,8 @@ public enum DefaultsKey {
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts"
/// Physical-mouse model (macOS): "capture" (pointer lock + relative, the default) or
/// "desktop" (uncaptured absolute pointer) the cross-client `mouse_mode`. Replaces the
/// never-shipped "punktfunk.cursorMode" (auto/always/never client-side-cursor setting,
/// which was hidden while disabled and had no readers).
public static let mouseMode = "punktfunk.mouseMode"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never".
public static let cursorMode = "punktfunk.cursorMode"
/// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by
/// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is
/// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across
@@ -316,41 +316,6 @@ final class PresentPacingTests: XCTestCase {
SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline)
}
// MARK: - Windowed present mechanism (the macOS DCP swapID-panic mitigation picker)
#if os(macOS)
/// The safe-present setting: ON/unset the validated transactional mitigation; an explicit
/// OFF the fast async path (the user accepted the affected-setup panic risk). The
/// PUNKTFUNK_WINDOWED_PRESENT env lever overrides both ways, `surface` is env-only (the
/// prototype mechanism), and garbage/empty env values are "unset", not an override.
func testWindowedPresentModeResolution() {
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: nil), .transaction,
"unset defaults to the panic mitigation")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: nil), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: nil), .async,
"an explicit opt-out gets the fast async path")
// The dev env lever wins over the setting, both directions.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "async"), .async)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "transaction"),
.transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "surface"), .surface,
"the surface prototype is reachable via env only")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "surface"), .surface)
// Garbage/empty env = unset.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: "garbage"), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: ""), .async)
}
#endif
// MARK: - Glass-gate depth
/// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue
+2 -11
View File
@@ -16,8 +16,6 @@
# PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games)
# PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect)
# PF_MGMT management-API port for --browse (optional; client defaults to 47990)
# PF_CONNECT_TIMEOUT connect budget in seconds (optional; the plugin stretches it after
# firing Wake-on-LAN so the connect survives the host's resume)
# PF_APPID flatpak app id (default io.unom.Punktfunk)
# PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH)
#
@@ -63,17 +61,10 @@ if [ -z "${PF_HOST:-}" ]; then
echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2
exit 2
fi
# Trailing args shared by both streaming execs. A stretched connect budget rides along when the
# plugin set one (it just fired Wake-on-LAN, so the host may still be resuming); an older flatpak
# without --connect-timeout ignores the flag harmlessly (hand-scanned argv).
set -- --fullscreen
if [ -n "${PF_CONNECT_TIMEOUT:-}" ]; then
set -- --connect-timeout "$PF_CONNECT_TIMEOUT" "$@"
fi
if [ -n "${PF_LAUNCH:-}" ]; then
# A pinned game: the id rides the session Hello and the host launches that title.
echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" "$@"
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" --fullscreen
fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" "$@"
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen
+3 -33
View File
@@ -10,17 +10,9 @@ import {
showModal,
staticClasses,
} from "@decky/ui";
import { definePlugin, routerHook, toaster } from "@decky/api";
import { definePlugin, routerHook } from "@decky/api";
import { FC } from "react";
import {
FaDownload,
FaLock,
FaLockOpen,
FaPlay,
FaPlus,
FaSyncAlt,
FaTv,
} from "react-icons/fa";
import { FaDownload, FaLock, FaLockOpen, FaPlay, FaSyncAlt, FaTv } from "react-icons/fa";
import { PluginErrorBoundary } from "./boundary";
import {
applyUpdate,
@@ -39,19 +31,7 @@ import {
import { streamPin } from "./library";
import { PunktfunkRoute, ROUTE } from "./page";
import { PairModal } from "./pair";
import { ensureGamepadUiShortcut, recreateShortcuts } from "./steam";
// Recovery action for "the Punktfunk library entry vanished" — recreates the visible shortcut.
// Deleting the shortcut (optionally + reinstalling the plugin) leaves a stale appId in Steam's
// CEF localStorage that self-heal fixes on the next mount, but this gives an in-session button
// that works even without a reload. Always ends in a toast so the tap has feedback.
async function recreatePunktfunkShortcut(): Promise<void> {
const appId = await recreateShortcuts();
toaster.toast({
title: "Punktfunk",
body: appId != null ? "Shortcut restored to your library" : "Couldn't create the shortcut",
});
}
import { ensureGamepadUiShortcut } from "./steam";
// ----------------------------------------------------------------------------------------
// QAM panel — quick status + entry into the full page + one-tap stream for known hosts
@@ -210,16 +190,6 @@ const QamPanel: FC = () => {
{checking ? "Checking…" : "Check for updates"}
</ButtonItem>
</PanelSectionRow>
<PanelSectionRow>
<ButtonItem
layout="below"
description="Missing the Punktfunk entry in your library? This puts it back."
onClick={() => void recreatePunktfunkShortcut()}
>
<FaPlus style={{ marginRight: "0.5em" }} />
Recreate library shortcut
</ButtonItem>
</PanelSectionRow>
</PanelSection>
</>
);
+10 -87
View File
@@ -55,29 +55,6 @@ declare const collectionStore:
| { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void }
| undefined;
// SteamUI's appStore indexes every registered app/shortcut by appId; a remembered appId whose
// overview is gone was deleted out from under us (the user removed the library entry). We must
// verify this because the remembered appId lives in Steam's CEF localStorage — which survives a
// plugin UNINSTALL/REINSTALL — so a manually-deleted shortcut otherwise leaves a dangling appId
// that the reuse path below silently repoints (SetShortcut* on a dead id is a no-op), and the
// entry never comes back.
declare const appStore:
| { GetAppOverviewByAppID?: (appId: number) => unknown | null }
| undefined;
/** True if a remembered appId still maps to a live Steam shortcut. When appStore is unavailable
* we can't tell, so assume it exists better to keep reusing than risk a duplicate library
* entry from a false "missing". A confident null means the shortcut was deleted recreate. */
function shortcutStillExists(appId: number): boolean {
try {
const get = appStore?.GetAppOverviewByAppID;
if (!get) return true; // no way to verify — preserve the reuse path
return get(appId) != null;
} catch {
return true;
}
}
/** Set a shortcut's library visibility (best-effort, deferred the overview registers a moment
* after AddShortcut). Hides the stateful stream shortcut; keeps the gamepad-UI one visible. */
function setShortcutHidden(appId: number, hidden: boolean): void {
@@ -93,9 +70,7 @@ function setShortcutHidden(appId: number, hidden: boolean): void {
};
// Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId).
// v3: CI zips through 0.17.1 shipped no assets/ at all, yet v2 was still recorded as applied
// on those installs — the bump makes them re-apply once on the first build that has the files.
const ART_VERSION = 3;
const ART_VERSION = 2;
function artKey(appId: number): string {
return `punktfunk:shortcutArt:${appId}`;
}
@@ -104,7 +79,7 @@ function artKey(appId: number): string {
* Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per
* appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call.
*/
async function applyArtwork(appId: number, isRetry = false): Promise<void> {
async function applyArtwork(appId: number): Promise<void> {
try {
if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) {
return;
@@ -116,29 +91,16 @@ async function applyArtwork(appId: number, isRetry = false): Promise<void> {
[art.logo, 2],
[art.gridwide, 3],
];
let applied = false;
for (const [data, assetType] of assets) {
if (data) {
await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType);
applied = true;
}
}
if (art.icon_path) {
SteamClient.Apps.SetShortcutIcon(appId, art.icon_path);
applied = true;
}
// Only record "done" when something actually landed — a plugin build whose assets/ is
// missing/empty must keep retrying on later mounts instead of poisoning the marker.
if (applied) {
localStorage.setItem(artKey(appId), `${ART_VERSION}`);
}
localStorage.setItem(artKey(appId), `${ART_VERSION}`);
} catch (e) {
// A shortcut fresh out of AddShortcut may not be registered yet (the same race
// setShortcutHidden defers around) — one deferred second attempt, then leave it to
// the next mount.
if (!isRetry) {
setTimeout(() => void applyArtwork(appId, true), 2500);
}
console.warn("punktfunk: shortcut artwork not applied", e);
}
}
@@ -195,9 +157,7 @@ async function ensureControllerConfig(): Promise<void> {
return;
}
const r = await applyControllerConfig(SHORTCUT_NAME);
// `ok` alone isn't done: with zero account configset dirs (fresh Steam) the backend
// succeeds without pointing any account at the template — keep retrying until one lands.
if (r?.ok && (r.applied ?? []).some((a) => a.startsWith("configset:"))) {
if (r?.ok) {
localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`);
} else {
console.warn("punktfunk: controller config not fully applied", r);
@@ -222,10 +182,8 @@ async function ensureStreamShortcut(): Promise<{ appId: number; runner: string }
const startDir = info.runner.replace(/\/[^/]*$/, ""); // the plugin's bin/ dir
void ensureControllerConfig(); // fire-and-forget — never blocks the launch
// Reuse the remembered shortcut only if it still exists — a stale appId (shortcut deleted, key
// outlived it across a reinstall) must fall through to AddShortcut, not be silently repointed.
const remembered = recall(STORAGE_KEY_STREAM);
if (remembered != null && shortcutStillExists(remembered)) {
if (remembered != null) {
SteamClient.Apps.SetShortcutExe(remembered, SHELL);
SteamClient.Apps.SetShortcutStartDir(remembered, startDir);
SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME);
@@ -261,11 +219,8 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
// home). %command% expands to the shortcut exe (/bin/sh); the wrapper rides behind as an arg.
const launchOpts = `PF_BROWSE=1 %command% "${info.runner}"`;
// Reuse the remembered entry only if it still exists; a stale appId (deleted shortcut whose
// localStorage key survived a plugin reinstall) falls through to AddShortcut so the visible
// library entry actually comes back instead of repointing a dead id.
let appId = recall(STORAGE_KEY_UI);
if (appId != null && shortcutStillExists(appId)) {
if (appId != null) {
SteamClient.Apps.SetShortcutExe(appId, SHELL);
SteamClient.Apps.SetShortcutStartDir(appId, startDir);
SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME);
@@ -284,30 +239,6 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
}
}
/**
* Force the visible "Punktfunk" library entry back into existence the recovery button for
* "my shortcut disappeared". Drops any remembered appId that no longer maps to a live shortcut
* (so it can't shadow a fresh AddShortcut), then re-ensures. Safe to press anytime: a shortcut
* that still exists is left in place (no duplicate); a missing one is recreated. Covers the case
* self-heal-on-mount can't deleting the shortcut WITHOUT reinstalling (no mount no ensure).
* Returns the (new or existing) visible appId, or null on failure.
*/
export async function recreateShortcuts(): Promise<number | null> {
for (const key of [STORAGE_KEY_STREAM, STORAGE_KEY_UI]) {
const id = recall(key);
if (id != null && !shortcutStillExists(id)) {
try {
localStorage.removeItem(artKey(id)); // stale art marker for the dead appId
localStorage.removeItem(key);
} catch {
/* ignore */
}
}
}
// Recreate the visible entry now; the hidden stream shortcut re-registers lazily on next launch.
return ensureGamepadUiShortcut();
}
/** Launch the stateless gamepad-UI shortcut (console home) from the plugin, e.g. a QAM button. */
export async function launchGamepadUi(): Promise<void> {
const appId = await ensureGamepadUiShortcut();
@@ -352,21 +283,13 @@ export async function launchStream(
opts: LaunchOpts = {},
): Promise<void> {
// Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now
// so it races with the shortcut setup (near-zero added latency); its outcome is needed below
// (the connect budget), and RunGame follows the await either way, so nothing is slower for it.
// so it races with the shortcut setup (near-zero added latency), and awaited just before RunGame.
// Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is
// known), and the connect that follows has its own retry window, so a failure never blocks launch.
const waking = wake(host, port).catch(() => ({ ok: false }));
const [{ appId, runner }, woke] = await Promise.all([ensureStreamShortcut(), waking]);
const { appId, runner } = await ensureStreamShortcut();
const target = port && port !== 9777 ? `${host}:${port}` : host;
const env = [`PF_HOST=${target}`];
// A magic packet actually went out (a MAC was known), so the host may be mid-resume from
// suspend — that takes far longer than the client's default 15 s connect budget. Stretch the
// budget so the client's wake-tolerant dial keeps retrying across the resume; against an
// already-awake host the connect still lands in under a second, so this costs nothing.
if (woke.ok) {
env.push("PF_CONNECT_TIMEOUT=75");
}
if (opts.browse) {
env.push("PF_BROWSE=1");
if (opts.mgmt) {
@@ -380,9 +303,9 @@ export async function launchStream(
env.push(`PF_LAUNCH=${opts.launchId}`);
}
// KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper
// script rides behind it as an argument and reads PF_* from the environment. The wake was
// awaited above, so the magic packet is out before the connect attempt.
// script rides behind it as an argument and reads PF_* from the environment.
SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`);
await waking; // ensure the magic packet is out before the connect attempt
SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100);
}
-1
View File
@@ -547,7 +547,6 @@ impl AppModel {
crate::video::decodable_codecs(), // codecs (unused by the probe, but honest)
0, // preferred_codec: no preference
None, // display_hdr: probe connect, nothing presents
0, // client_caps: probe connect, nothing renders a cursor
None, // launch: probe connect, no game
pin,
Some(identity),
+1 -40
View File
@@ -69,14 +69,6 @@ const TOUCH_MODE_CAPTIONS: &[&str] = &[
"The cursor jumps to your finger — a tap clicks there",
"Real multi-touch reaches the host — for touch-native apps",
];
/// Physical-mouse model values (persisted) + labels + dynamic captions — same idiom as
/// the touch rows. Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[&str] = &["capture", "desktop"];
const MOUSE_MODE_LABELS: &[&str] = &["Capture (games)", "Desktop (absolute)"];
const MOUSE_MODE_CAPTIONS: &[&str] = &[
"Pointer locks to the stream — relative motion, best for games",
"Pointer moves freely in and out — best for remote desktop work",
];
/// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page.
const APP_LICENSE: &str = concat!(
@@ -550,20 +542,6 @@ pub fn show(
set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]);
});
}
let mouse_row = ChoiceRow::new(
&dialog,
inline,
"Mouse input",
MOUSE_MODE_CAPTIONS[0],
MOUSE_MODE_LABELS,
);
{
let w = mouse_row.widget().clone();
mouse_row.connect_changed(move |i| {
let i = (i as usize).min(MOUSE_MODE_CAPTIONS.len() - 1);
set_row_subtitle(&w, MOUSE_MODE_CAPTIONS[i]);
});
}
let inhibit_row = adw::SwitchRow::builder()
.title("Capture system shortcuts")
.subtitle("Forward Alt+Tab, Super, … to the host while input is captured")
@@ -740,12 +718,6 @@ pub fn show(
touch_row.set_selected(touch_i as u32);
// set_selected never fires the changed hook, so seed the dynamic caption directly.
set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]);
let mouse_i = MOUSE_MODES
.iter()
.position(|&m| m == s.mouse_mode)
.unwrap_or(0);
mouse_row.set_selected(mouse_i as u32);
set_row_subtitle(mouse_row.widget(), MOUSE_MODE_CAPTIONS[mouse_i]);
let comp_i = COMPOSITORS
.iter()
.position(|&c| c == s.compositor)
@@ -816,7 +788,6 @@ pub fn show(
touch_group.add(touch_row.widget());
// Group titles are Pango markup — the ampersand must be an entity.
let kbm_group = group("Keyboard &amp; mouse", "");
kbm_group.add(mouse_row.widget());
kbm_group.add(&inhibit_row);
kbm_group.add(&invert_row);
input.add(&touch_group);
@@ -885,19 +856,9 @@ pub fn show(
s.render_scale =
RENDER_SCALES[(scale_row.selected() as usize).min(RENDER_SCALES.len() - 1)];
s.bitrate_kbps = (bitrate_row.value() * 1000.0) as u32;
// Keep a stored preference this table doesn't list (e.g. "switchpro" — valid to the
// session, hand-edited or written by another client): it displays as "Automatic", and
// writing that back would silently erase it just by opening + closing the dialog.
// Persist the row only when the user picked a non-Auto entry or the stored value was
// a listed one to begin with.
let pad_sel = (pad_row.selected() as usize).min(GAMEPADS.len() - 1);
if pad_sel != 0 || GAMEPADS.contains(&s.gamepad.as_str()) {
s.gamepad = GAMEPADS[pad_sel].to_string();
}
s.gamepad = GAMEPADS[(pad_row.selected() as usize).min(GAMEPADS.len() - 1)].to_string();
s.touch_mode =
TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string();
s.mouse_mode =
MOUSE_MODES[(mouse_row.selected() as usize).min(MOUSE_MODES.len() - 1)].to_string();
s.forward_pad = chosen_pin.borrow().clone();
s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)]
.to_string();
+13 -29
View File
@@ -458,11 +458,7 @@ async fn session(args: Args) -> Result<()> {
),
(None, None) => tracing::info!(%remote, "punktfunk/1 connected"),
}
let (mut send, recv) = conn.open_bi().await.context("open control stream")?;
// Frame every read on the control stream through the resumable reader, exactly as the client
// pump does: `clock_sync` bounds each read with a timeout, and a frame straddling two wakeups
// would otherwise leave the stream permanently misaligned for the rest of the run.
let mut recv = io::MsgReader::new(recv);
let (mut send, mut recv) = conn.open_bi().await.context("open control stream")?;
io::write_msg(
&mut send,
@@ -487,24 +483,14 @@ async fn session(args: Args) -> Result<()> {
// host/network split is exactly what it exists to report. Old hosts ignore the bit.
// PROBE_SEQ: the shared-core reassembler windows probe-space frames, so the probe
// qualifies for `--speed-test` bursts; without the bit the host declines them.
// STREAMED_AU: the same shared reassembler accepts sentinel-headed streamed
// blocks, and the probe is exactly the tool that measures the overlap win.
let mut caps = punktfunk_core::quic::VIDEO_CAP_HOST_TIMING
| punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ
| punktfunk_core::quic::VIDEO_CAP_STREAMED_AU;
| punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ;
if std::env::var_os("PUNKTFUNK_CLIENT_10BIT").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_10BIT;
}
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444;
}
// PUNKTFUNK_CLIENT_CHACHA20=1 advertises VIDEO_CAP_CHACHA20 — drives the
// host's ChaCha20-Poly1305 session-cipher resolution (the soft-AES armv7
// negotiation, design/chacha20-session-cipher.md §7) without a webOS build;
// the negotiated cipher is reported in the welcome log line below.
if std::env::var_os("PUNKTFUNK_CLIENT_CHACHA20").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_CHACHA20;
}
caps
},
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the
@@ -523,15 +509,12 @@ async fn session(args: Args) -> Result<()> {
// writes it into the virtual display's EDID (CTA HDR block), so the EDID-forwarding
// path can be validated headlessly (check the host's monitor caps / ADD log line).
display_hdr: punktfunk_core::client::display_hdr_env_override(),
// No CLIENT_CAP_CURSOR: this headless tool renders nothing — advertising it would
// just strip the pointer from the dumped bitstream.
client_caps: 0,
}
.encode(),
)
.await?;
let welcome =
Welcome::decode(&recv.read_msg().await?).map_err(|e| anyhow!("Welcome decode: {e:?}"))?;
let welcome = Welcome::decode(&io::read_msg(&mut recv).await?)
.map_err(|e| anyhow!("Welcome decode: {e:?}"))?;
tracing::info!(
mode = ?welcome.mode,
fec = ?welcome.fec,
@@ -545,11 +528,6 @@ async fn session(args: Args) -> Result<()> {
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format,
codec = codec_ext(welcome.codec),
cipher = if welcome.cipher == punktfunk_core::quic::CIPHER_CHACHA20_POLY1305 {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
"session offer"
);
@@ -651,7 +629,10 @@ async fn session(args: Args) -> Result<()> {
tracing::error!("Reconfigure write failed");
return;
}
match rr.read_msg().await.map(|b| Reconfigured::decode(&b)) {
match io::read_msg(&mut rr)
.await
.map(|b| Reconfigured::decode(&b))
{
Ok(Ok(ack)) if ack.accepted => {
tracing::info!(mode = ?ack.mode, "mode switch ACCEPTED")
}
@@ -704,7 +685,10 @@ async fn session(args: Args) -> Result<()> {
tracing::error!("SetBitrate write failed");
return;
}
match rr.read_msg().await.map(|b| BitrateChanged::decode(&b)) {
match io::read_msg(&mut rr)
.await
.map(|b| BitrateChanged::decode(&b))
{
Ok(Ok(ack)) => tracing::info!(
applied_kbps = ack.bitrate_kbps,
"BITRATE CHANGE acked by host"
@@ -766,7 +750,7 @@ async fn session(args: Args) -> Result<()> {
tracing::error!("ProbeRequest write failed");
return;
}
let res = match sr.read_msg().await.map(|b| ProbeResult::decode(&b)) {
let res = match io::read_msg(&mut sr).await.map(|b| ProbeResult::decode(&b)) {
Ok(Ok(r)) => r,
other => {
tracing::error!(?other, "bad ProbeResult");
+2 -6
View File
@@ -27,13 +27,9 @@ ui = ["dep:pf-console-ui", "dep:serde_json"]
# Same Linux+Windows gating as the rest of the client stack; elsewhere this is a stub
# binary.
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
# `default-features = false` on both: THIS crate's `pyrowave` feature (above) is the single
# switch that turns the wavelet codec on, and it enables it explicitly on each. Inheriting their
# defaults instead would make `--no-default-features` a lie — the Windows ARM64 leg builds that
# way precisely to skip the vendored PyroWave C++, which has no ARM64 SIMD path.
pf-presenter = { path = "../../crates/pf-presenter", default-features = false }
pf-presenter = { path = "../../crates/pf-presenter" }
pf-console-ui = { path = "../../crates/pf-console-ui", optional = true }
pf-client-core = { path = "../../crates/pf-client-core", default-features = false }
pf-client-core = { path = "../../crates/pf-client-core" }
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# The fake-library dev hook (`PUNKTFUNK_FAKE_LIBRARY`, browse mode) parses GameEntry JSON.
serde_json = { version = "1", optional = true }
-1
View File
@@ -158,7 +158,6 @@ pub fn run(target: Option<&str>) -> u8 {
v => v,
},
touch_mode: settings_at_start.touch_mode(),
mouse_mode: settings_at_start.mouse_mode(),
invert_scroll: settings_at_start.invert_scroll,
json_status,
on_connected: Some(Box::new(move |fingerprint: [u8; 32]| {
-6
View File
@@ -172,11 +172,6 @@ mod session_main {
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
// pump) pins one manually.
display_hdr: None,
// The presenter renders the host cursor locally in desktop mouse mode (M2 cursor
// channel); capture-mode sessions keep the composited cursor, so only advertise
// when the session STARTS in desktop mode. The host gates further (Linux portal
// compositors only).
cursor_forward: settings.mouse_mode() == trust::MouseMode::Desktop,
mic_enabled: settings.mic_enabled,
clipboard,
// The Settings preference (auto → VAAPI where it exists; the presenter
@@ -434,7 +429,6 @@ mod session_main {
v => v,
},
touch_mode: settings.touch_mode(),
mouse_mode: settings.mouse_mode(),
invert_scroll: settings.invert_scroll,
json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
+1 -11
View File
@@ -29,17 +29,7 @@ punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# The shared client service layer: the trust/settings stores (ONE `Settings` struct for the
# shell and the spawned session binary — src/trust.rs re-exports it) and the game-library
# data model (fetch + art pipeline) behind the library page.
#
# `default-features = false` drops pf-client-core's default `pyrowave`, which would otherwise
# build the vendored PyroWave C++ INTO THE SHELL — dead weight here (the shell never decodes;
# it only offers "pyrowave" as a codec preference string the session binary acts on) and fatal
# on ARM64, where Granite's math falls back to x86 SSE intrinsics and stops at
# `simd.hpp: #error "Implement me."`. This does NOT drop PyroWave from the Windows client:
# decode lives in the spawned punktfunk-session binary, whose own default enables the feature,
# and cargo's feature unification turns it back on for the shared pf-client-core whenever that
# binary is in the same build (x64). On the ARM64 leg both are built --no-default-features, so
# nothing enables it and the C++ is never compiled.
pf-client-core = { path = "../../crates/pf-client-core", default-features = false }
pf-client-core = { path = "../../crates/pf-client-core" }
# WinUI 3 UI via windows-reactor (a declarative React-like framework backed by WinUI). Its
# `build.rs` downloads the Windows App SDK NuGets and stages the bootstrap DLL + resources.pri
+3 -8
View File
@@ -59,14 +59,9 @@
</Application>
<!--
Second entry point: the couch/console UI, for an HTPC or a TV-attached box where the
desktop shell is the wrong first screen. Its own executable (punktfunk-console.exe)
because an MSIX Application entry cannot pass arguments to a full-trust exe; it hands
straight off to the session binary's controller-driven browse mode (host list,
pairing, settings, library) fullscreen.
NOTE: never write a double hyphen in this file. XML forbids it inside a comment, and
makepri rejects the whole manifest ("Appx manifest not found or is invalid") — which
is exactly how the console flag spelled out here broke the v0.15.0 MSIX build.
desktop shell is the wrong first screen. Same full-trust executable, launched with
`--console`, which hands straight off to the session binary's controller-driven
browse mode (host list, pairing, settings, library) fullscreen.
-->
<Application Id="PunktfunkConsole" Executable="punktfunk-console.exe"
EntryPoint="Windows.FullTrustApplication">
-18
View File
@@ -90,13 +90,6 @@ const TOUCH_MODES: &[(&str, &str)] = &[
("pointer", "Direct pointer"),
("touch", "Touch passthrough"),
];
/// Physical-mouse presets: `(stored value, display label)` — capture (pointer lock,
/// relative, for games) vs desktop (uncaptured absolute pointer, for remote desktop
/// work). Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[(&str, &str)] = &[
("capture", "Capture (games)"),
("desktop", "Desktop (absolute)"),
];
/// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to
/// auto-detect when the choice is unavailable. Only meaningful against a Linux host.
const COMPOSITORS: &[(&str, &str)] = &[
@@ -401,10 +394,6 @@ pub(crate) fn settings_page(
let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| {
s.touch_mode = TOUCH_MODES[i].0.to_string();
});
let (mouse_names, mouse_i) = presets(MOUSE_MODES, |v| *v == s.mouse_mode);
let mouse_combo = setting_combo(ctx, "Mouse input", mouse_names, mouse_i, |s, i| {
s.mouse_mode = MOUSE_MODES[i].0.to_string();
});
let invert_scroll_toggle =
setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| {
s.invert_scroll = on
@@ -553,13 +542,6 @@ pub(crate) fn settings_page(
out.extend(group(
Some("Keyboard & mouse"),
vec![
described(
mouse_combo,
"Capture locks the pointer to the stream and sends relative motion — \
best for games. Desktop leaves the pointer free to enter and leave \
the stream and sends absolute positions best for remote desktop \
work. Ctrl+Alt+Shift+M switches live.",
),
described(
shortcuts_toggle,
"Alt+Tab, the Windows key and friends reach the host while the stream \
-1
View File
@@ -56,7 +56,6 @@ pub fn run_speed_probe(
decodable_codecs(),
0, // preferred_codec: no preference
None, // display_hdr: probe connect, nothing presents
0, // client_caps: probe connect, nothing renders a cursor
None, // launch: no game
pin,
Some(identity),
-8
View File
@@ -30,12 +30,6 @@ pipewire = "0.9"
libc = "0.2"
# ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane).
tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
# XFixes cursor source for gamescope (remote-desktop-sweep Phase C): gamescope paints no
# `SPA_META_Cursor`, so the pointer never reaches the PipeWire node. We read the shape/hotspot/
# visibility from gamescope's nested Xwayland via XFixes instead and feed the existing cursor slot.
# `RustConnection` is the pure-Rust default (no libxcb link → no new C dependency on the host); the
# `xfixes` feature (auto-pulls `render` + `shape`) is what exposes GetCursorImage/SelectCursorInput.
x11rb = { version = "0.13", default-features = false, features = ["xfixes"] }
[target.'cfg(target_os = "windows")'.dependencies]
# The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header).
@@ -49,9 +43,7 @@ windows = { version = "0.62", features = [
"Win32_Graphics_Direct3D_Fxc",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
"Win32_Graphics_Gdi",
"Win32_System_LibraryLoader",
"Win32_System_StationsAndDesktops",
"Win32_System_Memory",
"Win32_System_Threading",
"Win32_UI_HiDpi",
-58
View File
@@ -24,16 +24,6 @@ use pf_frame::DmabufFrame;
pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>;
/// [`next_frame`](Self::next_frame) with a caller-chosen first-frame budget instead of the
/// backend's default. The pipeline retry loop shortens its FIRST attempt's wait: a PipeWire
/// stream connected while gamescope re-inits its headless takeover can negotiate a format,
/// reach `Streaming`, and still never receive a buffer — a fresh connect then delivers within
/// ~0.5 s, so waiting out the full default budget on a doomed stream just delays the retry
/// that fixes it. Backends without an internal wait budget ignore it (the default delegates).
fn next_frame_within(&mut self, _budget: std::time::Duration) -> Result<CapturedFrame> {
self.next_frame()
}
/// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal
@@ -69,34 +59,6 @@ pub trait Capturer: Send {
/// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet).
/// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram),
/// so the two stay a single source of truth. May change mid-session if the source is regraded.
/// The capture source's LIVE cursor state, when it arrives out-of-band from the frames
/// (the Windows IddCx hardware-cursor channel). Polled by the encode loop every tick and
/// preferred over `CapturedFrame::cursor` — with a hardware cursor, pointer-only moves
/// produce NO new frame, so the frame-attached overlay would go stale on a static desktop.
/// Default `None`: the Linux portal path attaches its cursor to frames instead.
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
None
}
/// LIVE cursor-render flip for a cursor-forward session (design/remote-desktop-sweep.md §8):
/// `on = true` — the client draws the pointer, keep it OUT of the video; `on = false` —
/// the capture mouse model, the pointer must be IN the video again. The Windows IDD
/// capturer implements the composite side ITSELF (slot-copy + alpha-blended quad from the
/// GDI poller) — a declared IddCx hardware cursor is irrevocable, so DWM can never be
/// handed the job back. Called every encode tick (implementations cache; steady state is
/// one compare). Default no-op: the Linux portal never bakes the pointer into frames —
/// the encode loop blends its overlay instead.
fn set_cursor_forward(&mut self, _on: bool) {}
/// Attach a gamescope cursor source (remote-desktop-sweep Phase C). gamescope paints no
/// `SPA_META_Cursor`, so [`cursor`](Self::cursor)'s slot stays empty — this hands the Linux
/// portal capturer gamescope's nested Xwayland `(DISPLAY, XAUTHORITY)` targets (it may run
/// several — one per `--xwayland-count`) so it reads the pointer shape/position over X11
/// (XFixes + QueryPointer), following whichever display is focused, and publishes it into that
/// same slot. Called once, after the capturer is built, only for gamescope sessions. Default
/// no-op: every non-gamescope capturer already has a cursor source.
fn attach_gamescope_cursor(&mut self, _targets: Vec<(String, Option<String>)>) {}
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None
}
@@ -287,12 +249,6 @@ pub struct ZeroCopyPolicy {
/// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only
/// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`.
pub pyrowave_session: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (the Linux raw Vulkan
/// Video backend on an H265/AV1 session — resolved by the host facade via
/// `pf_encode::linux_native_nv12_ok`). Gates whether the negotiation PREFERS gamescope's
/// producer-side NV12 pod: libav VAAPI (H264's backend) would misread the two-plane buffer,
/// so H264/GameStream/PyroWave sessions must never see NV12 frames.
pub native_nv12_session: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
@@ -395,16 +351,6 @@ pub type FrameChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync,
>;
/// Delivery closure for the v5 hardware-cursor channel (`IOCTL_SET_CURSOR_CHANNEL`) — same
/// facade contract as [`FrameChannelSender`]. `Some` also OPTS THE SESSION IN: the capturer
/// creates + delivers the cursor section only when the host hands it a sender (the negotiated
/// cursor-forward sessions), and the driver only declares the hardware cursor once that
/// delivery lands — so a plain session keeps DWM's composited pointer untouched.
#[cfg(target_os = "windows")]
pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")]
pub mod pwinit;
@@ -462,7 +408,6 @@ pub fn open_virtual_output(
allow_zerocopy: bool,
want_444: bool,
policy: ZeroCopyPolicy,
expect_exact_dims: bool,
) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::from_virtual_output(
remote_fd,
@@ -472,7 +417,6 @@ pub fn open_virtual_output(
allow_zerocopy,
want_444,
policy,
expect_exact_dims,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
@@ -490,7 +434,6 @@ pub fn open_idd_push(
pyrowave: bool,
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(
target,
@@ -500,7 +443,6 @@ pub fn open_idd_push(
pyrowave,
keepalive,
sender,
cursor_sender,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
+122 -435
View File
@@ -22,10 +22,6 @@
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy};
use anyhow::{anyhow, Context, Result};
// gamescope cursor source (remote-desktop-sweep Phase C) — feeds `cursor_live` from XFixes when
// the PipeWire node carries no `SPA_META_Cursor` (gamescope's does not).
mod xfixes_cursor;
use std::os::fd::OwnedFd;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, TryRecvError};
@@ -61,12 +57,6 @@ pub struct PortalCapturer {
/// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream
/// is `Streaming`.
stall_since: Option<std::time::Instant>,
/// The LIVE cursor overlay, published by the PipeWire thread from every buffer's
/// `SPA_META_Cursor` — including the cursor-only "corrupted" buffers that never become
/// frames. [`Capturer::cursor`] serves it so the encode loop's forwarder tracks pointer-only
/// motion on a static desktop; the frame-attached overlay alone goes stale between damage
/// frames (the same gap the Windows IddCx channel fills, and why the tick prefers LIVE).
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
/// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If
/// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
@@ -92,12 +82,6 @@ pub struct PortalCapturer {
/// is, releasing the compositor-side output via the keepalive's own `Drop`. `None` for the
/// portal source (its session ends with the portal thread's zbus connection).
_keepalive: Option<Box<dyn Send>>,
/// The gamescope XFixes cursor reader (remote-desktop-sweep Phase C), when this capturer
/// serves a gamescope node. `Some` after
/// [`attach_gamescope_cursor`](Capturer::attach_gamescope_cursor); its `Drop` stops the reader
/// thread, so it lives exactly as long as the capturer. `None` on the portal path (its cursor
/// comes from `SPA_META_Cursor`).
_gs_cursor: Option<xfixes_cursor::XFixesCursorSource>,
}
impl PortalCapturer {
@@ -130,17 +114,10 @@ impl PortalCapturer {
"ScreenCast portal session started; connecting PipeWire"
);
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok(spawn_pipewire(
Some(fd),
node_id,
None,
true,
false,
want_hdr,
policy,
false,
)?
.into_capturer(node_id, None))
Ok(
spawn_pipewire(Some(fd), node_id, None, true, false, want_hdr, policy)?
.into_capturer(node_id, None),
)
}
/// Build a capturer from an already-created virtual output's PipeWire node. The host facade
@@ -160,13 +137,11 @@ impl PortalCapturer {
allow_zerocopy: bool,
want_444: bool,
policy: ZeroCopyPolicy,
expect_exact_dims: bool,
) -> Result<PortalCapturer> {
tracing::info!(
node_id,
allow_zerocopy,
want_444,
expect_exact_dims,
"connecting PipeWire to virtual output"
);
// Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit
@@ -179,7 +154,6 @@ impl PortalCapturer {
want_444,
false,
policy,
expect_exact_dims,
)?
.into_capturer(node_id, Some(keepalive)))
}
@@ -202,8 +176,6 @@ struct PwHandles {
hdr_offer: bool,
/// See [`PortalCapturer::hdr_negotiated`].
hdr_negotiated: Arc<AtomicBool>,
/// See [`PortalCapturer::cursor_live`].
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>,
}
@@ -223,12 +195,10 @@ impl PwHandles {
vaapi_dmabuf: self.vaapi_dmabuf,
hdr_offer: self.hdr_offer,
hdr_negotiated: self.hdr_negotiated,
cursor_live: self.cursor_live,
node_id,
quit: Some(self.quit),
join: Some(self.join),
_keepalive: keepalive,
_gs_cursor: None,
}
}
}
@@ -236,7 +206,6 @@ impl PwHandles {
/// Spawn the PipeWire consumer thread for `node_id` (fd `Some` = portal remote, `None` =
/// default daemon) and return its [`PwHandles`]. `preferred` seeds the format negotiation's
/// default size/framerate — for Mutter virtual monitors this is what actually sizes the monitor.
#[allow(clippy::too_many_arguments)]
fn spawn_pipewire(
fd: Option<OwnedFd>,
node_id: u32,
@@ -255,12 +224,6 @@ fn spawn_pipewire(
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6).
policy: ZeroCopyPolicy,
// The producer's FIRST negotiation is for a sacrificial mode and a renegotiation to
// `preferred`'s dims is guaranteed to follow (KWin virtual outputs — see kwin.rs `create`):
// skip whole buffers until the negotiated size matches, so the pipeline never builds against
// the doomed birth mode. `false` everywhere else (Mutter SIZES the monitor from negotiation,
// gamescope fixates its own — gating those would starve legitimate first frames).
expect_exact_dims: bool,
) -> Result<PwHandles> {
// Frames flow from the pipewire thread over a small bounded channel.
let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8);
@@ -274,8 +237,6 @@ fn spawn_pipewire(
let broken_cb = broken.clone();
let hdr_negotiated = Arc::new(AtomicBool::new(false));
let hdr_negotiated_cb = hdr_negotiated.clone();
let cursor_live = Arc::new(std::sync::Mutex::new(None::<pf_frame::CursorOverlay>));
let cursor_live_cb = cursor_live.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope.
@@ -311,14 +272,12 @@ fn spawn_pipewire(
streaming_cb,
broken_cb,
hdr_negotiated_cb,
cursor_live_cb,
zerocopy,
want_444,
want_hdr,
preferred,
quit_rx,
policy,
expect_exact_dims,
) {
tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed");
}
@@ -333,7 +292,6 @@ fn spawn_pipewire(
vaapi_dmabuf,
hdr_offer: want_hdr,
hdr_negotiated,
cursor_live,
quit: quit_tx,
join,
})
@@ -341,29 +299,29 @@ fn spawn_pipewire(
impl Capturer for PortalCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
self.frame_within(Duration::from_secs(10))
}
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
// The PipeWire thread's live cursor slot (fed by every buffer's meta, frames or not) —
// lets the forwarder track pointer-only motion on a static desktop. See `cursor_live`.
// On a gamescope node the meta never arrives; the XFixes source (attached below) fills
// the same slot instead.
self.cursor_live.lock().ok().and_then(|slot| slot.clone())
}
fn attach_gamescope_cursor(&mut self, targets: Vec<(String, Option<String>)>) {
// gamescope paints no `SPA_META_Cursor`, so `cursor_live` would stay empty. Spawn the
// XFixes reader to publish gamescope's pointer into that SAME slot — `cursor()` above then
// serves it and the encode loop composites it, exactly like the portal path. It connects
// to every nested Xwayland and follows the focused one's pointer. A failure (no Xwayland /
// no XFixes) logs and leaves the slot empty = today's cursorless gamescope.
self._gs_cursor =
xfixes_cursor::XFixesCursorSource::spawn(targets, Arc::clone(&self.cursor_live));
}
fn next_frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
self.frame_within(budget)
// First frame can lag behind format negotiation; later frames arrive at ~fps. Wait in
// short slices so a GPU-import poison (worker death) fails the capture within ~0.5 s
// instead of sitting out the full first-frame budget.
let deadline = std::time::Instant::now() + Duration::from_secs(10);
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e),
}
}
}
fn supports_arrival_wait(&self) -> bool {
@@ -459,41 +417,9 @@ impl Capturer for PortalCapturer {
}
impl PortalCapturer {
/// The blocking first-frame wait behind [`Capturer::next_frame`] /
/// [`Capturer::next_frame_within`]. First frame can lag behind format negotiation; later
/// frames arrive at ~fps. Wait in short slices so a GPU-import poison (worker death) fails
/// the capture within ~0.5 s instead of sitting out the full first-frame budget.
fn frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
let deadline = std::time::Instant::now() + budget;
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e, budget),
}
}
}
/// The [`frame_within`](Self::frame_within) budget expired (or the thread ended) — turn it
/// into the diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(
&self,
err: RecvTimeoutError,
budget: Duration,
) -> Result<CapturedFrame> {
let within = budget.as_secs_f32();
/// The [`Capturer::next_frame`] budget expired (or the thread ended) — turn it into the
/// diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(&self, err: RecvTimeoutError) -> Result<CapturedFrame> {
match err {
RecvTimeoutError::Timeout => {
// Split the two black-screen root causes apart so the operator gets a cause, not
@@ -501,10 +427,9 @@ impl PortalCapturer {
// not (no acceptable format / node never emitted a param)?
if self.negotiated.load(Ordering::Relaxed) {
Err(anyhow!(
"no PipeWire frame within {within}s (node {}): format negotiated but no \
buffers arrived the compositor produced no frames (virtual output \
idle/unmapped, capture never started, or a stream bound during a \
compositor (re)start that will never deliver a reconnect fixes that)",
"no PipeWire frame within 10s (node {}): format negotiated but no buffers \
arrived the compositor produced no frames (virtual output idle/unmapped, \
or capture never started)",
self.node_id
))
} else if self.hdr_offer {
@@ -515,10 +440,10 @@ impl PortalCapturer {
// auto-reconnects) negotiates SDR instead of re-running this same timeout.
super::note_hdr_capture_failed();
Err(anyhow!(
"no PipeWire frame within {within}s (node {}): the compositor never \
accepted the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored \
monitor in HDR mode on GNOME 50+? Downgrading this host to SDR capture; \
reconnect to stream SDR",
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored monitor in \
HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \
to stream SDR",
self.node_id
))
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
@@ -527,15 +452,14 @@ impl PortalCapturer {
// retries on the CPU offer instead of failing this same negotiation forever.
pf_zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!(
"no PipeWire frame within {within}s (node {}): the compositor never \
accepted the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this \
host to the CPU capture path; the pipeline rebuild will renegotiate \
without dmabuf",
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this host to the \
CPU capture path; the pipeline rebuild will renegotiate without dmabuf",
self.node_id
))
} else {
Err(anyhow!(
"no PipeWire frame within {within}s (node {}): format negotiation never \
"no PipeWire frame within 10s (node {}): format negotiation never \
completed the compositor offered no format this consumer accepts \
(pixel-format/modifier mismatch) or the node never emitted a Format param",
self.node_id
@@ -900,7 +824,6 @@ mod pipewire {
VideoFormat::RGBA => PixelFormat::Rgba,
VideoFormat::RGB => PixelFormat::Rgb,
VideoFormat::BGR => PixelFormat::Bgr,
VideoFormat::NV12 => PixelFormat::Nv12,
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
@@ -928,21 +851,13 @@ mod pipewire {
/// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves,
/// so the GPU encoder re-uploads its cursor texture only on change.
serial: u64,
/// The compositor-reported hotspot — carried on the overlay for the cursor-forward
/// channel (the blend path uses the pre-adjusted `x`/`y` and never reads it).
hot_x: i32,
hot_y: i32,
}
impl CursorState {
/// A shareable overlay for the encode/forward paths, or `None` before the first bitmap
/// arrived. A HIDDEN pointer still yields `Some` (with `visible: false`): the
/// cursor-forward channel needs "known but hidden" — an app grabbed the pointer, the
/// client's relative-mode hint (M3) — which is a different fact from "no cursor yet".
/// The encode loop strips invisible overlays before any blend path sees the frame.
/// Cheap: clones an `Arc` + a few scalars.
/// A shareable overlay for the GPU encode paths (blended at encode time), or `None` when
/// there is nothing to draw. Cheap: clones an `Arc` + a few scalars.
fn overlay(&self) -> Option<pf_frame::CursorOverlay> {
if self.rgba.is_empty() {
if !self.visible || self.rgba.is_empty() {
return None;
}
Some(pf_frame::CursorOverlay {
@@ -952,9 +867,6 @@ mod pipewire {
h: self.bh,
rgba: self.rgba.clone(),
serial: self.serial,
hot_x: self.hot_x.max(0) as u32,
hot_y: self.hot_y.max(0) as u32,
visible: self.visible,
})
}
}
@@ -1005,21 +917,6 @@ mod pipewire {
dbg_log_n: u64,
/// Cursor-as-metadata state, composited into the CPU de-pad path (see `consume_frame`).
cursor: CursorState,
/// LIVE overlay slot shared with [`super::PortalCapturer::cursor_live`] — refreshed after
/// every `update_cursor_meta`, including from cursor-only buffers that never become frames.
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
/// `Some((w, h))` while the producer's negotiated size is a sacrificial birth mode and a
/// renegotiation to these dims is guaranteed (KWin virtual outputs — kwin.rs `create`):
/// `.process` skips whole buffers until the negotiated size matches, then clears this
/// (self-disarming — later legitimate resizes are unaffected). `None` = no gating.
expect_dims: Option<(u32, u32)>,
/// Buffers skipped by the `expect_dims` gate (rate-limits its log).
gate_skips: u64,
/// When the gate first held a buffer — after [`GATE_DEADLINE`] with no renegotiation the
/// gate disarms and accepts what the producer serves (degraded dims beat a session wedged
/// into the first-frame-timeout retry loop; the promised renegotiation normally lands
/// within a frame or two).
gate_since: Option<std::time::Instant>,
}
/// Consecutive tiled-import failures (worker alive, e.g. a per-buffer `EGL_BAD_MATCH`) before
@@ -1092,10 +989,10 @@ mod pipewire {
.into_inner())
}
/// Build a LINEAR/modifier DMA-BUF `EnumFormat` pod. Packed BGRx is the existing import path;
/// NV12 is gamescope's producer-side RGB→YUV path (opt-in during bring-up).
/// Build a BGRx dmabuf `EnumFormat` pod advertising the EGL-importable `modifiers` as a
/// mandatory enum Choice; the compositor fixates to one of them that it can allocate, which
/// we read back in `param_changed`.
fn build_dmabuf_format(
format: VideoFormat,
modifiers: &[u64],
preferred: Option<(u32, u32, u32)>,
) -> Result<Vec<u8>> {
@@ -1106,7 +1003,7 @@ mod pipewire {
pw::spa::param::ParamType::EnumFormat,
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, VideoFormat::BGRx),
pw::spa::pod::property!(
FormatProperties::VideoSize,
Choice,
@@ -1135,22 +1032,6 @@ mod pipewire {
pw::spa::utils::Fraction { num: 240, denom: 1 }
),
);
if format == VideoFormat::NV12 {
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorMatrix,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_MATRIX_BT709,
)),
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorRange,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_RANGE_16_235,
)),
});
}
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
@@ -1398,17 +1279,10 @@ mod pipewire {
value: pw::spa::pod::Value::Choice(pw::spa::pod::ChoiceValue::Int(
pw::spa::utils::Choice(
pw::spa::utils::ChoiceFlags::empty(),
// The max must cover the producer's offer or the Meta param silently
// fails to negotiate and NO buffer ever carries the meta region:
// Mutter offers a FIXED `SPA_POD_Int(CURSOR_META_SIZE(384, 384))`
// (meta-screen-cast-stream-src.c, GNOME 50) — a 256² max made the
// intersection empty, which cost the whole Linux cursor channel
// on-glass. 1024² is headroom, not an allocation: the negotiated
// region follows the producer's value.
pw::spa::utils::ChoiceEnum::Range {
default: meta_size(64, 64),
min: meta_size(1, 1),
max: meta_size(1024, 1024),
max: meta_size(256, 256),
},
),
)),
@@ -1436,25 +1310,21 @@ mod pipewire {
/// (which Mutter delivers as metadata-only "corrupted" buffers) still refresh the position.
fn update_cursor_meta(cursor: &mut CursorState, spa_buf: *mut spa::sys::spa_buffer) {
// SAFETY: `spa_buf` is the live buffer we still hold (dequeued, not yet requeued).
// `spa_buffer_find_meta` returns the `spa_meta` (type + byte `size` + `data` pointer) for
// `SPA_META_Cursor`, or null. We take `find_meta` rather than `find_meta_data` specifically
// to obtain the region's real `size`: the bitmap offset, pixel offset and stride read below
// are ALL producer-written, and without a bound against the actual region they drive
// out-of-bounds pointer arithmetic and an oversized `slice::from_raw_parts` — an OOB read
// that SIGSEGVs inside the PipeWire `.process` callback (a segfault `catch_unwind` cannot
// catch). Every offset below is validated against `region_size` with checked arithmetic,
// mirroring the fd-length guard the main frame path already applies to xdg-desktop-portal-wlr.
let meta = unsafe { spa::sys::spa_buffer_find_meta(spa_buf, spa::sys::SPA_META_Cursor) };
if meta.is_null() {
// `spa_buffer_find_meta_data` scans its metadata array for a `SPA_META_Cursor` of at least
// `size_of::<spa_meta_cursor>()` bytes and returns a pointer into that buffer's metadata
// (or null), valid until requeue. The size argument matches the struct the result is cast to.
let cur = unsafe {
spa::sys::spa_buffer_find_meta_data(
spa_buf,
spa::sys::SPA_META_Cursor,
std::mem::size_of::<spa::sys::spa_meta_cursor>(),
) as *const spa::sys::spa_meta_cursor
};
if cur.is_null() {
return;
}
// SAFETY: `meta` is non-null and points into the held buffer's metadata array.
let (region_size, data) = unsafe { ((*meta).size as usize, (*meta).data as *const u8) };
if data.is_null() || region_size < std::mem::size_of::<spa::sys::spa_meta_cursor>() {
return;
}
let cur = data as *const spa::sys::spa_meta_cursor;
// SAFETY: `region_size >= size_of::<spa_meta_cursor>()` checked above, so every field is in bounds.
// SAFETY: `cur` is non-null and points to a `spa_meta_cursor` of at least its own size
// inside the held buffer (guaranteed by the size arg above), so every field read is in bounds.
let (id, pos_x, pos_y, hot_x, hot_y, bmp_off) = unsafe {
(
(*cur).id,
@@ -1466,35 +1336,24 @@ mod pipewire {
)
};
if id == 0 {
// SPA contract: id 0 = "no cursor information", NOT "cursor hidden". Mutter only
// REWRITES a buffer's meta region when the cursor changed, so recycled buffers
// between damage frames carry a stale id-0 meta — treating that as hidden flickered
// the cursor off between hovers (on-glass round 5). Keep the last-known state; a
// pointer that really left/hid simply stops producing updates. (The M3 hidden hint
// loses its Mutter signal — Windows has its own CURSOR_SUPPRESSED source.)
// Compositor reports no visible pointer (e.g. a game grabbed/hid it).
cursor.visible = false;
return;
}
cursor.visible = true;
cursor.x = pos_x - hot_x;
cursor.y = pos_y - hot_y;
cursor.hot_x = hot_x;
cursor.hot_y = hot_y;
if bmp_off == 0 {
// Position-only update — keep the cached bitmap.
return;
}
let bmp_off = bmp_off as usize;
// The `spa_meta_bitmap` header must fit entirely inside the region before we read it —
// `bitmap_offset` is producer-controlled and otherwise reads past the metadata.
match bmp_off.checked_add(std::mem::size_of::<spa::sys::spa_meta_bitmap>()) {
Some(end) if end <= region_size => {}
_ => return,
}
// SAFETY: `bmp_off + size_of::<spa_meta_bitmap>() <= region_size` (checked directly above),
// so the header is fully in bounds; the producer places it aligned as before.
let bmp = unsafe { data.add(bmp_off) as *const spa::sys::spa_meta_bitmap };
// SAFETY: `bmp` is the in-bounds `spa_meta_bitmap` header validated just above; reading its
// scalar fields is sound.
// SAFETY: `bitmap_offset` is a byte offset from `cur` to a `spa_meta_bitmap`, which the
// producer placed inside the same meta region it sized for this cursor (>= the size we
// requested). The resulting pointer is in bounds and aligned for `spa_meta_bitmap`.
let bmp =
unsafe { (cur as *const u8).add(bmp_off as usize) as *const spa::sys::spa_meta_bitmap };
// SAFETY: `bmp` is the in-bounds, aligned `spa_meta_bitmap` pointer computed just above; the
// producer fully initialized this header, so reading its scalar fields is sound.
let (vfmt, bw, bh, stride, pix_off) = unsafe {
(
(*bmp).format,
@@ -1504,34 +1363,16 @@ mod pipewire {
(*bmp).offset as usize,
)
};
// Ignore empty or implausibly large bitmaps (the meta-size request covers <= 1024×1024;
// real cursors are ≤96px — the cursor channel downscales >120px for the wire anyway).
if bw == 0 || bh == 0 || bw > 1024 || bh > 1024 {
// Ignore empty or implausibly large bitmaps (we requested <= 256×256).
if bw == 0 || bh == 0 || bw > 256 || bh > 256 {
return;
}
let row = bw as usize * 4;
let stride = if stride < row { row } else { stride };
// `span` is the exact byte extent the strided loop reads: `stride·(bh-1) + row`. Compute it
// with checked arithmetic (a producer stride near `i32::MAX` would otherwise overflow) and
// require the whole pixel block `[bmp_off + pix_off, +span)` to lie inside the region before
// fabricating the slice — this is the check whose absence made the read go out of bounds.
let span = match stride
.checked_mul(bh as usize - 1)
.and_then(|v| v.checked_add(row))
{
Some(s) => s,
None => return,
};
match bmp_off
.checked_add(pix_off)
.and_then(|v| v.checked_add(span))
{
Some(end) if end <= region_size => {}
_ => return,
}
// SAFETY: `bmp_off + pix_off + span <= region_size` (checked directly above), so the slice
// is fully within the producer's meta region; `span` is exactly the strided loop's extent.
let src = unsafe { std::slice::from_raw_parts(data.add(bmp_off + pix_off), span) };
let span = stride * (bh as usize - 1) + row;
// SAFETY: the bitmap pixels live at `bmp + pix_off` for `span` bytes, within the
// producer-sized meta region. `span` is the exact extent the strided copy below reads.
let src = unsafe { std::slice::from_raw_parts((bmp as *const u8).add(pix_off), span) };
let mut rgba = vec![0u8; bw as usize * bh as usize * 4];
for y in 0..bh as usize {
for x in 0..bw as usize {
@@ -1737,8 +1578,8 @@ mod pipewire {
}
}
// Raw DMA-BUF passthrough: packed RGB is imported for GPU CSC; producer-native NV12 can
// be consumed by the Vulkan Video encoder without another color conversion.
// VAAPI zero-copy passthrough: hand the raw dmabuf straight to the encoder, which imports
// it into a VA surface and does RGB→NV12 on the GPU video engine. No CUDA importer here.
if ud.vaapi_passthrough {
if let Some(fmt) = ud.format {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
@@ -1746,41 +1587,9 @@ mod pipewire {
let chunk = datas[0].chunk();
let offset = chunk.offset();
let stride = chunk.stride().max(0) as u32;
// Native NV12 usually arrives as a two-plane SPA buffer over ONE buffer
// object; plane 1's chunk carries the REAL UV offset/stride (compositors
// may align the Y plane before UV). Pass it through instead of assuming
// contiguity. Each spa_data holds its own (dup'd) fd, so BO identity is
// by inode, not fd number; a genuinely two-BO frame cannot travel through
// the single-fd import — drop it with a diagnosis instead of streaming
// garbage chroma.
let plane1 =
if fmt == PixelFormat::Nv12 && datas.len() >= 2 && datas[1].fd() > 0 {
// SAFETY: zeroed `libc::stat` is a valid POD initializer; both fds are
// owned by the live PipeWire buffer for this callback, and `fstat`
// only writes the out-param structs, whose fields are read only after
// the `== 0` success checks.
let same_bo = unsafe {
let mut s0: libc::stat = std::mem::zeroed();
let mut s1: libc::stat = std::mem::zeroed();
libc::fstat(datas[0].fd() as i32, &mut s0) == 0
&& libc::fstat(datas[1].fd() as i32, &mut s1) == 0
&& (s0.st_dev, s0.st_ino) == (s1.st_dev, s1.st_ino)
};
if !same_bo {
warn_once(
"NV12 planes live in different buffer objects — frames \
dropped (single-fd import only)",
);
return;
}
let c1 = datas[1].chunk();
Some((c1.offset(), c1.stride().max(0) as u32))
} else {
None
};
// dup the fd so it survives the SPA buffer recycle — the encode thread
// imports it. Content stability across the brief import/encode window relies
// on the compositor's buffer-pool depth, like any zero-copy capture.
// imports it. (Content stability across the brief map+CSC window relies on
// the compositor's buffer-pool depth, like any zero-copy capture.)
// SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid
// for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd,
// touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1).
@@ -1807,10 +1616,9 @@ mod pipewire {
modifier: ud.modifier,
offset,
stride,
plane1,
}),
// Cursor-as-metadata is blended only by RGB→NV12 backends. Gamescope
// embeds its pointer in the produced pixels, so native NV12 has none.
// Cursor-as-metadata: the encoder blends this into its owned VA
// surface (raw dmabuf never touched).
cursor: ud.cursor.overlay(),
});
static ONCE: std::sync::atomic::AtomicBool =
@@ -1821,12 +1629,7 @@ mod pipewire {
h,
modifier = ud.modifier,
fourcc = format_args!("{:#010x}", fourcc),
source = if fmt == PixelFormat::Nv12 {
"producer-native NV12"
} else {
"packed RGB (encoder GPU CSC)"
},
"zero-copy: handing the raw DMA-BUF to the encoder"
"zero-copy: handing the raw dmabuf to the encoder (GPU import + CSC)"
);
}
return;
@@ -2106,9 +1909,6 @@ mod pipewire {
streaming: Arc<AtomicBool>,
broken: Arc<AtomicBool>,
hdr_negotiated: Arc<AtomicBool>,
// LIVE cursor publisher (see `PortalCapturer::cursor_live`): refreshed from every
// dequeued buffer's cursor meta, frames or not.
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
zerocopy: bool,
// 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert.
want_444: bool,
@@ -2120,9 +1920,6 @@ mod pipewire {
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6).
policy: ZeroCopyPolicy,
// See `spawn_pipewire`: the first negotiation is for a sacrificial mode; hold frames
// until the producer renegotiates to `preferred`'s dims.
expect_exact_dims: bool,
) -> Result<()> {
crate::pwinit::ensure_init();
@@ -2192,28 +1989,6 @@ mod pipewire {
// PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw
// dmabuf straight to the encoder.
let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough;
// Producer-side NV12 (default-on; PUNKTFUNK_PIPEWIRE_NV12=0 escapes): gamescope offers a
// one-fd LINEAR NV12 image when the consumer asks — its compositor pass does the RGB→YUV,
// and the Vulkan Video encoder imports the buffer as its encode source directly (no host
// CSC at all). `native_nv12_session` restricts this to sessions whose encoder can ingest
// it (Linux vulkan-encode H265/AV1 — never H264/libav-VAAPI, GameStream-resolve, or
// PyroWave, whose Vulkan compute CSC ingests packed RGB only). Raw passthrough is
// required because the CUDA importer expects packed RGB, and 4:4:4/HDR must not be
// silently subsampled/downconverted. Non-NV12 compositors (KWin/GNOME) simply match the
// packed-RGB fallback pod.
let prefer_native_nv12 = std::env::var("PUNKTFUNK_PIPEWIRE_NV12").as_deref() != Ok("0")
&& policy.native_nv12_session
&& backend_is_vaapi
&& vaapi_passthrough
&& !policy.pyrowave_session
&& !want_444
&& !want_hdr;
if prefer_native_nv12 {
tracing::info!(
"zero-copy: preferring gamescope producer-side NV12 LINEAR DMA-BUF (no host \
RGB CSC; PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation)"
);
}
// Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR
// (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via
// CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only:
@@ -2253,9 +2028,7 @@ mod pipewire {
tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path");
} else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() {
tracing::info!(
native_nv12_preferred = prefer_native_nv12,
"zero-copy: advertising LINEAR DMA-BUF for encoder import (native NV12 first \
when enabled, packed RGB fallback)"
"zero-copy: advertising LINEAR dmabuf for direct VAAPI import (GPU CSC)"
);
} else if want_dmabuf && !vaapi_passthrough {
tracing::info!(
@@ -2307,14 +2080,6 @@ mod pipewire {
linear_nv12_failed: false,
dbg_log_n: 0,
cursor: CursorState::default(),
cursor_live,
expect_dims: if expect_exact_dims {
preferred.map(|(w, h, _)| (w, h))
} else {
None
},
gate_skips: 0,
gate_since: None,
};
let stream = pw::stream::StreamBox::new(
@@ -2399,91 +2164,36 @@ mod pipewire {
}
})
.process(|stream, ud| {
// Latest-frame-only (OBS pattern): Mutter delivers buffers in bursts and recycles its
// pool; an older queued buffer carries a STALE frame. Drain all queued buffers, requeue
// the older ones, keep only the newest. This dequeue/requeue runs OUTSIDE the
// `catch_unwind` below — they are non-panicking C FFI pointer ops, and `newest` is
// requeued exactly once AFTER the panic-containing region. Previously the whole thing was
// inside the catch, so a caught panic (in `update_cursor_meta`/`consume_frame`) stranded
// `newest` forever, permanently shrinking the stream's fixed pool until capture wedged.
// SAFETY: `stream` is the live stream PipeWire passes into this `.process` callback on the
// loop thread; `dequeue_raw_buffer` returns a stream-owned `*mut pw_buffer` or null
// (null-checked), single-threaded so no concurrent access.
let mut newest = unsafe { stream.dequeue_raw_buffer() };
if newest.is_null() {
return;
}
let mut drained = 1u32;
loop {
// SAFETY: same stream/loop-thread contract; returns the next stream-owned buffer or null.
let next = unsafe { stream.dequeue_raw_buffer() };
if next.is_null() {
break;
}
// SAFETY: `newest` was dequeued from this stream and not yet requeued; we immediately
// overwrite it, so the requeued pointer is never touched again.
unsafe { stream.queue_raw_buffer(newest) };
newest = next;
drained += 1;
}
// Sacrificial-mode gate (kwin.rs `create`): until the producer renegotiates to the
// expected dims, every buffer — frame AND cursor meta, whose positions are in the
// doomed mode's space — belongs to the birth mode; consuming one would build the
// pipeline at the wrong size. Self-disarms on the first matching negotiation, or
// after `GATE_DEADLINE` without one — degraded dims beat wedging the session into
// the first-frame-timeout retry loop when the promised renegotiation never comes.
if let Some((ew, eh)) = ud.expect_dims {
/// The renegotiation normally lands within a frame or two of recording; well
/// past that, the producer is not going to deliver it (the on-glass case: the
/// real mode never actually applied) — stop starving the pipeline.
const GATE_DEADLINE: std::time::Duration = std::time::Duration::from_secs(3);
let sz = ud.info.size();
if sz.width == ew && sz.height == eh {
tracing::info!(
skipped = ud.gate_skips,
width = ew,
height = eh,
"producer renegotiated to the expected mode — frames flow"
);
ud.expect_dims = None;
} else if ud
.gate_since
.get_or_insert_with(std::time::Instant::now)
.elapsed()
> GATE_DEADLINE
{
tracing::warn!(
negotiated_w = sz.width,
negotiated_h = sz.height,
expected_w = ew,
expected_h = eh,
skipped = ud.gate_skips,
"producer never renegotiated to the expected mode — accepting its \
dims (session runs degraded rather than wedged)"
);
ud.expect_dims = None;
} else {
ud.gate_skips += 1;
if ud.gate_skips == 1 || ud.gate_skips.is_power_of_two() {
tracing::info!(
negotiated_w = sz.width,
negotiated_h = sz.height,
expected_w = ew,
expected_h = eh,
n = ud.gate_skips,
"holding frames until the producer renegotiates to the expected mode"
);
}
// SAFETY: `newest` was dequeued from this stream and not yet requeued;
// requeued exactly once here, then never touched (mirrors the null path).
unsafe { stream.queue_raw_buffer(newest) };
// PipeWire dispatches this from a C trampoline with no catch_unwind; a
// panic crossing that FFI boundary would abort the whole host. Contain it.
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
// Latest-frame-only (OBS pattern): Mutter delivers buffers in bursts and
// recycles its pool; an older queued buffer carries a STALE frame. Drain all
// queued buffers, requeue the older ones, keep only the newest.
// SAFETY: `stream` is the live stream PipeWire passes into this `.process` callback on
// the loop thread, where `pw_stream_dequeue_buffer` is the documented call. It returns
// a `*mut pw_buffer` owned by the stream (or null when the queue is drained),
// null-checked before any use. The loop is single-threaded, so no concurrent access.
let mut newest = unsafe { stream.dequeue_raw_buffer() };
if newest.is_null() {
return;
}
}
// PipeWire dispatches from a C trampoline with no catch_unwind; a panic crossing that FFI
// boundary would abort the whole host. Contain the inspect/consume work — the only Rust
// code here that can panic — and requeue `newest` unconditionally after it.
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let mut drained = 1u32;
loop {
// SAFETY: same stream/loop-thread contract as the dequeue above; each call returns
// the next stream-owned `*mut pw_buffer` or null (null-checked before use).
let next = unsafe { stream.dequeue_raw_buffer() };
if next.is_null() {
break;
}
// SAFETY: `newest` is a non-null `*mut pw_buffer` previously dequeued from this same
// stream and not yet requeued; `pw_stream_queue_buffer` hands ownership back to the
// stream. We immediately overwrite `newest = next`, so the requeued pointer is never
// touched again (no use-after-requeue). Loop thread, single-threaded.
unsafe { stream.queue_raw_buffer(newest) };
newest = next;
drained += 1;
}
// SAFETY: `newest` is the non-null buffer we still own (dequeued, not requeued);
// `.buffer` is a `*mut spa_buffer` field libpipewire populated. This is a single field
// load through a valid pointer — no mutation or aliasing.
@@ -2493,19 +2203,6 @@ mod pipewire {
// pointer-only movements as metadata-only "corrupted" buffers we drop for their
// frame, but their cursor meta is fresh and must still move our overlay.
update_cursor_meta(&mut ud.cursor, spa_buf);
// Publish the LIVE overlay (frames or not) so the encode loop's forwarder
// tracks pointer-only motion on a static desktop — the frame-attached overlay
// alone stales between damage frames. ONLY when we actually have one: a
// gamescope node carries no `SPA_META_Cursor`, so `overlay()` is always `None`
// here, and writing that would clobber — at frame rate — the `Some` the
// attached XFixes source publishes into this SAME slot, strobing the
// composited pointer on/off. Portal cursors are `None` only before the first
// bitmap (nothing to drop), and a HIDDEN pointer is still `Some(visible:false)`.
if let Some(overlay) = ud.cursor.overlay() {
if let Ok(mut slot) = ud.cursor_live.lock() {
*slot = Some(overlay);
}
}
// Inspect the newest buffer's header + first chunk for the diagnostic and the
// CORRUPTED skip. SPA_META_Header is optional — `hdr` may be null.
@@ -2575,18 +2272,19 @@ mod pipewire {
"capture: skipped a stale CORRUPTED/cursor buffer (GNOME)"
);
}
// Skip this stale/cursor buffer — `newest` is requeued unconditionally below.
// SAFETY: `newest` is the non-null buffer we own (dequeued, never requeued on this
// skip path); hand it back to the stream exactly once and return without touching it
// again. Loop thread inside `.process`.
unsafe { stream.queue_raw_buffer(newest) };
return;
}
consume_frame(ud, spa_buf);
// SAFETY: `consume_frame` has finished reading `spa_buf` (and the `datas` borrows derived
// from `newest`), so requeuing the owned `newest` exactly once here is sound — no
// use-after-requeue. Loop thread inside `.process`.
unsafe { stream.queue_raw_buffer(newest) };
}));
// Hand `newest` back to the stream exactly once, on EVERY path — normal, corrupted-skip,
// or a caught panic in the closure above. This single requeue is what keeps the fixed
// buffer pool from draining.
// SAFETY: all reads of `spa_buf`/`newest` (update_cursor_meta, consume_frame) completed
// inside the closure above; `newest` was dequeued from this stream and not yet requeued.
unsafe { stream.queue_raw_buffer(newest) };
if outcome.is_err() {
// In the per-frame `.process` callback: a deterministic panic (e.g. a bad
// format) would fire this every frame, so power-of-two throttle it — enough to
@@ -2670,18 +2368,7 @@ mod pipewire {
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
]
} else if want_dmabuf {
let mut pods = Vec::with_capacity(if prefer_native_nv12 { 2 } else { 1 });
if prefer_native_nv12 {
// First compatible consumer pod wins. Gamescope advertises NV12 and BGRx; pinning
// BT.709 limited here selects its RGB→NV12 shader with our bitstream colorimetry.
pods.push(build_dmabuf_format(VideoFormat::NV12, &[0], preferred)?);
}
pods.push(build_dmabuf_format(
VideoFormat::BGRx,
&modifiers,
preferred,
)?);
pods
vec![build_dmabuf_format(&modifiers, preferred)?]
} else {
vec![serialize_pod(obj)?]
};
@@ -1,366 +0,0 @@
//! XFixes cursor source for the gamescope capture path (remote-desktop-sweep Phase C).
//!
//! gamescope draws the pointer on a DRM hardware-cursor plane and its `paint_pipewire()`
//! deliberately excludes the cursor from the frame it feeds its built-in PipeWire node — so
//! `SPA_META_Cursor` never arrives and the ordinary [`cursor_live`](super::PortalCapturer) slot
//! stays empty (a KWin/GNOME session gets its cursor from that meta; gamescope can't embed one
//! either, its `set_hw_cursor` is inert). We instead read the pointer from gamescope's nested
//! Xwayland via X11 — the trick Sunshine uses — and publish a [`CursorOverlay`] into that same
//! slot, so the encoder blend composites the pointer into the video exactly like the portal path.
//!
//! **Multiple Xwaylands.** gamescope runs one Xwayland per `--xwayland-count` (Steam Gaming Mode
//! uses 2: one for Big Picture, one for the game). The pointer lives on whichever is FOCUSED — an
//! inactive display's pointer is frozen. So the source connects to ALL of them and each tick
//! follows the one whose pointer actually moved (gamescope routes input to the focused surface, so
//! exactly one moves at a time). It reads that display's shape too, since each Xwayland has its own
//! current cursor. This is why a single-display read froze the pointer the moment a game on the
//! OTHER Xwayland took focus.
//!
//! Two X sources per display, split by cost (Sunshine's split):
//! * **Position** — core `QueryPointer` on the root, polled fast. Cheap (a few-byte reply, no
//! bitmap), so it can out-pace the stream fps and keep the composited pointer smooth. It also
//! doubles as the focus signal (the display whose pointer moves is the active one).
//! * **Shape / hotspot / visibility** — `XFixesGetCursorImage`, refreshed only after an XFixes
//! `CursorNotify` (a real cursor change). A game hiding the pointer IS a cursor change → the
//! image comes back fully transparent → `visible: false`, which the encode loop strips before
//! any blend path draws it (so a grabbed pointer shows nothing, matching native gamescope).
use std::sync::{
atomic::{AtomicBool, Ordering},
Arc, Mutex,
};
use std::time::Duration;
use pf_frame::CursorOverlay;
use x11rb::connection::Connection;
use x11rb::errors::ReplyError;
use x11rb::protocol::xfixes::{self, ConnectionExt as _, GetCursorImageReply};
use x11rb::protocol::xproto::{ConnectionExt as _, QueryPointerReply, Window};
use x11rb::rust_connection::RustConnection;
/// Serializes the brief `XAUTHORITY` env swap around a connect (the var is process-global). Only
/// ever contended if two gamescope sessions start at once — rare, and the swap is microseconds.
static XAUTH_LOCK: Mutex<()> = Mutex::new(());
/// Position out-paces the stream fps (`POLL`); shape rides `CursorNotify` events drained each tick.
/// 4 ms ≈ 250 Hz matches the Windows GDI poller — the polled position IS the composited position
/// and must out-run a 240 fps session or the pointer stutters.
const POLL: Duration = Duration::from_millis(4);
/// A running XFixes cursor reader. Dropping it stops the worker thread and joins it, releasing the
/// X connections — so it lives exactly as long as the capturer that owns it.
pub(super) struct XFixesCursorSource {
stop: Arc<AtomicBool>,
join: Option<std::thread::JoinHandle<()>>,
}
impl XFixesCursorSource {
/// Connect to every gamescope nested Xwayland in `targets` (`(DISPLAY, XAUTHORITY)`) and start
/// publishing cursor overlays into `slot`, following the focused display's pointer. Returns
/// `None` — and logs — if NONE can be used (no X connection / no XFixes), so the caller
/// degrades to no gamescope cursor (today's behaviour) instead of failing the session.
pub(super) fn spawn(
targets: Vec<(String, Option<String>)>,
slot: Arc<Mutex<Option<CursorOverlay>>>,
) -> Option<Self> {
// Connect on the caller's thread so failures degrade cleanly and the displays are validated
// before we commit a thread.
let mut displays = Vec::new();
for (dpy, xauth) in targets {
match connect(&dpy, xauth.as_deref()) {
Ok((conn, root)) => displays.push(XDisplay::new(dpy, conn, root)),
Err(e) => tracing::warn!(
dpy = %dpy,
error = %e,
"gamescope cursor: skipping a nested Xwayland we can't use"
),
}
}
if displays.is_empty() {
tracing::warn!(
"gamescope cursor: no usable nested Xwayland — no in-video pointer this session \
(falls back to today's cursorless gamescope stream)"
);
return None;
}
let names: Vec<&str> = displays.iter().map(|d| d.name.as_str()).collect();
tracing::info!(
displays = ?names,
"gamescope cursor: XFixes source live — following the focused Xwayland's pointer"
);
let stop = Arc::new(AtomicBool::new(false));
let stop_worker = Arc::clone(&stop);
let join = std::thread::Builder::new()
.name("pf-gs-cursor".into())
.spawn(move || run(displays, slot, stop_worker))
.ok()?;
Some(XFixesCursorSource {
stop,
join: Some(join),
})
}
}
impl Drop for XFixesCursorSource {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(j) = self.join.take() {
let _ = j.join();
}
}
}
/// Open the X connection, negotiate XFixes, and select cursor-change events — returning the
/// connection + root window. `RustConnection` reads `XAUTHORITY` from the env at connect time only,
/// so set it under the lock (the host isn't a gamescope child), connect, then restore.
fn connect(dpy: &str, xauthority: Option<&str>) -> Result<(RustConnection, Window), String> {
let (conn, screen_num) = {
let _g = XAUTH_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let prev = std::env::var_os("XAUTHORITY");
if let Some(x) = xauthority {
std::env::set_var("XAUTHORITY", x);
}
let out = RustConnection::connect(Some(dpy));
match (&prev, xauthority) {
(Some(p), _) => std::env::set_var("XAUTHORITY", p),
(None, Some(_)) => std::env::remove_var("XAUTHORITY"),
(None, None) => {}
}
out.map_err(|e| format!("connect: {e}"))?
};
// XFixes ≥ 1 gives GetCursorImage / SelectCursorInput; ask for a modern minor, take what we get.
conn.xfixes_query_version(5, 0)
.map_err(ReplyError::from)
.and_then(|c| c.reply())
.map_err(|e| format!("XFixes unavailable: {e}"))?;
let root = conn
.setup()
.roots
.get(screen_num)
.ok_or_else(|| format!("no X screen {screen_num}"))?
.root;
// Wake the worker's event drain whenever the cursor shape changes (incl. hide/show).
conn.xfixes_select_cursor_input(root, xfixes::CursorNotifyMask::DISPLAY_CURSOR)
.map_err(ReplyError::from)
.and_then(|c| c.check())
.map_err(|e| format!("SelectCursorInput: {e}"))?;
let _ = conn.flush();
Ok((conn, root))
}
/// One gamescope Xwayland the source tracks.
struct XDisplay {
name: String,
conn: RustConnection,
root: Window,
/// Last polled pointer position — a change since the previous tick marks this display FOCUSED.
last_pos: Option<(i32, i32)>,
/// Cached cursor shape, refreshed only after this display's XFixes `CursorNotify`.
shape: Shape,
/// A `CursorNotify` (or first read) is pending — fetch the shape when this display is active.
need_shape: bool,
/// The X connection died (game/Xwayland exited) — skip it.
dead: bool,
}
impl XDisplay {
fn new(name: String, conn: RustConnection, root: Window) -> Self {
XDisplay {
name,
conn,
root,
last_pos: None,
shape: Shape::default(),
need_shape: true,
dead: false,
}
}
}
/// Cached cursor shape for one display.
#[derive(Default)]
struct Shape {
/// Straight-alpha RGBA (`w*h*4`, bytes R,G,B,A); empty before the first image arrives.
rgba: Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
/// XFixes' own per-display cursor serial — bumps on every shape change.
serial: u64,
/// A hidden pointer arrives as an all-transparent image; kept so a position-only tick preserves
/// the last known visibility.
visible: bool,
}
fn run(
mut displays: Vec<XDisplay>,
slot: Arc<Mutex<Option<CursorOverlay>>>,
stop: Arc<AtomicBool>,
) {
let mut active = 0usize;
// The overlay serial must bump whenever the DRAWN cursor changes — either the active display's
// shape OR which display is active (per-display XFixes serials aren't comparable across
// displays, so switching could reuse a number and the encoder would keep the old texture).
let mut out_serial = 0u64;
let mut last_key = (usize::MAX, u64::MAX);
let mut warned_image = false;
while !stop.load(Ordering::Relaxed) {
// 1) Poll every display's pointer; note which moved since last tick (the focus signal).
let mut active_moved = false;
let mut other_moved: Option<usize> = None;
for (i, d) in displays.iter_mut().enumerate() {
if d.dead {
continue;
}
// Drain pending events; the only ones selected are CursorNotify, so ANY event means
// "re-read this display's shape". poll_for_event never blocks.
loop {
match d.conn.poll_for_event() {
Ok(Some(_)) => d.need_shape = true,
Ok(None) => break,
Err(_) => {
d.dead = true;
break;
}
}
}
match fetch_pointer(&d.conn, d.root) {
Ok(p) if p.same_screen => {
let pos = (i32::from(p.root_x), i32::from(p.root_y));
let moved = d.last_pos.is_some_and(|lp| lp != pos);
d.last_pos = Some(pos);
if moved {
if i == active {
active_moved = true;
} else if other_moved.is_none() {
other_moved = Some(i);
}
}
}
Ok(_) => {} // pointer on another screen — keep the last position.
Err(_) => d.dead = true,
}
}
// 2) Switch focus: sticky to the active display while it moves (no flapping); otherwise
// follow another display that moved. If the active one died, fall to any live display.
if !active_moved {
if let Some(j) = other_moved {
active = j;
}
}
if displays.get(active).is_none_or(|d| d.dead) {
match displays.iter().position(|d| !d.dead) {
Some(k) => active = k,
None => {
std::thread::sleep(POLL); // all connections dead — idle until Drop.
continue;
}
}
}
// 3) Fetch the active display's shape if a CursorNotify (or a focus switch) left it stale.
if displays[active].need_shape {
match fetch_cursor_image(&displays[active].conn) {
Ok(img) => {
update_shape(&mut displays[active].shape, &img);
displays[active].need_shape = false;
}
Err(e) => {
if !warned_image {
warned_image = true;
tracing::warn!(error = %e, "gamescope cursor: GetCursorImage failed — retrying");
}
}
}
}
// 4) Publish the ACTIVE display's pointer + shape (or clear the slot when it has no cursor
// of its own — so a focus switch never leaves the other display's stale pointer showing).
let d = &displays[active];
let overlay = match (d.last_pos, d.shape.rgba.is_empty()) {
(Some((px, py)), false) => {
let key = (active, d.shape.serial);
if key != last_key {
out_serial += 1;
last_key = key;
}
Some(CursorOverlay {
// Top-left = pointer position hotspot (the overlay contract).
x: px - d.shape.hot_x as i32,
y: py - d.shape.hot_y as i32,
w: d.shape.w,
h: d.shape.h,
rgba: Arc::clone(&d.shape.rgba),
serial: out_serial,
hot_x: d.shape.hot_x,
hot_y: d.shape.hot_y,
visible: d.shape.visible,
})
}
_ => None,
};
if let Ok(mut s) = slot.lock() {
*s = overlay;
}
std::thread::sleep(POLL);
}
}
/// Update `shape` from a fresh `GetCursorImage` reply. A hidden pointer (all-transparent) keeps the
/// last bitmap (instant re-show) but flips visibility; the serial still bumps so the change shows.
fn update_shape(shape: &mut Shape, img: &GetCursorImageReply) {
let visible =
img.width > 0 && img.height > 0 && img.cursor_image.iter().any(|&p| (p >> 24) & 0xff != 0);
if visible {
shape.rgba = Arc::new(argb_premul_to_straight_rgba(&img.cursor_image));
shape.w = u32::from(img.width);
shape.h = u32::from(img.height);
shape.hot_x = u32::from(img.xhot);
shape.hot_y = u32::from(img.yhot);
}
shape.visible = visible;
shape.serial = u64::from(img.cursor_serial);
}
/// One request+reply — x11rb splits errors (the request is `ConnectionError`, `reply()` is
/// `ReplyError` which is `From<ConnectionError>`), so the request `?` converts into the reply error.
fn fetch_cursor_image(conn: &RustConnection) -> Result<GetCursorImageReply, ReplyError> {
conn.xfixes_get_cursor_image()?.reply()
}
fn fetch_pointer(conn: &RustConnection, root: Window) -> Result<QueryPointerReply, ReplyError> {
conn.query_pointer(root)?.reply()
}
/// XFixes cursor pixels are packed `0xAARRGGBB` with **premultiplied** alpha (the Xrender / Xcursor
/// convention). The overlay + both blend paths want **straight** alpha RGBA (R,G,B,A bytes), like
/// the `SPA_META_Cursor` path — so un-premultiply here. (If on-glass shows over-bright fringes the
/// source wasn't premultiplied after all; drop the divide.)
fn argb_premul_to_straight_rgba(argb: &[u32]) -> Vec<u8> {
let mut out = Vec::with_capacity(argb.len() * 4);
for &px in argb {
let a = (px >> 24) & 0xff;
let r = (px >> 16) & 0xff;
let g = (px >> 8) & 0xff;
let b = px & 0xff;
let (r, g, b) = match a {
0 => (0, 0, 0),
255 => (r, g, b),
a => (
((r * 255 + a / 2) / a).min(255),
((g * 255 + a / 2) / a).min(255),
((b * 255 + a / 2) / a).min(255),
),
};
out.extend_from_slice(&[r as u8, g as u8, b as u8, a as u8]);
}
out
}
+14 -211
View File
@@ -161,7 +161,7 @@ pub fn install_gpu_pref_hook() {
});
}
pub(crate) unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> {
unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> {
let mut blob: Option<ID3DBlob> = None;
let mut errs: Option<ID3DBlob> = None;
let r = D3DCompile(
@@ -194,7 +194,7 @@ pub(crate) unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> R
}
/// Fullscreen-triangle vertex shader for the HDR conversion pass (3 verts, no input layout).
pub(crate) const HDR_VS: &str = r"
const HDR_VS: &str = r"
struct VOut { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; };
VOut main(uint vid : SV_VertexID) {
float2 uv = float2((vid << 1) & 2, vid & 2);
@@ -308,9 +308,8 @@ float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
/// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`]
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format. (There is no runtime
/// fallback — the error propagates through `try_consume` and ends the session; the "R10 path" the
/// original design referenced was never kept.)
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format so the caller can fall
/// back to the existing R10 path.
pub(crate) struct HdrP010Converter {
vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader,
@@ -738,157 +737,14 @@ fn p010_reference(r: f64, g: f64, b: f64) -> (f64, f64, f64) {
/// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL.
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest() -> Result<()> {
hdr_p010_selftest_at(64, 64, None)
}
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_HARDWARE;
use windows::Win32::Graphics::Dxgi::IDXGIAdapter;
/// [`hdr_p010_selftest`] at an arbitrary even size and (optionally) on a specific GPU vendor
/// (PCI vendor id, e.g. `0x8086` Intel / `0x10de` NVIDIA / `0x1002` AMD). The size matters on
/// top of the 64×64 default because the field sessions run at capture resolutions whose height
/// is NOT 16-aligned (1080 → the encoder's align16 pool seam) and a driver may treat the planar
/// RTVs differently at real sizes; the vendor pin matters on dual-GPU boxes where the default
/// adapter is not the one the session encodes on.
/// Test support (used by pf-encode's live e2e): the 8 sRGB colour bars (white/yellow/cyan/green/
/// magenta/red/blue/black, sRGB 1.0 = scRGB 1.0 = 80 nits) as a w×h FP16 scRGB texture on the
/// adapter with `luid`, converted through the REAL [`HdrP010Converter`] into a P010 texture with
/// **`BIND_RENDER_TARGET` only, `MiscFlags` 0 — the exact bind profile of the IDD out-ring** (the
/// CPU-upload encoder tests can't use that profile, so only this path exercises "RTV-written P010
/// → encoder ingest copy"). Returns `(device, p010)`; expected decoded codes per bar are the
/// bars_pq2020 fixture's: (490,512,512) (478,423,518) (464,525,473) (450,432,476) (350,584,585)
/// (325,448,598) (226,650,535) (64,512,512).
#[cfg(target_os = "windows")]
#[doc(hidden)]
pub fn hdr_p010_convert_bars_on_luid(
luid: [u8; 8],
w: u32,
h: u32,
) -> Result<(ID3D11Device, ID3D11Texture2D)> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("bars pattern needs even non-zero dimensions, got {w}x{h}");
}
// sRGB primaries at full/zero channels: sRGB EOTF(1.0)=1.0, (0)=0 → the scRGB pattern is
// pure 0/1 floats and the PQ/BT.2020 reference codes above are exact.
const BARS: [(f32, f32, f32); 8] = [
(1.0, 1.0, 1.0),
(1.0, 1.0, 0.0),
(0.0, 1.0, 1.0),
(0.0, 1.0, 0.0),
(1.0, 0.0, 1.0),
(1.0, 0.0, 0.0),
(0.0, 0.0, 1.0),
(0.0, 0.0, 0.0),
];
let bar_w = (w / 8).max(1) as usize;
let mut fp16 = vec![0u16; (w * h * 4) as usize];
for y in 0..h as usize {
for x in 0..w as usize {
let (r, g, b) = BARS[(x / bar_w).min(7)];
let i = (y * w as usize + x) * 4;
fp16[i] = f32_to_f16(r);
fp16[i + 1] = f32_to_f16(g);
fp16[i + 2] = f32_to_f16(b);
fp16[i + 3] = f32_to_f16(1.0);
}
}
// SAFETY: same single-device/single-thread contract as `hdr_p010_selftest_at`; the FP16
// initial-data Vec outlives the synchronous CreateTexture2D; the returned COM handles own
// their references.
unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("dxgi factory")?;
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).context("adapter by luid")?;
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice(luid) for bars convert")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
let src_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R16G16B16A16_FLOAT,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: fp16.as_ptr() as *const c_void,
SysMemPitch: w * 8,
SysMemSlicePitch: 0,
};
let mut src_tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&src_desc, Some(&init), Some(&mut src_tex))
.context("CreateTexture2D(fp16 bars)")?;
let src_tex = src_tex.context("null src tex")?;
let mut src_srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&src_tex, None, Some(&mut src_srv))
.context("CreateShaderResourceView(fp16 bars)")?;
let src_srv = src_srv.context("null src srv")?;
// The IDD out-ring's exact profile: P010, RENDER_TARGET only, MiscFlags 0.
let p010_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
..Default::default()
};
let mut p010: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&p010_desc, None, Some(&mut p010))
.context("CreateTexture2D(P010 bars dst)")?;
let p010 = p010.context("null p010 tex")?;
let conv = HdrP010Converter::new(&device)?;
conv.convert(&device, &context, &src_srv, &p010, w, h)?;
Ok((device, p010))
}
}
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_UNKNOWN};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("hdr-p010-selftest needs even non-zero dimensions, got {w}x{h}");
}
// A grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → exact chroma
// comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus a couple
// of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
#[allow(non_snake_case)]
let (W, H) = (w, h);
// 64x64, even dims. A 4x4 grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform →
// exact chroma comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus
// a couple of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
const W: u32 = 64;
const H: u32 = 64;
const BLK: u32 = 16;
// (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values.
let named: [(&str, f32, f32, f32); 8] = [
@@ -941,36 +797,12 @@ pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
// `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are
// proven individually at the `read_u16` closure below.
unsafe {
// Device on the requested vendor's adapter (dual-GPU boxes encode on a specific one), else
// the default hardware GPU. Always says which adapter ran — a PASS is only meaningful for
// the GPU it actually tested.
let adapter: Option<IDXGIAdapter> = match vendor {
None => None,
Some(want) => {
let factory: IDXGIFactory1 = CreateDXGIFactory1().context("dxgi factory")?;
let mut found = None;
for i in 0.. {
let Ok(a) = factory.EnumAdapters(i) else {
break;
};
let desc = a.GetDesc().context("adapter desc")?;
if desc.VendorId == want {
found = Some(a);
break;
}
}
Some(found.with_context(|| format!("no adapter with vendor id {want:#x}"))?)
}
};
// Hardware D3D11 device (no adapter pin — the default GPU is fine for the self-test).
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
adapter.as_ref(),
if adapter.is_some() {
D3D_DRIVER_TYPE_UNKNOWN
} else {
D3D_DRIVER_TYPE_HARDWARE
},
None::<&IDXGIAdapter>,
D3D_DRIVER_TYPE_HARDWARE,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
@@ -982,22 +814,6 @@ pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
.context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
{
let dxgi: windows::Win32::Graphics::Dxgi::IDXGIDevice =
device.cast().context("device -> IDXGIDevice")?;
let desc = dxgi.GetAdapter().context("GetAdapter")?.GetDesc()?;
let name = String::from_utf16_lossy(
&desc.Description[..desc
.Description
.iter()
.position(|&c| c == 0)
.unwrap_or(desc.Description.len())],
);
println!(
"adapter: {name} (vendor {:#06x}, luid {:08x}:{:08x})",
desc.VendorId, desc.AdapterLuid.HighPart, desc.AdapterLuid.LowPart
);
}
// Source FP16 texture (initialized) + SRV.
let src_desc = D3D11_TEXTURE2D_DESC {
@@ -1359,16 +1175,3 @@ impl VideoConverter {
blt.context("VideoProcessorBlt")
}
}
#[cfg(test)]
mod hdr_selftests {
/// LIVE (needs the GPU): [`super::hdr_p010_selftest_at`] at the field capture size — 1080 is
/// NOT 16-aligned, and the planar-RTV write path is driver-specific per vendor. Pinned to the
/// Intel adapter (`0x8086`), so it runs on the Intel validation boxes and errors out cleanly
/// ("no adapter") elsewhere. `cargo test -p pf-capture -- --ignored hdr_p010 --nocapture`.
#[test]
#[ignore]
fn hdr_p010_selftest_intel_1080_live() {
super::hdr_p010_selftest_at(1920, 1080, Some(0x8086)).expect("hdr p010 selftest @1080");
}
}
+22 -401
View File
@@ -365,12 +365,6 @@ pub unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &str) ->
#[path = "idd_push/channel.rs"]
mod channel;
#[path = "idd_push/cursor.rs"]
mod cursor;
#[path = "idd_push/cursor_blend.rs"]
mod cursor_blend;
#[path = "idd_push/cursor_poll.rs"]
mod cursor_poll;
#[path = "idd_push/descriptor.rs"]
mod descriptor;
#[path = "idd_push/stall.rs"]
@@ -392,54 +386,6 @@ pub struct IddPushCapturer {
/// The sealed channel's handle-duplication broker (WUDFHost process + control device); used at open
/// and again on every ring recreate to deliver fresh duplicates.
broker: ChannelBroker,
/// The v5 hardware-cursor channel's host end (`Some` = delivered; the driver declared the
/// hardware cursor and seqlock-publishes into it). Survives ring recreates — the section is
/// independent of the frame ring's generation. With the channel delivered, the driver's
/// hardware cursor keeps DWM from compositing ANY cursor into the frame; the SHAPE now comes
/// from [`cursor_poll::CursorPoller`] (the IddCx query is alpha-only — see cursor_poll.rs),
/// and this shm read is the fallback if that poller dies.
cursor_shared: Option<cursor::CursorShared>,
/// The GDI cursor-shape poller (design §8): the overlay source while alive. `Some` when
/// `cursor_shared` is (both ride the negotiated cursor channel + successful delivery) — or
/// when `composite_forced` (no channel, but the target's sticky declare needs a blend source).
cursor_poll: Option<cursor_poll::CursorPoller>,
/// Retained delivery sender (`IOCTL_SET_CURSOR_CHANNEL`) for RE-delivery: a driver-side
/// monitor re-arrival (match-window re-arrival resize, a sibling session recreating the
/// shared slot) destroys the driver's cursor worker — the section here survives, so the
/// channel is re-delivered on ring recreates.
cursor_sender: Option<crate::CursorChannelSender>,
/// The CAPTURE mouse model is active — the HOST composites the pointer into the frame
/// (see cursor_blend.rs for why DWM cannot: a declared IddCx hardware cursor is forever).
composite_cursor: bool,
/// This session never negotiated the cursor channel but its target carries an IRREVOCABLE
/// hardware-cursor declare from an earlier session (`WinCaptureTarget::cursor_excluded`,
/// §8.6): DWM delivers pointer-free frames and no client draws the cursor, so the ONLY path
/// to a visible pointer is compositing here. Pins `composite_cursor` on — nothing may turn
/// it off (there is no channel to hand the pointer to).
composite_forced: bool,
/// The cursor-quad blend pass (lazy; per capture device). `None` after a build failure —
/// composite mode then degrades to pointer-less frames (warned once).
cursor_blend: Option<cursor_blend::CursorBlendPass>,
cursor_blend_failed: bool,
/// The frame-sized blend scratch (slot copy + cursor quad): texture + SRV + (w, h, fmt)
/// it was built for — rebuilt when the ring geometry changes.
blend_scratch: Option<(
ID3D11Texture2D,
ID3D11ShaderResourceView,
u32,
u32,
DXGI_FORMAT,
)>,
/// The (serial, x, y, visible) of the LAST blended pointer — the composite-regen change
/// key: pointer-only motion produces no driver publish (the declared hardware cursor
/// doesn't dirty frames), so `try_consume` regenerates from the last slot when this moves.
last_blend_key: Option<(u64, i32, i32, bool)>,
/// The ring slot of the last FRESH publish — the regen source.
last_slot: Option<usize>,
/// The target's SDR-white scale (vs 80 nits) for HDR cursor compositing — refreshed on
/// each blend-scratch rebuild (first use + ring geometry changes). 2.5 ≈ the Windows
/// SDR-brightness default; without it the composited cursor renders visibly dark on HDR.
sdr_white_scale: f32,
width: u32,
height: u32,
slots: Vec<HostSlot>,
@@ -657,7 +603,6 @@ impl IddPushCapturer {
/// instead of tearing the display down (audit §5.1 — no more 20 s black bail). "Failure" includes the
/// driver not attaching to the ring within a few seconds (e.g. a hybrid-GPU render mismatch).
#[allow(clippy::too_many_arguments)]
#[allow(clippy::too_many_arguments)]
pub fn open(
target: WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
@@ -666,20 +611,11 @@ impl IddPushCapturer {
pyrowave: bool,
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
pf_win_display::display_events::spawn_once();
match Self::open_inner(
target,
preferred,
client_10bit,
want_444,
pyrowave,
sender,
cursor_sender,
) {
match Self::open_inner(target, preferred, client_10bit, want_444, pyrowave, sender) {
Ok(mut me) => {
me._keepalive = keepalive;
Ok(me)
@@ -696,7 +632,6 @@ impl IddPushCapturer {
want_444: bool,
pyrowave: bool,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -719,7 +654,6 @@ impl IddPushCapturer {
pyrowave,
luid,
sender.clone(),
cursor_sender.clone(),
) {
Ok(me) => Ok(me),
Err(e) => {
@@ -753,7 +687,6 @@ impl IddPushCapturer {
pyrowave,
drv,
sender,
cursor_sender,
)
.context("IDD-push rebind to the driver's reported render adapter")
}
@@ -769,7 +702,6 @@ impl IddPushCapturer {
pyrowave: bool,
luid: LUID,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
) -> Result<Self> {
let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -1003,50 +935,6 @@ impl IddPushCapturer {
)
.context("deliver IDD-push frame channel to the driver")?;
// v5 hardware-cursor channel (M2c): create + deliver the CursorShm section. Failure
// is NON-fatal — the driver never declares the hardware cursor without this delivery,
// so the session degrades to today's composited pointer (and the forwarder simply
// never sees a live overlay).
let cursor_shared = cursor_sender.as_ref().and_then(|send_cursor| {
match cursor::CursorShared::create(target.target_id) {
Ok(cs) => {
// Deliver via the shared helper (also used for RE-delivery after a
// driver-side monitor re-arrival destroyed the worker).
deliver_cursor_channel(&broker, target.target_id, &cs, send_cursor)
.then_some(cs)
}
Err(e) => {
tracing::warn!(
"cursor section creation failed (composited cursor stays): {e:#}"
);
None
}
}
});
// No channel this session, but the target's sticky declare (an EARLIER session's —
// irrevocable, §8.6) keeps DWM's frames pointer-free with no client drawing either:
// the only visible pointer is the one composited here, so force composite mode on.
let composite_forced = target.cursor_excluded && cursor_sender.is_none();
if composite_forced {
tracing::info!(
target_id = target.target_id,
"target carries an irrevocable hardware-cursor declare from an earlier \
desktop-mode session and this session has no cursor channel the host \
composites the pointer into frames (forced, for the session's life)"
);
}
// The GDI shape poller rides the SAME gate as the delivered channel: with the driver's
// hardware cursor keeping the frame cursor-free, the poller supplies the full-fidelity
// shape (masked/monochrome included — the IddCx query can't; see cursor_poll.rs).
// Forced-composite sessions need it too — it is their only shape/position source.
let cursor_poll = (cursor_shared.is_some() || composite_forced).then(|| {
// Safety of the CCD call: read-only QueryDisplayConfig over owned locals (same
// call CursorShared::create makes) — already inside open_on's unsafe region.
let rect = pf_win_display::win_display::source_desktop_rect(target.target_id)
.unwrap_or((0, 0, i32::MAX, i32::MAX));
cursor_poll::CursorPoller::spawn(target.target_id, rect)
});
tracing::info!(
target_id = target.target_id,
wudf_pid = target.wudf_pid,
@@ -1105,17 +993,6 @@ impl IddPushCapturer {
last_seq: 0,
last_present: None,
status_logged: false,
cursor_shared,
cursor_poll,
cursor_sender,
composite_cursor: composite_forced,
composite_forced,
cursor_blend: None,
cursor_blend_failed: false,
blend_scratch: None,
last_blend_key: None,
last_slot: None,
sdr_white_scale: 1.0,
// Held from BEFORE the first-frame gate (the display must not idle off while we
// wait for the first compose) until the capturer drops with the session.
_display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(),
@@ -1460,24 +1337,10 @@ impl IddPushCapturer {
"IDD push: frame-channel re-delivery failed after ring recreate"
);
}
// Ring recreates ride display churn that can also have re-arrived the MONITOR driver-side
// (destroying its cursor worker with it) — re-deliver the surviving cursor section so the
// hardware-cursor declaration follows the CURRENT monitor generation.
if let (Some(cs), Some(send)) = (self.cursor_shared.as_ref(), self.cursor_sender.as_ref()) {
let _ = deliver_cursor_channel(&self.broker, self.target_id, cs, send);
}
self.blend_scratch = None; // ring geometry/format changed — rebuild at next blend
self.last_slot = None; // old-ring slot indices are meaningless now
self.last_seq = 0;
self.out_ring.clear(); // the output format changed → rebuild lazily at the new format
self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode
self.hdr_p010_conv = None;
// The PyroWave CSC is mode-baked too (BgraToYuvPlanes picks different SDR vs HDR shaders
// and R8/R8G8 vs R16/R16G16 outputs). Without this, a display_hdr flip (Downgrade point D:
// client_10bit=true but HDR couldn't enable at open) reused the stale SDR converter against
// the freshly HDR-formatted pyro ring — every frame corrupted. `ensure_pyro_conv` only
// builds when None, so it must be reset here like its siblings.
self.pyro_conv = None;
self.pyro_ring.clear(); // PyroWave two-plane ring is sized → rebuild at the new mode
self.pyro_last = None;
self.out_idx = 0;
@@ -1753,136 +1616,6 @@ impl IddPushCapturer {
Ok(Some((self.pyro_fence_handle, value)))
}
/// The (serial, x, y, visible) of the CURRENT polled cursor — the composite-regen change
/// key. `None` while the poller has no shape yet (or isn't running).
fn cursor_blend_key(&self) -> Option<(u64, i32, i32, bool)> {
self.cursor_poll
.as_ref()
.and_then(|p| p.read())
.map(|o| (o.serial, o.x, o.y, o.visible))
}
/// Composite the pointer for this convert: ensure the frame-sized blend scratch, copy the
/// slot into it, and alpha-blend the GDI poller's shape at its polled position. Returns the
/// scratch (texture + SRV) the conversion should read INSTEAD of the slot; `None` degrades
/// to the pointer-less slot (scratch/pass creation failed — warned once). A hidden pointer
/// blends nothing (the plain copy is the correct frame).
///
/// # Safety
/// D3D11 calls on the owning capture/encode thread's device + immediate context, called
/// while holding the slot's keyed mutex (the copy reads the slot).
unsafe fn prepare_blend_scratch(
&mut self,
slot_tex: &ID3D11Texture2D,
) -> Option<(ID3D11Texture2D, ID3D11ShaderResourceView)> {
let fmt = self.ring_format();
// (Re)build the scratch at the current ring geometry.
let stale = self
.blend_scratch
.as_ref()
.is_none_or(|(_, _, w, h, f)| (*w, *h, *f) != (self.width, self.height, fmt));
if stale {
self.blend_scratch = None;
let desc = D3D11_TEXTURE2D_DESC {
Width: self.width,
Height: self.height,
MipLevels: 1,
ArraySize: 1,
Format: fmt,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
..Default::default()
};
let mut tex: Option<ID3D11Texture2D> = None;
let built = self
.device
.CreateTexture2D(&desc, None, Some(&mut tex))
.ok()
.and(tex)
.and_then(|t| {
let mut srv: Option<ID3D11ShaderResourceView> = None;
self.device
.CreateShaderResourceView(&t, None, Some(&mut srv))
.ok()
.and(srv)
.map(|v| (t, v))
});
match built {
Some((t, v)) => {
self.blend_scratch = Some((t, v, self.width, self.height, fmt));
if self.display_hdr {
// Where DWM places SDR white on this HDR desktop — the composited
// cursor must match or it reads dark (~2.5x at the Windows default).
// Queried only here: scratch rebuilds are rare, and the CCD query
// contends on the display-config lock, which must stay OFF the
// per-frame path.
// Safety: read-only CCD query over owned locals (within unsafe fn).
let queried =
pf_win_display::win_display::sdr_white_level_scale(self.target_id);
self.sdr_white_scale = queried.unwrap_or(self.sdr_white_scale);
tracing::info!(
target_id = self.target_id,
queried = ?queried,
applied = self.sdr_white_scale,
"cursor composite: HDR SDR-white scale (1.0 = 80 nits; None = \
query failed keeping the prior value)"
);
}
}
None => {
if !self.cursor_blend_failed {
self.cursor_blend_failed = true;
tracing::warn!(
"cursor blend scratch creation failed — capture-model frames stay \
pointer-less this session"
);
}
return None;
}
}
}
let (tex, srv, ..) = self.blend_scratch.as_ref().expect("just ensured");
let (tex, srv) = (tex.clone(), srv.clone());
self.context.CopyResource(&tex, slot_tex);
// Blend the pointer (visible shapes only; hidden = the copy alone is the frame).
let overlay = self.cursor_poll.as_ref().and_then(|p| p.read());
self.last_blend_key = overlay.as_ref().map(|o| (o.serial, o.x, o.y, o.visible));
if let Some(ov) = overlay.filter(|o| o.visible) {
if self.cursor_blend.is_none() && !self.cursor_blend_failed {
match cursor_blend::CursorBlendPass::new(&self.device) {
Ok(p) => self.cursor_blend = Some(p),
Err(e) => {
self.cursor_blend_failed = true;
tracing::warn!(
"cursor blend pass build failed — capture-model frames stay \
pointer-less this session: {e:#}"
);
}
}
}
if let Some(pass) = self.cursor_blend.as_mut() {
// FP16 ring = scRGB linear composition (HDR): linearize the sRGB shape and
// scale it to the target's SDR white so it matches the desktop around it.
let scale = if self.display_hdr {
self.sdr_white_scale
} else {
0.0
};
if let Err(e) = pass.blend(&self.device, &self.context, &tex, &ov, scale) {
if !self.cursor_blend_failed {
self.cursor_blend_failed = true;
tracing::warn!("cursor blend draw failed — pointer-less frames: {e:#}");
}
}
}
}
Some((tex, srv))
}
fn try_consume(&mut self) -> Result<Option<CapturedFrame>> {
self.log_driver_status_once();
// Follow the display: a "Use HDR" flip recreates the ring at the matching format.
@@ -1943,25 +1676,9 @@ impl IddPushCapturer {
return Ok(None);
}
let seq = u64::from(tok.seq);
let mut slot = tok.slot as usize;
let fresh = seq != self.last_seq && slot < self.slots.len();
let mut regen = false;
if !fresh {
// Composite cursor model: pointer-only motion produces NO new publish (the declared
// hardware cursor never dirties the frame), so a static desktop would freeze the
// blended pointer. Regenerate from the LAST slot whenever the polled cursor state
// changed — the re-converted out-ring frame carries the pointer's new position.
let moved = self.composite_cursor
&& self.last_slot.is_some()
&& self.cursor_blend_key() != self.last_blend_key;
if !moved {
return Ok(None);
}
slot = self.last_slot.expect("checked above");
if slot >= self.slots.len() {
return Ok(None); // ring shrank across a recreate — wait for a fresh publish
}
regen = true;
let slot = tok.slot as usize;
if seq == self.last_seq || slot >= self.slots.len() {
return Ok(None);
}
// Build the ring + converter BEFORE acquiring the slot so nothing between Acquire and Release
// can `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot).
@@ -1995,31 +1712,18 @@ impl IddPushCapturer {
// Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the
// ~3 ms encode — NVENC reads the host out-ring slot, not the keyed-mutex slot), so the driver gets
// the slot back immediately and the encode of the PREVIOUS frame overlaps this convert.
// Clone the slot's COM interfaces (an AddRef each) so the guard borrows LOCALS, leaving
// `self` free for the composite blend prep inside the lock.
let (slot_tex, slot_srv, slot_mutex) = {
let s = &self.slots[slot];
(s.tex.clone(), s.srv.clone(), s.mutex.clone())
};
let s = &self.slots[slot];
// Acquire the slot's keyed mutex via a RAII guard, scoped to JUST the convert/copy below so it
// releases at the same point as the old hand-written `ReleaseSync` (the driver gets the slot back
// immediately, NOT held across the rest of `try_consume`) — but now leak-proof on any early return.
{
let Some(_lock) = KeyedMutexGuard::acquire(&slot_mutex, 0, 8) else {
let Some(_lock) = KeyedMutexGuard::acquire(&s.mutex, 0, 8) else {
return Ok(None);
};
// SAFETY: convert on the owning (encode) thread's immediate context, holding the slot lock.
// A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing
// the slot back to the driver.
unsafe {
// Composite cursor model: divert the convert input through the blend scratch —
// a slot copy with the pointer quad alpha-blended on top. `None` = compositing
// off or degraded (the conversion then reads the slot as always).
let blended = if self.composite_cursor {
self.prepare_blend_scratch(&slot_tex)
} else {
None
};
if self.pyrowave {
// PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate
// plane textures via the mode-aware CSC; the shared fence signalled just after
@@ -2027,17 +1731,22 @@ impl IddPushCapturer {
// convert. The composition format is pinned to the negotiated depth.
let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot");
if let Some(conv) = self.pyro_conv.as_ref() {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
conv.convert(&self.context, src, y_rtv, cbcr_rtv, self.width, self.height)?;
conv.convert(
&self.context,
&s.srv,
y_rtv,
cbcr_rtv,
self.width,
self.height,
)?;
}
} else if self.display_hdr {
// HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010.
if let Some(conv) = self.hdr_p010_conv.as_ref() {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
conv.convert(
&self.device,
&self.context,
src,
&s.srv,
out.as_ref().expect("out ring"),
self.width,
self.height,
@@ -2047,14 +1756,12 @@ impl IddPushCapturer {
// SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma
// RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain
// copy-engine move; the slot releases back to the driver immediately.
let src = blended.as_ref().map(|(t, _)| t).unwrap_or(&slot_tex);
self.context
.CopyResource(out.as_ref().expect("out ring"), src);
.CopyResource(out.as_ref().expect("out ring"), &s.tex);
} else {
// SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC.
if let Some(conv) = self.video_conv.as_ref() {
let src = blended.as_ref().map(|(t, _)| t).unwrap_or(&slot_tex);
conv.convert(src, out.as_ref().expect("out ring"))?;
conv.convert(&s.tex, out.as_ref().expect("out ring"))?;
}
}
}
@@ -2062,20 +1769,13 @@ impl IddPushCapturer {
}
self.out_idx = (i + 1) % ring_len;
self.last_seq = seq;
if fresh {
self.last_slot = Some(slot);
}
if let Some((y, _, cbcr, _)) = pyro_slot.as_ref() {
self.pyro_last = Some((y.clone(), cbcr.clone()));
} else {
self.last_present = Some((out.as_ref().expect("out ring").clone(), pf));
}
let now = Instant::now();
if regen {
// A regen re-encodes OLD desktop content at a new pointer position — it is not a
// fresh driver frame; feeding the freshness/stall bookkeeping would mask a dead
// driver and pollute stall attribution.
} else if self.recovering_since.take().is_some() {
if self.recovering_since.take().is_some() {
// A fresh frame resumed → recovered. The recovery gap is self-inflicted (ring
// recreate, already logged by the recreate path) — reset the stall watch so it
// doesn't read as a DWM stall.
@@ -2147,12 +1847,10 @@ impl IddPushCapturer {
}
}
}
if !regen {
self.last_fresh = now; // feeds the driver-death watch
}
// Build the frame. For PyroWave the encode input is the Y plane
// (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence
// after the convert above. SAFETY: on the owning capture/encode thread.
self.last_fresh = now; // feeds the driver-death watch
// Build the frame. For PyroWave the encode input is the Y plane
// (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence
// after the convert above. SAFETY: on the owning capture/encode thread.
let (texture, pyro) = if let Some((y, _, cbcr, _)) = pyro_slot {
// SAFETY: on the owning capture/encode thread holding the immediate context.
let (fence_handle, fence_value) =
@@ -2163,7 +1861,6 @@ impl IddPushCapturer {
cbcr,
fence_handle,
fence_value,
ring_gen: self.generation,
}),
)
} else {
@@ -2222,7 +1919,6 @@ impl IddPushCapturer {
cbcr: dst_cbcr,
fence_handle,
fence_value,
ring_gen: self.generation,
}),
}),
cursor: None,
@@ -2277,82 +1973,7 @@ impl std::fmt::Display for AttachTexFail {
impl std::error::Error for AttachTexFail {}
/// Duplicate `cs`'s section into the driver's WUDFHost and send `IOCTL_SET_CURSOR_CHANNEL`.
/// `true` = the driver adopted it (worker declared per its `cursor_forward_on` state). Shared by
/// the open-time delivery and every RE-delivery (ring recreate / flip NOT_FOUND) — the request is
/// idempotent driver-side (a replaced worker is stopped + joined).
fn deliver_cursor_channel(
broker: &ChannelBroker,
target_id: u32,
cs: &cursor::CursorShared,
send_cursor: &crate::CursorChannelSender,
) -> bool {
// SAFETY: `cs.section_handle()` borrows the section mapping `cs` owns (live across this
// synchronous call); the broker's WUDFHost process handle is live for the broker's lifetime.
let value = match unsafe { broker.dup_into_public(cs.section_handle()) } {
Ok(v) => v,
Err(e) => {
tracing::warn!("cursor section duplication failed (composited cursor stays): {e:#}");
return false;
}
};
let req = pf_driver_proto::control::SetCursorChannelRequest {
target_id,
_pad: 0,
header_handle: value,
};
match send_cursor(&req) {
Ok(()) => {
tracing::info!(
target_id,
"IDD push(host): cursor channel delivered — driver declares the hardware cursor"
);
true
}
Err(e) => {
broker.close_remote_public(value);
tracing::warn!("cursor channel delivery failed (composited cursor stays): {e:#}");
false
}
}
}
impl Capturer for IddPushCapturer {
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
// A LIVE poller is the sole source — even while it still reports `None` (pre-first-shape):
// falling back to the shm mid-session would interleave two serial namespaces and poison
// the client's shape cache. The shm read only serves a poller that failed to start/died.
if let Some(p) = &self.cursor_poll {
if p.alive() {
return p.read();
}
}
self.cursor_shared.as_mut().and_then(|c| c.read())
}
fn set_cursor_forward(&mut self, on: bool) {
// The composite (capture) model is implemented HOST-side: the driver's hardware cursor
// stays declared for the session's whole life — the only dependable state (there is NO
// working un-declare; see cursor_blend.rs) — keeping every frame pointer-free, and the
// capturer blends the GDI poller's shape into the frame itself. No driver round-trip.
// `composite_forced` (a channel-less session on a sticky-declared target) is pinned ON:
// with no client drawing, un-compositing would erase the pointer entirely.
let composite = (!on && self.cursor_shared.is_some()) || self.composite_forced;
if self.composite_cursor != composite {
self.composite_cursor = composite;
self.last_blend_key = None; // regenerate immediately at the current pointer state
tracing::info!(
composite,
"cursor render model: host compositing {}",
if composite {
"ON (capture model — blending the pointer into frames)"
} else {
"OFF (client draws locally)"
}
);
}
}
fn next_frame(&mut self) -> Result<CapturedFrame> {
let deadline = Instant::now() + Duration::from_secs(20);
loop {
@@ -105,22 +105,6 @@ impl ChannelBroker {
Ok(out.0 as usize as u64)
}
/// Duplicate the cursor section into WUDFHost (v5 cursor channel) with the same
/// least-privilege section rights as the frame header. Thin `pub(super)` face over
/// [`dup_into`](Self::dup_into) for the cursor-delivery path in `open_on`.
///
/// # Safety
/// `h` must be a live handle of the current process.
pub(super) unsafe fn dup_into_public(&self, h: HANDLE) -> Result<u64> {
// SAFETY: forwarded contract — `h` is live per this fn's own contract.
unsafe { self.dup_into(h, Some(SECTION_MAP_RW)) }
}
/// [`close_remote`](Self::close_remote) for the cursor-delivery failure path.
pub(super) fn close_remote_public(&self, value: u64) {
self.close_remote(value);
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) {
@@ -1,194 +0,0 @@
//! Host side of the v5 hardware-cursor channel (remote-desktop-sweep M2c): the capturer creates
//! an unnamed [`CursorShm`] section, delivers it to the pf-vdisplay driver (which declares an
//! IddCx hardware cursor — DWM then EXCLUDES the pointer from the frames we consume), and reads
//! the driver's seqlock publishes here at encode-tick pace, converting them into the same
//! [`pf_frame::CursorOverlay`] the Linux portal path produces — everything downstream (the
//! cursor forwarder, the wire, the client renderer) is shared.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use pf_driver_proto::cursor::{
CursorShm, CURSOR_MAGIC, CURSOR_SHAPE_BYTES, CURSOR_SHAPE_MAX, CURSOR_SHAPE_OFFSET,
CURSOR_SHM_SIZE, CURSOR_TYPE_MASKED_COLOR,
};
use std::sync::atomic::AtomicU32;
/// The host end of one monitor's cursor channel: the section (we created it — the mapping stays
/// valid for the capturer's life) plus the reader's conversion cache.
pub(super) struct CursorShared {
section: MappedSection,
/// The monitor's desktop origin — IddCx reports positions in DESKTOP coordinates; the
/// overlay wants frame-relative. Fetched at attach (the virtual monitor's placement is
/// stable for the session; a topology change recreates the pipeline anyway).
origin: (i32, i32),
/// Conversion cache: the last `shape_id` whose pixels were converted, and the result.
/// Position-only updates (the common case) reuse it — a refcount bump, no pixel work.
cached_id: u32,
cached: Option<ConvertedShape>,
}
struct ConvertedShape {
rgba: std::sync::Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
}
impl CursorShared {
/// Create + initialize the section (magic stamped, seq even/zero). The returned handle is
/// the section itself (owned by `self`); the caller duplicates it into the WUDFHost.
pub(super) fn create(target_id: u32) -> Result<CursorShared> {
// SAFETY: plain FFI. Unnamed pagefile-backed section, host-lifetime owned; the view is
// mapped once and unmapped never (the capturer's life = the session's life).
let section = unsafe {
let map = CreateFileMappingW(
INVALID_HANDLE_VALUE,
None,
PAGE_READWRITE,
0,
CURSOR_SHM_SIZE as u32,
PCWSTR::null(),
)
.context("CreateFileMapping(cursor)")?;
let map = OwnedHandle::from_raw_handle(map.0 as _);
let view = MapViewOfFile(
HANDLE(map.as_raw_handle()),
FILE_MAP_ALL_ACCESS,
0,
0,
CURSOR_SHM_SIZE,
);
if view.Value.is_null() {
bail!("MapViewOfFile failed for the cursor section");
}
let shm = view.Value.cast::<CursorShm>();
std::ptr::write_bytes(view.Value.cast::<u8>(), 0, CURSOR_SHM_SIZE);
// Magic LAST-ish (the driver validates it at adopt; seq 0 = even = consistent).
std::sync::atomic::fence(Ordering::Release);
(*shm).magic = CURSOR_MAGIC;
MappedSection { handle: map, view }
};
// Desktop origin of this monitor's source — for the desktop→frame coordinate shift.
// SAFETY: `source_desktop_rect` only runs the CCD QueryDisplayConfig FFI over owned
// locals (same call the compose-kick path makes).
let rect = unsafe { pf_win_display::win_display::source_desktop_rect(target_id) };
let origin = rect.map(|(x, y, _w, _h)| (x, y)).unwrap_or((0, 0));
Ok(CursorShared {
section,
origin,
cached_id: 0,
cached: None,
})
}
/// The section handle for the broker's duplication into the WUDFHost.
pub(super) fn section_handle(&self) -> HANDLE {
HANDLE(self.section.handle.as_raw_handle())
}
/// Seqlock-read the driver's latest publish → a frame-relative [`pf_frame::CursorOverlay`].
/// `None` until the first publish lands (or while the pointer has never been seen). A hidden
/// pointer returns `Some` with `visible: false` — the forwarder turns that into the client's
/// relative-mode hint, exactly like the Linux path.
pub(super) fn read(&mut self) -> Option<pf_frame::CursorOverlay> {
let shm = self.section.ptr::<CursorShm>();
// SAFETY: the view spans CURSOR_SHM_SIZE for self's lifetime; seq is 4-aligned in the
// fixed layout (offset 4).
let seq = unsafe { &*std::ptr::addr_of!((*shm).seq).cast::<AtomicU32>() };
for _ in 0..64 {
let s1 = seq.load(Ordering::Acquire);
if s1 == 0 {
return None; // no publish yet
}
if s1 & 1 != 0 {
std::hint::spin_loop();
continue; // writer mid-update
}
// SAFETY: header reads within the mapped view; consistency is validated by the
// seq re-check below (a torn read is discarded and retried).
let hdr = unsafe { std::ptr::read_volatile(shm) };
// Shape pixels: convert only when the OS minted a new shape id.
if hdr.visible != 0 && hdr.shape_id != self.cached_id {
let rows = hdr.height.min(CURSOR_SHAPE_MAX) as usize;
let width = hdr.width.min(CURSOR_SHAPE_MAX) as usize;
let pitch = (hdr.pitch as usize).min(CURSOR_SHAPE_BYTES / rows.max(1));
let mut raw = vec![0u8; rows * pitch];
// SAFETY: the shape region spans CURSOR_SHAPE_BYTES from CURSOR_SHAPE_OFFSET
// inside the mapped view; `rows * pitch` is clamped to it above.
unsafe {
std::ptr::copy_nonoverlapping(
self.section.ptr::<u8>().add(CURSOR_SHAPE_OFFSET),
raw.as_mut_ptr(),
rows * pitch,
);
}
// Discard the copy if the writer raced us mid-shape (seq moved) — retry.
if seq.load(Ordering::Acquire) != s1 {
continue;
}
self.cached = Some(convert_shape(&hdr, &raw, width, rows, pitch));
self.cached_id = hdr.shape_id;
} else if seq.load(Ordering::Acquire) != s1 {
continue;
}
let shape = self.cached.as_ref()?;
return Some(pf_frame::CursorOverlay {
x: hdr.x - self.origin.0,
y: hdr.y - self.origin.1,
w: shape.w,
h: shape.h,
rgba: shape.rgba.clone(),
serial: u64::from(hdr.shape_id),
hot_x: shape.hot_x,
hot_y: shape.hot_y,
visible: hdr.visible != 0,
});
}
None // persistent tearing (writer wedged mid-seq) — skip this tick
}
}
/// Convert the OS's 32-bpp pitch-strided shape rows into the overlay's packed straight RGBA.
/// ALPHA cursors are BGRA with straight per-pixel alpha (swap R↔B). MASKED_COLOR approximates:
/// alpha 0x00 = opaque color pixel; 0xFF = an XOR pixel we cannot honor client-side — rendered
/// as a translucent mid-gray so inversion cursors stay visible instead of vanishing.
fn convert_shape(
hdr: &CursorShm,
raw: &[u8],
width: usize,
rows: usize,
pitch: usize,
) -> ConvertedShape {
let masked = hdr.cursor_type == CURSOR_TYPE_MASKED_COLOR;
let mut rgba = Vec::with_capacity(width * rows * 4);
for y in 0..rows {
let row = &raw[y * pitch..];
for x in 0..width {
let o = x * 4;
if o + 4 > row.len() {
rgba.extend_from_slice(&[0, 0, 0, 0]);
continue;
}
let (b, g, r, a) = (row[o], row[o + 1], row[o + 2], row[o + 3]);
if masked {
if a == 0 {
rgba.extend_from_slice(&[r, g, b, 0xFF]);
} else {
rgba.extend_from_slice(&[0x80, 0x80, 0x80, 0xB4]);
}
} else {
rgba.extend_from_slice(&[r, g, b, a]);
}
}
}
ConvertedShape {
rgba: std::sync::Arc::new(rgba),
w: width as u32,
h: rows as u32,
hot_x: hdr.hot_x.min(width.saturating_sub(1) as u32),
hot_y: hdr.hot_y.min(rows.saturating_sub(1) as u32),
}
}
@@ -1,216 +0,0 @@
//! Host-side cursor compositing for the CAPTURE mouse model (design/remote-desktop-sweep.md §8).
//!
//! Why the host draws it: once a monitor has ever declared an IddCx hardware cursor, DWM will
//! not composite the software cursor back into its frames — there is no un-declare DDI (the
//! empty-caps re-setup is rejected `STATUS_INVALID_PARAMETER`), and a successful same-mode
//! re-commit with the driver's re-declare provably suppressed still leaves the pointer excluded
//! (all observed on-glass, 26100). So the driver keeps its hardware cursor declared for the
//! session's whole life — the state that works — and when the client flips to the capture model
//! the HOST composites the pointer into the frame itself: a slot→scratch copy plus one
//! alpha-blended quad (the GDI poller's full-fidelity shape at its polled position), entirely
//! GPU-side on the capture device, before the normal conversion runs from the scratch.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use windows::core::s;
use windows::Win32::Graphics::Direct3D::D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST;
use windows::Win32::Graphics::Direct3D11::{
ID3D11BlendState, ID3D11Buffer, ID3D11PixelShader, ID3D11SamplerState, ID3D11VertexShader,
D3D11_BIND_CONSTANT_BUFFER, D3D11_BLEND_DESC, D3D11_BLEND_INV_SRC_ALPHA, D3D11_BLEND_ONE,
D3D11_BLEND_OP_ADD, D3D11_BLEND_SRC_ALPHA, D3D11_BUFFER_DESC, D3D11_COMPARISON_NEVER,
D3D11_CPU_ACCESS_WRITE, D3D11_FILTER_MIN_MAG_MIP_LINEAR, D3D11_MAPPED_SUBRESOURCE,
D3D11_MAP_WRITE_DISCARD, D3D11_RENDER_TARGET_BLEND_DESC, D3D11_SAMPLER_DESC,
D3D11_SUBRESOURCE_DATA, D3D11_TEXTURE_ADDRESS_CLAMP, D3D11_USAGE_DYNAMIC, D3D11_VIEWPORT,
};
use windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT_R8G8B8A8_UNORM;
/// Straight-alpha sample of the cursor bitmap. `linear_scale` = 0 passes sRGB through (SDR
/// ring); non-zero linearizes sRGB→scRGB AND multiplies by the target's SDR-white scale
/// (`sdr_white_level_scale` — 1.0 would put cursor-white at 80 nits, visibly DARKER than the
/// surrounding SDR desktop content DWM composes at the user's SDR-brightness setting).
const CURSOR_PS: &str = r"
Texture2D<float4> tx : register(t0);
SamplerState sm : register(s0);
cbuffer C : register(b0) { float linear_scale; float3 pad; };
float4 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_Target {
float4 c = tx.Sample(sm, uv);
if (linear_scale != 0.0) {
c.rgb = pow(abs(c.rgb), 2.2) * linear_scale;
}
return c;
}
";
/// The cursor-quad blend pass + its shape-texture cache. One per capturer (device-scoped).
pub(super) struct CursorBlendPass {
vs: ID3D11VertexShader,
ps: ID3D11PixelShader,
sampler: ID3D11SamplerState,
blend: ID3D11BlendState,
cbuf: ID3D11Buffer,
cbuf_scale: Option<f32>,
/// The uploaded shape (serial-keyed): SRV + dims in host pixels.
shape: Option<(u64, ID3D11ShaderResourceView, u32, u32)>,
}
impl CursorBlendPass {
pub(super) unsafe fn new(device: &ID3D11Device) -> Result<Self> {
let vsb = crate::dxgi::compile_shader(crate::dxgi::HDR_VS, s!("main"), s!("vs_5_0"))?;
let psb = crate::dxgi::compile_shader(CURSOR_PS, s!("main"), s!("ps_5_0"))?;
let mut vs = None;
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
let mut ps = None;
device.CreatePixelShader(&psb, None, Some(&mut ps))?;
let sd = D3D11_SAMPLER_DESC {
// LINEAR: the quad is drawn 1:1 in frame pixels, so this only matters at the
// half-texel edges; linear keeps them soft instead of ringing.
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
ComparisonFunc: D3D11_COMPARISON_NEVER,
MaxLOD: f32::MAX,
..Default::default()
};
let mut sampler = None;
device.CreateSamplerState(&sd, Some(&mut sampler))?;
// Straight-alpha over: dst.rgb = src.rgb*a + dst.rgb*(1-a); keep dst alpha.
let mut bd = D3D11_BLEND_DESC::default();
bd.RenderTarget[0] = D3D11_RENDER_TARGET_BLEND_DESC {
BlendEnable: true.into(),
SrcBlend: D3D11_BLEND_SRC_ALPHA,
DestBlend: D3D11_BLEND_INV_SRC_ALPHA,
BlendOp: D3D11_BLEND_OP_ADD,
SrcBlendAlpha: D3D11_BLEND_ONE,
DestBlendAlpha: D3D11_BLEND_ONE,
BlendOpAlpha: D3D11_BLEND_OP_ADD,
RenderTargetWriteMask: 0x0F,
};
let mut blend = None;
device.CreateBlendState(&bd, Some(&mut blend))?;
let cbd = D3D11_BUFFER_DESC {
ByteWidth: 16, // float to_linear + float3 pad
Usage: D3D11_USAGE_DYNAMIC,
BindFlags: D3D11_BIND_CONSTANT_BUFFER.0 as u32,
CPUAccessFlags: D3D11_CPU_ACCESS_WRITE.0 as u32,
..Default::default()
};
let mut cbuf = None;
device.CreateBuffer(&cbd, None, Some(&mut cbuf))?;
Ok(Self {
vs: vs.context("cursor blend vs")?,
ps: ps.context("cursor blend ps")?,
sampler: sampler.context("cursor blend sampler")?,
blend: blend.context("cursor blend state")?,
cbuf: cbuf.context("cursor blend cbuf")?,
cbuf_scale: None,
shape: None,
})
}
/// Upload `ov`'s bitmap if its serial is new; reuse the cached SRV otherwise.
unsafe fn ensure_shape(
&mut self,
device: &ID3D11Device,
ov: &pf_frame::CursorOverlay,
) -> Result<()> {
if self.shape.as_ref().is_some_and(|(s, ..)| *s == ov.serial) {
return Ok(());
}
if ov.rgba.len() < (ov.w as usize) * (ov.h as usize) * 4 || ov.w == 0 || ov.h == 0 {
bail!("malformed cursor overlay ({}x{})", ov.w, ov.h);
}
let desc = D3D11_TEXTURE2D_DESC {
Width: ov.w,
Height: ov.h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R8G8B8A8_UNORM,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: ov.rgba.as_ptr().cast(),
SysMemPitch: ov.w * 4,
SysMemSlicePitch: 0,
};
let mut tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&init), Some(&mut tex))
.context("CreateTexture2D(cursor shape)")?;
let tex = tex.context("null cursor shape texture")?;
let mut srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&tex, None, Some(&mut srv))
.context("CreateShaderResourceView(cursor shape)")?;
self.shape = Some((ov.serial, srv.context("null cursor shape srv")?, ov.w, ov.h));
Ok(())
}
/// Alpha-blend `ov` onto `dst` (frame-sized, RENDER_TARGET-capable — the blend scratch).
/// `linear_scale`: 0 = SDR passthrough; non-zero = the frame is FP16 scRGB (HDR
/// composition) — linearize and scale to the target's SDR white. The quad is placed purely
/// via the viewport (the fullscreen-triangle VS fills whatever viewport is set), clipped by
/// the target automatically.
pub(super) unsafe fn blend(
&mut self,
device: &ID3D11Device,
ctx: &ID3D11DeviceContext,
dst: &ID3D11Texture2D,
ov: &pf_frame::CursorOverlay,
linear_scale: f32,
) -> Result<()> {
self.ensure_shape(device, ov)?;
let (_, srv, w, h) = self.shape.as_ref().expect("shape just ensured");
if self.cbuf_scale != Some(linear_scale) {
let cb: [f32; 4] = [linear_scale, 0.0, 0.0, 0.0];
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
if ctx
.Map(&self.cbuf, 0, D3D11_MAP_WRITE_DISCARD, 0, Some(&mut mapped))
.is_ok()
{
std::ptr::copy_nonoverlapping(cb.as_ptr(), mapped.pData as *mut f32, cb.len());
ctx.Unmap(&self.cbuf, 0);
}
self.cbuf_scale = Some(linear_scale);
}
let mut rtv: Option<ID3D11RenderTargetView> = None;
device
.CreateRenderTargetView(dst, None, Some(&mut rtv))
.context("CreateRenderTargetView(cursor blend scratch)")?;
let rtv = rtv.context("null cursor blend rtv")?;
ctx.OMSetRenderTargets(Some(&[Some(rtv)]), None);
ctx.OMSetBlendState(&self.blend, None, 0xffff_ffff);
ctx.VSSetShader(&self.vs, None);
ctx.PSSetShader(&self.ps, None);
ctx.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
ctx.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
ctx.PSSetConstantBuffers(0, Some(&[Some(self.cbuf.clone())]));
ctx.IASetInputLayout(None);
ctx.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
// Placement IS the viewport: the VS fills it, the OS clips it to the target.
let vp = D3D11_VIEWPORT {
TopLeftX: ov.x as f32,
TopLeftY: ov.y as f32,
Width: *w as f32,
Height: *h as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
};
ctx.RSSetViewports(Some(&[vp]));
ctx.Draw(3, 0);
// Unbind so the scratch can be bound as a conversion INPUT without a hazard warning.
ctx.OMSetRenderTargets(None, None);
let none_srv: [Option<ID3D11ShaderResourceView>; 1] = [None];
ctx.PSSetShaderResources(0, Some(&none_srv));
Ok(())
}
}
@@ -1,445 +0,0 @@
//! GDI cursor poller — the Windows cursor-SHAPE source for the cursor-forward channel
//! (design/remote-desktop-sweep.md §8, the M2c redesign).
//!
//! Why not the IddCx hardware-cursor query (the v5 `CursorShm` path, now the fallback): it is
//! alpha-only BY DESIGN — `IDDCX_CURSOR_SHAPE_TYPE` has no monochrome value, the OS pre-converts
//! monochrome to masked-color (IDDCX_CURSOR_CAPS docs), and masked-color delivery is dead code on
//! modern builds (proven on-glass at every `ColorXorCursorSupport` level; no public evidence of a
//! MASKED_COLOR delivery anywhere). The driver keeps its hardware cursor declared at XOR FULL
//! purely so DWM EXCLUDES every cursor type from the IDD frame.
//!
//! Why not DXGI Desktop Duplication `GetFramePointerShape`: its `PointerPosition.Visible` goes
//! stale when the cursor moves only via injected input on current Win11 (Sunshine #5293 — exactly
//! a Punktfunk session's topology), it burns one of the session's four duplication slots, and its
//! per-output metadata on IDD monitors has conflicting field reports. The GDI path below is the
//! metadata-forwarding-remote-desktop pattern (RustDesk, WebRTC/Chrome Remote Desktop, OBS): the
//! cursor is per-session global state in win32k, readable cross-process, and `CURSOR_SHOWING` is
//! the logical visibility — immune to all of the above.
//!
//! Works because the capture host runs as SYSTEM *inside the interactive session* on
//! `winsta0\default` (the service supervisor retargets the token — `windows/service.rs`
//! `spawn_host`), so the poller thread sees the session's cursor directly; no helper process.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use windows::Win32::Graphics::Gdi::{
DeleteObject, GetDC, GetDIBits, GetObjectW, ReleaseDC, BITMAP, BITMAPINFO, BITMAPINFOHEADER,
BI_RGB, DIB_RGB_COLORS, HBITMAP, HDC,
};
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, OpenInputDesktop, SetThreadDesktop, DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS,
HDESK,
};
use windows::Win32::UI::HiDpi::{
SetThreadDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
};
use windows::Win32::UI::WindowsAndMessaging::{
CopyIcon, DestroyIcon, GetCursorInfo, GetIconInfo, CURSORINFO, HICON, ICONINFO,
};
/// `CURSORINFO.flags` bits (WindowsAndMessaging): the pointer is logically shown /
/// touch-or-pen-suppressed. Named locally so the visibility rule below reads as the docs do.
const CURSOR_SHOWING: u32 = 0x1;
const CURSOR_SUPPRESSED: u32 = 0x2;
/// A converted shape: the cache the per-tick overlay is assembled from. `rgba` is `Arc` so the
/// slot publish (and every downstream frame attach) is a refcount bump.
struct Shape {
rgba: std::sync::Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
serial: u64,
}
/// Off-thread GDI cursor poller. Samples `GetCursorInfo` at ~60 Hz, rasterises the `HCURSOR` only
/// when its handle value changes, and publishes a ready [`pf_frame::CursorOverlay`] snapshot; the
/// capture thread's per-tick cost is one uncontended mutex read + an `Arc` clone
/// (same split as [`DescriptorPoller`], and for the same reason: user32/gdi32 calls have no place
/// on the capture/encode thread).
pub(super) struct CursorPoller {
slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl CursorPoller {
/// ~250 Hz: the polled position is ALSO the composite-blend position (capture model), so
/// it must out-pace the fastest session — at 16 ms a 240 fps stream re-used a stale
/// position for ~4 consecutive frames and the composited pointer visibly stuttered
/// against the video. A tick is one `GetCursorInfo` syscall (rasterisation only on shape
/// change), so 250 Hz is still negligible CPU.
const INTERVAL: Duration = Duration::from_millis(4);
/// Unconditional input-desktop reattach cadence — catches secure-desktop (UAC/lock) switches
/// without a failure signal (`GetCursorInfo` on a stale desktop *succeeds* with stale data).
const REATTACH: Duration = Duration::from_secs(2);
/// Spawn the poller for the virtual display `target_id`. `rect` = the target's desktop rect
/// (`source_desktop_rect` order: x, y, w, h) — cursor positions are desktop-global; the
/// overlay wants frame-relative, and a pointer outside the rect reports `visible: false`
/// (per-output semantics, matching the driver shm path and the Linux portal).
pub(super) fn spawn(target_id: u32, rect: (i32, i32, i32, i32)) -> Self {
let slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>> = Arc::new(Mutex::new(None));
let stop = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t) = (slot.clone(), stop.clone());
let thread = std::thread::Builder::new()
.name("pf-cursor-poll".into())
.spawn(move || run(target_id, rect, &slot_t, &stop_t))
.ok();
if thread.is_none() {
tracing::warn!("cursor poller thread spawn failed — cursor falls back to driver shm");
}
Self { slot, stop, thread }
}
/// The latest overlay snapshot (`None` until the first successful shape rasterisation).
pub(super) fn read(&self) -> Option<pf_frame::CursorOverlay> {
self.slot.lock().unwrap_or_else(|p| p.into_inner()).clone()
}
/// Whether the worker thread is (still) alive — `false` degrades the capturer to the shm read.
pub(super) fn alive(&self) -> bool {
self.thread.as_ref().is_some_and(|t| !t.is_finished())
}
}
impl Drop for CursorPoller {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(t) = self.thread.take() {
let _ = t.join(); // worker sleeps ≤ INTERVAL — a bounded join
}
}
}
/// The poll loop. Owns the thread's input-desktop binding and the shape cache.
fn run(
target_id: u32,
rect: (i32, i32, i32, i32),
slot: &Mutex<Option<pf_frame::CursorOverlay>>,
stop: &AtomicBool,
) {
// Physical-pixel coordinates on this thread regardless of the process's DPI awareness:
// `rect` comes from CCD (always physical), and a DPI-virtualized `GetCursorInfo` position
// would land in the wrong frame pixel on any scaled display. Thread-scoped, so the rest of
// the host is untouched.
// SAFETY: takes and returns only a by-value context handle; affects this thread only.
let _ = unsafe { SetThreadDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) };
let mut desktop = DesktopBinding::default();
desktop.reattach(); // best-effort: already on winsta0\default if this fails
let mut last_attach = Instant::now();
let mut shape: Option<Shape> = None;
let mut cached_handle: isize = 0;
let mut failed_handle: isize = 0; // don't re-rasterise a failing handle every tick
let mut serial: u64 = 0;
let mut logged_live = false;
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(CursorPoller::INTERVAL);
if last_attach.elapsed() >= CursorPoller::REATTACH {
last_attach = Instant::now();
desktop.reattach();
}
let mut ci = CURSORINFO {
cbSize: std::mem::size_of::<CURSORINFO>() as u32,
..Default::default()
};
// SAFETY: `ci` is a live, correctly-sized out-param for this synchronous call; no pointer
// escapes it.
if unsafe { GetCursorInfo(&mut ci) }.is_err() {
// Desktop went away under us (secure-desktop switch mid-call) — rebind and retry
// next tick; the slot keeps its last snapshot meanwhile.
desktop.reattach();
last_attach = Instant::now();
continue;
}
let flags = ci.flags.0;
let showing = flags & CURSOR_SHOWING != 0 && flags & CURSOR_SUPPRESSED == 0;
// Rasterise on handle change only (position-only ticks are a header update). Hidden
// cursors keep the cached shape — the forwarder's hidden-but-known contract needs the
// bitmap to have been seen. v1: animated cursors publish their first frame (the OBS
// behavior); frame cycling via DrawIconEx istep is a known follow-up.
let handle = ci.hCursor.0 as isize;
if showing && handle != 0 && handle != cached_handle && handle != failed_handle {
match rasterize(ci.hCursor) {
Some((rgba, w, h, hot_x, hot_y)) => {
serial += 1;
shape = Some(Shape {
rgba: std::sync::Arc::new(rgba),
w,
h,
hot_x,
hot_y,
serial,
});
cached_handle = handle;
failed_handle = 0;
if !logged_live {
logged_live = true;
tracing::info!(
target_id,
"cursor poller live — GDI shape source publishing (serial 1: {w}x{h})"
);
}
}
None => {
// The owning app may have destroyed the cursor mid-read; keep the previous
// shape and don't hammer this handle again until it changes.
failed_handle = handle;
}
}
}
let overlay = shape.as_ref().map(|s| {
let (px, py) = (ci.ptScreenPos.x - rect.0, ci.ptScreenPos.y - rect.1);
let in_rect = px >= 0 && py >= 0 && px < rect.2 && py < rect.3;
pf_frame::CursorOverlay {
// Overlay x/y = bitmap top-left (reported position hotspot), frame pixels.
x: px - s.hot_x as i32,
y: py - s.hot_y as i32,
w: s.w,
h: s.h,
rgba: s.rgba.clone(),
serial: s.serial,
hot_x: s.hot_x,
hot_y: s.hot_y,
visible: showing && in_rect,
}
});
*slot.lock().unwrap_or_else(|p| p.into_inner()) = overlay;
}
}
/// The thread's owned input-desktop handle — the [`SendInputInjector`] reattach model
/// (`pf-inject` sendinput.rs): keep the current binding, swap on demand, close exactly once.
#[derive(Default)]
struct DesktopBinding(Option<HDESK>);
impl DesktopBinding {
fn reattach(&mut self) {
const GENERIC_ALL: u32 = 0x1000_0000;
// SAFETY: `OpenInputDesktop`/`SetThreadDesktop`/`CloseDesktop` take only by-value args.
// `OpenInputDesktop` yields an owned `HDESK` only on `Ok`; it is either installed (and the
// previously-owned handle closed exactly once) or closed on failure — no handle is leaked
// or used after close. `SetThreadDesktop` rebinds only this calling thread (which owns
// no windows/hooks, so the rebind cannot fail on that account).
unsafe {
match OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
) {
Ok(h) => {
if SetThreadDesktop(h).is_ok() {
if let Some(old) = self.0.replace(h) {
let _ = CloseDesktop(old);
}
} else {
let _ = CloseDesktop(h);
}
}
Err(_) => { /* not privileged for this desktop; stay put */ }
}
}
}
}
impl Drop for DesktopBinding {
fn drop(&mut self) {
if let Some(h) = self.0.take() {
// SAFETY: `h` is our owned desktop handle, closed exactly once here.
let _ = unsafe { CloseDesktop(h) };
}
}
}
/// Rasterise `hcursor` to straight-alpha RGBA: `(rgba, w, h, hot_x, hot_y)`. `None` on any
/// failure (caller keeps the previous shape).
fn rasterize(hcursor: windows::Win32::UI::WindowsAndMessaging::HCURSOR) -> RasterOut {
// CopyIcon first: the owning process can destroy its HCURSOR between GetCursorInfo and the
// reads below; the copy is ours (the OBS/WebRTC guard).
// SAFETY: `HICON(hcursor.0)` reinterprets the cursor handle as an icon handle (cursors ARE
// icons in user32); CopyIcon yields an owned HICON we destroy below.
let Ok(icon) = (unsafe { CopyIcon(HICON(hcursor.0)) }) else {
return None;
};
let mut ii = ICONINFO::default();
// SAFETY: `ii` is a live out-param. On Ok it hands us COPIES of the mask/color bitmaps —
// both deleted below (GDI-handle leak otherwise).
let got = unsafe { GetIconInfo(icon, &mut ii) };
let out = if got.is_ok() { convert(&ii) } else { None };
// SAFETY: deleting the two bitmap copies GetIconInfo returned (null-safe: DeleteObject on a
// null HGDIOBJ fails harmlessly) and the icon copy — each exactly once.
unsafe {
let _ = DeleteObject(ii.hbmColor.into());
let _ = DeleteObject(ii.hbmMask.into());
let _ = DestroyIcon(icon);
}
out.map(|(rgba, w, h)| {
let hot_x = ii.xHotspot.min(w.saturating_sub(1));
let hot_y = ii.yHotspot.min(h.saturating_sub(1));
(rgba, w, h, hot_x, hot_y)
})
}
type RasterOut = Option<(Vec<u8>, u32, u32, u32, u32)>;
/// Convert the ICONINFO bitmaps to straight RGBA. Two families:
/// - color (`hbmColor` set): 32bpp BGRA; if the alpha channel is entirely empty (old-style
/// cursors) the AND mask supplies it (mask bit 1 = transparent).
/// - monochrome (`hbmColor` null): `hbmMask` is DOUBLE height — AND plane over XOR plane, the
/// WebRTC truth table: (0,0) black, (0,1) white, (1,0) transparent, (1,1) invert. Invert
/// pixels — unrepresentable in straight alpha — become opaque black with a white outline
/// grown into adjacent transparency (the WebRTC approximation; keeps the I-beam legible on
/// any background, which the old translucent-gray stand-in did not).
fn convert(ii: &ICONINFO) -> Option<(Vec<u8>, u32, u32)> {
// SAFETY: GetDC(None) yields the screen DC, released below on every path; it is only used
// as the GetDIBits reference DC.
let dc = unsafe { GetDC(None) };
let result = (|| {
if !ii.hbmColor.is_invalid() {
let color = read_bitmap_32(dc, ii.hbmColor)?;
let (w, h) = (color.w as u32, color.h as u32);
let mut rgba = bgra_to_rgba(&color.bgra);
if rgba.chunks_exact(4).all(|p| p[3] == 0) {
// Alpha-less color cursor: transparency lives in the AND mask.
let mask = read_bitmap_32(dc, ii.hbmMask)?;
if mask.w != color.w || mask.h < color.h {
return None;
}
for (px, m) in rgba.chunks_exact_mut(4).zip(mask.bgra.chunks_exact(4)) {
px[3] = if m[0] != 0 { 0 } else { 0xFF }; // mask white (AND=1) = transparent
}
}
Some((rgba, w, h))
} else {
let mask = read_bitmap_32(dc, ii.hbmMask)?;
if mask.h < 2 || mask.h % 2 != 0 {
return None;
}
let (w, h) = (mask.w as usize, (mask.h / 2) as usize);
let row = w * 4;
let (and_plane, xor_plane) = mask.bgra.split_at(h * row);
let mut rgba = vec![0u8; w * h * 4];
let mut invert = vec![false; w * h];
for i in 0..w * h {
let (a, x) = (and_plane[i * 4] != 0, xor_plane[i * 4] != 0);
let px = &mut rgba[i * 4..i * 4 + 4];
match (a, x) {
(false, false) => px.copy_from_slice(&[0, 0, 0, 0xFF]),
(false, true) => px.copy_from_slice(&[0xFF, 0xFF, 0xFF, 0xFF]),
(true, false) => {} // transparent (already zeroed)
(true, true) => {
px.copy_from_slice(&[0, 0, 0, 0xFF]);
invert[i] = true;
}
}
}
// White outline around invert regions so the (now black) shape survives dark
// backgrounds: any transparent 8-neighbor of an invert pixel turns opaque white.
for y in 0..h as i32 {
for x in 0..w as i32 {
if !invert[(y * w as i32 + x) as usize] {
continue;
}
for (dx, dy) in NEIGHBORS {
let (nx, ny) = (x + dx, y + dy);
if nx < 0 || ny < 0 || nx >= w as i32 || ny >= h as i32 {
continue;
}
let o = (ny * w as i32 + nx) as usize * 4;
if rgba[o + 3] == 0 {
rgba[o..o + 4].copy_from_slice(&[0xFF, 0xFF, 0xFF, 0xFF]);
}
}
}
}
Some((rgba, w as u32, h as u32))
}
})();
// SAFETY: releasing the screen DC obtained above, exactly once.
unsafe {
ReleaseDC(None, dc);
}
result
}
const NEIGHBORS: [(i32, i32); 8] = [
(-1, -1),
(0, -1),
(1, -1),
(-1, 0),
(1, 0),
(-1, 1),
(0, 1),
(1, 1),
];
struct RawBitmap {
w: i32,
h: i32,
/// 32bpp top-down BGRA rows, `w*h*4` (monochrome sources arrive expanded: 0x00/0xFF channels).
bgra: Vec<u8>,
}
/// Read any GDI bitmap as 32bpp top-down via `GetDIBits` (which performs the 1bpp→32bpp
/// expansion for the mask planes).
fn read_bitmap_32(dc: HDC, hbm: HBITMAP) -> Option<RawBitmap> {
let mut bm = BITMAP::default();
// SAFETY: `bm` is a live out-param sized exactly as passed; GetObjectW only writes into it.
let n = unsafe {
GetObjectW(
hbm.into(),
std::mem::size_of::<BITMAP>() as i32,
Some((&mut bm as *mut BITMAP).cast()),
)
};
if n == 0 || bm.bmWidth <= 0 || bm.bmHeight <= 0 || bm.bmWidth > 512 || bm.bmHeight > 1024 {
return None; // 512/1024: sanity caps (256² is the wire max; XL accessibility ≤ that)
}
let (w, h) = (bm.bmWidth, bm.bmHeight);
let mut info = BITMAPINFO {
bmiHeader: BITMAPINFOHEADER {
biSize: std::mem::size_of::<BITMAPINFOHEADER>() as u32,
biWidth: w,
biHeight: -h, // top-down
biPlanes: 1,
biBitCount: 32,
biCompression: BI_RGB.0,
..Default::default()
},
..Default::default()
};
let mut buf = vec![0u8; (w as usize) * (h as usize) * 4];
// SAFETY: `buf` spans exactly `h` rows of `w` 32bpp pixels as described by `info`; both are
// live locals for this synchronous call, `hbm` is a live bitmap not selected into any DC
// (fresh GetIconInfo copies).
let rows = unsafe {
GetDIBits(
dc,
hbm,
0,
h as u32,
Some(buf.as_mut_ptr().cast()),
&mut info,
DIB_RGB_COLORS,
)
};
(rows != 0).then_some(RawBitmap { w, h, bgra: buf })
}
fn bgra_to_rgba(bgra: &[u8]) -> Vec<u8> {
let mut out = bgra.to_vec();
for px in out.chunks_exact_mut(4) {
px.swap(0, 2);
}
out
}
+4 -10
View File
@@ -876,16 +876,10 @@ impl Worker {
);
return;
};
let pref = match self.pad_info(id) {
// Steam Input's virtual pad standing in front of the Deck's built-in controls (the
// only-pad-forwarded case, [`Self::forwarded_ids`]): declare the DECK kind, not the
// wrapper's Xbox 360 identity. [`Self::auto_pref`] already resolves the SESSION
// default this way, but a current host honors the per-pad arrival over the session
// default — so without this the host builds an X-Box 360 pad on a real Deck.
Some(p) if p.steam_virtual && is_steam_deck() => GamepadPref::SteamDeck,
Some(p) => p.pref,
None => GamepadPref::Xbox360,
};
let pref = self
.pad_info(id)
.map(|p| p.pref)
.unwrap_or(GamepadPref::Xbox360);
match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) {
Ok(pad) => {
let mut slot = Slot::new(id, index, pref, pad);
+2 -22
View File
@@ -44,13 +44,6 @@ pub struct SessionParams {
/// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The
/// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`.
pub clipboard: bool,
/// Advertise `quic::CLIENT_CAP_CURSOR`: this embedder renders the host cursor locally
/// (the presenter's cursor channel, design/remote-desktop-sweep.md M2), so the host may
/// stop compositing the pointer into the video. Only set when the embedder actually
/// draws it (the SDL presenter in desktop mouse mode) — a session that advertises it
/// without rendering streams with NO visible cursor. The host answers `HOST_CAP_CURSOR`
/// when its capture can forward (Linux portal, not gamescope/Windows).
pub cursor_forward: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
/// `video::Decoder::new`).
pub decoder: String,
@@ -262,11 +255,6 @@ fn pump(
// This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an
// A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600).
punktfunk_core::client::display_hdr_env_override().or(params.display_hdr),
if params.cursor_forward {
punktfunk_core::quic::CLIENT_CAP_CURSOR
} else {
0
},
params.launch.clone(),
params.pin,
Some(params.identity),
@@ -459,16 +447,8 @@ fn pump(
// every ~816 ms at 60120 Hz anyway, so this rarely times out mid-stream).
match connector.next_frame(Duration::from_millis(20)) {
Ok(frame) => {
// The `received` point: reassembly COMPLETION, stamped by the core session as
// the AU crossed poll_frame (ABI v9). Stamping here at the hand-off pull instead
// would fold the pre-decode queue wait into `host+network` — a client-side
// standing backlog masquerading as network latency (the 2026-07 two-pair
// investigation). 0 = a core predating the stamp; fall back to the pull instant.
let received_ns = if frame.received_ns > 0 {
frame.received_ns
} else {
now_ns()
};
// The `received` point: AU fully reassembled, in hand, before decode.
let received_ns = now_ns();
// fps / goodput count every received AU (spec), decoded or not.
frames_n += 1;
bytes_n += frame.data.len() as u64;
-62
View File
@@ -325,11 +325,6 @@ pub fn connect_reject_message(reason: punktfunk_core::reject::RejectReason) -> S
"Client and host versions don't match — update both to the same release.".into()
}
R::Busy => "The host is busy with another session.".into(),
R::SetupFailed => {
"The host accepted the connection but couldn't start the stream — the host's log \
(web console Log) has the cause."
.into()
}
}
}
@@ -461,48 +456,6 @@ impl TouchMode {
}
}
/// How a physical mouse drives the host — the desktop-sweep mouse model
/// (design/remote-desktop-sweep.md M1). Stored stringly in [`Settings::mouse_mode`] so the
/// file stays readable; parsed with [`MouseMode::from_name`].
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum MouseMode {
/// Pointer lock (relative deltas, hidden cursor) — the game model, and the default:
/// the only cursor you see is the host's.
Capture,
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion goes on the wire as absolute positions through the letterbox. The remote
/// desktop model. Requires a host injector with absolute support (not gamescope).
Desktop,
}
impl MouseMode {
/// Cycle/picker order (also the settings pickers' option order).
pub const ALL: [MouseMode; 2] = [MouseMode::Capture, MouseMode::Desktop];
/// Parse the persisted name, defaulting to `Capture` for unset/unknown values.
pub fn from_name(s: &str) -> MouseMode {
match s {
"desktop" => MouseMode::Desktop,
_ => MouseMode::Capture,
}
}
/// The persisted name (the inverse of [`from_name`](Self::from_name)).
pub fn as_name(self) -> &'static str {
match self {
MouseMode::Capture => "capture",
MouseMode::Desktop => "desktop",
}
}
pub fn label(self) -> &'static str {
match self {
MouseMode::Capture => "Capture (games)",
MouseMode::Desktop => "Desktop (absolute)",
}
}
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)]
@@ -537,12 +490,6 @@ pub struct Settings {
/// stores load as trackpad.
#[serde(default = "default_touch_mode")]
pub touch_mode: String,
/// How a physical mouse drives the host: a [`MouseMode`] name — `"capture"` (default,
/// pointer lock + relative) or `"desktop"` (uncaptured absolute pointer). Read at
/// connect via [`Settings::mouse_mode`]. `default` so pre-existing stores load as
/// capture — today's behavior.
#[serde(default = "default_mouse_mode")]
pub mouse_mode: String,
/// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured.
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
@@ -630,10 +577,6 @@ fn default_touch_mode() -> String {
"trackpad".into()
}
fn default_mouse_mode() -> String {
"capture".into()
}
fn default_true() -> bool {
true
}
@@ -661,10 +604,6 @@ impl Settings {
TouchMode::from_name(&self.touch_mode)
}
pub fn mouse_mode(&self) -> MouseMode {
MouseMode::from_name(&self.mouse_mode)
}
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() {
@@ -692,7 +631,6 @@ impl Default for Settings {
forward_pad: String::new(),
compositor: "auto".into(),
touch_mode: "trackpad".into(),
mouse_mode: "capture".into(),
inhibit_shortcuts: true,
mic_enabled: false,
audio_channels: 2,
+2 -42
View File
@@ -10,7 +10,7 @@ use crate::screens::{Ctx, Outbox};
use crate::theme::{Fonts, DIM, W};
use crate::widgets::{ListMsg, MenuList, RowSpec};
use pf_client_core::gamepad::{MenuEvent, MenuPulse};
use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use pf_client_core::trust::{StatsVerbosity, TouchMode};
use skia_safe::{Canvas, Rect};
/// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale
@@ -29,11 +29,10 @@ enum RowId {
Pad,
PadType,
Touch,
Mouse,
Stats,
}
const ROWS: [RowId; 14] = [
const ROWS: [RowId; 13] = [
RowId::Resolution,
RowId::Refresh,
RowId::Bitrate,
@@ -46,7 +45,6 @@ const ROWS: [RowId; 14] = [
RowId::Pad,
RowId::PadType,
RowId::Touch,
RowId::Mouse,
RowId::Stats,
];
@@ -253,7 +251,6 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
"Touch mode",
s.touch_mode().label().into(),
),
RowId::Mouse => (None, "Mouse mode", s.mouse_mode().label().into()),
RowId::Stats => (
Some("Interface"),
"Statistics overlay",
@@ -295,11 +292,6 @@ fn detail(id: RowId) -> &'static str {
"How the touchscreen drives the host: Trackpad (relative cursor), \
Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)."
}
RowId::Mouse => {
"How a physical mouse drives the host: Capture locks the pointer (relative, \
for games), Desktop leaves it free and sends absolute positions. \
Ctrl+Alt+Shift+M switches live while streaming."
}
RowId::Stats => {
"How much the overlay shows: Compact (one line) → Normal → Detailed. \
Ctrl+Alt+Shift+S cycles it live while streaming."
@@ -375,11 +367,6 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, TouchMode::ALL.len(), delta, wrap)
.map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string())
}
RowId::Mouse => {
let cur = MouseMode::ALL.iter().position(|m| *m == s.mouse_mode());
step_option(cur, MouseMode::ALL.len(), delta, wrap)
.map(|i| s.mouse_mode = MouseMode::ALL[i].as_name().to_string())
}
RowId::Stats => {
let cur = StatsVerbosity::ALL
.iter()
@@ -523,33 +510,6 @@ mod tests {
assert_eq!(ctx.settings.touch_mode, "trackpad");
}
#[test]
fn mouse_mode_steps_and_wraps() {
let (mut settings, pads) = ctx_parts();
assert_eq!(settings.mouse_mode, "capture");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
// Capture → Desktop, then a step past the end is a boundary.
assert!(
!adjust(RowId::Mouse, -1, false, &mut ctx),
"already first = thud"
);
assert!(adjust(RowId::Mouse, 1, false, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "desktop");
assert!(!adjust(RowId::Mouse, 1, false, &mut ctx), "last = thud");
// A wraps back to the first.
assert!(adjust(RowId::Mouse, 1, true, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "capture");
}
#[test]
fn unknown_value_snaps_to_first() {
let (mut settings, pads) = ctx_parts();
+9 -211
View File
@@ -66,30 +66,7 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// too ([`MIN_DRIVER_PROTOCOL_VERSION`]) and simply falls back to the re-arrival resize against it;
/// a v4 driver serving an older (v3-asserting) host fails that host's strict handshake — ship
/// driver+host together, as ever.
/// v5: ADDITIVE — the IddCx HARDWARE CURSOR channel (remote-desktop-sweep M2c):
/// [`control::AddRequest::hw_cursor`] (the former tail `_reserved` — same size, same offsets)
/// asks the driver to declare a hardware cursor for this monitor (DWM then EXCLUDES the pointer
/// from the desktop frame and delivers shape/position out-of-band), and
/// [`control::IOCTL_SET_CURSOR_CHANNEL`] delivers the host-created [`cursor::CursorShm`] section
/// the driver's cursor thread seqlock-publishes into. Nothing existing changed; the host gates
/// the feature on the handshake-reported version (`>= 5`) and keeps composited-cursor behavior
/// against older drivers.
/// v6: ADDITIVE — [`control::IOCTL_SET_CURSOR_FORWARD`] (the mid-stream cursor-render flip,
/// remote-desktop-sweep §8): the client's mouse-model flip (un)declares the hardware cursor on a
/// LIVE monitor, so the capture mouse model gets DWM's composited pointer back (full fidelity)
/// and the desktop model gets exclusion + forwarding. Nothing existing changed; against a v5
/// driver the unknown IOCTL fails and the host logs + keeps the declared-at-ADD behavior.
/// v6 tail ext (no bump, the `AddRequest` luminance-tail discipline):
/// [`control::AddReply::cursor_excluded`] — the driver reports whether the ADDed monitor's OS
/// target already carries a hardware-cursor declare from an earlier session. A declare is
/// IRREVOCABLE for the target's life (remote-desktop-sweep §8.6, proven on-glass): DWM never
/// composites the software cursor back into that target's frames, and the sticky state survives
/// monitor REMOVE→ADD because the host hands every client a STABLE target id. The host uses the
/// flag to self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// cursor channel — without it those sessions are silently cursor-less. Both skews degrade
/// cleanly: an old driver writes only the 20-byte reply prefix (host reads `0` = unknown/clean),
/// an old host retrieves a 20-byte buffer (driver writes just the prefix).
pub const PROTOCOL_VERSION: u32 = 6;
pub const PROTOCOL_VERSION: u32 = 4;
/// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on
/// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the
@@ -133,22 +110,6 @@ pub mod control {
/// identity (saved per-monitor DPI) and the driver's swap-chain/stash machinery survive. A v3
/// driver fails this unknown IOCTL → the host falls back to the re-arrival resize.
pub const IOCTL_UPDATE_MODES: u32 = ctl_code(0x907);
/// Deliver a monitor's hardware-cursor channel (v5): the handle VALUE of the unnamed
/// [`cursor::CursorShm`](crate::cursor) file mapping the host duplicated into the WUDFHost
/// (same delivery model as [`IOCTL_SET_FRAME_CHANNEL`], no event — the host polls the
/// seqlock at its encode-tick pace). Sent once after ADD, only for a monitor whose
/// [`AddRequest::hw_cursor`] was set. The driver maps it, calls
/// `IddCxMonitorSetupHardwareCursor`, and starts its cursor-query thread. Input
/// [`SetCursorChannelRequest`].
pub const IOCTL_SET_CURSOR_CHANNEL: u32 = ctl_code(0x908);
/// Flip a LIVE monitor's hardware-cursor declaration (v6, the mid-stream cursor-render
/// flip): `enable = 1` re-declares (`IddCxMonitorSetupHardwareCursor` — DWM excludes the
/// pointer, the query/shape machinery resumes), `enable = 0` un-declares (the driver stops
/// re-declaring on mode commits and asks the OS to revert to the software cursor — DWM
/// composites the pointer into the frame, the pre-channel behavior the capture mouse
/// model wants). Only meaningful for a monitor whose cursor channel was delivered. Input
/// [`SetCursorForwardRequest`].
pub const IOCTL_SET_CURSOR_FORWARD: u32 = ctl_code(0x909);
/// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns
/// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this
@@ -186,13 +147,9 @@ pub mod control {
/// The client display's min luminance in MILLI-nits (0.001 cd/m² — the CTA min-luminance
/// range lives well below 1 nit) → Desired Content Min Luminance. `0` = unknown.
pub min_luminance_millinits: u32,
/// Non-zero = declare an IddCx HARDWARE CURSOR for this monitor (v5, remote-desktop-sweep
/// M2c): DWM stops compositing the pointer into the frame and the driver publishes
/// shape/position into the [`cursor::CursorShm`](crate::cursor) section delivered by
/// [`IOCTL_SET_CURSOR_CHANNEL`]. Byte-compatible with the old tail `_reserved` (offset 36):
/// an un-upgraded driver ignores it (cursor stays composited — the host already gates on
/// the handshake version, this is defense in depth), an un-upgraded host sends `0` (off).
pub hw_cursor: u32,
/// Pads the `u64`-aligned struct to a multiple of 8 (Pod forbids implicit tail padding);
/// free expansion room for the next appended field.
pub _reserved: u32,
}
/// [`AddRequest`]'s size before the client-HDR luminance tail — the prefix an un-upgraded
@@ -217,21 +174,8 @@ pub mod control {
/// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a
/// WUDFHost restart between sessions can never leave the host duplicating into a dead process.
pub wudf_pid: u32,
/// Non-zero = this monitor's OS target already carries an IRREVOCABLE hardware-cursor
/// declare from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer
/// from every frame on this target, forever, and a session without the cursor channel must
/// self-composite (GDI poller + blend) or stream a cursor-less desktop. Appended after
/// [`ADD_REPLY_LEGACY_SIZE`] under the same dual-size discipline as the `AddRequest`
/// luminance tail: an un-upgraded driver writes only the legacy prefix (the host's
/// zero-initialized buffer then reads `0` = unknown/clean), an un-upgraded host retrieves a
/// legacy-size buffer (the driver writes just the prefix).
pub cursor_excluded: u32,
}
/// [`AddReply`]'s size before the `cursor_excluded` tail — the prefix an un-upgraded driver
/// writes and an un-upgraded host retrieves (see the field docs).
pub const ADD_REPLY_LEGACY_SIZE: usize = 20;
/// `IOCTL_REMOVE` input.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
@@ -309,29 +253,6 @@ pub mod control {
/// at the compile-time maximum; `ring_len` says how many entries are live).
pub const RING_LEN_USIZE: usize = RING_LEN as usize;
/// `IOCTL_SET_CURSOR_CHANNEL` input (v5): the hardware-cursor section for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorChannelRequest {
/// The OS target id from [`AddReply`] — which monitor this channel belongs to.
pub target_id: u32,
pub _pad: u32,
/// The [`cursor::CursorShm`](crate::cursor) file-mapping handle VALUE, already duplicated
/// into the driver's WUDFHost process ([`AddReply::wudf_pid`]).
pub header_handle: u64,
}
/// `IOCTL_SET_CURSOR_FORWARD` input (v6): the mid-stream cursor-render flip for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorForwardRequest {
/// The OS target id from [`AddReply`] — which monitor to flip.
pub target_id: u32,
/// `1` = declare the hardware cursor (exclude + forward), `0` = un-declare (DWM
/// composites — the capture mouse model).
pub enable: u32,
}
// Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already
// rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!`
// asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss.
@@ -348,18 +269,13 @@ pub mod control {
assert!(offset_of!(AddRequest, max_luminance_nits) == ADD_REQUEST_LEGACY_SIZE);
assert!(offset_of!(AddRequest, max_frame_avg_nits) == 28);
assert!(offset_of!(AddRequest, min_luminance_millinits) == 32);
// v5: the former tail `_reserved` — same offset, same total size (rename-only).
assert!(offset_of!(AddRequest, hw_cursor) == 36);
assert!(size_of::<AddRequest>() == 40);
assert!(size_of::<AddReply>() == 24);
assert!(size_of::<AddReply>() == 20);
assert!(offset_of!(AddReply, adapter_luid_low) == 0);
assert!(offset_of!(AddReply, adapter_luid_high) == 4);
assert!(offset_of!(AddReply, target_id) == 8);
assert!(offset_of!(AddReply, resolved_monitor_id) == 12);
assert!(offset_of!(AddReply, wudf_pid) == 16);
// The cursor-excluded tail starts exactly at the legacy boundary (prefix-compat).
assert!(offset_of!(AddReply, cursor_excluded) == ADD_REPLY_LEGACY_SIZE);
assert!(size_of::<SetFrameChannelRequest>() == 32 + 8 * RING_LEN_USIZE);
assert!(offset_of!(SetFrameChannelRequest, target_id) == 0);
@@ -372,13 +288,6 @@ pub mod control {
assert!(size_of::<RemoveRequest>() == 8);
assert!(offset_of!(RemoveRequest, session_id) == 0);
assert!(size_of::<SetCursorChannelRequest>() == 16);
assert!(offset_of!(SetCursorChannelRequest, target_id) == 0);
assert!(offset_of!(SetCursorChannelRequest, header_handle) == 8);
assert!(size_of::<SetCursorForwardRequest>() == 8);
assert!(offset_of!(SetCursorForwardRequest, target_id) == 0);
assert!(offset_of!(SetCursorForwardRequest, enable) == 4);
assert!(size_of::<UpdateModesRequest>() == 24);
assert!(offset_of!(UpdateModesRequest, session_id) == 0);
assert!(offset_of!(UpdateModesRequest, width) == 8);
@@ -1051,84 +960,6 @@ pub mod mouse {
};
}
/// The v5 hardware-cursor channel (remote-desktop-sweep M2c): one unnamed file mapping per
/// monitor, host-created, delivered by handle value ([`control::IOCTL_SET_CURSOR_CHANNEL`]).
/// The DRIVER's cursor thread (woken by its IddCx `hNewCursorDataAvailable` event) seqlock-writes
/// shape + position + visibility; the HOST reads at its encode-tick pace — no event crosses the
/// boundary. Writer: bump [`CursorShm::seq`] to ODD, write fields (+ shape bytes when the OS said
/// the shape changed), bump to EVEN. Reader: read seq (retry while odd), copy, re-read seq —
/// unchanged ⇒ consistent snapshot. Position-only updates never touch the shape bytes, so a
/// reader that skips unchanged `shape_id`s never copies torn pixels.
pub mod cursor {
use bytemuck::{Pod, Zeroable};
/// First field of [`CursorShm`] — `b"PFCU"` little-endian; anything else = not attached yet.
pub const CURSOR_MAGIC: u32 = u32::from_le_bytes(*b"PFCU");
/// Max cursor side (px) the driver declares to the OS (`IDDCX_CURSOR_CAPS::MaxX/MaxY`) and
/// the section's shape buffer is sized for. Windows XL accessibility cursors top out here;
/// the host's wire forwarder downscales to its own cap anyway.
pub const CURSOR_SHAPE_MAX: u32 = 256;
/// Shape-buffer bytes: 32-bpp at the declared max.
pub const CURSOR_SHAPE_BYTES: usize = (CURSOR_SHAPE_MAX * CURSOR_SHAPE_MAX * 4) as usize;
/// Byte offset of the shape pixels inside the section (one cache-line-ish header).
pub const CURSOR_SHAPE_OFFSET: usize = 64;
/// Total section size.
pub const CURSOR_SHM_SIZE: usize = CURSOR_SHAPE_OFFSET + CURSOR_SHAPE_BYTES;
/// `IDDCX_CURSOR_SHAPE_TYPE` values mirrored for the host (the driver writes the OS value
/// verbatim into [`CursorShm::cursor_type`]).
pub const CURSOR_TYPE_MASKED_COLOR: u32 = 1;
pub const CURSOR_TYPE_ALPHA: u32 = 2;
/// The section header (the shape pixels follow at [`CURSOR_SHAPE_OFFSET`]). `x`/`y` are the
/// shape's TOP-LEFT in desktop coordinates (the IddCx `IDARG_OUT_QUERY_HWCURSOR::X/Y`
/// convention — position hotspot, can be negative); `shape_id` is the OS's per-set counter
/// (bumps on every shape set, the overlay serial); pixels are the OS's 32-bpp rows at
/// `pitch` bytes (BGRA for ALPHA; color+mask for MASKED_COLOR — the host converts).
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct CursorShm {
pub magic: u32,
/// Seqlock: odd = writer mid-update.
pub seq: u32,
pub visible: u32,
pub cursor_type: u32,
pub x: i32,
pub y: i32,
pub shape_id: u32,
pub width: u32,
pub height: u32,
pub pitch: u32,
pub hot_x: u32,
pub hot_y: u32,
/// Reserved expansion room up to [`CURSOR_SHAPE_OFFSET`].
pub _reserved: [u32; 4],
}
// Layout is load-bearing across the process boundary — pin it.
const _: () = {
use core::mem::{offset_of, size_of};
assert!(size_of::<CursorShm>() == 64);
assert!(size_of::<CursorShm>() <= CURSOR_SHAPE_OFFSET);
assert!(offset_of!(CursorShm, magic) == 0);
assert!(offset_of!(CursorShm, seq) == 4);
assert!(offset_of!(CursorShm, visible) == 8);
assert!(offset_of!(CursorShm, cursor_type) == 12);
assert!(offset_of!(CursorShm, x) == 16);
assert!(offset_of!(CursorShm, y) == 20);
assert!(offset_of!(CursorShm, shape_id) == 24);
assert!(offset_of!(CursorShm, width) == 28);
assert!(offset_of!(CursorShm, height) == 32);
assert!(offset_of!(CursorShm, pitch) == 36);
assert!(offset_of!(CursorShm, hot_x) == 40);
assert!(offset_of!(CursorShm, hot_y) == 44);
};
}
#[cfg(test)]
mod tests {
use super::*;
@@ -1248,7 +1079,7 @@ mod tests {
max_luminance_nits: 800,
max_frame_avg_nits: 400,
min_luminance_millinits: 50, // 0.05 nits
hw_cursor: 1,
_reserved: 0,
};
let bytes = bytemuck::bytes_of(&req);
assert_eq!(bytes.len(), 40);
@@ -1278,19 +1109,14 @@ mod tests {
target_id: 262,
resolved_monitor_id: 7,
wudf_pid: 4242,
cursor_excluded: 1,
};
let rbytes = bytemuck::bytes_of(&reply);
assert_eq!(rbytes.len(), 24);
assert_eq!(rbytes.len(), 20);
assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply);
// resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible.
assert_eq!(rbytes[12..16], 7u32.to_le_bytes());
// The v2 duplication-target pid trails at offset 16.
assert_eq!(rbytes[16..20], 4242u32.to_le_bytes());
// The cursor-excluded tail rides after the legacy boundary; an un-upgraded driver writes
// only the prefix, so a zero-filled tail reads as "unknown/clean" (see the field docs).
assert_eq!(rbytes[20..24], 1u32.to_le_bytes());
assert_eq!(control::ADD_REPLY_LEGACY_SIZE, 20);
}
#[test]
@@ -1335,39 +1161,11 @@ mod tests {
req
);
assert_eq!(bytes[8..12], 2560u32.to_le_bytes());
// The compat window: v4v6 are additive over v3, so the host floor stays at 3.
assert_eq!(PROTOCOL_VERSION, 6);
// The compat window: v4 is additive over v3, so the host floor stays one below.
assert_eq!(PROTOCOL_VERSION, 4);
assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3);
}
#[test]
fn cursor_shm_layout_is_pinned() {
use cursor::*;
// The header must leave the shape offset intact whatever grows inside `_reserved`.
assert_eq!(core::mem::size_of::<CursorShm>(), 64);
assert_eq!(CURSOR_SHM_SIZE, 64 + 256 * 256 * 4);
assert_eq!(CURSOR_MAGIC, u32::from_le_bytes(*b"PFCU"));
// Seqlock snapshot discipline survives a bytemuck roundtrip.
let hdr = CursorShm {
magic: CURSOR_MAGIC,
seq: 2,
visible: 1,
cursor_type: CURSOR_TYPE_ALPHA,
x: -3,
y: 7,
shape_id: 42,
width: 32,
height: 32,
pitch: 128,
hot_x: 4,
hot_y: 5,
_reserved: [0; 4],
};
let bytes = bytemuck::bytes_of(&hdr);
assert_eq!(*bytemuck::from_bytes::<CursorShm>(bytes), hdr);
assert_eq!(bytes[16..20], (-3i32).to_le_bytes());
}
#[test]
fn gamepad_names_and_magics_are_stable() {
assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0");
-5
View File
@@ -25,11 +25,6 @@ tracing = "0.1"
# A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`).
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[target.'cfg(target_os = "windows")'.dev-dependencies]
# The QSV live e2e drives the REAL HdrP010Converter output (an RTV-written, ring-profile P010
# texture) into the encoder — the one seam the CPU-upload tests can't reach.
pf-capture = { path = "../pf-capture" }
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
# Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms.
openh264 = "0.9"
-101
View File
@@ -32,46 +32,6 @@ pub struct EncodedFrame {
pub chunk_aligned: bool,
}
/// One slice-boundary chunk of an encoded AU, emitted by a chunked-poll backend
/// ([`Encoder::poll_chunk`], latency plan §7 LN1): the encoder hands out completed slices while
/// the rest of the frame is still encoding, so packetize/FEC/pacing can overlap the encode tail.
/// The chunks of one AU concatenate to exactly the bytes [`Encoder::poll`] would have returned,
/// and every cut lands on an Annex-B NAL boundary (slice starts). AU-level metadata
/// (`pts_ns`/`keyframe`/`recovery_anchor`/`chunk_aligned`) is authoritative on the FIRST chunk
/// (`first`) — the host opens the wire frame from it; `last` closes the AU. `keyframe` on a
/// non-final chunk is the encoder's own prediction (exact under the P-only/infinite-GOP config —
/// the driver only ever emits an IDR we asked for); the final chunk re-checks it against the
/// driver's reported picture type.
pub struct AuChunk {
pub data: Vec<u8>,
pub pts_ns: u64,
pub keyframe: bool,
/// See [`EncodedFrame::recovery_anchor`].
pub recovery_anchor: bool,
/// See [`EncodedFrame::chunk_aligned`].
pub chunk_aligned: bool,
/// Opens the AU (carries the authoritative AU metadata).
pub first: bool,
/// Closes the AU (the concatenation is complete; the encoder's in-flight slot is released).
pub last: bool,
}
impl AuChunk {
/// A whole AU as a single self-closing chunk — what every non-chunked backend's
/// [`Encoder::poll_chunk`] default emits, so a chunk consumer needs no per-backend fork.
pub fn whole(f: EncodedFrame) -> Self {
AuChunk {
data: f.data,
pts_ns: f.pts_ns,
keyframe: f.keyframe,
recovery_anchor: f.recovery_anchor,
chunk_aligned: f.chunk_aligned,
first: true,
last: true,
}
}
}
/// Codec selection negotiated with the client.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Codec {
@@ -259,11 +219,6 @@ pub struct EncoderCaps {
/// A hardware encoder. One per session; runs on the encode thread.
pub trait Encoder: Send {
/// Submit one captured frame for encoding. Lifetime contract: the caller must keep `frame`
/// (and its GPU payload) alive until this frame's AU has been returned by
/// [`poll`](Self::poll) — a stream-ordered backend (Linux direct-NVENC's IO-stream binding)
/// may still be reading the payload asynchronously after `submit` returns. Both host encode
/// loops already hold the frame across their poll drain; new callers must do the same.
fn submit(&mut self, frame: &CapturedFrame) -> Result<()>;
/// [`submit`](Self::submit) with the **wire frame index** this frame's AU will carry — the
/// number the packetizer stamps on it and the client's loss reports/RFI requests name. The
@@ -309,37 +264,8 @@ pub trait Encoder: Send {
fn invalidate_ref_frames(&mut self, _first_frame: i64, _last_frame: i64) -> bool {
false
}
/// Escalate into a pipelined (two-thread) retrieve mode under sustained GPU contention — the
/// encoder analog of the capturer depth escalation: AUs ride ~one loop tick behind (`poll`
/// may return `None` while an encode is in flight) in exchange for capture/submit no longer
/// serializing on the encode wait. Returns whether pipelined retrieve is (now) active; the
/// switch may be deferred to the next safe point internally. `false` from the default impl =
/// unsupported — the session loop stops asking. De-escalation is a v2 item everywhere.
fn set_pipelined(&mut self, _on: bool) -> bool {
false
}
/// Pull the next encoded AU if one is ready.
fn poll(&mut self) -> Result<Option<EncodedFrame>>;
/// Whether [`poll_chunk`](Self::poll_chunk) currently emits sub-AU chunks — i.e. the LIVE
/// session has slice-level readback armed (Linux direct-NVENC with the
/// `PUNKTFUNK_NVENC_SLICES` and `PUNKTFUNK_NVENC_SUBFRAME` knobs on a sync depth-1
/// retrieve). Dynamic, not static: a pipelined-retrieve escalation or a session rebuild can
/// turn it off — re-query per AU, never cache across frames. `false` (the default) means
/// `poll_chunk` degrades to one whole-AU chunk per frame.
fn supports_chunked_poll(&self) -> bool {
false
}
/// Pull the next slice-boundary chunk of the oldest in-flight AU (latency plan §7 LN1).
/// Semantics when chunking is live: BLOCKS until the next chunk is readable, and the final
/// (`last`) chunk blocks exactly like [`poll`](Self::poll) does — the depth-1 pump treats
/// `None` as re-poll-next-tick, so a non-blocking tail would ride the AU one tick late (the
/// `6dc195f9` Vulkan bug class). `Ok(None)` only when no AU is in flight. Each AU must be
/// drained through ONE method: calling `poll` on a partially-chunked AU is a caller bug (the
/// backend errors rather than double-emit bytes). Default: delegates to `poll`, wrapping the
/// whole AU as a single `first && last` chunk.
fn poll_chunk(&mut self) -> Result<Option<AuChunk>> {
Ok(self.poll()?.map(AuChunk::whole))
}
/// Tear the underlying hardware encoder down and rebuild it in place, keeping the session's
/// negotiated parameters — the encode-stall watchdog's recovery lever (a wedged AMF/QSV
/// driver stops emitting AUs or accepting frames without ever returning an error). Returns
@@ -367,16 +293,6 @@ pub trait Encoder: Send {
/// flagged [`EncodedFrame::chunk_aligned`] and the session marks them on the wire.
/// Default: no-op (the H.26x backends' bitstreams cannot be cut losslessly).
fn set_wire_chunking(&mut self, _shard_payload: usize) {}
/// How many frames the CAPTURER guarantees the encoder may hold in flight before it starts
/// reusing an input texture (`Capturer::pipeline_depth`). Backends that encode the capturer's
/// textures IN PLACE — no `CopyResource` — must not pipeline deeper than this: the capturer
/// rotates its output ring per delivered frame with no regard for encode completion, so a
/// deeper pipeline lets it overwrite a texture mid-encode. That is visual corruption (torn or
/// mixed frames), not UB, so it fails silently and intermittently.
///
/// Called once by the session glue after the capturer is known; a backend that copies its
/// input, or is synchronous, ignores it. Default: no-op.
fn set_input_ring_depth(&mut self, _depth: usize) {}
/// Signal end-of-stream. After this, drain the remaining AUs with [`poll`](Self::poll)
/// until it returns `None` — NVENC buffers frames internally even at `delay=0`.
fn flush(&mut self) -> Result<()>;
@@ -496,23 +412,6 @@ mod tests {
}
}
/// The whole-AU chunk (every non-chunked backend's `poll_chunk` shape) must carry the AU's
/// metadata verbatim and be self-closing (`first && last`).
#[test]
fn whole_au_chunk_is_self_closing() {
let c = AuChunk::whole(EncodedFrame {
data: vec![0, 0, 0, 1, 0x40],
pts_ns: 42,
keyframe: true,
recovery_anchor: true,
chunk_aligned: false,
});
assert_eq!(c.data, vec![0, 0, 0, 1, 0x40]);
assert_eq!(c.pts_ns, 42);
assert!(c.keyframe && c.recovery_anchor && !c.chunk_aligned);
assert!(c.first && c.last);
}
/// Wire round-trip and the stats label stay in lockstep with the `quic::CODEC_*` bits.
#[test]
fn codec_wire_roundtrip_and_label() {
@@ -0,0 +1,105 @@
// Cursor-overlay blend kernels for the CUDA/NVENC path (cursor-as-metadata). The cursor bitmap is
// straight-alpha RGBA, row-packed (stride = curW*4). Blended into the encoder-OWNED NVENC input
// surface — never the compositor's dmabuf. One thread per cursor pixel (ARGB / YUV444) or per 2x2
// chroma block (NV12). Coefficients are BT.709 limited, matching rgb2yuv.comp so the cursor colour
// matches the rest of the frame regardless of which zero-copy backend encodes it.
//
// Build (regenerate cursor_blend.ptx after editing):
// nvcc -ptx -arch=compute_75 cursor_blend.cu -o cursor_blend.ptx
// PTX is JIT'd by the driver forward to the actual GPU, so a compute_75 (Turing) baseline runs on
// every Turing-or-newer NVENC GPU. (CUDA 13's nvcc no longer targets pre-Turing archs.)
typedef unsigned char u8;
__device__ __forceinline__ u8 blend8(int dst, int src, int a) {
return (u8)((src * a + dst * (255 - a)) / 255);
}
// Packed 4-byte surface. NVENC's ARGB format stores bytes B,G,R,A in memory; the cursor is R,G,B,A.
extern "C" __global__ void blend_argb(
u8* surf, int pitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int cx = blockIdx.x * blockDim.x + threadIdx.x;
int cy = blockIdx.y * blockDim.y + threadIdx.y;
if (cx >= curW || cy >= curH) return;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) return;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) return;
u8* d = surf + (size_t)py * pitch + (size_t)px * 4;
d[0] = blend8(d[0], s[2], a); // B <- cursor B
d[1] = blend8(d[1], s[1], a); // G <- cursor G
d[2] = blend8(d[2], s[0], a); // R <- cursor R
}
// Planar YUV444: three full-res planes stacked at base, base+plane, base+2*plane (plane=pitch*surfH).
extern "C" __global__ void blend_yuv444(
u8* base, int pitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int cx = blockIdx.x * blockDim.x + threadIdx.x;
int cy = blockIdx.y * blockDim.y + threadIdx.y;
if (cx >= curW || cy >= curH) return;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) return;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) return;
float R = s[0], G = s[1], B = s[2];
int Y = (int)(16.0f + 0.1826f * R + 0.6142f * G + 0.0620f * B + 0.5f);
int U = (int)(128.0f - 0.1006f * R - 0.3386f * G + 0.4392f * B + 0.5f);
int V = (int)(128.0f + 0.4392f * R - 0.3989f * G - 0.0403f * B + 0.5f);
size_t plane = (size_t)pitch * surfH;
u8* yp = base + (size_t)py * pitch + px;
u8* up = base + plane + (size_t)py * pitch + px;
u8* vp = base + 2 * plane + (size_t)py * pitch + px;
*yp = blend8(*yp, Y, a);
*up = blend8(*up, U, a);
*vp = blend8(*vp, V, a);
}
// NV12: full-res Y plane + interleaved half-res UV plane. One thread per 2x2 luma block; each blends
// up to four Y samples and one (alpha-weighted) UV sample.
extern "C" __global__ void blend_nv12(
u8* yb, int yPitch, u8* uvb, int uvPitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int bx = blockIdx.x * blockDim.x + threadIdx.x;
int by = blockIdx.y * blockDim.y + threadIdx.y;
int base_cx = bx * 2, base_cy = by * 2;
if (base_cx >= curW || base_cy >= curH) return;
float ua = 0.0f, va = 0.0f, wa = 0.0f;
int cnt = 0;
for (int j = 0; j < 2; j++) {
for (int i = 0; i < 2; i++) {
int cx = base_cx + i, cy = base_cy + j;
if (cx >= curW || cy >= curH) continue;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) continue;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) continue;
float R = s[0], G = s[1], B = s[2];
int Y = (int)(16.0f + 0.1826f * R + 0.6142f * G + 0.0620f * B + 0.5f);
u8* yp = yb + (size_t)py * yPitch + px;
*yp = blend8(*yp, Y, a);
ua += (128.0f - 0.1006f * R - 0.3386f * G + 0.4392f * B) * a;
va += (128.0f + 0.4392f * R - 0.3989f * G - 0.0403f * B) * a;
wa += a;
cnt++;
}
}
if (wa <= 0.0f || cnt == 0) return;
// The chroma sample covering this block's top-left surface pixel.
int uvx = (ox + base_cx) / 2;
int uvy = (oy + base_cy) / 2;
if (uvx < 0 || uvy < 0 || uvx * 2 >= surfW || uvy * 2 >= surfH) return;
int U = (int)(ua / wa + 0.5f);
int V = (int)(va / wa + 0.5f);
int amean = (int)(wa / cnt + 0.5f);
u8* uv = uvb + (size_t)uvy * uvPitch + (size_t)uvx * 2;
uv[0] = blend8(uv[0], U, amean);
uv[1] = blend8(uv[1], V, amean);
}
@@ -0,0 +1,576 @@
//
// Generated by NVIDIA NVVM Compiler
//
// Compiler Build ID: CL-38244171
// Cuda compilation tools, release 13.3, V13.3.73
// Based on NVVM 7.0.1
//
.version 9.3
.target sm_75
.address_size 64
// .globl blend_argb
.visible .entry blend_argb(
.param .u64 blend_argb_param_0,
.param .u32 blend_argb_param_1,
.param .u32 blend_argb_param_2,
.param .u32 blend_argb_param_3,
.param .u64 blend_argb_param_4,
.param .u32 blend_argb_param_5,
.param .u32 blend_argb_param_6,
.param .u32 blend_argb_param_7,
.param .u32 blend_argb_param_8
)
{
.reg .pred %p<10>;
.reg .b16 %rs<2>;
.reg .b32 %r<34>;
.reg .b64 %rd<17>;
ld.param.u64 %rd2, [blend_argb_param_0];
ld.param.u32 %r5, [blend_argb_param_1];
ld.param.u32 %r6, [blend_argb_param_2];
ld.param.u32 %r7, [blend_argb_param_3];
ld.param.u64 %rd3, [blend_argb_param_4];
ld.param.u32 %r8, [blend_argb_param_5];
ld.param.u32 %r11, [blend_argb_param_6];
ld.param.u32 %r9, [blend_argb_param_7];
ld.param.u32 %r10, [blend_argb_param_8];
mov.u32 %r12, %ntid.x;
mov.u32 %r13, %ctaid.x;
mov.u32 %r14, %tid.x;
mad.lo.s32 %r1, %r13, %r12, %r14;
mov.u32 %r15, %ntid.y;
mov.u32 %r16, %ctaid.y;
mov.u32 %r17, %tid.y;
mad.lo.s32 %r2, %r16, %r15, %r17;
setp.ge.s32 %p1, %r1, %r8;
setp.ge.s32 %p2, %r2, %r11;
or.pred %p3, %p1, %p2;
@%p3 bra $L__BB0_4;
add.s32 %r3, %r1, %r9;
add.s32 %r4, %r2, %r10;
or.b32 %r18, %r4, %r3;
setp.lt.s32 %p4, %r18, 0;
setp.ge.s32 %p5, %r3, %r6;
or.pred %p6, %p5, %p4;
setp.ge.s32 %p7, %r4, %r7;
or.pred %p8, %p7, %p6;
@%p8 bra $L__BB0_4;
mad.lo.s32 %r19, %r2, %r8, %r1;
mul.wide.s32 %rd4, %r19, 4;
cvta.to.global.u64 %rd5, %rd3;
add.s64 %rd1, %rd5, %rd4;
ld.global.u8 %rs1, [%rd1+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB0_4;
cvt.u32.u16 %r20, %rs1;
mul.wide.s32 %rd6, %r4, %r5;
mul.wide.s32 %rd7, %r3, 4;
add.s64 %rd8, %rd6, %rd7;
cvta.to.global.u64 %rd9, %rd2;
add.s64 %rd10, %rd9, %rd8;
ld.global.u8 %r21, [%rd10];
ld.global.u8 %r22, [%rd1+2];
xor.b32 %r23, %r20, 255;
mul.lo.s32 %r24, %r23, %r21;
mad.lo.s32 %r25, %r22, %r20, %r24;
mul.wide.u32 %rd11, %r25, -2139062143;
shr.u64 %rd12, %rd11, 39;
st.global.u8 [%rd10], %rd12;
ld.global.u8 %r26, [%rd10+1];
ld.global.u8 %r27, [%rd1+1];
mul.lo.s32 %r28, %r23, %r26;
mad.lo.s32 %r29, %r27, %r20, %r28;
mul.wide.u32 %rd13, %r29, -2139062143;
shr.u64 %rd14, %rd13, 39;
st.global.u8 [%rd10+1], %rd14;
ld.global.u8 %r30, [%rd10+2];
ld.global.u8 %r31, [%rd1];
mul.lo.s32 %r32, %r23, %r30;
mad.lo.s32 %r33, %r31, %r20, %r32;
mul.wide.u32 %rd15, %r33, -2139062143;
shr.u64 %rd16, %rd15, 39;
st.global.u8 [%rd10+2], %rd16;
$L__BB0_4:
ret;
}
// .globl blend_yuv444
.visible .entry blend_yuv444(
.param .u64 blend_yuv444_param_0,
.param .u32 blend_yuv444_param_1,
.param .u32 blend_yuv444_param_2,
.param .u32 blend_yuv444_param_3,
.param .u64 blend_yuv444_param_4,
.param .u32 blend_yuv444_param_5,
.param .u32 blend_yuv444_param_6,
.param .u32 blend_yuv444_param_7,
.param .u32 blend_yuv444_param_8
)
{
.reg .pred %p<10>;
.reg .b16 %rs<5>;
.reg .f32 %f<16>;
.reg .b32 %r<49>;
.reg .b64 %rd<14>;
ld.param.u64 %rd2, [blend_yuv444_param_0];
ld.param.u32 %r5, [blend_yuv444_param_1];
ld.param.u32 %r6, [blend_yuv444_param_2];
ld.param.u32 %r7, [blend_yuv444_param_3];
ld.param.u64 %rd3, [blend_yuv444_param_4];
ld.param.u32 %r8, [blend_yuv444_param_5];
ld.param.u32 %r11, [blend_yuv444_param_6];
ld.param.u32 %r9, [blend_yuv444_param_7];
ld.param.u32 %r10, [blend_yuv444_param_8];
mov.u32 %r12, %ntid.x;
mov.u32 %r13, %ctaid.x;
mov.u32 %r14, %tid.x;
mad.lo.s32 %r1, %r13, %r12, %r14;
mov.u32 %r15, %ntid.y;
mov.u32 %r16, %ctaid.y;
mov.u32 %r17, %tid.y;
mad.lo.s32 %r2, %r16, %r15, %r17;
setp.ge.s32 %p1, %r1, %r8;
setp.ge.s32 %p2, %r2, %r11;
or.pred %p3, %p1, %p2;
@%p3 bra $L__BB1_4;
add.s32 %r3, %r1, %r9;
add.s32 %r4, %r2, %r10;
or.b32 %r18, %r4, %r3;
setp.lt.s32 %p4, %r18, 0;
setp.ge.s32 %p5, %r3, %r6;
or.pred %p6, %p5, %p4;
setp.ge.s32 %p7, %r4, %r7;
or.pred %p8, %p7, %p6;
@%p8 bra $L__BB1_4;
mad.lo.s32 %r19, %r2, %r8, %r1;
mul.wide.s32 %rd4, %r19, 4;
cvta.to.global.u64 %rd5, %rd3;
add.s64 %rd1, %rd5, %rd4;
ld.global.u8 %rs1, [%rd1+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB1_4;
cvt.u32.u16 %r20, %rs1;
ld.global.u8 %rs2, [%rd1];
cvt.rn.f32.u16 %f1, %rs2;
ld.global.u8 %rs3, [%rd1+1];
cvt.rn.f32.u16 %f2, %rs3;
ld.global.u8 %rs4, [%rd1+2];
cvt.rn.f32.u16 %f3, %rs4;
fma.rn.f32 %f4, %f1, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f5, %f2, 0f3F1D3C36, %f4;
fma.rn.f32 %f6, %f3, 0f3D7DF3B6, %f5;
add.f32 %f7, %f6, 0f3F000000;
cvt.rzi.s32.f32 %r21, %f7;
fma.rn.f32 %f8, %f1, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f9, %f2, 0fBEAD5CFB, %f8;
fma.rn.f32 %f10, %f3, 0f3EE0DED3, %f9;
add.f32 %f11, %f10, 0f3F000000;
cvt.rzi.s32.f32 %r22, %f11;
fma.rn.f32 %f12, %f1, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f13, %f2, 0fBECC3C9F, %f12;
fma.rn.f32 %f14, %f3, 0fBD25119D, %f13;
add.f32 %f15, %f14, 0f3F000000;
cvt.rzi.s32.f32 %r23, %f15;
mul.wide.s32 %rd6, %r4, %r5;
cvt.s64.s32 %rd7, %r3;
add.s64 %rd8, %rd6, %rd7;
cvta.to.global.u64 %rd9, %rd2;
add.s64 %rd10, %rd9, %rd8;
ld.global.u8 %r24, [%rd10];
mul.lo.s32 %r25, %r21, %r20;
xor.b32 %r26, %r20, 255;
mad.lo.s32 %r27, %r26, %r24, %r25;
mul.hi.s32 %r28, %r27, -2139062143;
add.s32 %r29, %r28, %r27;
shr.u32 %r30, %r29, 31;
shr.u32 %r31, %r29, 7;
add.s32 %r32, %r31, %r30;
st.global.u8 [%rd10], %r32;
mul.wide.s32 %rd11, %r7, %r5;
add.s64 %rd12, %rd10, %rd11;
ld.global.u8 %r33, [%rd12];
mul.lo.s32 %r34, %r22, %r20;
mad.lo.s32 %r35, %r26, %r33, %r34;
mul.hi.s32 %r36, %r35, -2139062143;
add.s32 %r37, %r36, %r35;
shr.u32 %r38, %r37, 31;
shr.u32 %r39, %r37, 7;
add.s32 %r40, %r39, %r38;
st.global.u8 [%rd12], %r40;
add.s64 %rd13, %rd12, %rd11;
ld.global.u8 %r41, [%rd13];
mul.lo.s32 %r42, %r23, %r20;
mad.lo.s32 %r43, %r26, %r41, %r42;
mul.hi.s32 %r44, %r43, -2139062143;
add.s32 %r45, %r44, %r43;
shr.u32 %r46, %r45, 31;
shr.u32 %r47, %r45, 7;
add.s32 %r48, %r47, %r46;
st.global.u8 [%rd13], %r48;
$L__BB1_4:
ret;
}
// .globl blend_nv12
.visible .entry blend_nv12(
.param .u64 blend_nv12_param_0,
.param .u32 blend_nv12_param_1,
.param .u64 blend_nv12_param_2,
.param .u32 blend_nv12_param_3,
.param .u32 blend_nv12_param_4,
.param .u32 blend_nv12_param_5,
.param .u64 blend_nv12_param_6,
.param .u32 blend_nv12_param_7,
.param .u32 blend_nv12_param_8,
.param .u32 blend_nv12_param_9,
.param .u32 blend_nv12_param_10
)
{
.reg .pred %p<43>;
.reg .b16 %rs<17>;
.reg .f32 %f<108>;
.reg .b32 %r<123>;
.reg .b64 %rd<35>;
ld.param.u64 %rd11, [blend_nv12_param_0];
ld.param.u32 %r21, [blend_nv12_param_1];
ld.param.u64 %rd10, [blend_nv12_param_2];
ld.param.u32 %r22, [blend_nv12_param_3];
ld.param.u32 %r23, [blend_nv12_param_4];
ld.param.u32 %r24, [blend_nv12_param_5];
ld.param.u64 %rd12, [blend_nv12_param_6];
ld.param.u32 %r25, [blend_nv12_param_7];
ld.param.u32 %r26, [blend_nv12_param_8];
ld.param.u32 %r27, [blend_nv12_param_9];
ld.param.u32 %r28, [blend_nv12_param_10];
cvta.to.global.u64 %rd1, %rd11;
cvta.to.global.u64 %rd2, %rd12;
mov.u32 %r29, %ntid.x;
mov.u32 %r30, %ctaid.x;
mov.u32 %r31, %tid.x;
mad.lo.s32 %r32, %r30, %r29, %r31;
mov.u32 %r33, %ntid.y;
mov.u32 %r34, %ctaid.y;
mov.u32 %r35, %tid.y;
mad.lo.s32 %r36, %r34, %r33, %r35;
shl.b32 %r1, %r32, 1;
shl.b32 %r2, %r36, 1;
setp.ge.s32 %p1, %r1, %r25;
setp.ge.s32 %p2, %r2, %r26;
or.pred %p3, %p1, %p2;
mov.f32 %f102, 0f00000000;
mov.f32 %f103, 0f00000000;
mov.f32 %f104, 0f00000000;
@%p3 bra $L__BB2_19;
cvt.s64.s32 %rd3, %r21;
add.s32 %r3, %r2, %r28;
setp.ge.s32 %p4, %r3, %r24;
mul.lo.s32 %r4, %r2, %r25;
mul.wide.s32 %rd4, %r3, %r21;
add.s32 %r5, %r1, %r27;
or.b32 %r38, %r5, %r3;
setp.lt.s32 %p5, %r38, 0;
mov.u32 %r121, 0;
setp.ge.s32 %p6, %r5, %r23;
or.pred %p7, %p6, %p5;
or.pred %p8, %p4, %p7;
@%p8 bra $L__BB2_4;
add.s32 %r40, %r1, %r4;
mul.wide.s32 %rd13, %r40, 4;
add.s64 %rd5, %rd2, %rd13;
ld.global.u8 %rs1, [%rd5+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB2_4;
cvt.u32.u16 %r42, %rs1;
ld.global.u8 %rs5, [%rd5];
cvt.rn.f32.u16 %f31, %rs5;
ld.global.u8 %rs6, [%rd5+1];
cvt.rn.f32.u16 %f32, %rs6;
ld.global.u8 %rs7, [%rd5+2];
cvt.rn.f32.u16 %f33, %rs7;
fma.rn.f32 %f34, %f31, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f35, %f32, 0f3F1D3C36, %f34;
fma.rn.f32 %f36, %f33, 0f3D7DF3B6, %f35;
add.f32 %f37, %f36, 0f3F000000;
cvt.rzi.s32.f32 %r43, %f37;
cvt.s64.s32 %rd14, %r5;
add.s64 %rd15, %rd4, %rd14;
add.s64 %rd16, %rd1, %rd15;
ld.global.u8 %r44, [%rd16];
mul.lo.s32 %r45, %r43, %r42;
xor.b32 %r46, %r42, 255;
mad.lo.s32 %r47, %r46, %r44, %r45;
mul.hi.s32 %r48, %r47, -2139062143;
add.s32 %r49, %r48, %r47;
shr.u32 %r50, %r49, 31;
shr.u32 %r51, %r49, 7;
add.s32 %r52, %r51, %r50;
st.global.u8 [%rd16], %r52;
fma.rn.f32 %f38, %f31, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f39, %f32, 0fBEAD5CFB, %f38;
fma.rn.f32 %f40, %f33, 0f3EE0DED3, %f39;
cvt.rn.f32.u16 %f104, %rs1;
fma.rn.f32 %f102, %f40, %f104, 0f00000000;
fma.rn.f32 %f41, %f31, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f42, %f32, 0fBECC3C9F, %f41;
fma.rn.f32 %f43, %f33, 0fBD25119D, %f42;
fma.rn.f32 %f103, %f43, %f104, 0f00000000;
mov.u32 %r121, 1;
$L__BB2_4:
add.s32 %r7, %r1, 1;
setp.ge.s32 %p10, %r7, %r25;
@%p10 bra $L__BB2_8;
add.s32 %r8, %r7, %r27;
or.b32 %r53, %r8, %r3;
setp.lt.s32 %p12, %r53, 0;
setp.ge.s32 %p13, %r8, %r23;
or.pred %p14, %p13, %p12;
or.pred %p15, %p4, %p14;
@%p15 bra $L__BB2_8;
add.s32 %r54, %r7, %r4;
mul.wide.s32 %rd17, %r54, 4;
add.s64 %rd6, %rd2, %rd17;
ld.global.u8 %rs2, [%rd6+3];
setp.eq.s16 %p16, %rs2, 0;
@%p16 bra $L__BB2_8;
cvt.u32.u16 %r55, %rs2;
ld.global.u8 %rs8, [%rd6];
cvt.rn.f32.u16 %f44, %rs8;
ld.global.u8 %rs9, [%rd6+1];
cvt.rn.f32.u16 %f45, %rs9;
ld.global.u8 %rs10, [%rd6+2];
cvt.rn.f32.u16 %f46, %rs10;
fma.rn.f32 %f47, %f44, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f48, %f45, 0f3F1D3C36, %f47;
fma.rn.f32 %f49, %f46, 0f3D7DF3B6, %f48;
add.f32 %f50, %f49, 0f3F000000;
cvt.rzi.s32.f32 %r56, %f50;
cvt.s64.s32 %rd18, %r8;
add.s64 %rd19, %rd4, %rd18;
add.s64 %rd20, %rd1, %rd19;
ld.global.u8 %r57, [%rd20];
mul.lo.s32 %r58, %r56, %r55;
xor.b32 %r59, %r55, 255;
mad.lo.s32 %r60, %r59, %r57, %r58;
mul.hi.s32 %r61, %r60, -2139062143;
add.s32 %r62, %r61, %r60;
shr.u32 %r63, %r62, 31;
shr.u32 %r64, %r62, 7;
add.s32 %r65, %r64, %r63;
st.global.u8 [%rd20], %r65;
fma.rn.f32 %f51, %f44, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f52, %f45, 0fBEAD5CFB, %f51;
fma.rn.f32 %f53, %f46, 0f3EE0DED3, %f52;
cvt.rn.f32.u16 %f54, %rs2;
fma.rn.f32 %f102, %f53, %f54, %f102;
fma.rn.f32 %f55, %f44, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f56, %f45, 0fBECC3C9F, %f55;
fma.rn.f32 %f57, %f46, 0fBD25119D, %f56;
fma.rn.f32 %f103, %f57, %f54, %f103;
add.f32 %f104, %f104, %f54;
add.s32 %r121, %r121, 1;
$L__BB2_8:
add.s32 %r11, %r2, 1;
setp.ge.s32 %p17, %r11, %r26;
add.s32 %r12, %r11, %r28;
add.s32 %r13, %r4, %r25;
cvt.s64.s32 %rd21, %r12;
mul.lo.s64 %rd7, %rd21, %rd3;
@%p17 bra $L__BB2_12;
setp.ge.s32 %p18, %r12, %r24;
or.b32 %r66, %r5, %r12;
setp.lt.s32 %p19, %r66, 0;
or.pred %p21, %p6, %p19;
or.pred %p22, %p18, %p21;
@%p22 bra $L__BB2_12;
add.s32 %r67, %r1, %r13;
mul.wide.s32 %rd22, %r67, 4;
add.s64 %rd8, %rd2, %rd22;
ld.global.u8 %rs3, [%rd8+3];
setp.eq.s16 %p23, %rs3, 0;
@%p23 bra $L__BB2_12;
cvt.u32.u16 %r68, %rs3;
ld.global.u8 %rs11, [%rd8];
cvt.rn.f32.u16 %f58, %rs11;
ld.global.u8 %rs12, [%rd8+1];
cvt.rn.f32.u16 %f59, %rs12;
ld.global.u8 %rs13, [%rd8+2];
cvt.rn.f32.u16 %f60, %rs13;
fma.rn.f32 %f61, %f58, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f62, %f59, 0f3F1D3C36, %f61;
fma.rn.f32 %f63, %f60, 0f3D7DF3B6, %f62;
add.f32 %f64, %f63, 0f3F000000;
cvt.rzi.s32.f32 %r69, %f64;
cvt.s64.s32 %rd23, %r5;
add.s64 %rd24, %rd7, %rd23;
add.s64 %rd25, %rd1, %rd24;
ld.global.u8 %r70, [%rd25];
mul.lo.s32 %r71, %r69, %r68;
xor.b32 %r72, %r68, 255;
mad.lo.s32 %r73, %r72, %r70, %r71;
mul.hi.s32 %r74, %r73, -2139062143;
add.s32 %r75, %r74, %r73;
shr.u32 %r76, %r75, 31;
shr.u32 %r77, %r75, 7;
add.s32 %r78, %r77, %r76;
st.global.u8 [%rd25], %r78;
fma.rn.f32 %f65, %f58, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f66, %f59, 0fBEAD5CFB, %f65;
fma.rn.f32 %f67, %f60, 0f3EE0DED3, %f66;
cvt.rn.f32.u16 %f68, %rs3;
fma.rn.f32 %f102, %f67, %f68, %f102;
fma.rn.f32 %f69, %f58, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f70, %f59, 0fBECC3C9F, %f69;
fma.rn.f32 %f71, %f60, 0fBD25119D, %f70;
fma.rn.f32 %f103, %f71, %f68, %f103;
add.f32 %f104, %f104, %f68;
add.s32 %r121, %r121, 1;
$L__BB2_12:
or.pred %p26, %p17, %p10;
@%p26 bra $L__BB2_16;
setp.ge.s32 %p27, %r12, %r24;
add.s32 %r16, %r7, %r27;
or.b32 %r79, %r16, %r12;
setp.lt.s32 %p28, %r79, 0;
setp.ge.s32 %p29, %r16, %r23;
or.pred %p30, %p29, %p28;
or.pred %p31, %p27, %p30;
@%p31 bra $L__BB2_16;
add.s32 %r80, %r7, %r13;
mul.wide.s32 %rd26, %r80, 4;
add.s64 %rd9, %rd2, %rd26;
ld.global.u8 %rs4, [%rd9+3];
setp.eq.s16 %p32, %rs4, 0;
@%p32 bra $L__BB2_16;
cvt.u32.u16 %r81, %rs4;
ld.global.u8 %rs14, [%rd9];
cvt.rn.f32.u16 %f72, %rs14;
ld.global.u8 %rs15, [%rd9+1];
cvt.rn.f32.u16 %f73, %rs15;
ld.global.u8 %rs16, [%rd9+2];
cvt.rn.f32.u16 %f74, %rs16;
fma.rn.f32 %f75, %f72, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f76, %f73, 0f3F1D3C36, %f75;
fma.rn.f32 %f77, %f74, 0f3D7DF3B6, %f76;
add.f32 %f78, %f77, 0f3F000000;
cvt.rzi.s32.f32 %r82, %f78;
cvt.s64.s32 %rd27, %r16;
add.s64 %rd28, %rd7, %rd27;
add.s64 %rd29, %rd1, %rd28;
ld.global.u8 %r83, [%rd29];
mul.lo.s32 %r84, %r82, %r81;
xor.b32 %r85, %r81, 255;
mad.lo.s32 %r86, %r85, %r83, %r84;
mul.hi.s32 %r87, %r86, -2139062143;
add.s32 %r88, %r87, %r86;
shr.u32 %r89, %r88, 31;
shr.u32 %r90, %r88, 7;
add.s32 %r91, %r90, %r89;
st.global.u8 [%rd29], %r91;
fma.rn.f32 %f79, %f72, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f80, %f73, 0fBEAD5CFB, %f79;
fma.rn.f32 %f81, %f74, 0f3EE0DED3, %f80;
cvt.rn.f32.u16 %f82, %rs4;
fma.rn.f32 %f102, %f81, %f82, %f102;
fma.rn.f32 %f83, %f72, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f84, %f73, 0fBECC3C9F, %f83;
fma.rn.f32 %f85, %f74, 0fBD25119D, %f84;
fma.rn.f32 %f103, %f85, %f82, %f103;
add.f32 %f104, %f104, %f82;
add.s32 %r121, %r121, 1;
$L__BB2_16:
setp.eq.s32 %p33, %r121, 0;
setp.le.f32 %p34, %f104, 0f00000000;
or.pred %p35, %p34, %p33;
@%p35 bra $L__BB2_19;
shr.u32 %r92, %r5, 31;
add.s32 %r93, %r5, %r92;
shr.s32 %r19, %r93, 1;
setp.lt.s32 %p36, %r3, -1;
setp.lt.s32 %p37, %r5, -1;
or.pred %p38, %p37, %p36;
and.b32 %r94, %r93, -2;
setp.ge.s32 %p39, %r94, %r23;
or.pred %p40, %p38, %p39;
shr.u32 %r95, %r3, 31;
add.s32 %r96, %r3, %r95;
shr.s32 %r20, %r96, 1;
and.b32 %r97, %r96, -2;
setp.ge.s32 %p41, %r97, %r24;
or.pred %p42, %p40, %p41;
@%p42 bra $L__BB2_19;
div.rn.f32 %f86, %f102, %f104;
add.f32 %f87, %f86, 0f3F000000;
cvt.rzi.s32.f32 %r98, %f87;
div.rn.f32 %f88, %f103, %f104;
add.f32 %f89, %f88, 0f3F000000;
cvt.rzi.s32.f32 %r99, %f89;
cvt.rn.f32.s32 %f90, %r121;
div.rn.f32 %f91, %f104, %f90;
add.f32 %f92, %f91, 0f3F000000;
cvt.rzi.s32.f32 %r100, %f92;
mul.wide.s32 %rd30, %r20, %r22;
mul.wide.s32 %rd31, %r19, 2;
add.s64 %rd32, %rd30, %rd31;
cvta.to.global.u64 %rd33, %rd10;
add.s64 %rd34, %rd33, %rd32;
ld.global.u8 %r101, [%rd34];
mul.lo.s32 %r102, %r100, %r98;
mov.u32 %r103, 255;
sub.s32 %r104, %r103, %r100;
mad.lo.s32 %r105, %r104, %r101, %r102;
mul.hi.s32 %r106, %r105, -2139062143;
add.s32 %r107, %r106, %r105;
shr.u32 %r108, %r107, 31;
shr.u32 %r109, %r107, 7;
add.s32 %r110, %r109, %r108;
st.global.u8 [%rd34], %r110;
ld.global.u8 %r111, [%rd34+1];
mul.lo.s32 %r112, %r100, %r99;
mad.lo.s32 %r113, %r104, %r111, %r112;
mul.hi.s32 %r114, %r113, -2139062143;
add.s32 %r115, %r114, %r113;
shr.u32 %r116, %r115, 31;
shr.u32 %r117, %r115, 7;
add.s32 %r118, %r117, %r116;
st.global.u8 [%rd34+1], %r118;
$L__BB2_19:
ret;
}
+3 -3
View File
@@ -826,7 +826,7 @@ impl NvencEncoder {
(*f).linesize[i] as usize,
)
});
pf_zerocopy::cuda::copy_yuv444_to_device(buf, dsts, true)
pf_zerocopy::cuda::copy_yuv444_to_device(buf, dsts)
} else if self.want_444 {
ffi::av_frame_free(&mut f);
bail!(
@@ -839,11 +839,11 @@ impl NvencEncoder {
let y_pitch = (*f).linesize[0] as usize;
let uv_ptr = (*f).data[1] as pf_zerocopy::cuda::CUdeviceptr;
let uv_pitch = (*f).linesize[1] as usize;
pf_zerocopy::cuda::copy_nv12_to_device(buf, y_ptr, y_pitch, uv_ptr, uv_pitch, true)
pf_zerocopy::cuda::copy_nv12_to_device(buf, y_ptr, y_pitch, uv_ptr, uv_pitch)
} else {
let dst_ptr = (*f).data[0] as pf_zerocopy::cuda::CUdeviceptr;
let dst_pitch = (*f).linesize[0] as usize;
pf_zerocopy::cuda::copy_device_to_device(buf, dst_ptr, dst_pitch, true)
pf_zerocopy::cuda::copy_device_to_device(buf, dst_ptr, dst_pitch)
};
if let Err(e) = copy_res {
ffi::av_frame_free(&mut f);
File diff suppressed because it is too large Load Diff
+1 -18
View File
@@ -1176,24 +1176,7 @@ impl Encoder for PyroWaveEncoder {
fn submit(&mut self, frame: &CapturedFrame) -> Result<()> {
// SAFETY: single-threaded encoder; `encode_frame` records/submits on handles this
// struct owns and waits its own fence before touching results.
let r = unsafe { self.encode_frame(frame) };
if r.is_err() {
// `encode_frame` opens the recording window early and has several fallible steps
// inside it (cursor prep, dmabuf import, format mapping, the CPU-RGB staging path,
// an unsupported-payload bail, and the encode call itself). Every one returns with
// `self.cmd` still RECORDING, and nothing downstream repairs it — there is exactly
// one `begin_command_buffer` in this file and `reset()`/`Drop` never touch `cmd` —
// so the NEXT frame would call `begin` on a recording buffer, which is invalid usage.
// Legal here on every path: the pool carries RESET_COMMAND_BUFFER and the buffer is
// not pending (we never reached the submit, or the submit itself failed).
// SAFETY: `self.cmd` is owned by this encoder and, on these paths, not in flight.
unsafe {
let _ = self
.device
.reset_command_buffer(self.cmd, vk::CommandBufferResetFlags::empty());
}
}
r
unsafe { self.encode_frame(frame) }
}
fn caps(&self) -> EncoderCaps {
+29 -91
View File
@@ -37,11 +37,9 @@ pub(crate) fn fourcc_to_vk(fourcc: u32) -> Option<vk::Format> {
const AR24: u32 = 0x3432_5241; // ARGB8888
const XB24: u32 = 0x3432_4258; // XBGR8888
const AB24: u32 = 0x3432_4241; // ABGR8888
const NV12: u32 = 0x3231_564e; // DRM_FORMAT_NV12
match fourcc {
XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM),
XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM),
NV12 => Some(vk::Format::G8_B8R8_2PLANE_420_UNORM),
_ => None,
}
}
@@ -79,82 +77,39 @@ pub(crate) unsafe fn import_rgb_dmabuf(
d: &pf_frame::DmabufFrame,
cw: u32,
ch: u32,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
import_rgb_dmabuf_as(
device,
ext_fd,
mem_props,
d,
cw,
ch,
vk::ImageUsageFlags::SAMPLED,
None,
)
}
/// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list.
/// Despite the historical name, this also imports gamescope's one-fd LINEAR NV12: the UV
/// subresource layout comes from the producer's plane-1 chunk when it reported one, falling
/// back to the shared-stride contiguous-plane contract.
#[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn import_rgb_dmabuf_as(
device: &ash::Device,
ext_fd: &ash::khr::external_memory_fd::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
d: &pf_frame::DmabufFrame,
cw: u32,
ch: u32,
usage: vk::ImageUsageFlags,
profile_list: Option<&mut vk::VideoProfileListInfoKHR>,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
use anyhow::Context;
use std::os::fd::IntoRawFd;
let fmt = fourcc_to_vk(d.fourcc)
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
d.offset as u64 + d.stride as u64 * ch as u64,
d.stride as u64,
));
vec![
vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64),
vk::SubresourceLayout::default()
.offset(uv_offset)
.row_pitch(uv_stride),
]
} else {
vec![vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64)]
};
let plane = [vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64)];
let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default()
.drm_format_modifier(d.modifier)
.plane_layouts(&planes);
.plane_layouts(&plane);
let mut ext = vk::ExternalMemoryImageCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT);
let mut ci = vk::ImageCreateInfo::default()
.image_type(vk::ImageType::TYPE_2D)
.format(fmt)
.extent(vk::Extent3D {
width: cw,
height: ch,
depth: 1,
})
.mip_levels(1)
.array_layers(1)
.samples(vk::SampleCountFlags::TYPE_1)
.tiling(vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT)
.usage(usage)
.sharing_mode(vk::SharingMode::EXCLUSIVE)
.initial_layout(vk::ImageLayout::UNDEFINED)
.push_next(&mut ext)
.push_next(&mut drm);
if let Some(pl) = profile_list {
ci = ci.push_next(pl);
}
let img = device.create_image(&ci, None)?;
let img = device.create_image(
&vk::ImageCreateInfo::default()
.image_type(vk::ImageType::TYPE_2D)
.format(fmt)
.extent(vk::Extent3D {
width: cw,
height: ch,
depth: 1,
})
.mip_levels(1)
.array_layers(1)
.samples(vk::SampleCountFlags::TYPE_1)
.tiling(vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT)
.usage(vk::ImageUsageFlags::SAMPLED)
.sharing_mode(vk::SharingMode::EXCLUSIVE)
.initial_layout(vk::ImageLayout::UNDEFINED)
.push_next(&mut ext)
.push_next(&mut drm),
None,
)?;
// dup the fd; Vulkan takes ownership of the dup on a successful import.
let dup = d.fd.try_clone().context("dup dmabuf fd")?.into_raw_fd();
let fd_props = {
@@ -228,8 +183,7 @@ pub(crate) unsafe fn make_plain_image(
None,
)?;
let req = device.get_image_memory_requirements(img);
// Unwind on failure: callers (the encoders' open paths) only ever see the completed triple.
let mem = match device.allocate_memory(
let mem = device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(find_mem(
@@ -238,24 +192,8 @@ pub(crate) unsafe fn make_plain_image(
vk::MemoryPropertyFlags::DEVICE_LOCAL,
)),
None,
) {
Ok(m) => m,
Err(e) => {
device.destroy_image(img, None);
return Err(e.into());
}
};
if let Err(e) = device.bind_image_memory(img, mem, 0) {
device.destroy_image(img, None);
device.free_memory(mem, None);
return Err(e.into());
}
match make_view(device, img, fmt, 0) {
Ok(view) => Ok((img, mem, view)),
Err(e) => {
device.destroy_image(img, None);
device.free_memory(mem, None);
Err(e)
}
}
)?;
device.bind_image_memory(img, mem, 0)?;
let view = make_view(device, img, fmt, 0)?;
Ok((img, mem, view))
}
@@ -1,82 +0,0 @@
//! Vendored `VK_VALVE_video_encode_rgb_conversion` bindings — the RGB→YCbCr encode-source
//! extension (Vulkan 1.4.327; RADV since Mesa 26.0, hardware-gated on the VCN EFC front-end
//! conversion block). Our pinned `ash 0.38.0+1.3.281` predates it entirely; same vendoring
//! rationale as [`vk_av1_encode`](super::vk_av1_encode) — definitions copied from the registry
//! so the layouts are correct-by-construction, chained via raw `p_next`. Consumed by
//! `vulkan_video.rs`: B0 probes + logs availability (design/vulkan-rgb-direct-encode.md);
//! B1 makes the captured BGRx dmabuf the direct encode source with EFC doing the 709-narrow CSC.
#![allow(dead_code)]
use ash::vk;
use std::ffi::{c_void, CStr};
pub const EXTENSION_NAME: &CStr = c"VK_VALVE_video_encode_rgb_conversion";
// ---------- struct-type (VkStructureType) values — construct via `stype` ----------
pub const ST_PHYSICAL_DEVICE_FEATURES: i32 = 1_000_390_000;
pub const ST_CAPABILITIES: i32 = 1_000_390_001;
pub const ST_PROFILE_INFO: i32 = 1_000_390_002;
pub const ST_SESSION_CREATE_INFO: i32 = 1_000_390_003;
// `VkVideoEncodeRgbModelConversionFlagBitsVALVE`
pub const MODEL_RGB_IDENTITY: u32 = 0x01;
pub const MODEL_YCBCR_IDENTITY: u32 = 0x02;
pub const MODEL_YCBCR_709: u32 = 0x04;
pub const MODEL_YCBCR_601: u32 = 0x08;
pub const MODEL_YCBCR_2020: u32 = 0x10;
// `VkVideoEncodeRgbRangeCompressionFlagBitsVALVE`
pub const RANGE_FULL: u32 = 0x01;
pub const RANGE_NARROW: u32 = 0x02;
// `VkVideoEncodeRgbChromaOffsetFlagBitsVALVE`
pub const CHROMA_OFFSET_COSITED_EVEN: u32 = 0x01;
pub const CHROMA_OFFSET_MIDPOINT: u32 = 0x02;
/// `VkPhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE` — chain into
/// `VkPhysicalDeviceFeatures2` (query) / `VkDeviceCreateInfo` (enable).
#[repr(C)]
pub struct PhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE {
pub s_type: vk::StructureType,
pub p_next: *mut c_void,
pub video_encode_rgb_conversion: vk::Bool32,
}
/// `VkVideoEncodeRgbConversionCapabilitiesVALVE` — chain into the
/// `vkGetPhysicalDeviceVideoCapabilitiesKHR` output when the queried profile carries
/// [`VideoEncodeProfileRgbConversionInfoVALVE`]; reports which conversions the HW does.
#[repr(C)]
pub struct VideoEncodeRgbConversionCapabilitiesVALVE {
pub s_type: vk::StructureType,
pub p_next: *mut c_void,
pub rgb_models: u32,
pub rgb_ranges: u32,
pub x_chroma_offsets: u32,
pub y_chroma_offsets: u32,
}
/// `VkVideoEncodeProfileRgbConversionInfoVALVE` — part of the video-profile *identity*: every
/// consumer of the profile (caps query, format query, session, image profile lists) must carry
/// the same chain.
#[repr(C)]
pub struct VideoEncodeProfileRgbConversionInfoVALVE {
pub s_type: vk::StructureType,
pub p_next: *const c_void,
pub perform_encode_rgb_conversion: vk::Bool32,
}
/// `VkVideoEncodeSessionRgbConversionCreateInfoVALVE` — chain into
/// `VkVideoSessionCreateInfoKHR`; single-bit selections of the conversion actually performed.
#[repr(C)]
pub struct VideoEncodeSessionRgbConversionCreateInfoVALVE {
pub s_type: vk::StructureType,
pub p_next: *const c_void,
pub rgb_model: u32,
pub rgb_range: u32,
pub x_chroma_offset: u32,
pub y_chroma_offset: u32,
}
/// `vk::StructureType` for a raw `ST_*` constant above.
#[inline]
pub fn stype(raw: i32) -> vk::StructureType {
vk::StructureType::from_raw(raw)
}
File diff suppressed because it is too large Load Diff
-69
View File
@@ -35,38 +35,6 @@ pub(super) fn codec_guid(codec: Codec) -> nv::GUID {
}
}
/// Resolved per-frame slice count for a session (latency plan §7 LN1, Phase 3): the
/// `PUNKTFUNK_NVENC_SLICES` env override wins (1..=32; **1 = the explicit single-slice
/// escape**, needed now that a backend can default higher), else the backend's
/// `default_slices` — 4 on Linux direct-NVENC since the Phase-3 default-on, 1 everywhere else
/// (the Windows async path is deliberately untouched). H.264/HEVC only (AV1 partitions via
/// tiles). ONE parse shared by the config author ([`apply_low_latency_config`] via
/// [`LowLatencyConfig::slices`]) and the Linux backend's chunked-poll arming, so the two can
/// never disagree about whether a session is multi-slice.
pub(super) fn resolve_slices(codec: Codec, default_slices: u32) -> u32 {
if !matches!(codec, Codec::H264 | Codec::H265) {
return 1;
}
std::env::var("PUNKTFUNK_NVENC_SLICES")
.ok()
.and_then(|s| s.parse::<u32>().ok())
.filter(|n| (1..=32).contains(n))
.unwrap_or(default_slices)
}
/// Resolved sub-frame readback (`enableSubFrameWrite` + `reportSliceOffsets`; sync sessions
/// only, see [`build_init_params`]): `PUNKTFUNK_NVENC_SUBFRAME` tri-state — `0` = never (the
/// default-on escape), `1` = force (even where the caps probe says unsupported — an operator
/// explicitly testing), unset = the backend's `default_on` (Linux direct-NVENC passes its
/// SUBFRAME_READBACK caps-probe result since Phase 3; Windows passes `false`).
pub(super) fn resolve_subframe(default_on: bool) -> bool {
match std::env::var("PUNKTFUNK_NVENC_SUBFRAME").as_deref() {
Ok("0") => false,
Ok("1") => true,
_ => default_on,
}
}
/// Reference-frame DPB depth when RFI is supported (Apollo uses 5). A deeper DPB lets an invalidated
/// reference fall back to an older still-valid frame instead of a full IDR; `numRefL0 = 1` keeps each
/// P-frame single-reference for low latency. Also the window the backends' `invalidate_ref_frames`
@@ -98,9 +66,6 @@ pub(super) struct LowLatencyConfig {
pub hdr: bool,
/// This GPU supports reference-frame invalidation (a deeper DPB for graceful loss recovery).
pub rfi_supported: bool,
/// Resolved per-frame slice count ([`resolve_slices`] — env override, else the backend
/// default). ≤ 1 leaves the preset's single slice untouched.
pub slices: u32,
}
/// Author the shared `NV_ENC_INITIALIZE_PARAMS` (P1/ULL preset, PTD, the session dimensions/rate)
@@ -117,7 +82,6 @@ pub(super) fn build_init_params(
cfg: &mut nv::NV_ENC_CONFIG,
split_mode: u32,
enable_async: bool,
subframe: bool,
) -> nv::NV_ENC_INITIALIZE_PARAMS {
let mut init = nv::NV_ENC_INITIALIZE_PARAMS {
version: nv::NV_ENC_INITIALIZE_PARAMS_VER,
@@ -137,16 +101,6 @@ pub(super) fn build_init_params(
};
// splitEncodeMode is a C bitfield — set via the generated accessor, not a struct field.
init.set_splitEncodeMode(split_mode);
// Sub-frame readback (latency plan §7 LN1; default-on for Linux direct-NVENC since Phase 3 —
// the caller resolves `subframe` via [`resolve_subframe`] + its caps probe): the driver
// writes each slice into the output buffer as it completes and reports per-slice offsets, so
// a sync-mode consumer can read slices out while the frame is still encoding. Pair with
// multi-slice (a single-slice frame yields nothing to read early). `reportSliceOffsets`
// requires `enableEncodeAsync = 0`, so async (Windows) sessions never arm.
if !enable_async && subframe {
init.set_enableSubFrameWrite(1);
init.set_reportSliceOffsets(1);
}
init
}
@@ -164,9 +118,6 @@ pub(super) unsafe fn apply_low_latency_config(cfg: &mut nv::NV_ENC_CONFIG, c: Lo
cfg.gopLength = nv::NVENC_INFINITE_GOPLENGTH;
cfg.frameIntervalP = 1;
cfg.rcParams.rateControlMode = nv::NV_ENC_PARAMS_RC_MODE::NV_ENC_PARAMS_RC_CBR;
// Explicit zero reorder delay: with P-only + no lookahead there is no reordering to buffer,
// but pin the bit so no preset/driver default can ever slip a frame of reorder delay in.
cfg.rcParams.set_zeroReorderDelay(1);
let bps = c.bitrate.min(u32::MAX as u64) as u32;
cfg.rcParams.averageBitRate = bps;
cfg.rcParams.maxBitRate = bps;
@@ -195,26 +146,6 @@ pub(super) unsafe fn apply_low_latency_config(cfg: &mut nv::NV_ENC_CONFIG, c: Lo
Codec::PyroWave => unreachable!("PyroWave never opens the direct-NVENC backend"),
}
// Multi-slice frames (latency plan §7 LN1): `c.slices` splits every frame into N slices
// (sliceMode 3 = "N slices per frame"), the unit sub-frame readback ships early and loss
// concealment can discard independently. Costs ~1-2 % bitrate in slice headers. H.264/HEVC
// only — AV1 partitions via tiles, not slices (the resolver already returns 1 there).
// Default 4 on Linux direct-NVENC (Phase 3), env-only elsewhere; ≤ 1 keeps the preset's
// single slice.
if let Some(n) = Some(c.slices).filter(|n| *n >= 2) {
match c.codec {
Codec::H264 => {
cfg.encodeCodecConfig.h264Config.sliceMode = 3;
cfg.encodeCodecConfig.h264Config.sliceModeData = n;
}
Codec::H265 => {
cfg.encodeCodecConfig.hevcConfig.sliceMode = 3;
cfg.encodeCodecConfig.hevcConfig.sliceModeData = n;
}
Codec::Av1 | Codec::PyroWave => {}
}
}
// Chroma + bit depth. Full-chroma 4:4:4 (HEVC Range Extensions, chromaFormatIDC=3 under the FREXT
// profile) takes precedence and composes with 10-bit (Main 4:4:4 10); it needs a full-chroma-
// capable input. Otherwise 10-bit selects Main10 (HEVC) or the AV1 output depth — stamping the
+6 -43
View File
@@ -37,8 +37,7 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::nvenc_core::{
apply_low_latency_config, build_init_params, codec_guid, resolve_slices, resolve_subframe,
LowLatencyConfig, NvStatusExt, RFI_DPB,
apply_low_latency_config, build_init_params, codec_guid, LowLatencyConfig, NvStatusExt, RFI_DPB,
};
use super::nvenc_status;
use super::{ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
@@ -410,12 +409,6 @@ pub struct NvencD3d11Encoder {
events: Vec<usize>,
/// Async mode: the retrieve thread + its channels (`None` = classic same-thread sync retrieve).
async_rt: Option<AsyncRetrieve>,
/// The capturer's `pipeline_depth` (`set_input_ring_depth`). This backend encodes the
/// capturer's textures IN PLACE, so it is a HARD ceiling on async in-flight depth: the
/// capturer rotates its ring per delivered frame regardless of encode completion, so
/// pipelining deeper lets it overwrite a texture mid-encode (torn frames). `None` until the
/// session glue reports it — treated as "unknown, don't pipeline past the env cap".
input_ring_depth: Option<usize>,
/// `NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT` from the caps probe — gates the async retrieve mode.
async_supported: bool,
/// (bitstream, mapped input resource to unmap after retrieval, pts_ns, recovery-anchor) per
@@ -512,7 +505,6 @@ impl NvencD3d11Encoder {
bitstreams: Vec::new(),
events: Vec::new(),
async_rt: None,
input_ring_depth: None,
async_supported: false,
pending: VecDeque::new(),
frame_idx: 0,
@@ -745,10 +737,6 @@ impl NvencD3d11Encoder {
av1_input_depth_minus8: if ten_bit_in { 2 } else { 0 },
hdr: self.hdr,
rfi_supported: self.rfi_supported,
// Env-only on Windows (default single slice): the Phase-3 default-on is a
// Linux-direct-NVENC decision — the Windows async path stays untouched, and a
// Windows operator opting in must choose slices+sync over async retrieve.
slices: resolve_slices(self.codec, 1),
},
);
Ok(cfg)
@@ -796,9 +784,6 @@ impl NvencD3d11Encoder {
&mut cfg,
split_mode,
enable_async,
// Windows: env opt-in only ("1"), never a default — and build_init_params
// additionally refuses it on an async session.
resolve_subframe(false),
);
match (api().initialize_encoder)(enc, &mut init).nv_ok() {
@@ -1171,21 +1156,11 @@ impl Encoder for NvencD3d11Encoder {
// index, which is non-zero on a mid-session encoder rebuild's first frame.
let opening = self.next == 0;
// Async backpressure: never hand NVENC an output bitstream that is still in flight, and
// keep in-flight depth within the capturer's texture ring. At the cap, block on the OLDEST
// completion (the retrieve thread is already waiting on its event) before submitting more —
// bounding depth exactly like the sync path's per-tick blocking poll, just `cap` deep
// instead of 1.
//
// The ring term is the one that matters for correctness: `async_inflight_cap()` is only the
// output-bitstream-pool ceiling plus an env knob, and consults NOTHING about the capturer,
// despite this comment previously claiming otherwise. Since this backend encodes the
// capturer's textures in place, exceeding the capturer's declared `pipeline_depth` lets it
// rotate a texture out from under a live encode — torn frames, silently.
let cap = match self.input_ring_depth {
Some(d) => async_inflight_cap().min(d.max(1)),
None => async_inflight_cap(),
};
while self.async_rt.is_some() && self.pending.len() >= cap {
// keep in-flight depth within the capturer's texture ring (see `async_inflight_cap`). At
// the cap, block on the OLDEST completion (the retrieve thread is already waiting on its
// event) before submitting more — bounding depth exactly like the sync path's per-tick
// blocking poll, just `cap` deep instead of 1.
while self.async_rt.is_some() && self.pending.len() >= async_inflight_cap() {
let done = {
let rt = self.async_rt.as_mut().expect("checked in loop condition");
rt.done_rx
@@ -1361,17 +1336,6 @@ impl Encoder for NvencD3d11Encoder {
self.submit(frame)
}
fn set_input_ring_depth(&mut self, depth: usize) {
// This backend registers and encodes the capturer's textures in place (no CopyResource),
// so the capturer's ring depth is a hard ceiling on how deep async may pipeline.
self.input_ring_depth = Some(depth);
tracing::debug!(
depth,
env_cap = async_inflight_cap(),
"NVENC: capturer input-ring depth reported — async in-flight bounded by the smaller"
);
}
fn request_keyframe(&mut self) {
self.force_kf = true;
}
@@ -1576,7 +1540,6 @@ impl Encoder for NvencD3d11Encoder {
&mut cfg,
self.split_mode,
self.session_async,
resolve_subframe(false),
),
..Default::default()
};
+6 -57
View File
@@ -43,9 +43,6 @@ const BS_SLACK: usize = 256 * 1024;
/// (a desktop-switch device recreate), in which case the stale imports are evicted + destroyed.
const IMPORT_CACHE_CAP: usize = 8;
/// Plane-import cache key: the texture's COM address plus the extent it was imported at.
type PlaneKey = (isize, u32, u32);
// --- Vulkan enum values not surfaced by pyrowave-sys' bindgen (only enums *reachable* from the
// pyrowave C API are generated; these plain #define / flags-typedef values are stable spec
// constants). bindgen renders every reachable Vulkan enum as a `u32` type alias, so these u32
@@ -139,12 +136,8 @@ pub struct PyroWaveEncoder {
// Imported plane textures, cached by the out-ring texture's raw pointer (stable per ring slot):
// the full-res R8 Y plane and the half-res R8G8 CbCr plane, imported SEPARATELY (a single planar
// NV12 import is unreliable on NVIDIA at arbitrary sizes).
/// The capturer ring generation the cached plane imports below belong to. A recreate bumps it,
/// and every cached import is destroyed — the COM addresses they are keyed on can be recycled
/// by the allocator after a recreate, so identity cannot rest on the pointer alone.
ring_gen: Option<u32>,
y_images: Vec<(PlaneKey, pw::pyrowave_image)>,
cbcr_images: Vec<(PlaneKey, pw::pyrowave_image)>,
y_images: Vec<(isize, pw::pyrowave_image)>,
cbcr_images: Vec<(isize, pw::pyrowave_image)>,
width: u32,
height: u32,
@@ -275,7 +268,6 @@ impl PyroWaveEncoder {
pw_dev,
pw_enc,
sync: std::ptr::null_mut(),
ring_gen: None,
y_images: Vec::new(),
cbcr_images: Vec::new(),
width,
@@ -359,16 +351,10 @@ impl PyroWaveEncoder {
///
/// # Safety
/// Same contract as [`import_plane`].
/// Keyed on `(texture address, width, height)` rather than the bare address: the COM pointer
/// carries no reference here, so a released texture's address can be recycled by a later
/// allocation and return an import describing the WRONG surface. Folding the extent in means a
/// recycled address at a different size can never alias. (A recycle at the SAME size is still
/// possible in principle — the complete fix is to key on the capturer's ring generation, which
/// needs that generation plumbed onto `PyroFrameShare`.)
unsafe fn cached_plane(
cache: &mut Vec<(PlaneKey, pw::pyrowave_image)>,
cache: &mut Vec<(isize, pw::pyrowave_image)>,
make: impl FnOnce() -> Result<pw::pyrowave_image>,
key: PlaneKey,
key: isize,
) -> Result<pw::pyrowave_image> {
if let Some((_, img)) = cache.iter().find(|(k, _)| *k == key) {
return Ok(*img);
@@ -437,21 +423,6 @@ impl PyroWaveEncoder {
!self.pw_enc.is_null(),
"pyrowave: encode after a failed reset (encoder was destroyed and not rebuilt)"
);
// The plane textures are imported at the encoder's CONFIGURED extent, not the frame's, so a
// capture that changed size would be read under a stale `VkImageCreateInfo`. This is
// reachable without any client Reconfigure: the IDD capturer autonomously recreates its ring
// on a confirmed display-descriptor change (e.g. a fullscreen game mode-setting the virtual
// display). Refuse instead — the session must reopen the encoder at the new mode. Mirrors
// the guard the QSV and AMF backends already carry.
anyhow::ensure!(
frame.width == self.width && frame.height == self.height,
"pyrowave: captured frame {}x{} != encoder {}x{} (the capturer recreated its ring at a \
new mode the encoder must be reopened)",
frame.width,
frame.height,
self.width,
self.height
);
let FramePayload::D3d11(d3d) = &frame.payload else {
bail!("pyrowave (Windows) needs a D3D11 frame (the capturer must be in pyrowave mode)")
};
@@ -460,25 +431,6 @@ impl PyroWaveEncoder {
in pyrowave mode (session_plan::output_format must set OutputFormat::pyrowave)",
)?;
// Ring recreate ⇒ every cached plane import belongs to textures that no longer exist. Their
// COM addresses can be handed back out by the allocator, so a pointer-keyed hit could return
// an image bound to freed memory. Flush on the generation change rather than relying on the
// address (or the FIFO cap) to notice.
if self.ring_gen != Some(share.ring_gen) {
if self.ring_gen.is_some() {
tracing::info!(
from = ?self.ring_gen,
to = share.ring_gen,
cached = self.y_images.len() + self.cbcr_images.len(),
"pyrowave: capturer recreated its ring — flushing stale plane imports"
);
}
for (_, img) in self.y_images.drain(..).chain(self.cbcr_images.drain(..)) {
pw::pyrowave_image_destroy(img);
}
self.ring_gen = Some(share.ring_gen);
}
// Import the fence whenever this encoder has no timeline yet — the first frame, OR a fresh
// encoder after a client mode-switch rebuild (the capturer passes the persistent handle on
// every frame precisely so a rebuilt encoder can re-import it).
@@ -513,7 +465,7 @@ impl PyroWaveEncoder {
};
let pw_dev = self.pw_dev;
let y_img = {
let key = (d3d.texture.as_raw() as isize, w, h);
let key = d3d.texture.as_raw() as isize;
let tex = &d3d.texture;
Self::cached_plane(
&mut self.y_images,
@@ -522,7 +474,7 @@ impl PyroWaveEncoder {
)?
};
let cbcr_img = {
let key = (share.cbcr.as_raw() as isize, cw, ch);
let key = share.cbcr.as_raw() as isize;
let tex = &share.cbcr;
Self::cached_plane(
&mut self.cbcr_images,
@@ -1024,9 +976,6 @@ mod tests {
cbcr: cbcr_tex,
fence_handle: Some(fence_handle.0 as isize),
fence_value: 1,
// One synthetic ring for the whole case: a constant generation exercises the
// steady-state cache-hit path (a changing one would flush every frame).
ring_gen: 1,
}),
}),
cursor: None,
+11 -291
View File
@@ -478,14 +478,10 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b
});
// Colour signalling, written UNCONDITIONALLY (mirrors nvenc_core.rs): the input is already
// CSC'd to a specific matrix — BT.709 limited for SDR (the capture-side VideoConverter),
// BT.2020 PQ for HDR (HdrP010Converter) — so the stream must say so. An SDR stream without a
// colour description leaves the choice to the decoder's "unspecified" default, and
// Moonlight/third-party/Android-vendor decoders default to 601 at sub-HD → mis-rendered
// colours. (10-bit sessions are the HDR path on Windows — same coupling as NVENC.)
// HDR signalling (10-bit sessions are the HDR path on Windows — same coupling as NVENC):
// BT.2020/PQ colour description + the source's mastering/CLL grade at every IDR.
let hdr = cfg.ten_bit && cfg.codec != Codec::H264;
let vsi = {
let vsi = hdr.then(|| {
// SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtVideoSignalInfo> = Box::new(unsafe { std::mem::zeroed() });
b.Header.BufferId = vpl::MFX_EXTBUFF_VIDEO_SIGNAL_INFO as u32;
@@ -493,17 +489,11 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b.VideoFormat = 5; // unspecified
b.VideoFullRange = 0;
b.ColourDescriptionPresent = 1;
if hdr {
b.ColourPrimaries = 9; // BT.2020
b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ)
b.MatrixCoefficients = 9; // BT.2020 non-constant
} else {
b.ColourPrimaries = 1; // BT.709
b.TransferCharacteristics = 1; // BT.709
b.MatrixCoefficients = 1; // BT.709
}
Some(b)
};
b.ColourPrimaries = 9; // BT.2020
b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ)
b.MatrixCoefficients = 9; // BT.2020 non-constant
b
});
let mastering = cfg.hdr_meta.filter(|_| hdr).map(|m| {
// SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtMasteringDisplayColourVolume> =
@@ -724,16 +714,7 @@ pub struct QsvEncoder {
/// `EncoderCaps::supports_rfi` and all per-frame marking/forcing below.
ltr_active: bool,
/// The wire frame index stored in each LTR slot (`None` = never marked).
///
/// This mirrors the HARDWARE DPB, so an entry must not be cleared merely because we distrust
/// it: nulling issues no VPL call, and the encoder keeps the frame marked long-term until that
/// `LongTermIdx` is re-marked or an IDR flushes it. Distrust is recorded in `ltr_tainted`
/// instead, so the rejection list can still NAME the entry the hardware is holding.
ltr_slots: [Option<i64>; NUM_LTR_SLOTS],
/// Per-slot taint from `invalidate_ref_frames`' sweep: the mark is still live in the hardware
/// DPB but was encoded inside the client's corrupt window, so it may not anchor a recovery —
/// it must be REJECTED instead. Cleared wherever the slot is re-marked or the DPB is flushed.
ltr_tainted: [bool; NUM_LTR_SLOTS],
next_ltr_slot: usize,
ltr_mark_interval: i64,
/// Set by `invalidate_ref_frames`: the slot the next submitted frame force-references.
@@ -808,7 +789,6 @@ impl QsvEncoder {
ir_active: false,
ltr_active: false,
ltr_slots: [None; NUM_LTR_SLOTS],
ltr_tainted: [false; NUM_LTR_SLOTS],
next_ltr_slot: 0,
ltr_mark_interval: ltr_mark_interval(fps),
pending_force: None,
@@ -933,7 +913,6 @@ impl QsvEncoder {
self.ltr_active = ltr_active;
self.ir_active = ir_active;
self.ltr_slots = [None; NUM_LTR_SLOTS];
self.ltr_tainted = [false; NUM_LTR_SLOTS];
self.next_ltr_slot = 0;
self.pending_force = None;
self.hdr_applied = self.hdr_meta;
@@ -1059,7 +1038,6 @@ impl Encoder for QsvEncoder {
// An IDR voids the decoder's reference buffers — drop stale slots and any
// queued force; the mark cadence below re-anchors on the IDR itself.
self.ltr_slots = [None; NUM_LTR_SLOTS];
self.ltr_tainted = [false; NUM_LTR_SLOTS]; // the IDR flushed the DPB with them
self.next_ltr_slot = 0;
self.pending_force = None;
} else if self.ltr_test_force_at == Some(cur_idx) {
@@ -1075,9 +1053,7 @@ impl Encoder for QsvEncoder {
// emptied the slot since the force was queued. An empty slot means there is
// nothing clean to re-reference — the frame must ship as a plain P WITHOUT the
// `recovery_anchor` tag (the client lifts its post-loss freeze on that tag).
// The slot is no longer emptied by the sweep, so test the taint flag too — a
// tainted slot is exactly the "nothing clean to re-reference" case.
if let Some(idx) = self.ltr_slots[slot].filter(|_| !self.ltr_tainted[slot]) {
if let Some(idx) = self.ltr_slots[slot] {
force_ltr = Some((slot, idx));
recovery_anchor = true;
}
@@ -1085,9 +1061,6 @@ impl Encoder for QsvEncoder {
if force_ltr.is_none() && (forced || cur_idx % self.ltr_mark_interval == 0) {
let slot = self.next_ltr_slot;
self.ltr_slots[slot] = Some(cur_idx);
// Re-marking replaces the hardware's LongTermIdx: the tainted frame is gone from
// the DPB and this slot is clean again.
self.ltr_tainted[slot] = false;
self.next_ltr_slot = (self.next_ltr_slot + 1) % NUM_LTR_SLOTS;
mark_slot = Some(slot);
}
@@ -1350,23 +1323,13 @@ impl Encoder for QsvEncoder {
// loss ships corruption as the recovery anchor, and every subsequent mark re-samples
// the soup — the sustained-loss field failure where the picture never healed. Dropped
// slots stay dropped; the cadence re-marks a clean frame within ~1/4 s.
//
// Mark tainted rather than clearing: `ltr_slots` mirrors the HARDWARE DPB, and nulling an
// entry issues no VPL call — the frame stays marked long-term in the encoder. Clearing it
// made the rejection list below (which iterates the post-sweep mirror and only names `Some`
// slots) silently SKIP the one entry the sweep exists to distrust, so the recovery frame
// could still predict from it. With two slots the "exactly one swept" case is the modal
// one, and it was the broken one.
for (slot, marked) in self.ltr_slots.iter().enumerate() {
for marked in self.ltr_slots.iter_mut() {
if marked.is_some_and(|idx| idx >= first) {
self.ltr_tainted[slot] = true;
*marked = None;
}
}
let mut best: Option<(usize, i64)> = None;
for (slot, marked) in self.ltr_slots.iter().enumerate() {
if self.ltr_tainted[slot] {
continue; // still in the DPB, but encoded inside the corrupt window
}
if let Some(idx) = *marked {
if idx < first && best.is_none_or(|(_, b)| idx > b) {
best = Some((slot, idx));
@@ -1491,7 +1454,6 @@ impl Encoder for QsvEncoder {
self.ltr_active = ltr;
self.ir_active = ir;
self.ltr_slots = [None; NUM_LTR_SLOTS];
self.ltr_tainted = [false; NUM_LTR_SLOTS];
self.next_ltr_slot = 0;
self.pending_force = None;
if let Some(inner) = self.inner.as_mut() {
@@ -2004,246 +1966,4 @@ mod tests {
"the bitrate retarget emitted a keyframe (StartNewSequence leak)"
);
}
/// FULL-CHAIN colour check at the field capture size: a known P010 colour-bar source at
/// 1920x1080 — whose height is NOT 16-aligned, so the ingest `CopySubresourceRegion` copies
/// into a 1920x1088 runtime pool surface whose chroma plane sits at a DIFFERENT row offset
/// than the source's (the seam no 640x480 test exercises) — encoded to Main10 HEVC and
/// dumped to `%TEMP%\pf_qsv_1080_bars.h265` for off-box decode verification against the
/// same codes. On-box this asserts stream shape; the pixel verdict needs a decoder.
#[test]
fn qsv_live_p010_1080_colorbars_dump() {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE,
D3D11_SDK_VERSION, D3D11_SUBRESOURCE_DATA, D3D11_USAGE_DEFAULT,
};
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_P010, DXGI_SAMPLE_DESC};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
// (Y, Cb, Cr) 10-bit limited codes for the 8 sRGB bars white/yellow/cyan/green/magenta/
// red/blue/black at 80-nit SDR white under PQ/BT.2020 — the same math as pf-capture's
// `p010_reference` (and the bars_pq2020 client fixture). Stored MSB-aligned (`<<6`).
const BARS: [(u16, u16, u16); 8] = [
(490, 512, 512),
(478, 423, 518),
(464, 525, 473),
(450, 432, 476),
(350, 584, 585),
(325, 448, 598),
(226, 650, 535),
(64, 512, 512),
];
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
// P010 initial data: plane 0 = H rows of W u16 luma; plane 1 = H/2 rows of W u16
// (interleaved Cb,Cr pairs), same pitch. Bars are vertical: bar index = x / (W/8).
let bar_w = (W / 8) as usize;
let mut init = vec![0u16; (W as usize) * (H as usize + H as usize / 2)];
for y in 0..H as usize {
for x in 0..W as usize {
init[y * W as usize + x] = BARS[(x / bar_w).min(7)].0 << 6;
}
}
let chroma_base = (W as usize) * (H as usize);
for cy in 0..(H as usize / 2) {
for cx in 0..(W as usize / 2) {
let (_, cb, cr) = BARS[((cx * 2) / bar_w).min(7)];
init[chroma_base + cy * W as usize + cx * 2] = cb << 6;
init[chroma_base + cy * W as usize + cx * 2 + 1] = cr << 6;
}
}
// SAFETY: self-contained harness on one thread/device (same contract as `drive_live`);
// the initial-data pointer outlives the synchronous CreateTexture2D that reads it.
let (device, tex) = unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(imp.luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(imp.luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().expect("dxgi factory");
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).expect("intel adapter");
let mut device = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
windows::Win32::Foundation::HMODULE::default(),
Default::default(),
None,
D3D11_SDK_VERSION,
Some(&mut device),
None,
None,
)
.expect("d3d11 device on intel adapter");
let device: ID3D11Device = device.expect("device");
let desc = D3D11_TEXTURE2D_DESC {
Width: W,
Height: H,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
CPUAccessFlags: 0,
MiscFlags: 0,
};
let data = D3D11_SUBRESOURCE_DATA {
pSysMem: init.as_ptr() as *const std::ffi::c_void,
SysMemPitch: W * 2,
SysMemSlicePitch: 0,
};
let mut t: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&data), Some(&mut t))
.expect("bar texture");
(device.clone(), t.expect("texture"))
};
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
let mut keyframes = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
assert!(keyframes >= 1, "expected an IDR in the dump");
let path = std::env::temp_dir().join("pf_qsv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {} AUs ({} bytes, {keyframes} keyframes) to {}",
aus,
stream.len(),
path.display()
);
}
/// The PRODUCTION host chain minus the IDD ring: the REAL `HdrP010Converter` renders the 8
/// sRGB bars into a ring-profile P010 texture (`BIND_RENDER_TARGET` only — RTV-written, not
/// CPU-uploaded) on the VPL implementation's own adapter, and THAT texture goes through the
/// unaligned-height ingest copy into a Main10 encode. Dumped to
/// `%TEMP%\pf_qsv_conv_1080_bars.h265`; expected decode codes = the bars_pq2020 fixture set
/// (see `hdr_p010_convert_bars_on_luid`).
#[test]
fn qsv_live_hdr_converter_e2e_1080_dump() {
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
let (device, tex) = pf_capture::dxgi::hdr_p010_convert_bars_on_luid(imp.luid, W, H)
.expect("converter bars");
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
let path = std::env::temp_dir().join("pf_qsv_conv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {aus} AUs ({} bytes) to {}",
stream.len(),
path.display()
);
}
}
+9 -107
View File
@@ -129,9 +129,6 @@ pub fn open_video(
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
// The session may hand this encoder cursor bitmaps to composite (cursor-as-metadata
// captures). Backends whose fast path can't blend (Vulkan EFC RGB-direct) key off it.
cursor_blend: bool,
) -> Result<Box<dyn Encoder>> {
let (inner, backend) = open_video_backend(
codec,
@@ -143,7 +140,6 @@ pub fn open_video(
cuda,
bit_depth,
chroma,
cursor_blend,
)?;
// Record what this session encodes on (the mgmt API's "currently used GPU"): the backend label
// is reported by `open_video_backend` from the branch that ACTUALLY opened — not re-derived by
@@ -207,34 +203,15 @@ impl Encoder for TrackedEncoder {
fn invalidate_ref_frames(&mut self, first_frame: i64, last_frame: i64) -> bool {
self.inner.invalidate_ref_frames(first_frame, last_frame)
}
// Forwarded for the same reason as `set_wire_chunking` below — the unforwarded default
// (`false` = "backend can't pipeline, stop asking") silently killed the §7 LN3 contention
// escalation for every session, since the host loop only ever holds the wrapped box.
fn set_pipelined(&mut self, on: bool) -> bool {
self.inner.set_pipelined(on)
}
// The classic TrackedEncoder trap: a defaulted trait method that isn't forwarded
// silently no-ops through the wrapper (bit the direct-NVENC work, then THIS — the
// §4.4 chunking probe run hit the default while the plan said Some(1408)).
fn set_wire_chunking(&mut self, shard_payload: usize) {
self.inner.set_wire_chunking(shard_payload)
}
// Forwarded for the same reason as `set_wire_chunking` above — an unforwarded default here
// would silently leave the in-place backends pipelining past the capturer's ring.
fn set_input_ring_depth(&mut self, depth: usize) {
self.inner.set_input_ring_depth(depth)
}
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
self.inner.poll()
}
// Both chunked-poll methods forwarded (the same trap class): the defaults would report
// "not chunked" and wrap whole AUs, silently discarding the sub-frame overlap.
fn supports_chunked_poll(&self) -> bool {
self.inner.supports_chunked_poll()
}
fn poll_chunk(&mut self) -> Result<Option<AuChunk>> {
self.inner.poll_chunk()
}
fn reset(&mut self) -> bool {
self.inner.reset()
}
@@ -268,9 +245,7 @@ fn open_video_backend(
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
cursor_blend: bool,
) -> Result<(Box<dyn Encoder>, &'static str)> {
let _ = cursor_blend; // consumed only by the Linux vulkan-encode + direct-NVENC arms below
validate_dimensions(codec, width, height)?;
// Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`)
// and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero.
@@ -328,15 +303,8 @@ fn open_video_backend(
&& vulkan_encode_enabled()
&& !(bit_depth == 10 && format.is_hdr_rgb10())
{
match vulkan_video::VulkanVideoEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
cursor_blend,
) {
match vulkan_video::VulkanVideoEncoder::open(codec, width, height, fps, bitrate_bps)
{
Ok(e) => {
tracing::info!(
codec = ?codec,
@@ -345,32 +313,12 @@ fn open_video_backend(
);
return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan"));
}
// Native NV12 (PUNKTFUNK_PIPEWIRE_NV12 capture) has no VAAPI fallback:
// libav's dmabuf lane would import the two-plane buffer as packed RGB
// (silent garbage) and its CPU lane bails per frame — die crisply instead.
Err(e) if format == PixelFormat::Nv12 => {
return Err(e.context(
"Vulkan Video open failed on a native-NV12 capture \
no VAAPI fallback exists; set PUNKTFUNK_PIPEWIRE_NV12=0 to \
restore the packed-RGB negotiation",
));
}
Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"Vulkan Video encode open failed — falling back to libav VAAPI"
),
}
}
// Same rule when the Vulkan backend was never eligible (H264 session,
// PUNKTFUNK_VULKAN_ENCODE=0, or a build without the feature).
if format == PixelFormat::Nv12 {
anyhow::bail!(
"native NV12 capture requires the Vulkan Video encoder (HEVC/AV1 \
session, --features vulkan-encode, PUNKTFUNK_VULKAN_ENCODE not 0) this \
session resolved to libav VAAPI; set PUNKTFUNK_PIPEWIRE_NV12=0 to restore \
the packed-RGB negotiation"
);
}
vaapi::VaapiEncoder::open(
codec,
format,
@@ -394,7 +342,6 @@ fn open_video_backend(
cuda,
bit_depth,
chroma,
cursor_blend,
)
.map(|e| (e, "nvenc"))
};
@@ -410,16 +357,8 @@ fn open_video_backend(
"the Vulkan Video encoder supports HEVC + AV1; the session negotiated {codec:?}"
);
}
vulkan_video::VulkanVideoEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
cursor_blend,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "vulkan"))
vulkan_video::VulkanVideoEncoder::open(codec, width, height, fps, bitrate_bps)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "vulkan"))
}
#[cfg(not(feature = "vulkan-encode"))]
{
@@ -730,15 +669,12 @@ fn open_nvenc_probed(
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
cursor_blend: bool,
) -> Result<Box<dyn Encoder>> {
#[cfg(not(feature = "nvenc"))]
let _ = cursor_blend; // consumed by the direct-SDK arm below
// Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA
// capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those
// keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own
// level-ceiling binary search at session open), so it skips the probe-loop stepping below.
// Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA
// capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those
// keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own
// level-ceiling binary search at session open), so it skips the probe-loop stepping below.
#[cfg(feature = "nvenc")]
if cuda && nvenc_direct_enabled() {
tracing::info!(
@@ -755,7 +691,6 @@ fn open_nvenc_probed(
cuda,
bit_depth,
chroma,
cursor_blend,
)?) as Box<dyn Encoder>);
}
// The silent-degrade trap: a build without `--features nvenc` compiles the direct-SDK
@@ -846,33 +781,6 @@ fn vulkan_encode_enabled() -> bool {
.unwrap_or(true)
}
/// Whether THIS session's encoder can ingest a producer-native NV12 capture: only the raw
/// Vulkan Video backend does (libav VAAPI would misread the two-plane buffer as packed RGB —
/// [`open_video`] refuses the combination), so the session's codec must be one it encodes and
/// the backend must be eligible to open. The host facade threads the verdict into the capture
/// negotiation (`OutputFormat::nv12_native` → `ZeroCopyPolicy::native_nv12_session`), which
/// then PREFERS gamescope's producer-side NV12 pod (default-on; `PUNKTFUNK_PIPEWIRE_NV12=0`
/// escapes at the capture gate).
#[cfg(target_os = "linux")]
pub fn linux_native_nv12_ok(codec: Codec) -> bool {
#[cfg(feature = "vulkan-encode")]
{
matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
// NVENC/PyroWave prefs never open the Vulkan Video backend; every other pref
// (auto/vaapi/amd/intel/vulkan) tries it first on AMD/Intel — see [`open_video`].
&& !matches!(
pf_host_config::config().encoder_pref.as_str(),
"nvenc" | "nvidia" | "cuda" | "pyrowave"
)
}
#[cfg(not(feature = "vulkan-encode"))]
{
let _ = codec;
false
}
}
/// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA
/// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT
/// create a CUDA context (that would allocate GPU state on every host that merely *might* be
@@ -1430,12 +1338,6 @@ mod vulkan_video;
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[path = "enc/linux/vk_av1_encode.rs"]
mod vk_av1_encode;
// Vendored `VK_VALVE_video_encode_rgb_conversion` bindings (host-only) — RGB encode source with
// the VCN EFC front-end doing the CSC (design/vulkan-rgb-direct-encode.md). ash 0.38 predates
// the extension; same vendoring rationale as `vk_av1_encode`.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[path = "enc/linux/vk_valve_rgb.rs"]
mod vk_valve_rgb;
// Small ash leaf helpers shared by the Linux Vulkan encode backends (dmabuf import, image/memory
// utilities) — extracted from `vulkan_video.rs` when the PyroWave backend arrived.
#[cfg(all(
-11
View File
@@ -32,11 +32,6 @@ pub struct WinCaptureTarget {
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32,
/// The ADD reply flagged this target as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer from its
/// frames forever, so a session WITHOUT the cursor channel must self-composite (the IDD-push
/// capturer's forced-composite gate) or the streamed desktop has no cursor at all.
pub cursor_excluded: bool,
}
/// The PyroWave (Windows) zero-copy sharing payload attached to a captured frame: the SECOND plane
@@ -57,12 +52,6 @@ pub struct PyroFrameShare {
/// The fence value the capturer signalled after THIS frame's convert. The encoder's Vulkan
/// acquire waits on it, so the wavelet read is ordered after the D3D11 CSC.
pub fence_value: u64,
/// The capturer's ring generation, bumped every time it recreates its texture ring. The
/// PyroWave encoder caches its plane imports keyed on the texture's COM address, which carries
/// no reference — after a recreate those addresses can be recycled by the allocator, so a
/// cached import may describe a texture that no longer exists. The encoder flushes its import
/// cache whenever this changes, making cache identity independent of allocator behaviour.
pub ring_gen: u32,
}
/// A GPU-resident captured texture (the Windows zero-copy path: NVENC/AMF/QSV encode it in place;
+10 -49
View File
@@ -105,15 +105,13 @@ pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
// Linux native NV12 capture (gamescope PipeWire): one LINEAR dmabuf with contiguous Y then
// interleaved UV, exposed under DRM_FORMAT_NV12.
Nv12 => drm_fourcc_code(b"NV12"),
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020).
X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010
X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
// Rgb10a2/P010 are Windows formats; Yuv444 is OUR convert output, never a capture source.
Rgb | Bgr | Rgb10a2 | P010 | Yuv444 => return None,
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format.
Rgb | Bgr | Rgb10a2 | Nv12 | P010 | Yuv444 => return None,
})
}
@@ -146,18 +144,6 @@ pub struct OutputFormat {
/// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the
/// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture).
pub pyrowave: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (Linux raw Vulkan Video
/// backend on an H265/AV1 session — see `pf_encode::linux_native_nv12_ok`). The Linux capture
/// negotiation only offers gamescope the NV12 pod when this is set: libav VAAPI (the H264
/// codec's backend, and the fallback family) would misread the two-plane buffer as packed
/// RGB. Always `false` on Windows (the IDD-push capturer owns its own formats).
pub nv12_native: bool,
/// The session negotiated the cursor-forward channel (remote-desktop-sweep M2c): on Windows
/// the IDD-push capturer creates + delivers the driver's hardware-cursor section, so DWM
/// stops compositing the pointer into the frames and the capturer surfaces it via
/// `Capturer::cursor()` instead. Ignored on Linux (the portal's `SPA_META_Cursor` already
/// separates the pointer; the session plan's `cursor_blend` gate handles the rest).
pub hw_cursor: bool,
}
impl OutputFormat {
@@ -175,13 +161,6 @@ impl OutputFormat {
chroma_444: false,
// GameStream never negotiates PyroWave (native punktfunk/1 only).
pyrowave: false,
// GameStream/spike sessions never negotiate the cursor channel.
hw_cursor: false,
// Conservative: the GameStream + spike paths don't resolve the codec here, and a
// Moonlight client may negotiate H264 (whose VAAPI backend can't ingest NV12) — so
// they never prefer the producer-native NV12 pod. The punktfunk/1 plane opts in via
// `SessionPlan::output_format()`, which knows the codec.
nv12_native: false,
}
}
}
@@ -204,16 +183,6 @@ pub struct CursorOverlay {
pub rgba: std::sync::Arc<Vec<u8>>,
/// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves.
pub serial: u64,
/// Hotspot (the pixel that IS the pointer position) within `w`×`h`. The blend paths ignore
/// it (`x`/`y` are already hotspot-adjusted); the cursor-forward channel ships it to the
/// client so a locally-drawn OS cursor points with the right pixel.
pub hot_x: u32,
pub hot_y: u32,
/// Compositor-reported pointer visibility. `false` = an app on the host grabbed/hid the
/// pointer — the cursor-forward channel turns that into the client's relative-mode hint
/// (remote-desktop-sweep M3). The encode loop STRIPS invisible overlays before the frame
/// reaches any blend path, so encoders may keep treating `Some` as "draw it".
pub visible: bool,
}
/// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of
@@ -232,26 +201,18 @@ pub struct CapturedFrame {
pub cursor: Option<CursorOverlay>,
}
/// A captured frame still living in a DMA-BUF. Packed RGB uses one plane. Native Linux NV12
/// (gamescope PipeWire) travels in ONE fd: Y starts at `offset`, and the interleaved UV plane
/// lives at `plane1`'s offset/stride when the producer reported them — else at the contiguous
/// fallback `offset + stride * frame_height` with the shared `stride`.
///
/// A captured frame still living in a single-plane packed-RGB dmabuf (the VAAPI zero-copy path).
/// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be
/// imported there without the compositor's buffer being closed underneath it. Content stability
/// across the brief import window relies on the compositor's buffer pool depth, like any zero-copy
/// capture.
/// imported into a VA surface there without the compositor's buffer being closed underneath it.
/// (Content stability across the brief import window relies on the compositor's buffer pool depth,
/// same as any zero-copy capture — the VAAPI importer copies into its own NV12 surface promptly.)
#[cfg(target_os = "linux")]
pub struct DmabufFrame {
pub fd: std::os::fd::OwnedFd,
/// DRM FourCC (`XR24` for BGRx, `NV12` for native 4:2:0).
/// DRM FourCC of the packed-RGB plane (e.g. `XR24` for BGRx).
pub fourcc: u32,
/// DRM format modifier the compositor allocated (0 = LINEAR).
pub modifier: u64,
/// Second-plane `(offset, stride)` within the SAME fd, when the producer reported one (the
/// PipeWire buffer's plane-1 chunk — NV12's interleaved UV). `None` falls back to the
/// contiguous-plane contract above. Always `None` for single-plane packed RGB.
pub plane1: Option<(u32, u32)>,
pub offset: u32,
pub stride: u32,
}
@@ -264,8 +225,8 @@ pub enum FramePayload {
/// The dmabuf has already been imported + copied into this owned device buffer.
#[cfg(target_os = "linux")]
Cuda(pf_zerocopy::DeviceBuffer),
/// A raw DMA-BUF: packed RGB for the existing GPU CSC paths, or native NV12 from a producer
/// such as gamescope. The encoder imports it without a host copy.
/// A raw packed-RGB dmabuf — the AMD/Intel (VAAPI) zero-copy path. The encoder imports it into
/// a VA surface and does RGB→NV12 on the GPU video engine (no host CSC, no upload).
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame.
-17
View File
@@ -63,13 +63,6 @@ pub struct HostConfig {
/// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0.
/// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth).
pub four_four_four: bool,
/// `PUNKTFUNK_CHACHA20` — host policy gate for the negotiated ChaCha20-Poly1305 session
/// cipher (design/chacha20-session-cipher.md). **Default ON** (pure rollout safety — perf-only,
/// both AEADs are full-strength): the host merely *allows* it — a session only seals with
/// ChaCha when the client advertised `VIDEO_CAP_CHACHA20` (set by soft-AES armv7 clients,
/// e.g. webOS TVs, whose GCM decrypt caps at ~100 Mbps); everyone else stays AES-128-GCM.
/// `PUNKTFUNK_CHACHA20=0`/`false`/`off`/`no` disables.
pub chacha20: bool,
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
pub perf: bool,
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic).
@@ -154,16 +147,6 @@ impl HostConfig {
)
})
.unwrap_or(true),
// Default ON, explicit-off grammar (the client's VIDEO_CAP_CHACHA20 bit is the real
// per-session switch; see the field doc).
chacha20: val("PUNKTFUNK_CHACHA20")
.map(|s| {
!matches!(
s.trim().to_ascii_lowercase().as_str(),
"0" | "false" | "off" | "no"
)
})
.unwrap_or(true),
perf: flag("PUNKTFUNK_PERF"),
video_source: val("PUNKTFUNK_VIDEO_SOURCE"),
compositor: val("PUNKTFUNK_COMPOSITOR"),
-6
View File
@@ -36,12 +36,6 @@ pub fn vk_to_evdev(vk: u8) -> Option<u16> {
0x2D => Some(110), // VK_INSERT -> KEY_INSERT
0x2E => Some(111), // VK_DELETE -> KEY_DELETE
// --- Consumer/media keys (Android TV remotes, keyboard media rows) ---
0xB0 => Some(163), // VK_MEDIA_NEXT_TRACK -> KEY_NEXTSONG
0xB1 => Some(165), // VK_MEDIA_PREV_TRACK -> KEY_PREVIOUSSONG
0xB2 => Some(166), // VK_MEDIA_STOP -> KEY_STOPCD
0xB3 => Some(164), // VK_MEDIA_PLAY_PAUSE -> KEY_PLAYPAUSE
// --- Generic modifiers ---
0x10 => Some(42), // VK_SHIFT -> KEY_LEFTSHIFT
0x11 => Some(29), // VK_CONTROL -> KEY_LEFTCTRL
@@ -424,9 +424,6 @@ impl InputInjector for KwinFakeInjector {
self.fake.touch_up(event.code);
self.fake.touch_frame();
}
// fake_input can only press host-layout keycodes — no committed-text path (the
// HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => {}
// Gamepads are injected through uinput, not the compositor.
InputKind::GamepadState
| InputKind::GamepadButton
+30 -205
View File
@@ -109,7 +109,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
// Keep `_rd`/`_session` bound for the whole loop — dropping the portal session closes the
// EIS connection. Bound the setup so a headless approval dialog (un-bypassed grant) can't
// hang the worker forever.
let (_keepalive, context, mut events, output_hint) = match tokio::time::timeout(
let (_keepalive, context, mut events) = match tokio::time::timeout(
Duration::from_secs(30),
connect(source),
)
@@ -130,7 +130,6 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
tracing::info!("libei: EIS connected — awaiting devices");
let mut state = EiState::new();
state.output_hint = output_hint;
// Watchdog: a healthy EIS server adds + resumes an input device within a beat of the handshake.
// If none has resumed by this deadline, the connection is dead-on-arrival (stale/half-ready
// gamescope socket the handshake passed but no real server is behind) — exit so the next
@@ -178,29 +177,21 @@ type Connected = (
Box<dyn Send>,
ei::Context,
reis::tokio::EiConvertEventStream,
// The compositor's output size ("WxH" relay-file hint) — the scale target for absolute
// coordinates when the EIS advertises only a degenerate region (gamescope). `None` for
// the portal/Mutter paths, whose regions are real.
Option<(u32, u32)>,
);
/// Reach an EIS server per `source` and run the EI sender handshake.
async fn connect(source: EiSource) -> Result<Connected> {
let (keepalive, stream, output_hint): (Box<dyn Send>, UnixStream, Option<(u32, u32)>) =
match source {
EiSource::Portal => {
let (rd, session, fd) = connect_portal().await?;
(Box::new((rd, session)), UnixStream::from(fd), None)
}
EiSource::MutterEis => {
let (keepalive, fd) = connect_mutter().await?;
(keepalive, UnixStream::from(fd), None)
}
EiSource::SocketPathFile(file) => {
let (stream, hint) = connect_socket_file(&file).await?;
(Box::new(()), stream, hint)
}
};
let (keepalive, stream): (Box<dyn Send>, UnixStream) = match source {
EiSource::Portal => {
let (rd, session, fd) = connect_portal().await?;
(Box::new((rd, session)), UnixStream::from(fd))
}
EiSource::MutterEis => {
let (keepalive, fd) = connect_mutter().await?;
(keepalive, UnixStream::from(fd))
}
EiSource::SocketPathFile(file) => (Box::new(()), connect_socket_file(&file).await?),
};
let context = ei::Context::new(stream).map_err(|e| anyhow!("reis EI context: {e}"))?;
// Bound the handshake. `UnixStream::connect` to a socket *file* succeeds the moment the path
// exists, but a stale/half-ready gamescope (its socket created early in startup, or left behind
@@ -215,7 +206,7 @@ async fn connect(source: EiSource) -> Result<Connected> {
anyhow!("EI handshake timed out (EIS server not responding — stale/half-ready socket?)")
})?
.map_err(|e| anyhow!("EI handshake: {e}"))?;
Ok((keepalive, context, events, output_hint))
Ok((keepalive, context, events))
}
/// Open a RemoteDesktop portal session (pointer + keyboard) and obtain the EIS socket fd.
@@ -303,10 +294,8 @@ async fn connect_mutter() -> Result<(Box<dyn Send>, std::os::fd::OwnedFd)> {
/// Poll `file` for the EIS socket path (the gamescope backend relays `LIBEI_SOCKET` there once
/// the nested app launches), then connect. A bare name is resolved against `XDG_RUNTIME_DIR`,
/// mirroring libei's own `LIBEI_SOCKET` semantics. Line 2 of the relay file, when present,
/// carries the compositor's output size as `WxH` — returned as the absolute-coordinate scale
/// hint (gamescope's EIS region is degenerate, so the geometry can't come from the protocol).
async fn connect_socket_file(file: &std::path::Path) -> Result<(UnixStream, Option<(u32, u32)>)> {
/// mirroring libei's own `LIBEI_SOCKET` semantics.
async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
// The relay file is rewritten each session with the CURRENT gamescope's `LIBEI_SOCKET`, and the
// socket may not be `listen()`ing the instant its name appears — or the file may briefly still
// hold a prior, now-dead session's name (the host-lifetime injector reconnecting between
@@ -330,12 +319,7 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<(UnixStream, Opti
));
}
if let Ok(s) = std::fs::read_to_string(file) {
let mut file_lines = s.lines();
let name = file_lines.next().unwrap_or("").trim();
let hint = file_lines.next().and_then(|l| {
let (w, h) = l.trim().split_once('x')?;
Some((w.parse::<u32>().ok()?, h.parse::<u32>().ok()?))
});
let name = s.trim();
if !name.is_empty() {
let full = if name.starts_with('/') {
std::path::PathBuf::from(name)
@@ -350,7 +334,7 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<(UnixStream, Opti
logged = name.to_string();
}
match UnixStream::connect(&full) {
Ok(stream) => return Ok((stream, hint)),
Ok(stream) => return Ok(stream),
// Refused = socket file exists but no listener yet (or a dead session);
// NotFound = path not created yet. Both heal once the live gamescope's EIS is
// up — retry. Anything else (e.g. permission) is a real failure.
@@ -374,25 +358,6 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<(UnixStream, Opti
}
/// One EI device and its emulation state.
/// Pick the region to map absolute coordinates into: the one whose logical size matches the
/// streamed mode (the session's virtual output). The device advertises one region per logical
/// monitor, and blindly taking `first()` next to a physical monitor put the pointer — and every
/// click — on whichever output the compositor happened to announce first (on-glass: GNOME with a
/// dummy HDMI beside the virtual primary; the seat cursor never entered the streamed monitor, so
/// neither embedded nor metadata cursor capture could see it). Size is the only key available
/// today: regions carry no output name, and matching the screencast stream's `mapping_id` needs
/// the stream id plumbed across crates (follow-up). Two same-sized monitors stay ambiguous.
fn region_for_mode(
regions: &[reis::event::Region],
w: f32,
h: f32,
) -> Option<&reis::event::Region> {
regions
.iter()
.find(|r| r.width as f32 == w && r.height as f32 == h)
.or_else(|| regions.first())
}
struct DeviceSlot {
device: reis::event::Device,
/// The device is resumed (allowed to emit). Devices arrive paused and may pause again.
@@ -421,22 +386,6 @@ struct EiState {
held_keys: Vec<u32>,
held_buttons: Vec<u32>,
held_touches: Vec<u32>,
/// The touch id currently degraded to the absolute pointer ([`EiState::degrade_touch`]) —
/// the "primary finger" while the EIS has no touchscreen device. `None` between touches.
degraded_touch: Option<u32>,
/// The compositor's output size (relay-file "WxH" hint) — scale target for absolute
/// coordinates when the device's region is degenerate/absent (gamescope). Without it the
/// fallback is raw client pixels, correct only when the stream runs at the output's size.
output_hint: Option<(u32, u32)>,
}
/// Is this EIS region a plausible OUTPUT geometry — something to map normalized coordinates
/// into? gamescope advertises a degenerate `(0,0,INT32_MAX,INT32_MAX)` "everything" region on
/// its virtual input device, meaning "absolute coordinates are raw"; normalizing into it
/// explodes a center tap to x≈1e9, which the compositor clamps to the far corner. 16384
/// comfortably covers real multi-monitor layouts while rejecting the sentinel.
fn sane_region(r: &reis::event::Region) -> bool {
r.width > 0 && r.height > 0 && r.width <= 16_384 && r.height <= 16_384
}
/// Stable small index per [`InputKind`] for the `seen_kinds` bitmask.
@@ -457,7 +406,6 @@ fn kind_bit(kind: InputKind) -> u32 {
InputKind::GamepadState => 12,
InputKind::GamepadRemove => 13,
InputKind::GamepadArrival => 14,
InputKind::TextInput => 15,
};
1 << i
}
@@ -474,8 +422,6 @@ impl EiState {
held_keys: Vec::new(),
held_buttons: Vec::new(),
held_touches: Vec::new(),
degraded_touch: None,
output_hint: None,
}
}
@@ -484,10 +430,6 @@ impl EiState {
/// normal [`EiState::inject`] path so the compositor sees proper key-up / button-up /
/// touch-up frames before the devices disappear.
fn release_all(&mut self, ctx: &ei::Context) {
// A degraded touch is held as a synthesized left button (in `held_buttons`), so the
// loop below releases it — but the primary-finger latch must clear too, or the NEXT
// session's first TouchDown reads as a second finger and is ignored.
self.degraded_touch = None;
let (keys, buttons, touches) = (
std::mem::take(&mut self.held_keys),
std::mem::take(&mut self.held_buttons),
@@ -564,14 +506,6 @@ impl EiState {
keyboard = dev.has_capability(DeviceCapability::Keyboard),
button = dev.has_capability(DeviceCapability::Button),
scroll = dev.has_capability(DeviceCapability::Scroll),
// One region per logical monitor — which one absolute coords map into is
// resolved per event (`region_for_mode`); log them so a mis-mapped pointer
// (cursor/clicks on the wrong output) is diagnosable from the journal.
regions = ?dev
.regions()
.iter()
.map(|r| format!("{}x{}+{}+{}", r.width, r.height, r.x, r.y))
.collect::<Vec<_>>(),
"libei: device RESUMED (now emittable)"
);
}
@@ -603,85 +537,8 @@ impl EiState {
}
}
/// Degrade touch to a single-finger ABSOLUTE POINTER on compositors whose EIS never
/// creates a touchscreen device (gamescope's "Gamescope Virtual Input" advertises
/// pointer/pointer_abs/button but no touch — observed live; headless KWin the same).
/// Down = abs-move + left press, move = abs-move, up = left release — synthesized through
/// the normal [`EiState::inject`] Mouse* paths so region mapping, held-state tracking and
/// [`EiState::release_all`] all apply. Only the FIRST finger drives the pointer; later
/// fingers are ignored (a pointer has no second contact — a pinch degrades to a drag).
fn degrade_touch(&mut self, ev: &InputEvent, ctx: &ei::Context) {
const GS_BUTTON_LEFT: u32 = 1;
match ev.kind {
InputKind::TouchDown => {
if self.degraded_touch.is_some() {
return; // secondary finger — single-pointer degradation
}
self.degraded_touch = Some(ev.code);
static NOTED: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(false);
if !NOTED.swap(true, std::sync::atomic::Ordering::Relaxed) {
tracing::info!(
"compositor's EIS has no touchscreen device — degrading touch to a \
single-finger absolute pointer (tap = left click; multi-touch \
gestures unavailable)"
);
}
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
self.inject(
&InputEvent {
kind: InputKind::MouseButtonDown,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
InputKind::TouchMove if self.degraded_touch == Some(ev.code) => {
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
}
InputKind::TouchUp if self.degraded_touch == Some(ev.code) => {
self.degraded_touch = None;
self.inject(
&InputEvent {
kind: InputKind::MouseButtonUp,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
_ => {}
}
}
/// Translate and emit one client input event, committing it as a single `frame`.
fn inject(&mut self, ev: &InputEvent, ctx: &ei::Context) {
// No ei_touchscreen device but an absolute pointer exists → degrade rather than drop
// (the game-mode "touch simply does not work" trap). Checked per event, not latched:
// a touchscreen device appearing later (compositor restart, capability change) takes
// over seamlessly on the next touch.
if matches!(
ev.kind,
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp
) && self.device_for(DeviceCapability::Touch).is_none()
&& self.device_for(DeviceCapability::PointerAbsolute).is_some()
{
self.degrade_touch(ev, ctx);
return;
}
let cap = match ev.kind {
InputKind::MouseMove => DeviceCapability::Pointer,
InputKind::MouseMoveAbs => DeviceCapability::PointerAbsolute,
@@ -696,9 +553,6 @@ impl EiState {
| InputKind::GamepadAxis
| InputKind::GamepadRemove
| InputKind::GamepadArrival => return, // uinput path (later)
// libei presses keycodes against the server's negotiated keymap — no committed-text
// path (the HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => return,
};
self.injected += 1;
let n = self.injected;
@@ -751,33 +605,16 @@ impl EiState {
InputKind::MouseMoveAbs => {
let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32;
match slot.interface::<ei::PointerAbsolute>() {
Some(p) if w > 0.0 && h > 0.0 => {
// Map the normalized client position into the region matching the streamed
// output's mode (`region_for_mode` picks the right one on a multi-monitor
// EIS). gamescope's "Gamescope Virtual Input" advertises a degenerate
// (0,0,INT32_MAX,INT32_MAX) region meaning "coordinates are raw" —
// `sane_region` rejects it (normalizing into it explodes a center tap to
// x≈1e9, clamped to the far corner), so a non-matching / insane region
// falls to the output hint (correct across a resolution mismatch), then
// raw client pixels as the last resort.
match (
slot.interface::<ei::PointerAbsolute>(),
slot.regions().first(),
) {
(Some(p), Some(region)) if w > 0.0 && h > 0.0 => {
// Map the normalized client position into the device's first region.
let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let (x, y) = match region_for_mode(slot.regions(), w, h)
.filter(|r| sane_region(r))
{
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
// Degenerate/absent region: scale into the relay-file output hint
// (correct even when the client streams at a different resolution
// than the session runs); raw client pixels as the last resort.
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
let x = region.x as f32 + nx * region.width as f32;
let y = region.y as f32 + ny * region.height as f32;
p.motion_absolute(x, y);
}
_ => emitted = false,
@@ -843,23 +680,12 @@ impl EiState {
InputKind::TouchDown | InputKind::TouchMove => {
let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32;
match slot.interface::<ei::Touchscreen>() {
Some(t) if w > 0.0 && h > 0.0 => {
match (slot.interface::<ei::Touchscreen>(), slot.regions().first()) {
(Some(t), Some(region)) if w > 0.0 && h > 0.0 => {
let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
// Same region-selection + degenerate fallback ladder as MouseMoveAbs.
let (x, y) = match region_for_mode(slot.regions(), w, h)
.filter(|r| sane_region(r))
{
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
let x = region.x as f32 + nx * region.width as f32;
let y = region.y as f32 + ny * region.height as f32;
if ev.kind == InputKind::TouchDown {
t.down(ev.code, x, y);
} else {
@@ -877,8 +703,7 @@ impl EiState {
| InputKind::GamepadButton
| InputKind::GamepadAxis
| InputKind::GamepadRemove
| InputKind::GamepadArrival
| InputKind::TextInput => emitted = false,
| InputKind::GamepadArrival => emitted = false,
}
if emitted {
@@ -17,7 +17,7 @@
use anyhow::{bail, Context, Result};
use std::mem::size_of;
use std::os::fd::RawFd;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
@@ -196,45 +196,6 @@ impl Drop for GadgetFd {
}
}
/// The signal used to break a worker thread out of a blocking raw_gadget ioctl at teardown.
/// `EVENT_FETCH`/`EP_WRITE` are `wait_event_interruptible` in the kernel with no timeout and no
/// `O_NONBLOCK` honouring, and closing the fd cannot wake a thread already inside the ioctl (the
/// in-flight syscall holds a reference to the struct file). A signal is the only reliable lever:
/// delivered with a no-op, non-`SA_RESTART` handler it forces the ioctl to return `EINTR`, after
/// which the loop's top-of-iteration `running` check exits. `SIGUSR1` is unused elsewhere in this
/// process; the handler is a no-op, so a stray `SIGUSR1` becomes harmless rather than fatal.
const WAKE_SIGNAL: libc::c_int = libc::SIGUSR1;
/// Install the no-op `WAKE_SIGNAL` handler exactly once. Crucially `sa_flags = 0` (no `SA_RESTART`)
/// so a delivered signal makes the interruptible ioctl return `EINTR` instead of auto-restarting.
fn install_wake_handler() {
static ONCE: std::sync::Once = std::sync::Once::new();
ONCE.call_once(|| {
extern "C" fn noop(_: libc::c_int) {}
// SAFETY: installing a well-formed `sigaction` with an empty mask and a valid no-op handler
// for a single signal; touches only this process's disposition for `WAKE_SIGNAL`.
unsafe {
let mut sa: libc::sigaction = std::mem::zeroed();
// Via `*const ()`: casting a function item straight to an integer is what
// `clippy::function_casts_as_integer` rejects, and the pointer hop is the documented
// way to spell it. `sa_sigaction` is a `usize`-typed handler slot, so the value is
// unchanged.
sa.sa_sigaction = noop as *const () as usize;
libc::sigemptyset(&mut sa.sa_mask);
sa.sa_flags = 0;
libc::sigaction(WAKE_SIGNAL, &sa, std::ptr::null_mut());
}
});
}
/// Lets `Drop` wake a specific worker thread parked in a blocking ioctl. `tid` is the thread's
/// `pthread_self()` (0 until it starts); `done` is set right before the thread returns, so `Drop`
/// stops signalling a thread that has already exited.
struct Waker {
tid: Arc<AtomicU64>,
done: Arc<AtomicBool>,
}
/// A virtual Steam Deck presented over the USB gadget subsystem. Dropping it stops the threads and
/// closes the gadget (the kernel tears down the device).
pub struct SteamDeckGadget {
@@ -242,7 +203,6 @@ pub struct SteamDeckGadget {
feedback: Arc<Mutex<super::steam_proto::SteamFeedback>>,
running: Arc<AtomicBool>,
threads: Vec<JoinHandle<()>>,
wakers: Vec<Waker>,
_fd: Arc<GadgetFd>,
seq: u32,
}
@@ -283,18 +243,6 @@ impl SteamDeckGadget {
let ctrl_ep = Arc::new(std::sync::atomic::AtomicI32::new(-1));
let configured = Arc::new(AtomicBool::new(false));
// The teardown wake path (see `WAKE_SIGNAL`) needs the handler installed before any thread
// can park in a blocking ioctl.
install_wake_handler();
let ctrl_waker = Waker {
tid: Arc::new(AtomicU64::new(0)),
done: Arc::new(AtomicBool::new(false)),
};
let stream_waker = Waker {
tid: Arc::new(AtomicU64::new(0)),
done: Arc::new(AtomicBool::new(false)),
};
// Control thread: enumerate + answer every control transfer.
let control = {
let fd = fd.clone();
@@ -302,15 +250,10 @@ impl SteamDeckGadget {
let ctrl_ep = ctrl_ep.clone();
let configured = configured.clone();
let feedback = feedback.clone();
let tid = ctrl_waker.tid.clone();
let done = ctrl_waker.done.clone();
std::thread::Builder::new()
.name("pf-deck-gadget-ctrl".into())
.spawn(move || {
// SAFETY: `pthread_self` is always valid on the calling thread.
tid.store(unsafe { libc::pthread_self() } as u64, Ordering::SeqCst);
control_loop(fd, running, ctrl_ep, configured, feedback, serial, unit_id);
done.store(true, Ordering::SeqCst);
control_loop(fd, running, ctrl_ep, configured, feedback, serial, unit_id)
})
.context("spawn gadget control thread")?
};
@@ -321,16 +264,9 @@ impl SteamDeckGadget {
let ctrl_ep = ctrl_ep.clone();
let configured = configured.clone();
let report = report.clone();
let tid = stream_waker.tid.clone();
let done = stream_waker.done.clone();
std::thread::Builder::new()
.name("pf-deck-gadget-stream".into())
.spawn(move || {
// SAFETY: `pthread_self` is always valid on the calling thread.
tid.store(unsafe { libc::pthread_self() } as u64, Ordering::SeqCst);
stream_loop(fd, running, ctrl_ep, configured, report);
done.store(true, Ordering::SeqCst);
})
.spawn(move || stream_loop(fd, running, ctrl_ep, configured, report))
.context("spawn gadget stream thread")?
};
@@ -339,7 +275,6 @@ impl SteamDeckGadget {
feedback,
running,
threads: vec![control, stream],
wakers: vec![ctrl_waker, stream_waker],
_fd: fd,
seq: 0,
})
@@ -367,32 +302,6 @@ impl SteamDeckGadget {
impl Drop for SteamDeckGadget {
fn drop(&mut self) {
self.running.store(false, Ordering::SeqCst);
// The control thread spends steady state parked in a blocking `EVENT_FETCH` ioctl that only
// tests `running` at the top of its loop, so clearing the flag is not enough — it must be
// signalled out of the syscall (see `WAKE_SIGNAL`). Without this the join below can hang the
// caller (the session input thread, via `PadSlots::sweep`) indefinitely. Retry until each
// thread reports done, to cover the race where the signal lands just before the thread
// re-enters the ioctl; bounded (~1 s) so a genuinely stuck thread can't wedge teardown either.
for _ in 0..200 {
let mut all_done = true;
for w in &self.wakers {
if w.done.load(Ordering::SeqCst) {
continue;
}
all_done = false;
let tid = w.tid.load(Ordering::SeqCst);
if tid != 0 {
// SAFETY: the thread is joinable and not yet joined (join runs after this loop),
// so `tid` names a live pthread; `pthread_kill` on a finished-but-unjoined thread
// is defined (returns ESRCH), never UB.
unsafe { libc::pthread_kill(tid as libc::pthread_t, WAKE_SIGNAL) };
}
}
if all_done {
break;
}
std::thread::sleep(std::time::Duration::from_millis(5));
}
for t in self.threads.drain(..) {
let _ = t.join();
}
-96
View File
@@ -98,26 +98,9 @@ pub struct WlrootsInjector {
keyboard: ZwpVirtualKeyboardV1,
xkb_state: xkb::State,
_keymap_file: std::fs::File, // keep the memfd alive for the compositor's mmap
/// Dedicated committed-text device ([`InputKind::TextInput`]), created on first use.
text: Option<TextKeyboard>,
start: Instant,
}
/// Cap on distinct characters the dynamic text keymap holds before it restarts from scratch
/// (keycodes grow upward from 9; xkb tops out at 255, so stay well under).
const TEXT_KEYMAP_MAX: usize = 200;
/// The dedicated **text** virtual keyboard: types committed IME text (`InputKind::TextInput`,
/// one Unicode scalar per event) by growing a keymap of Unicode keysyms on demand and pressing
/// the character's keycode — the `wtype` model. A separate `zwp_virtual_keyboard` so keymap
/// re-uploads never disturb the main device's layout/modifier state that VK key events ride on.
struct TextKeyboard {
keyboard: ZwpVirtualKeyboardV1,
/// Characters in keycode order: `chars[i]` types on wire keycode `i + 1` (xkb `i + 9`).
chars: Vec<char>,
_keymap_file: Option<std::fs::File>, // keep the memfd alive for the compositor's mmap
}
impl WlrootsInjector {
pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env()
@@ -188,7 +171,6 @@ impl WlrootsInjector {
keyboard,
xkb_state,
_keymap_file: file,
text: None,
start: Instant::now(),
})
}
@@ -197,54 +179,6 @@ impl WlrootsInjector {
self.start.elapsed().as_millis() as u32
}
/// Type one committed-text Unicode scalar on the dedicated text device (created lazily),
/// growing its keymap when the character is new. Control characters are dropped — Enter,
/// Backspace and Tab ride the VK key-event path.
fn type_text(&mut self, cp: u32) -> Result<()> {
let Some(ch) = char::from_u32(cp) else {
return Ok(()); // lone surrogate / out of range
};
if ch.is_control() {
return Ok(());
}
if self.text.is_none() {
let (Some(mgr), Some(seat)) =
(self.globals.keyboard_mgr.clone(), self.globals.seat.clone())
else {
return Ok(());
};
let kb = mgr.create_virtual_keyboard(&seat, &self.queue.handle(), ());
self.text = Some(TextKeyboard {
keyboard: kb,
chars: Vec::new(),
_keymap_file: None,
});
}
let t = self.now_ms();
let text = self.text.as_mut().expect("created above");
let code = match text.chars.iter().position(|&c| c == ch) {
Some(i) => (i + 1) as u32,
None => {
if text.chars.len() >= TEXT_KEYMAP_MAX {
text.chars.clear(); // restart the map; old codes are re-assigned lazily
}
text.chars.push(ch);
let keymap_str = text_keymap(&text.chars);
let file = memfd_with(&keymap_str)?;
text.keyboard.keymap(
1, /* XKB_V1 */
file.as_fd(),
keymap_str.len() as u32 + 1,
);
text._keymap_file = Some(file);
text.chars.len() as u32
}
};
text.keyboard.key(t, code, 1);
text.keyboard.key(t, code, 0);
Ok(())
}
/// Update xkb state for a key and tell the compositor the resulting modifier mask.
fn send_modifiers(&mut self, evdev: u16, down: bool) {
let kc = xkb::Keycode::new(evdev as u32 + 8); // evdev -> xkb keycode
@@ -320,9 +254,6 @@ impl InputInjector for WlrootsInjector {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
}
}
InputKind::TextInput => {
self.type_text(event.code)?;
}
InputKind::GamepadState
| InputKind::GamepadButton
| InputKind::GamepadAxis
@@ -340,33 +271,6 @@ impl InputInjector for WlrootsInjector {
}
}
/// Build a minimal xkb keymap whose keycode `i + 9` (wire code `i + 1`) types `chars[i]`, using
/// Unicode keysym names (`U<hex>` — xkbcommon resolves them for any scalar, emoji included).
/// Types/compat `include "complete"` mirrors `wtype`'s generated keymap — proven on wlroots
/// compositors, and the system XKB data is present (the main keymap compiled from it in `open`).
fn text_keymap(chars: &[char]) -> String {
use std::fmt::Write as _;
let mut keycodes = String::new();
let mut symbols = String::new();
for (i, ch) in chars.iter().enumerate() {
let _ = writeln!(keycodes, " <T{i}> = {};", i + 9);
let _ = writeln!(symbols, " key <T{i}> {{ [ U{:04X} ] }};", *ch as u32);
}
format!(
"xkb_keymap {{\n\
xkb_keycodes \"punktfunk-text\" {{\n\
minimum = 8;\n\
maximum = {};\n\
{keycodes}\
}};\n\
xkb_types \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_compatibility \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_symbols \"punktfunk-text\" {{\n{symbols} }};\n\
}};\n",
chars.len() + 9,
)
}
/// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd).
fn memfd_with(s: &str) -> Result<std::fs::File> {
let name = b"punktfunk-keymap\0";
@@ -299,20 +299,13 @@ pub(super) fn create_swdevice(p: &SwDeviceProfile) -> Result<(HSWDEVICE, Option<
let event = unsafe { CreateEventW(None, true, false, PCWSTR::null())? };
// `result` starts as E_FAIL, NOT S_OK: if the wait below times out, a zero-initialised HRESULT
// would read as success and mask the failure (found by the 2026-07 driver-health audit).
// HEAP-allocated, deliberately: `sw_create_cb` writes `result` + up to 127 u16 of instance id
// through this pointer and then `SetEvent`s. The wait below is bounded (10 s), so on a wedged-PnP
// timeout the callback may still be PENDING — a stack context would be popped and a late callback
// would corrupt whatever the input thread put there next, and SetEvent a closed/recycled handle.
// On the timeout path we therefore LEAK the box and leave the event open (a one-off ~264 B + one
// HANDLE, only on that rare path) so a late callback always writes to live memory.
let ctx = Box::into_raw(Box::new(SwCreateCtx {
let mut ctx = SwCreateCtx {
event,
result: E_FAIL,
instance_id: [0; 128],
}));
// SAFETY: info + the buffers outlive the call; `ctx` is a live heap allocation that outlives every
// path below (reclaimed only where the callback provably ran). windows-rs returns the HSWDEVICE
// (the C out-param) as the Result value.
};
// SAFETY: info + the buffers + ctx outlive the call (we wait on the event before returning);
// windows-rs returns the HSWDEVICE (the C out-param) as the Result value.
let hsw = match unsafe {
SwDeviceCreate(
w!("punktfunk"),
@@ -320,15 +313,13 @@ pub(super) fn create_swdevice(p: &SwDeviceProfile) -> Result<(HSWDEVICE, Option<
&info,
None,
Some(sw_create_cb),
Some(ctx as *const c_void),
Some(&mut ctx as *mut SwCreateCtx as *const c_void),
)
} {
Ok(h) => h,
Err(e) => {
// SAFETY: the call failed, so no callback was registered and `ctx` is ours to reclaim;
// `event` is valid and unreferenced.
// SAFETY: event is valid.
unsafe {
drop(Box::from_raw(ctx));
let _ = CloseHandle(event);
}
return Err(anyhow!("SwDeviceCreate failed: {e}"));
@@ -337,22 +328,17 @@ pub(super) fn create_swdevice(p: &SwDeviceProfile) -> Result<(HSWDEVICE, Option<
// Block until PnP finishes enumerating (the callback signals), then check its result.
// SAFETY: event is valid.
let wait = unsafe { WaitForSingleObject(event, 10_000) };
// SAFETY: event is valid.
unsafe {
let _ = CloseHandle(event);
}
if wait != WAIT_OBJECT_0 {
// Timed out: the callback may still fire. Intentionally leak `ctx` AND leave `event` open so
// its eventual write + SetEvent target live memory/handle rather than freed ones.
// SAFETY: hsw is the handle SwDeviceCreate returned.
unsafe { SwDeviceClose(hsw) };
return Err(anyhow!(
"SwDeviceCreate enumeration callback never fired (10s) — PnP may be wedged"
));
}
// The callback ran (it is what signalled the event), so nothing else will touch `ctx`/`event`.
// SAFETY: `ctx` came from `Box::into_raw` above and is reclaimed exactly once here; `event` is
// valid and no longer referenced by a pending callback.
let ctx = unsafe {
let _ = CloseHandle(event);
Box::from_raw(ctx)
};
if ctx.result.is_err() {
// SAFETY: hsw is the handle SwDeviceCreate returned.
unsafe { SwDeviceClose(hsw) };
@@ -62,7 +62,7 @@ impl Ds4WinPad {
std::ptr::write_unaligned(base as *mut u32, SHM_MAGIC);
}
let inst = format!("pf_ds4_{index}");
let (hsw, instance_id) = create_swdevice(&SwDeviceProfile {
let (hsw, instance_id) = match create_swdevice(&SwDeviceProfile {
instance: &inst,
container_tag: 0x5046_4453, // "PFDS"
container_index: index,
@@ -70,13 +70,13 @@ impl Ds4WinPad {
usb_vid_pid: "VID_054C&PID_09CC",
usb_mi: None,
description: "punktfunk Virtual DualShock 4",
})?; // Propagate, do NOT swallow — see below.
let (hsw, instance_id) = (Some(hsw), instance_id);
// Swallowing a create failure here (the previous behaviour) latched the pad slot to
// `Some(pad)` with no live devnode: `PadSlots::ensure` short-circuits on `is_some()` and
// `gate.on_success()` cleared the backoff, so the create-gate that exists precisely to
// self-heal a transient PnP failure never retried. The game saw no controller for the whole
// session unless the client unplugged the pad. Matches the XUSB sibling, which propagates.
}) {
Ok((h, id)) => (Some(h), id),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "SwDeviceCreate failed; DualShock 4 devnode unavailable");
(None, None)
}
};
let _sw = hsw.map(super::gamepad_raii::SwDevice::new);
// Bounded eager delivery — for the DS4 this is what closes the identity race: the driver
// must read `device_type = 1` from the delivered DATA section before hidclass asks it for
@@ -82,16 +82,12 @@ fn create_swdevice(index: u8) -> Result<(HSWDEVICE, Option<String>)> {
let event = unsafe { CreateEventW(None, true, false, PCWSTR::null())? };
// `result` starts as E_FAIL, NOT S_OK: if the wait below times out, a zero-initialised HRESULT
// would read as success and mask the failure (found by the 2026-07 driver-health audit).
// HEAP-allocated for the same reason as the DualSense sibling: the callback writes through this
// pointer and SetEvents, and the wait below is bounded — a stack context would be popped while a
// late callback still holds it. On the timeout path the box is deliberately leaked and the event
// left open so a late write/SetEvent always targets live memory/handle.
let ctx = Box::into_raw(Box::new(SwCreateCtx {
let mut ctx = SwCreateCtx {
event,
result: E_FAIL,
instance_id: [0; 128],
}));
// SAFETY: info + buffers outlive the call; `ctx` is a live heap allocation outliving every path.
};
// SAFETY: info + buffers + ctx outlive the call (we wait on the event before returning).
let hsw = match unsafe {
SwDeviceCreate(
w!("punktfunk"),
@@ -99,14 +95,13 @@ fn create_swdevice(index: u8) -> Result<(HSWDEVICE, Option<String>)> {
&info,
None,
Some(sw_create_cb),
Some(ctx as *const c_void),
Some(&mut ctx as *mut SwCreateCtx as *const c_void),
)
} {
Ok(h) => h,
Err(e) => {
// SAFETY: the call failed, so no callback is pending and `ctx` is ours to reclaim.
// SAFETY: event is valid.
unsafe {
drop(Box::from_raw(ctx));
let _ = CloseHandle(event);
}
return Err(anyhow!("SwDeviceCreate(pf_xusb) failed: {e}"));
@@ -114,20 +109,17 @@ fn create_swdevice(index: u8) -> Result<(HSWDEVICE, Option<String>)> {
};
// SAFETY: event valid; block until PnP finishes enumerating, then check the callback result.
let wait = unsafe { WaitForSingleObject(event, 10_000) };
// SAFETY: event is valid.
unsafe {
let _ = CloseHandle(event);
}
if wait != WAIT_OBJECT_0 {
// Timed out — intentionally leak `ctx` and leave `event` open (see above).
// SAFETY: hsw is the handle SwDeviceCreate returned.
unsafe { SwDeviceClose(hsw) };
return Err(anyhow!(
"SwDeviceCreate(pf_xusb) enumeration callback never fired (10s) — PnP may be wedged"
));
}
// The callback ran (it signalled the event), so nothing else will touch `ctx`/`event`.
// SAFETY: `ctx` came from `Box::into_raw` and is reclaimed exactly once here.
let ctx = unsafe {
let _ = CloseHandle(event);
Box::from_raw(ctx)
};
if ctx.result.is_err() {
// SAFETY: hsw is the handle SwDeviceCreate returned.
unsafe { SwDeviceClose(hsw) };
@@ -27,10 +27,10 @@ use windows::Win32::System::StationsAndDesktops::{
use windows::Win32::UI::Input::KeyboardAndMouse::{
GetKeyboardLayout, MapVirtualKeyExW, SendInput, HKL, INPUT, INPUT_0, INPUT_KEYBOARD,
INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE,
KEYEVENTF_UNICODE, MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL,
MOUSEEVENTF_LEFTDOWN, MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP,
MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK,
MOUSEEVENTF_WHEEL, MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY,
MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL, MOUSEEVENTF_LEFTDOWN,
MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE,
MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL,
MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY,
};
use windows::Win32::UI::WindowsAndMessaging::{
GetForegroundWindow, GetSystemMetrics, GetWindowThreadProcessId, SM_CXVIRTUALSCREEN,
@@ -297,33 +297,6 @@ impl InputInjector for SendInputInjector {
};
self.send(&[key(ki)])
}
InputKind::TextInput => {
// Committed IME text: one Unicode scalar per event, injected as
// `KEYEVENTF_UNICODE` packets (wScan = UTF-16 unit, no scancode/layout involved
// — the receiving app gets the character verbatim via WM_CHAR). An astral-plane
// scalar (emoji) is its surrogate pair, each unit down+up in order — exactly how
// Windows expects supplementary characters from unicode injection.
let Some(ch) = char::from_u32(event.code) else {
return Ok(()); // lone surrogate / out of range — drop
};
if ch.is_control() {
return Ok(()); // control chars ride the VK path (Enter/Backspace/Tab)
}
let mut units = [0u16; 2];
let mut inputs: Vec<INPUT> = Vec::with_capacity(4);
for &unit in ch.encode_utf16(&mut units).iter() {
for flags in [KEYEVENTF_UNICODE, KEYEVENTF_UNICODE | KEYEVENTF_KEYUP] {
inputs.push(key(KEYBDINPUT {
wVk: VIRTUAL_KEY(0),
wScan: unit,
dwFlags: flags,
time: 0,
dwExtraInfo: 0,
}));
}
}
self.send(&inputs)
}
// Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op.
InputKind::GamepadButton
| InputKind::GamepadAxis
@@ -66,7 +66,7 @@ impl DeckWinPad {
std::ptr::write_unaligned(base as *mut u32, SHM_MAGIC);
}
let inst = format!("pf_deck_{index}");
let (hsw, instance_id) = create_swdevice(&SwDeviceProfile {
let (hsw, instance_id) = match create_swdevice(&SwDeviceProfile {
instance: &inst,
container_tag: 0x5046_4453, // "PFDS"
container_index: index,
@@ -77,8 +77,13 @@ impl DeckWinPad {
// spike's run-1 failure).
usb_mi: Some(2),
description: "punktfunk Virtual Steam Deck",
})?; // Propagate — swallowing latched the slot to a pad with no devnode (see the DS4 twin).
let (hsw, instance_id) = (Some(hsw), instance_id);
}) {
Ok((h, i)) => (Some(h), i),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "SwDeviceCreate failed; Steam Deck devnode unavailable");
(None, None)
}
};
let _sw = hsw.map(super::gamepad_raii::SwDevice::new);
// Bounded eager delivery — the driver must read `device_type = 3` before hidclass asks
// it for descriptors, or the pad would enumerate with the default DualSense identity.
-23
View File
@@ -174,29 +174,6 @@ pub fn default_backend() -> Backend {
Backend::Unsupported
}
/// Whether the session's inject backend can type **committed text**
/// ([`InputKind::TextInput`] — see `HOST_CAP_TEXT_INPUT`): Windows always (`KEYEVENTF_UNICODE`);
/// Linux only on the wlroots backend (a dedicated virtual keyboard with a dynamically-grown
/// Unicode keymap) — KWin fake-input/libei/gamescope can only press keycodes of the host layout.
/// Consulted at Welcome time to advertise the cap; a mid-session backend switch away from a
/// capable one just degrades to dropped text events (input is lossy by design).
#[cfg(target_os = "windows")]
pub fn text_input_supported() -> bool {
true
}
/// See the Windows variant: Linux types text only through the wlroots virtual-keyboard backend.
#[cfg(target_os = "linux")]
pub fn text_input_supported() -> bool {
matches!(default_backend(), Backend::WlrVirtual)
}
/// No injector ⇒ no text.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn text_input_supported() -> bool {
false
}
#[path = "inject/service.rs"]
mod service;
pub use service::InjectorService;
+1 -5
View File
@@ -11,11 +11,7 @@ repository.workspace = true
# Same Linux+Windows gating as the rest of the client stack (dmabuf import is the one
# Linux-only module — see lib.rs).
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
# `default-features = false`: the PyroWave decode backend is turned on through THIS crate's own
# `pyrowave` feature (which re-exports it below), never by inheriting the dependency's default.
# Otherwise a consumer that deliberately builds us without `pyrowave` still drags the vendored
# C++ in — fatal on Windows ARM64, where Granite has no SIMD path.
pf-client-core = { path = "../pf-client-core", default-features = false }
pf-client-core = { path = "../pf-client-core" }
# AVVkFrame access (Vulkan Video frames: live sync state under the frames lock).
pf-ffvk = { path = "../pf-ffvk" }
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
-125
View File
@@ -1,125 +0,0 @@
//! Client-side cursor rendering (design/remote-desktop-sweep.md M2): the host forwards the
//! pointer's SHAPE (reliable control stream, cached by serial) and per-frame STATE (lossy
//! `0xD0` — position/visibility), and WE draw it as a real OS cursor — pointer feel stops
//! paying the video round-trip (the Parsec/RDP model). Active only when the session
//! negotiated it (`HOST_CAP_CURSOR` in the Welcome — the host stopped compositing then) and
//! only applied while the DESKTOP mouse model is engaged: under capture the pointer is
//! relative-locked (SDL hides it) and games draw their own cursor in-frame.
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR};
use sdl3::mouse::{Cursor, MouseUtil, SystemCursor};
use std::collections::HashMap;
use std::time::Duration;
/// Shape serials cached at most — cursors cycle through a handful of shapes (arrow, I-beam,
/// resize…); a runaway host can't grow the map past this (the cache resets, shapes re-arrive
/// on the reliable stream via the serial-miss path).
const SHAPE_CACHE_MAX: usize = 64;
pub struct CursorChannel {
/// The Welcome carried `HOST_CAP_CURSOR` — the host forwards instead of compositing.
negotiated: bool,
/// Serial → built OS cursor. An SDL `Cursor` must outlive its `set()`, so the cache owns
/// every shape ever applied this session (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, Cursor>,
/// The serial currently installed via `Cursor::set` (`None` = default/system cursor).
installed: Option<u32>,
/// Latest `0xD0` state (latest-wins across a drained batch).
state: Option<CursorState>,
}
impl CursorChannel {
pub fn new(connector: &NativeClient) -> CursorChannel {
let negotiated = connector.host_caps() & HOST_CAP_CURSOR != 0;
if negotiated {
tracing::info!("cursor channel negotiated — host cursor renders locally");
}
CursorChannel {
negotiated,
shapes: HashMap::new(),
installed: None,
state: None,
}
}
/// Whether the host forwards the cursor this session (it no longer composites one).
pub fn negotiated(&self) -> bool {
self.negotiated
}
/// The latest drained `0xD0` state — the run loop reads `relative_hint` off it for the
/// M3 host-driven mode flip (and `x`/`y` as the reappear position when leaving relative).
pub fn state(&self) -> Option<CursorState> {
self.state
}
/// Drain the two planes and apply the newest state — once per run-loop iteration.
/// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then
/// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns
/// it, and released the system cursor must look normal.
pub fn pump(&mut self, connector: &NativeClient, mouse: &MouseUtil, desktop_active: bool) {
if !self.negotiated {
return;
}
while let Ok(shape) = connector.next_cursor_shape(Duration::ZERO) {
if self.shapes.len() >= SHAPE_CACHE_MAX {
// Degenerate host: reset — live shapes re-install via the serial-miss path.
self.shapes.clear();
self.installed = None;
}
let mut data = shape.rgba;
let built = sdl3::surface::Surface::from_data(
&mut data,
shape.w as u32,
shape.h as u32,
shape.w as u32 * 4,
sdl3::pixels::PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())
.and_then(|surf| {
Cursor::from_surface(&surf, shape.hot_x as i32, shape.hot_y as i32)
.map_err(|e| e.to_string())
});
match built {
Ok(cursor) => {
// A re-sent serial replaces its entry; force re-install if it's current.
if self.installed == Some(shape.serial) {
self.installed = None;
}
self.shapes.insert(shape.serial, cursor);
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
}
while let Ok(st) = connector.next_cursor_state(Duration::ZERO) {
self.state = Some(st); // latest wins
}
if !desktop_active {
// Capture mode / released: hand the cursor back to the system default so a
// released pointer over the window doesn't wear the host's shape.
if self.installed.take().is_some() {
Cursor::from_system(SystemCursor::Arrow)
.map(|c| c.set())
.ok();
}
return;
}
let Some(st) = self.state else { return };
if st.visible() && self.installed != Some(st.serial) {
if let Some(cursor) = self.shapes.get(&st.serial) {
cursor.set();
self.installed = Some(st.serial);
}
// Serial miss: the (reliable) shape hasn't landed yet — keep the previous
// cursor for the RTT rather than flashing default.
}
// Visibility follows the host (a host app hid its pointer ⇒ ours hides too). Queried,
// not shadowed, so apply_capture's own show/hide calls can never desync us.
if mouse.is_cursor_showing() != st.visible() {
mouse.show_cursor(st.visible());
}
}
}
+3 -77
View File
@@ -14,17 +14,10 @@
//! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are
//! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would
//! otherwise send a datagram per event).
//!
//! The DESKTOP mouse model (design/remote-desktop-sweep.md M1) reuses this same engage/
//! release state but never locks the pointer: the local cursor moves freely (hidden over
//! the window — the host's composited cursor is the one you see) and motion goes on the
//! wire as absolute positions through the letterbox (`MouseMoveAbs`, latest-wins per loop
//! iteration). Requires a host injector with absolute support — gamescope's EIS is
//! relative-only, so sessions there are pinned to capture ([`Capture::new`] `abs_ok`).
use crate::keymap_sdl;
use crate::touch::{Abs, Act, Gestures};
use pf_client_core::trust::{MouseMode, TouchMode};
use pf_client_core::trust::TouchMode;
use punktfunk_core::client::NativeClient;
use punktfunk_core::input::{InputEvent, InputKind};
use std::collections::{HashMap, HashSet};
@@ -48,13 +41,6 @@ pub struct Capture {
held_buttons: HashSet<u32>,
/// Relative motion not yet on the wire, summed per loop iteration.
pending_rel: (i32, i32),
/// Desktop-model position not yet on the wire, latest-wins per loop iteration.
pending_abs: Option<Abs>,
/// The desktop (absolute, uncaptured) mouse model is active. Flipped live by the
/// Ctrl+Alt+Shift+M chord; never true unless `abs_ok`.
desktop: bool,
/// The host injector accepts `MouseMoveAbs` (any compositor but gamescope).
abs_ok: bool,
/// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space —
/// precision surfaces deliver sub-unit deltas; truncating each event drops the tail.
scroll_acc: (f64, f64),
@@ -84,14 +70,10 @@ fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, fl
}
impl Capture {
/// `abs_ok` = the host injector accepts absolute pointer events; without it the
/// desktop model is unavailable and `mouse_mode` silently resolves to capture.
pub fn new(
connector: Arc<NativeClient>,
touch_mode: TouchMode,
invert_scroll: bool,
mouse_mode: MouseMode,
abs_ok: bool,
) -> Capture {
Capture {
connector,
@@ -100,9 +82,6 @@ impl Capture {
held_keys: HashSet::new(),
held_buttons: HashSet::new(),
pending_rel: (0, 0),
pending_abs: None,
desktop: abs_ok && mouse_mode == MouseMode::Desktop,
abs_ok,
scroll_acc: (0.0, 0.0),
touch_slots: HashMap::new(),
touch_mode,
@@ -115,38 +94,6 @@ impl Capture {
self.captured
}
/// The desktop (absolute, uncaptured) mouse model is active.
pub fn desktop(&self) -> bool {
self.desktop
}
/// Flip capture ⇄ desktop (the Ctrl+Alt+Shift+M chord). `None` = the host can't take
/// absolute pointer events (gamescope), so the chord has nothing to offer; otherwise
/// the new desktop state. Motion gathered under the old model never crosses modes.
pub fn toggle_desktop(&mut self) -> Option<bool> {
if !self.abs_ok {
return None;
}
self.desktop = !self.desktop;
self.pending_rel = (0, 0);
self.pending_abs = None;
Some(self.desktop)
}
/// Set the mouse model directly (the M3 host-driven flip — `relative_hint` says a host
/// app grabbed/hid the pointer, so run relative; hint clear = back to absolute). Same
/// gating and motion hygiene as [`toggle_desktop`](Self::toggle_desktop); returns whether
/// the model actually changed.
pub fn set_desktop(&mut self, on: bool) -> bool {
if !self.abs_ok || self.desktop == on {
return false;
}
self.desktop = on;
self.pending_rel = (0, 0);
self.pending_abs = None;
true
}
/// Whether a regained focus should re-engage: yes unless the user released
/// deliberately (the chord keeps its meaning across an Alt-Tab).
pub fn should_reengage(&self) -> bool {
@@ -170,7 +117,6 @@ impl Capture {
return false;
}
self.pending_rel = (0, 0); // never flush motion gathered while captured
self.pending_abs = None;
for vk in self.held_keys.drain() {
send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0);
}
@@ -186,42 +132,22 @@ impl Capture {
true
}
/// Forward the coalesced motion, if any — one datagram per loop iteration. Only one
/// of the two stores is ever populated (the run loop routes by [`desktop`](Self::desktop)).
/// Forward the coalesced motion delta, if any — one datagram per loop iteration.
pub fn flush_motion(&mut self) {
let (dx, dy) = std::mem::take(&mut self.pending_rel);
if dx != 0 || dy != 0 {
send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0);
}
if let Some(a) = self.pending_abs.take() {
send(
&self.connector,
InputKind::MouseMoveAbs,
0,
a.x,
a.y,
Self::touch_flags(a.w, a.h),
);
}
}
/// Relative motion (SDL relative mouse mode delivers raw deltas while locked).
pub fn on_motion(&mut self, xrel: f32, yrel: f32) {
if self.captured && !self.desktop {
if self.captured {
self.pending_rel.0 += xrel as i32;
self.pending_rel.1 += yrel as i32;
}
}
/// Desktop-model motion: the cursor's position mapped into the letterboxed content
/// rect. Latest-wins — intermediate positions carry no information the final one
/// doesn't (unlike deltas, which must sum).
pub fn on_motion_abs(&mut self, abs: Abs) {
if self.captured && self.desktop {
self.pending_abs = Some(abs);
}
}
pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) {
if !self.captured {
return;

Some files were not shown because too many files have changed in this diff Show More