Compare commits

...
Author SHA1 Message Date
enricobuehlerandClaude Opus 5 28f8fc71c4 refactor(pf-encode): split vulkan_video's construction tail into vk_build.rs (WP7.5)
ci / web (push) Successful in 56s
ci / rust-arm64 (push) Failing after 2m18s
ci / docs-site (push) Successful in 1m2s
android / android (push) Failing after 4m7s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
apple / swift (push) Successful in 5m21s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m10s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 5m40s
deb / build-publish-client-arm64 (push) Successful in 7m57s
docker / build-push-arm64cross (push) Successful in 10s
deb / build-publish (push) Successful in 10m6s
docker / deploy-docs (push) Successful in 24s
deb / build-publish-host (push) Successful in 10m30s
arch / build-publish (push) Successful in 13m57s
windows-host / package (push) Successful in 14m6s
ci / rust (push) Successful in 22m9s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m30s
apple / screenshots (push) Successful in 24m30s
The ~820-line tail of free builders — make_frame/make_frame_csc/
make_frame_common, make_video_image, probe_rgb_direct, the H.265/AV1
parameter-set writers and the AV1 bit-writer — moves to a #[path] child
module, the amf_sys.rs shape: the child sees the parent's private items
(Frame and friends), so the split costs zero visibility churn. Six
parent-called items went pub(super); five stay child-private (dead_code
is per-item and each is used within the child). vulkan_video.rs drops
5,292 → 4,489 lines and the construction unsafe gets its own review
surface; steady-state encode logic stays in the parent.

⚠ Trap recorded for future child-module splits: inline `use super::X`
statements INSIDE moved fn bodies silently change meaning (super shifts
one level) — vk_av1_encode/vk_valve_rgb imports needed crate:: paths.

Proven on-glass, not just compiled: all 8 vulkan GPU smokes green under
the validation layers on the 780M post-split (H.265 + AV1, RGB-direct,
CSC, CPU paths — every moved constructor exercised).

nvenc_cuda.rs and qsv.rs are DECLINED the same treatment, with evidence
in the handoff doc: no equivalent self-contained seam — their candidate
regions are ~150-line loader/accessor clusters interleaved with the
encoders' own state types, and a thin-forwarder impl split is exactly
the churn a no-defect phase penalizes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 00:08:03 +02:00
enricobuehlerandClaude Opus 5 bc5ec5105c refactor(pf-encode): one Linux backend resolver, consumed by dispatch AND mirrors (WP7.6)
The Linux backend decision existed as open_video_backend's string match
plus five partial hand-copies (the zero-copy-plane gate, the pyro
advertisement gate, the software advertisement pin, the vulkan pref
ceiling, resolved_backend_is_gpu). Windows solved this long ago —
windows_resolved_backend() is consumed by its dispatch AND its mirrors,
with labels still stamped at the open sites. Linux now has the twin:
resolve_linux_backend (pure, lazy auto probe) + linux_resolved_backend
(config wrapper, unknown→auto exactly as every mirror's old `_` arm).

⚠ This deliberately DEVIATES from the audit's shadow-assertion
prescription, on both critics' findings: the shadow had no execution
venue (open_video had ZERO test call sites; shipped hosts are --release)
— unfalsifiable ceremony — and the file's own Windows half proves
dispatch-consumes-resolver is safe: the mgmt record is protected by the
label-from-the-open-site convention, not by resolver avoidance. One
consumed table beats two tables plus an inert cross-check.

Preservation riders from the critique, all applied:
- pref_ceiling KEEPS its cfg!(vulkan-encode) branch (the resolver is
  feature-blind; dropping it re-creates advertise-then-die-at-open).
- linux_zero_copy_is_vaapi's Vulkan|Software arm preserves the old `_`
  fallthrough EXACTLY — the vulkan-on-NVIDIA capture-plane mismatch and
  the software twin are FILED in the design doc, not fixed here.
- resolved_backend_is_gpu splits as linux + not(any(windows, linux)) —
  never a target list, so no exotic target loses the fn.
- The auto probe is lazy (impl FnOnce) — explicit prefs stay zero-probe
  (/serverinfo polls through these mirrors), pinned by a panicking
  closure in the resolver test.

New coverage: the full alias table pinned, and a GPU-free dispatch test
through the real open_video_backend (software arm, vendored openh264) —
its first test call site anywhere. Config-latch seam guarded loudly.

Cross-crate mirrors recorded out of scope in the design doc:
session_plan::resolve_encoder, gamestream/serverinfo.rs
base_codec_mode_support, capture.rs's "pyrowave" string.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 00:08:03 +02:00
enricobuehlerandClaude Opus 5 cb69cd3b8c refactor(pf-encode): move the AMF C-ABI mirror to its own file (WP7.4)
The ~410-line hand-mirrored AMF vtable ABI was already isolated in an
inline `mod sys` — the remaining value of the audit's 'best split in
the crate' is the FILE boundary: amf.rs drops 3379 → 2965 lines and the
pure unsafe-FFI surface (25 unsafe fns, zero policy) is reviewable in
isolation, which is the crate's stated review goal.

Pure move: `#[path = "amf_sys.rs"] mod sys;` keeps the module name and
every `sys::` call site byte-identical; the module was self-contained
(only `use std::ffi::c_void`, no super:: references). The banner became
the file's module docs; contents de-indented one level — rustfmt-clean
on the first check, which is what proves the move byte-exact.

Gated on .173: all five clippy combos + the amf-qsv,qsv test leg (38
passed — the live AMF matrix on real VCN silicon among them).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 00:08:03 +02:00
enricobuehlerandClaude Opus 5 9491cc8759 refactor(pf-encode): extract the range-family RFI recovery policy (WP7.2)
The two direct-NVENC backends carried hand-copied twins of the same
loss-recovery decision: range validity, covering-range dedup, DPB
window, clamp — ~30 duplicated lines each. The decision now lives once
as nvenc_core::plan_range_recovery (the range half of WP7.2; the slot
half is enc/rfi.rs), pure and unit-tested; each backend keeps its
session gate, its unsafe per-timestamp driver loop, and its state
stores.

The step order is load-bearing and now pinned by tests: the covering
dedup runs with the UNCLAMPED last and BEFORE the DPB window (a covered
re-ask never touches the driver even when the range has since aged out
of the DPB), the boundary at next_ts - RFI_DPB is inclusive, and the
Invalidate carries the CLAMPED last — which is also what the caller
records in last_rfi_range, exactly as the inline code stored it. A
driver failure mid-loop still returns false with NO range recorded and
no anchor armed. Decline deliberately clears nothing (neither twin
touched pending_anchor on decline — same shape as Vulkan's non-clear,
opposite of AMF/QSV; do not harmonize).

The exact-cover → Covered test records EXISTING behavior including that
a covered range survives a forced IDR with zero driver calls — a
recorded fact, not an endorsement. RFI_DPB's import leaves both twins:
its only per-backend use was the arithmetic that moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 00:08:03 +02:00
enricobuehlerandClaude Opus 5 9f1e648e4e refactor(pf-encode): extract the slot-family RFI recovery policy (WP7.2)
AMF (user-LTR bitfield), QSV (mfxExtRefListCtrl) and Vulkan Video (the
app-owned DPB slot table) each hand-implemented the same loss-recovery
decision: distrust every reference encoded at-or-after the loss start,
anchor on the newest one strictly older. Three copies had already
diverged once — the fecbec2d taint sweep reached AMF/QSV a commit before
the Vulkan backend was carved out, and Vulkan shipped without it. The
decision now lives once in enc/rfi.rs, pure and unit-tested (this path
had zero coverage and is the loss-recovery path); every mechanism —
how a force is applied, how distrust is persisted — stays in its
backend.

Behavior-preservation notes, each critic-verified against the shipped
code:
- Callers feed only currently-trusted references; taint (>= loss) and
  anchor (< loss) predicates are disjoint, so pre-sweep-view pick ==
  post-sweep-table pick on every input. Tie-break preserved (first
  entry wins == the strict '>' all three used, ascending slot order).
- plan_slot_recovery delegates its pick half to pick_anchor, which
  keeps both items live on every leg (pick_anchor's only external
  caller is Linux+vulkan-encode; dead_code is an item lint) and makes
  'anchor chosen from the taint snapshot' structural.
- The decline arms are deliberately NOT uniform and stay put: AMF/QSV
  clear an un-consumed pending_force; Vulkan leaves pending_loss armed
  (a stale arm re-resolves at frame-build into the healing IDR) — now
  pinned in comments on both sides.
- Vulkan's frame-build site is a pick-only re-run by design (the arm
  carries the loss start, not the slot); pick_recovery_slot is deleted
  and its tests migrated 1:1 into the shared module.

Test-count deltas are expected: the rfi tests now run on every Windows
combo (amf.rs is featureless there) and on the Linux feature leg.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 00:08:03 +02:00
enricobuehlerandClaude Opus 5 d28fb1282b test(pf-encode): guard TrackedEncoder's forwarding completeness (WP7.7, cheap half)
A defaulted Encoder method that TrackedEncoder doesn't forward silently
no-ops through the wrapper — the host loop only ever holds the wrapped
box, so the feature dies for every session with nothing in the logs. The
trap has bitten three times (set_wire_chunking's §4.4 chunking probe,
set_pipelined's LN3 escalation, applied_bitrate_bps's ABR truth), and
every Phase 7 consolidation that adds a trait method re-arms it.

Source-text parse of the trait and the forwarding impl (both top-level
rustfmt items: block ends at the first column-0 brace, method names sit
on 'fn '-prefixed lines), then set equality — the reverse direction is
already a compile error, so equality == completeness. Mutation-verified:
removing set_wire_chunking's forward fails naming exactly that method.
16/16 today.

Limit stated in the test doc: name-set equality only — a forward whose
body delegates to the WRONG inner method still passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 00:08:03 +02:00
enricobuehler 94b818da28 fix(client/pyrowave): declare the decode planes' real format — 10-bit sessions decoded through an R8 view of R16 planes
apple / swift (push) Successful in 5m14s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m32s
arch / build-publish (push) Failing after 7m6s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
decky / build-publish (push) Successful in 18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m53s
android / android (push) Successful in 12m56s
docker / deploy-docs (push) Successful in 24s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m41s
docker / build-push-arm64cross (push) Successful in 4m15s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 7m44s
deb / build-publish-host (push) Successful in 10m43s
deb / build-publish-client-arm64 (push) Successful in 9m8s
apple / screenshots (push) Successful in 22m58s
flatpak / build-publish (push) Successful in 6m56s
deb / build-publish (push) Successful in 8m58s
ci / rust (push) Successful in 22m10s
ci / rust-arm64 (push) Successful in 9m22s
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 1m13s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 12m48s
ci / bench (push) Successful in 6m3s
The HDR/4:4:4 leg (188edde2, shipped 0.14.0) taught the client's plane
ring to allocate R16_UNORM on a 10-bit session, but the
pyrowave_image_view structs handed to decode_gpu_buffer kept the
original SDR shape: image_format/view_format hardcoded R8_UNORM, chroma
extents hardcoded half-res. pyrowave wraps our VkImage under the
DECLARED format and vkCreateImageView's its storage view from it
(pyrowave_c.cpp WrappedViewBuffers::wrap) — an R8 view over a
non-MUTABLE R16 image is invalid Vulkan that release drivers execute
anyway: the view's addressing covers half the surface, so the iDWT's
8-bit codes fuse pairwise into 16-bit texels (structured tile-scrambled
garbage) and the never-written remainder samples as all-plane zeros
(saturated green). Field report 2026-07-25: RX 9070 XT host → 780M
client, total corruption on every 10-bit PyroWave session regardless of
bitrate/refresh, wire counters spotless; milder shapes of the same UB
on NVIDIA clients; Arc validation never saw it because it ran SDR
(R8 ring — declaration consistent by accident).

Declare what the ring really built: format from hdr16 (R16/R8), chroma
extents from chroma444 (full-/half-res) — exactly what upstream's own
pyrowave_image_get_image_view fills from the image itself. Host
encoders audited clean: Windows keys plane formats off hdr16 already,
Linux is 8-bit end to end.
2026-07-25 21:39:30 +02:00
enricobuehlerandClaude Opus 5 3e7828522d fix(tray): drop the conflicting-host warning — the tray is the wrong place for it
ci / web (push) Successful in 59s
ci / docs-site (push) Successful in 1m6s
ci / bench (push) Successful in 7m0s
deb / build-publish (push) Failing after 7m35s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
deb / build-publish-host (push) Failing after 8m6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
android / android (push) Successful in 13m38s
ci / rust-arm64 (push) Successful in 12m28s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 14m10s
windows-host / package (push) Successful in 10m56s
deb / build-publish-client-arm64 (push) Successful in 8m36s
docker / build-push-arm64cross (push) Successful in 4m37s
apple / swift (push) Successful in 5m28s
ci / rust (push) Successful in 22m58s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 13m9s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m0s
apple / screenshots (push) Successful in 23m51s
The tray led its tooltip with "⚠ conflicting host: Sunshine" and (on Linux)
went NeedsAttention for a Sunshine that was merely *installed* — never
started, never listening, harmless. An always-on warning over a dormant
package is a false alarm on every poll, and the tray is the wrong surface
for a one-shot install-time observation anyway.

Removes the `conflicts` field from the tray's `Summary`, the tooltip prefix,
`has_conflicts()`, and the ksni attention branch that fed on it. The host
still detects and reports conflicts where that belongs — the startup
`punktfunk::detect` warning, the `detect-conflicts` subcommand, and the
`/api/v1/local/summary` field, which the tray must (and does, per the new
test) keep tolerating on the wire.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 20:53:44 +02:00
enricobuehlerandClaude Fable 5 ffc7aec91a feat(encode/pyrowave): log which GPU pyrowave picked — the selection stays put, by decision
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m12s
apple / swift (push) Successful in 5m20s
ci / rust (push) Failing after 5m55s
android / android (push) Failing after 5m59s
deb / build-publish (push) Failing after 5m1s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 17s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 7m17s
ci / rust-arm64 (push) Successful in 9m40s
docker / deploy-docs (push) Successful in 26s
docker / build-push-arm64cross (push) Successful in 4m9s
deb / build-publish-host (push) Successful in 10m32s
arch / build-publish (push) Failing after 12m34s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 5m58s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 5m50s
deb / build-publish-client-arm64 (push) Failing after 7m10s
windows-host / package (push) Successful in 17m27s
apple / screenshots (push) Successful in 23m26s
WP4.5's device-selection half, closed as the observability intermediate after
TWO selection designs died in adversarial review:

- Attempt #1 (d26bcf05, withdrawn): match pf_gpu::selected_gpu(). Its Linux
  auto arm answers "the NVIDIA GPU" whenever /dev/nvidiactl exists, moving
  the encoder off the iGPU that can import the compositor's dmabufs on an
  Intel-compositor + NVIDIA-present laptop — import failures feed the
  process-wide raw-dmabuf latch, which never un-latches.
- Attempt #2 (this session, withdrawn before commit): anchor on the
  PUNKTFUNK_RENDER_NODE-else-renderD128 node via VK_EXT_physical_device_drm.
  Render minors are driver-BIND-ORDER artifacts, not display topology: on
  the common AMD-iGPU + NVIDIA-display desktop, in-tree amdgpu binds before
  out-of-tree nvidia, so the anchor deterministically picks the idle iGPU
  while the compositor allocates on NVIDIA — the same latch, opposite
  polarity, behind a success-looking log.

The correct oracle is evidence of which device ALLOCATED the capture
buffers — producer identity from the capture negotiation, threaded per
session into this open. Until that plumbing exists, selection stays
first-usable, both call sites still share one selector (pure over the
device list, so capture_modifiers and open_inner cannot diverge — including
across an in-place resize's re-open, which does not renegotiate capture),
and the open logs ONE greppable line: picked vendor/device, the anchor node
and its owner (DRM render major/minor, VK_EXT_pci_bus_info fallback), and
the console's selected GPU. A wrong-device session on a multi-GPU host used
to be completely invisible; a field report can now show it. No WARN arm on
purpose: the wrong-pick direction inverts between the laptop and desktop
topologies, so a mismatch is not evidence of a wrong pick, and a warning
that fires forever on healthy hosts teaches people to ignore warnings.

Decision recorded against the audit's framing: manual console GPU selection
stays unhonored by pyrowave on Linux (the Windows twin honors it) — honoring
console-mutable state without per-session threading is what made attempt #1
unsafe.

Verified on the 780M: the line resolves all three identities (1002:15bf x3,
DRM-props match live); full pyrowave on-glass suite green; selection
behaviour byte-for-byte unchanged.

WP4.5 (device half).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 19:22:28 +02:00
enricobuehlerandClaude Fable 5 46935bf0a5 fix(encode/vulkan): VBR instead of CBR — the driver was stuffing ~98% filler into every calm stream
Second attempt at WP6.3; the first (ce543668) was withdrawn after its tight
CBR window measured as a 36x bandwidth regression (97% filler NALs). The
correction that unlocked this one: that measurement's BASELINE row was an
8-frame artifact. Under the shipped 1000ms/500ms CBR window a calm stream
overflows the CPB once the initial fill drains (~30 frames at 10Mbps/60fps)
and RADV then pads every frame to the exact rate share, forever. Measured on
the 780M (1280x720@60, 10Mbps, calm content): 64 frames = 97.5% filler;
300 frames = 5.63MB at 98.5% filler where this commit ships 83KB at 0%.
AV1: 99.6% -> 0%. The status quo was ~the full target bitrate of zeros on
every idle AMD/Intel Vulkan-encode desktop and Steam Deck — and stuffing to
exactly the target permanently satisfied the ABR calm brake
(actual >= 3/4 * current), the ratchet WP6.3's withdrawal feared from the
tight window, live in the shipped code all along.

The fix reads VkVideoEncodeCapabilitiesKHR::rateControlModes (previously
ignored — rateControlMode was hardcoded CBR with no capability check) and
installs VBR with average == max plus the house ~1-frame window
(vbv_window_ms, PUNKTFUNK_VBV_FRAMES-scaled) when the driver advertises VBR.
VBR permits underspend, the exact missing degree of freedom: Vulkan exposes
no filler-suppression control (AMF's filler_data=false / NVENC's default-off
have no VK equivalent), so the MODE is the only lever. CBR-only drivers keep
the loose window untouched — tightening it under CBR just starts the stuffing
30 frames earlier. Drivers advertising neither mode (ANV per current Mesa)
keep the pre-existing CBR install, now WARN-logged.

No pacing claim, deliberately: burst A/B on the 780M is byte-identical
between 1000ms CBR and 17ms VBR (max AU 1.19MB in both) — this firmware
ignores the window for QP decisions entirely. The payload is filler
elimination. PUNKTFUNK_VULKAN_RC=cbr|vbr is the field escape hatch and the
on-box A/B control (two withdrawn attempts bought that insurance).

Also on the same caps struct: maxBitrate is now read and clamps open +
retarget (RADV reports 1 Gbps — within 5% of the 4K120 ABR targets), and
applied_bitrate_bps() reports the encoder-side truth (pending-first, so the
session loop's read right after reconfigure_bitrate sees the clamp) — without
it a binding clamp would feed the ABR a phantom base, the trap the trait doc
names.

And the one-frame VUID-vkCmdBeginVideoCodingKHR-pBeginInfo-08254 violation
found in the withdrawal review: record_submit promoted a pending retarget
into self.bitrate BEFORE recording whenever first_frame was set, so after a
mid-stream reset() (which preserves the pending rate and rc_installed) the
begin-coding declaration named a rate the session had not installed — and the
two triggers, ABR retarget and the stall watchdog, correlate. Now the
declaration always names the session's current rate and the RESET install
carries the pending one via its own struct; promotion stays in
post_submit_bookkeeping. The extended validation-layer test reproduces the
retarget-then-reset coincidence: exactly one 08254 on the pre-fix build, zero
on this one (RADV PHOENIX, on glass).

Gates: docker amd64 legs green; Windows .173 seven legs (34 passed);
.25 full-suite parity vs origin/main (identical CUDA-only failures) + all 9
vulkan on-glass tests + validation layers clean.

WP6.3, plus WP7.1's ms-form half (vbv_window_ms).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 19:22:12 +02:00
enricobuehlerandClaude Opus 5 fdded5b8c3 fix(encode/pyrowave): refuse a frame that isn't the session's mode
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 55s
ci / bench (push) Failing after 6m37s
ci / rust (push) Failing after 6m43s
deb / build-publish-host (push) Failing after 5m44s
arch / build-publish (push) Failing after 6m45s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
decky / build-publish (push) Successful in 18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 7s
docker / build-push-arm64cross (push) Successful in 9s
docker / deploy-docs (push) Successful in 23s
ci / rust-arm64 (push) Successful in 9m47s
deb / build-publish (push) Successful in 9m9s
android / android (push) Successful in 11m59s
deb / build-publish-client-arm64 (push) Successful in 8m44s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 8m27s
windows-host / package (push) Successful in 17m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m58s
apple / swift (push) Successful in 6m51s
apple / screenshots (push) Successful in 26m43s
PyroWave never checked frame dimensions against the session, and it applies no
alignment — `width`/`height` are the negotiated mode verbatim — so a mismatched
frame was encoded edge-smeared or cropped, silently, forever.

Every other Linux backend already refuses exactly this, with this shape, in
`submit`: libav-NVENC (`linux/mod.rs`), VAAPI (`vaapi.rs`) and openh264 (`sw.rs`)
all carry the same `ensure!`. PyroWave was the only one that didn't. That is the
justification; an earlier draft cited `vulkan_video.rs` instead, which is the
weaker precedent — it bails in its Dmabuf arms only, and its CSC path and CPU arm
have no dimension guard at all.

Mostly this is a wrong-picture bug and not a memory-safety one: `rgb2yuv.comp`
clamps every fetch with `min(p, textureSize - 1)` and the CPU arm uploads
`min(len, need)` into a session-sized image. But it also closes a narrow real
hazard that was not in the filing: `import_cached` keys on `(st_dev, st_ino)` and
returns the cached `VkImage` on a hit WITHOUT rechecking the extent, and unlike
the capture side it is never cleared on a renegotiation — so a dmabuf inode
recycled across a shrinking renegotiation would hand the encoder an image sized
for the old, larger allocation. This check closes that route.

⚠ Recorded at the code because it changes the failure mode, not just the
detection: a mismatch is NOT always transient. A compositor-initiated PipeWire
renegotiation updates the capturer's size in place and signals nothing the encode
loop reads, so it can be a permanent new steady state — and `reset()` reopens at
the same dimensions by construction, so the host's five-reset budget cannot
recover (~3.1 s of frozen stream, then the session ends) where before it would
have streamed on with a wrong picture. For a real mode change that trade is
clearly right; for a 16-row KWin mismatch it is not, and the proper fix is for
the host to classify this error as a PIPELINE rebuild rather than an encoder
reset. Filed, not done here.

The device-selection half of WP4.5 was written, reviewed and WITHDRAWN — see the
handoff doc. Matching the selected render GPU regresses the hybrid
Intel-compositor + NVIDIA-present topology this project has a live field report
for, because `selected_gpu()` answers NVIDIA whenever `/dev/nvidiactl` exists
regardless of where capture actually runs.

WP4.5 (dimension half).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 17:56:38 +02:00
enricobuehlerandClaude Opus 5 9fe9cbbf07 perf(encode/vulkan): build the padded RGB frame in the staging memory, not beside it
The RGB-direct CPU-upload path allocated and zero-filled a whole padded frame on
every submit, filled it row by row, then memcpy'd the whole thing into the mapped
staging buffer. The zero-fill was entirely dead: the row loop writes every byte
of every row, and rows past the source re-copy the last source row. So the frame
paid for an allocation, a page-fault storm over fresh pages, a full zero-fill,
and two full-frame copies where one would do.

Map first, write the padded rows straight into the mapping. Nothing reads back
from the destination — the row source is always the caller's buffer, never the
staging memory — so writing into (write-combined) host memory costs nothing
extra, and `make_host_buffer` allocates HOST_COHERENT, so the writes need no
flush before the transfer reads them.

Measured on real RDNA3 780M silicon, release build, `submit` alone, 400 frames
x 3 rounds:

- **1920x1080** (rows padded to 1088): p50 2315 -> **1725 us** (-25%),
  p99 2819 -> **1972 us** (-30%), and the spread (p99-min) tightens 739 -> 483 us.
- **1366x768 -> 1408x768** (BOTH axes padded, so the column tail loop runs):
  p50 1013 -> **926 us** (-8%), p99 1303 -> **1110 us** (-15%). This is the one
  case where the new code could have been slower — 4-byte stores straight into
  write-combined memory — and it is not.
- **CONTROL, 1280x720** (64x16-aligned, so the branch is never entered):
  p50 692 vs 692 us. No delta, which is what makes the two above attributable to
  this change rather than to anything else on the branch.

The branch is reached by any RGB-direct session at a mode that is not 64x16
aligned, whenever capture delivers CPU frames (the default Linux capture path is
dmabuf and never enters it). 1080p qualifies, since 1080 aligns to 1088. ⚠ An
earlier draft of this message claimed "33 MB at 4K" — that is wrong: 3840 and
2160 are both already aligned, so 4K UHD never enters this branch at all. The
large case is an ultrawide like 3440x1440 -> 3456x1440, ~20 MB.

Three things this deliberately does NOT do:

- The extent guards stay scoped to the `pad` branch. The CSC path deliberately
  supports a source SMALLER than the encode extent — its shader clamps at sample
  time — so a guard hoisted above the branch would break it.
- Every fallible step stays above the map. An error raised between `map_memory`
  and `unmap_memory` would strand the mapping for the life of the slot's staging
  buffer, and the next frame's `map_memory` on it then violates
  VUID-vkMapMemory-memory-00678. (The filed "two exits above the map leak Vulkan
  objects" hazard is separately already gone: `47a23bec` moved that unwind into
  `make_host_buffer`.)
- It adds guards rather than removing them: a zero source axis made `sh - 1`
  underflow, and "cannot fail after the map" has to be true by construction.

Three corrections to an earlier draft, all found by review of that draft:

- The `dw*dh*4 == need` precondition was a `debug_assert!` placed BELOW the map.
  That is wrong twice: it made the only check on the slice length vanish from the
  builds that ship, and a fired assert would have unwound past `unmap_memory` —
  the exact failure the bullet above says was designed out. It is now a real
  checked `?` above the map.
- `need` was `(iw * ih * 4) as u64`: a u32 multiply widened after the fact, which
  agreed with the usize slice length only up to ~32768x32768. The old code's
  `min(need)` was a hard backstop against exactly that and the rewrite dropped
  it. Now widened before the multiply, matching `read_slot`'s existing discipline.
- The extent guard now runs BEFORE the payload-length guard, so the usize
  `sw * sh * 4` cannot overflow on a garbage frame header.

WP6.2(a).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 17:56:22 +02:00
enricobuehlerandClaude Opus 5 e680096c6a perf(encode): stop re-reading the environment on every submit and poll
`std::env::var` was on three per-frame paths. Measured on `.173` (Windows, 57
environment variables, 2M iterations): **121.9 ns** per call for the NVENC
in-flight cap and **114.9 ns** per call for the ffmpeg poll spin, against
**0.9 ns** once memoized. On Linux, 32 ns → 1.5 ns.

⚠ Those numbers deflate the filing, and that is worth recording: the audit
ranked this as a hot-path defect, but ~120 ns/frame is ~0.003% of a frame
budget. The fix is still right — it is free, and it takes a global environment
lock off the encode thread — but nobody should schedule it ahead of anything on
the strength of the "hot path" framing.

The severity RANKING was also inverted, and the measurement confirms why. The
site the audit called worst — `nvenc_cuda`'s backpressure loop condition — costs
a default session nothing, because the condition short-circuits on
`async_rt.is_some()` and the default session never engages the two-thread
retrieve. The one that actually pays every frame is Windows `submit`, which
consults `async_inflight_cap()` in BOTH arms of the ring-depth match, sync mode
included, where the result is then thrown away. Windows `poll` is the second
unconditional one, and the audit ranked it last.

Memoized inside each helper rather than latched into a session field. Nothing in
the workspace mutates these variables at runtime (enumerated: no `set_var` for
either key anywhere in first-party code, and the Windows service's arbitrary-key
`host.env` loader runs in the SCM supervisor, which re-execs the host as a child
and never opens an encoder itself). A field would instead change WHEN the value
is read — and the Windows `cap` composes the env with `input_ring_depth`, which
`set_input_ring_depth` may change after open, so freezing that half would
reintroduce the in-place-overwrite bug the ring term exists to prevent.

Two deliberate behaviour changes ride along on `PUNKTFUNK_FFWIN_POLL_MS`, so
"behaviour-preserving" describes the memoization only, not this whole commit:

- **A 1000 ms ceiling.** The reachable hazard was never the overflow — that
  needed `ms >= 1.8e16` — it was a slipped digit: `=100000000` was a 27.7-hour
  spin of the encode thread.
- **`.trim()`, now on all three parsers.** An earlier draft applied the house
  rule (WP7.8) to one of the three, which left `PUNKTFUNK_NVENC_ASYNC=" 1 "`
  working while `PUNKTFUNK_NVENC_ASYNC_DEPTH=" 6 "` silently fell back to 4 —
  and memoization would have frozen that silent fallback for the process
  lifetime. Reachable: the Windows `host.env` loader trims around `=` before
  stripping quotes, so `=" 2 "` yields a value with inner spaces.

⚠ Correction to an earlier draft of this message, which asserted that the audit's
`saturating_mul` proposal would relocate an overflow panic into release builds.
That is FALSE and the code comment now says so: `Duration::from_micros(u64::MAX)`
is ~1.8e13 seconds, six orders of magnitude below `Duration`'s ceiling, so
`Instant + Duration` neither overflows nor panics (measured, with and without
debug assertions). `saturating_mul` is still wrong, for a different reason — it
sets a deadline ~584,000 years out on a spin that provably never produces the
owed AU, i.e. it wedges the encode thread permanently. A hang, not a panic.

WP6.1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 17:55:59 +02:00
enricobuehler 0044649b19 fix(encode/nvenc): codec-gate the HEVC 4:4:4 union write, and stop it eating the 10-bit arm
android / android (push) Successful in 12m39s
windows-host / package (push) Successful in 10m31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
decky / build-publish (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
arch / build-publish (push) Successful in 18m50s
docker / deploy-docs (push) Successful in 12s
docker / build-push-arm64cross (push) Successful in 5m55s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m21s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m29s
ci / docs-site (push) Successful in 1m11s
ci / web (push) Successful in 1m11s
apple / swift (push) Successful in 5m39s
ci / bench (push) Successful in 6m55s
ci / rust-arm64 (push) Failing after 8m55s
ci / rust (push) Successful in 21m27s
deb / build-publish (push) Successful in 8m45s
deb / build-publish-client-arm64 (push) Successful in 9m9s
deb / build-publish-host (push) Successful in 11m26s
apple / screenshots (push) Successful in 26m31s
`encodeCodecConfig` is a C union, so the `hevcConfig` writes in the 4:4:4 branch are only
meaningful on an HEVC session — on H.264 or AV1 they reinterpret that codec's own config bytes.
The branch was gated purely on `chroma_444 && full_chroma_input` with no codec test, and stayed
non-UB only because `lib.rs` degrades 4:4:4 for non-HEVC codecs: a two-file invariant with
nothing asserting it, on the path BOTH direct-NVENC backends take.

The audit filed that much. What it did not note is that the same shape hides a second bug:
this is an `if`/`else if`, so a non-HEVC session that arrived with `chroma_444` set took the
HEVC branch and skipped the per-codec bit-depth arm entirely — ending up with neither HEVC
4:4:4 (wrong for it) nor its own 10-bit configuration (simply absent). AV1 would have lost
`pixelBitDepthMinus8`/`inputPixelBitDepthMinus8` silently. Non-HEVC now falls through to the
arm that knows what to do with it, and an unexpected request is logged rather than swallowed.

Adds two tests that need no GPU — `apply_low_latency_config` is pure config authoring, so an
AV1 session can assert it never receives the HEVC FREXT profile GUID (an INVALID_PARAM at
open) and that its own depth still lands, with an HEVC case guarding the good path.

⚠ `NV_ENC_CONFIG` must NOT be `mem::zeroed` in those tests: `frameFieldMode`/`mvPrecision` are
C enums whose discriminants start at 1, so all-zero is not a valid value and Rust's zero-init
check ABORTS the process (SIGABRT, caught by the Linux gate). They seed it from `Default` the
same way `build_config` does before overwriting from the driver preset. Worth knowing before
writing any further test against these SDK types.

Verified: Linux gate L1-L4 green (39 tests, the two new ones among them) and the full Windows
gate on .173 — 7 legs. Note the Windows test leg runs `--features qsv`, so these tests only
execute on the Linux leg; the Windows legs prove the change compiles in all five feature
combinations.
2026-07-25 16:52:40 +02:00
enricobuehler b3f8803ea3 fix(encode/qsv): stop advertising intra-refresh the driver silently dropped
`ir_active` was `cfg.intra_refresh && set.co2.is_some()` — i.e. "we asked for it", not "we got
it". Both `Query` and `Init` can return MFX_WRN_INCOMPATIBLE_VIDEO_PARAM, a WARNING the open
path deliberately accepts, while dropping the intra-refresh wave on the floor.

That lie is not cosmetic. `ir_active` feeds `EncoderCaps::intra_refresh`, so the session
advertises gradual refresh to the client; the client then stops asking for the IDRs it would
otherwise request on packet loss, and a lost frame gets concealed instead of repaired. The
stream degrades exactly when recovery matters.

Now confirmed against the driver: a best-effort `GetVideoParam` with a CodingOption2 buffer
chained on, read back for `IntRefType`. Deliberately a SEPARATE query rather than a buffer
attached to the existing `BufferSizeInKB` call (which is what the audit finding suggested) —
that value backs every bitstream allocation, and a runtime that disliked the chained buffer
would take it down with the readback. A failure here costs only the verdict and resolves
conservatively (trust the request), so the worst case is the previous behaviour.

Verified on Intel UHD 750 (.42), which is where the Intel on-glass run originally caught this:
with PUNKTFUNK_INTRA_REFRESH=1, H.264 now logs "silently dropped intra-refresh
(GetVideoParam reports IntRefType=0)" and advertises it OFF, while H.265 on the same GPU
stays ON — it is genuinely active there. Per-codec, so it could never have been decided
statically. 8 live QSV tests pass.

Gated on .173: clippy -D warnings at nvenc,amf-qsv,qsv (host + pf-encode --all-targets),
amf-qsv without qsv, qsv alone, no-features, cargo test --features qsv, rustfmt — all green.
qsv.rs is Windows-only so the Linux legs do not compile it.

Closes WP3.2(c), the last Phase 3 item that was hardware-blocked.
2026-07-25 16:52:40 +02:00
enricobuehlerandClaude Fable 5 47a23bec12 fix(encode/vulkan): unwind every open/import leak, and serve 24-bpp CPU instead of dying on it
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
arch / build-publish (push) Successful in 12m42s
windows-host / package (push) Successful in 18m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m6s
docker / deploy-docs (push) Successful in 32s
docker / build-push-arm64cross (push) Successful in 6m26s
apple / swift (push) Successful in 6m42s
deb / build-publish-host (push) Failing after 5m39s
android / android (push) Failing after 6m36s
deb / build-publish-client-arm64 (push) Failing after 5m35s
deb / build-publish (push) Successful in 9m40s
ci / rust (push) Successful in 22m26s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 55s
ci / bench (push) Successful in 5m54s
ci / rust-arm64 (push) Successful in 11m18s
apple / screenshots (push) Successful in 27m18s
Phase 5's Linux half (audit WP5.1 + WP5.4), each item shaped by the
review that rejected the obvious fix:

Dmabuf import unwind (vk_util): every failure after create_image leaked
the VkImage, and the dup'd dmabuf fd leaked as a raw i32. The sharp edge
is that a SUCCESSFUL vkAllocateMemory transfers fd ownership to Vulkan
(vkFreeMemory closes it), so the naive close-on-error is a double close
that clobbers whatever unrelated descriptor recycled the number. The dup
now lives in an OwnedFd released exactly in the allocate-success arm;
every other path drops it once, and bind/view failures free image+memory.

PyroWave open unwind: open_inner had ~20 fallible steps that each leaked
everything before them (instance, device, pyrowave objects, the whole
CSC pipeline). Rather than a parallel teardown guard — whose reviewed
hazards were a null-unsafe pyrowave_encoder_destroy and a drifting
duplicate of Drop — Self is now constructed right after create_device
with every later resource null, and the existing Drop (wait-idle first,
pw_enc null-guarded, delete-nullptr and VK_NULL_HANDLE destroys are
no-ops) is the single unwind path for error and normal teardown alike.
The ensure_cpu_rgb staging twins (create/allocate/bind, both backends)
and the RGB-direct make_view pair get the same discipline via a shared
make_host_buffer. Observed on hardware: 32 forced import failures, zero
fd drift (the new import_failure_leaks_no_fds smoke on RADV).

24-bpp CPU service (WP5.4): pixel_to_vk had no mapping for the packed
Rgb/Bgr the PipeWire portal negotiates, so a session committed to a path
the backend could not serve and died at its first frame. The filed
open-gate was rejected as a half-mirror — the dmabuf axis is keyed by
fourcc at submit, unknowable at open — so instead the CPU axis is
SERVED: a 3-to-4 expand at the staging upload (normalize_cpu_rgb, the
CPU twin of WP1.4's swscale expand), order-preserving for the CSC
samplers and BGRA-forced for the RGB-direct encode source, whose session
pictureFormat is B8G8R8A8 — the on-glass run caught R-first sources
violating VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207, a mismatch that
predates this change for plain Rgbx CPU sources. The dmabuf axis feeds
pf-zerocopy's raw-dmabuf degrade latch (3efbe416) from both Vulkan
import caches — deterministic refusals flip capture to CPU delivery,
which now serves every format capture produces; transient OOM and
native-NV12 sessions are excluded.

RECORDING-state hygiene: the fallible recording prefix (cursor prep,
import, staging) now resets the command buffer on error instead of
leaving it RECORDING for the next begin to trip
VUID-vkBeginCommandBuffer-commandBuffer-00049; PyroWave's submit-level
blanket reset is replaced by the same scoped shape, closing its
fence-timeout hole where a reset hit a PENDING buffer
(VUID-vkResetCommandBuffer-commandBuffer-00045).

Verified: docker linux/amd64 4-leg gate green; RADV 780M on-glass under
the validation layers at 1920x1080 — all 10 smokes pass (including the
new 24-bpp CSC + RGB-direct legs and both channel orders), decoded
colour truth checked with ffmpeg, and the only remaining validation
messages are the two documented VALVE-extension layer gaps plus a
pre-existing pyrowave shader/feature mismatch now filed in the planning
doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 16:51:27 +02:00
enricobuehlerandClaude Fable 5 e0e845b7df fix(encode/pyrowave): pin the NT-handle import contract to consume-on-success-only
import_plane closed the shared NT handle on every pyrowave_image_create
failure, believing pyrowave only consumes it on success. The vendored
truth was messier: Granite's allocator closed by-reference handles
UNCONDITIONALLY after the first vkAllocateMemory — success AND failure —
while failures before the allocator (validation, vkCreateImage, no
memory type) left the handle open, and both classes surface as the same
error code. So the call site could not know whether to close, and an
allocate-stage failure double-closed a possibly-recycled handle value
(audit WP5.3). The filed fix (DuplicateHandle + close the original
unconditionally) traded the double close for a guaranteed leak on every
pre-allocate failure and left the ambiguity in place.

Patch 0006 fixes the callee instead: the allocator never closes a
caller's handle, and pyrowave_image_create consumes it exactly at its
success return — which is what pyrowave.h ("take ownership and close the
HANDLE on import") documents, and what Granite's semaphore import
already did, so import_fence was correct as written and the two paths
now share one contract. The close-on-failure in import_plane is thereby
correct on EVERY path, including a vkBindImageMemory failure after a
successful allocate. Bonus fix recorded in the patch: the allocator's
block-recycling retry loop used to re-run vkAllocateMemory with
import_info still pointing at the just-closed handle.

Both-platform gates green; pyrowave_win_smoke (.173, 34/34) re-validates
the live import path against the rebuilt vendored C++.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 16:51:00 +02:00
enricobuehlerandClaude Fable 5 d6d4408c8e fix(encode/nvenc): teardown that tells wedged from routine, and a session budget that can heal
Three Windows NVENC teardown/accounting defects (audit WP5.2, marked
BLOCKING — the obvious fixes were reviewed and rejected before this):

Event-handle leak: init_session pushed each completion event to
self.events only AFTER nvEncRegisterAsyncEvent, so a failed registration
leaked the Win32 handle. Push first — the init-failure path already runs
teardown, which closes everything in the list (unregistering the one
never-registered event is a harmless error return).

Wedged drain: teardown drains the retrieve thread's backlog, and each
queued job could burn its full 5 s completion wait — cap x 5 s on the
encode thread, paid again on every rebuild of the stall-recovery ladder.
The rejected fix (a shutdown flag) abandoned the backlog on EVERY
teardown, turning the routine drain into destroy-while-encoding. Instead
the wedge evidence stays where it lives: after one full-budget timeout in
the retrieve loop, later jobs wait a 250 ms slice; a success resets the
latch. Routine teardown is byte-identical to before — nothing is ever
abandoned — and a first timeout is already encoder-fatal, so short
slices behind an established wedge change only how long teardown blocks.

Session-unit accounting: LIVE_SESSION_UNITS was refunded even when
destroy_encoder FAILED, drifting the budget low and over-admitting
parallel displays. The rejected fix (never refund on failure) let one
transient wedge episode poison admission until a host restart. Now the
refund follows PROOF: destroy succeeded, or its status says the driver
holds no session (device gone/TDR). Ambiguous failures park the handle —
units still charged, fail-closed, pinning the D3D11 device the driver
session references — and init_session retries the destroy while zero
sessions are live, under a gate that serializes every session open
against the reap so a recycled handle address can never alias a live
session. The charge lasts exactly as long as the driver keeps refusing
the destroy, which is the definition of still-leaked. Admission stays a
lock-free atomic load.

Both-platform gates green (docker linux/amd64 4 legs; .173 all 7 legs).
The retry-destroy-on-a-failed-handle assumption is documented at the
reap: the SDK defines no retry semantics either way.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 16:50:46 +02:00
enricobuehler 3efbe4164e fix(capture): capture CPU frames once the encoder proves it can't import dmabufs
audit / cargo-audit (push) Successful in 2m13s
audit / bun-audit (push) Failing after 13s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m3s
windows-host / package (push) Successful in 10m18s
ci / bench (push) Successful in 6m41s
android / android (push) Successful in 12m26s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
arch / build-publish (push) Successful in 12m30s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m10s
ci / rust-arm64 (push) Successful in 11m46s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m17s
deb / build-publish (push) Successful in 11m17s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m13s
deb / build-publish-host (push) Successful in 12m35s
deb / build-publish-client-arm64 (push) Successful in 8m23s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m5s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7m40s
apple / swift (push) Successful in 5m28s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8m6s
docker / deploy-docs (push) Successful in 14s
ci / rust (push) Successful in 21m44s
docker / build-push-arm64cross (push) Successful in 7m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m14s
flatpak / build-publish (push) Successful in 6m38s
release / apple (push) Successful in 29m56s
apple / screenshots (push) Successful in 25m36s
A dmabuf import the GPU driver refuses is refused identically on every
retry, but the only recovery above it was the encode-stall ladder: five
in-place encoder rebuilds, then the video session ends. So a host whose
driver will not take what its compositor allocates lost every session
on its first frame, and every reconnect repeated it — while the very
same host streamed fine with `PUNKTFUNK_ZEROCOPY=0`. The software knew
how to run that machine and never chose to.

Latch it, exactly as the sibling CUDA-import path already does after
repeated worker deaths: three consecutive import failures with no frame
in between disable the raw-dmabuf passthrough for the host process, and
capture negotiates CPU frames from the next session on. Three sits
below the encoder's rebuild budget, so the latch is set before the
session it doomed ends — one bad session, then a working (if slower)
host, with a log line saying which and why instead of an operator
having to find an environment variable.

Only the two stages that ARE the import are counted — the buffersrc
push of the DRM-PRIME descriptor and the buffersink pull where `hwmap`
maps it into a VA surface. `avcodec_send_frame` is deliberately left
out: that one is the encoder stalling, which the in-place rebuild
exists to recover, and taking zero-copy away permanently over a
transient fault would be a bad trade.

The latch lives in pf-zerocopy because it is the leaf both sides can
see — the capture→encode edge is one-way by design, so pf-capture
cannot ask pf-encode anything.
2026-07-25 15:13:11 +02:00
enricobuehler f32207a6ba fix(encode/vaapi): tell libva the dmabuf's real size, not zero
The zero-copy DRM-PRIME descriptor declared `objects[0].size = 0`, on
the belief that ffmpeg would work the real size out. It does not: both
of its import paths pass the value straight through to libva —
`prime_desc.objects[i].size` on the PRIME_2 path, `buffer_desc.data_size`
on the legacy fallback it tries next — so every VA driver we have ever
handed a dmabuf to was told the backing object was empty and left to
derive the size itself.

The drivers this path has run on (radeonsi, modern Intel iHD) derive it
correctly, which is why nobody noticed. A Gen9 Intel host does not get
that far: `vaCreateSurfaces` answers VA_STATUS_ERROR_ALLOCATION_FAILED
on the first frame and every frame after it, `av_buffersink_get_frame`
returns EIO, and five in-place encoder rebuilds later the video session
is over. That host cannot stream at all with zero-copy on.

`lseek(SEEK_END)` is the standard dma-buf size query and the same one
this tree's Vulkan bridge already performs on these very fds. A kernel
that refuses it leaves the old 0 rather than costing a frame that might
still have encoded.

Whether this alone fixes that host is unconfirmed — the descriptor was
wrong either way, and it is the first thing the driver reads.
2026-07-25 15:13:11 +02:00
enricobuehler 7c82a72ecd fix(zerocopy): find the NVIDIA render node instead of assuming renderD128
The EGL importer — the head of the CUDA/NVENC zero-copy path — opened
`/dev/dri/renderD128` and hoped. On a single-GPU host that is the
NVIDIA node and it always worked. On a hybrid laptop the iGPU is bound
first, so renderD128 is Intel: we built a GBM device on Mesa, asked it
for a pbuffer-capable OpenGL config, got none (Mesa's GBM platform
advertises only window-capable ones — gbm_surface is the point of that
platform there), and reported

    zerocopy worker init failed: no EGL config for OpenGL

on a machine whose NVIDIA EGL stack was demonstrably fine, naming
neither the device nor the reason. `PUNKTFUNK_RENDER_NODE` was no
escape either: this path never read it.

Pick the node by what it is — the first `/dev/dri/renderD*` whose
sysfs PCI vendor is 0x10de, the same identification the crate's Vulkan
bridge already uses for its physical device. A local scan rather than a
`pf_gpu` dependency, because this crate is a leaf whose worker is its
own process, and `pf_gpu::linux_render_node` answers a different
question anyway: it follows the operator's GPU *preference*, which may
legitimately name the iGPU. What is needed here is not which GPU to use
but where CUDA is. `PUNKTFUNK_ZEROCOPY_RENDER_NODE` overrides, and no
NVIDIA node found keeps the historical guess — sysfs may just not be
mounted, and the CUDA context is what properly fails on a host with no
NVIDIA at all.

Then stop the config query being able to fail this way: retry without
the surface-type constraint, which is honest because we never create an
EGLSurface — `eglMakeCurrent` runs surfaceless. And name the node in
every error and in the ready line, so the next hybrid host diagnoses
itself.
2026-07-25 15:13:11 +02:00
enricobuehler 6571a26aa9 fix(scripting): park the idle runner instead of pinning a core
`punktfunk-scripting` with nothing to run pinned a full CPU core
indefinitely — in exactly the state its own systemd unit documents as
inert ("the runner does nothing until you add scripts or install
plugins"). A field report on a 7.7 GB laptop had it at 99.9% CPU for
two hours straight, `strace` showing a bare `clock_gettime` loop and no
other syscall at all, which starved the compositor badly enough to
blank the physical display.

Nothing in the runner was looping. `Effect.runFork` registers nothing
with the event loop — effect's keep-alive timer lives in
`Runtime.makeRunMain`, which runner-cli deliberately doesn't use
because its shutdown is the bespoke two-signal one — and a bun process
whose only pending work is an unresolved promise busy-spins rather than
blocking. With units running there is normally a socket or timer to
park on; with none there is nothing at all, so the no-op state was the
one state that span.

Hold one idle handle for the runner's lifetime and drop it when the
unit tree ends on its own. Measured on the packaged bun (1.3.14), three
seconds idle with empty scripts/plugins dirs: 3098 ms of CPU before,
280 ms after (bun's startup, then nothing).

The regression is a property of the process rather than of any function
in it, so the test spawns the real CLI and reads the child's CPU time,
asserting it also logged and stayed alive — a crashed runner burns no
CPU either and must not pass.
2026-07-25 15:13:11 +02:00
enricobuehler 8578141d43 fix(encode/vulkan): AV1 at unaligned modes was violating two VUIDs on every frame
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
android / android (push) Successful in 15m38s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
arch / build-publish (push) Successful in 16m13s
windows-host / package (push) Successful in 10m20s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m20s
apple / swift (push) Successful in 5m36s
docker / deploy-docs (push) Successful in 23s
docker / build-push-arm64cross (push) Successful in 8m41s
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 1m8s
ci / bench (push) Successful in 6m49s
deb / build-publish-client-arm64 (push) Successful in 7m31s
deb / build-publish (push) Successful in 11m40s
ci / rust-arm64 (push) Successful in 12m11s
deb / build-publish-host (push) Successful in 12m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m59s
ci / rust (push) Successful in 21m45s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m51s
apple / screenshots (push) Successful in 25m22s
Found by running the smokes at 1920x1080 under the validation layers while chasing the three
items left over from WP4.2. AV1 forbids the encode source's `codedExtent` differing from the
sequence header without `FRAME_SIZE_OVERRIDE`
(VUID-vkCmdEncodeVideoKHR-flags-10324), or from the reference slots without
`MOTION_VECTOR_SCALING` (`-10325`). RADV PHOENIX advertises NEITHER — and RGB-direct is the
default on EFC hosts with true-extent the default at unaligned modes, so plain 1080p AV1 was
tripping both on every frame: source 1920x1080 against an app-aligned 1920x1088 header and
DPB. Measured 16 violations per 8-frame run; the CSC path had none.

The fix is to make all three agree at the RENDER size rather than the aligned one — an
unpadded coded size is valid on this hardware, so the coded frame simply IS the visible
frame:
  - the AV1 sequence header follows the render size when true-extent is active (joining
    native NV12, which already authors true-size headers for the same reason);
  - the DPB setup and reference slots carry `src_extent` instead of the aligned `ext2d`.
    `src_extent` already collapses to `ext2d` whenever true-extent is off, so every other
    configuration is untouched;
  - `render_and_frame_size_different` now compares render against the DECLARED source extent
    instead of the aligned size, or true-extent would have claimed a mismatch that no longer
    exists.

Fixing only the header is not enough and is actively misleading: it clears -10324 and
immediately exposes -10325, because the mismatch has moved to the reference slots rather than
gone. Both had to move together.

This keeps the EFC fast path. Two alternatives were implemented, measured and rejected on the
way here: falling back to the compute CSC costs the zero-copy the B2 work existed to deliver,
and routing to the padded-copy staging trades these two VUIDs for
VUID-VkImageCreateInfo-pNext-06811 — `pad_img`'s extra TRANSFER_SRC usage is not in the
profile-advertised set, measured 8x per session on HEVC-padded too, so that is a pre-existing
defect of the padded path and not somewhere to route a default session.

HEVC is deliberately untouched: it has no equivalent constraint (its crop rides the
conformance window), it measures zero violations, and its aligned-SPS path is the validated
one.

On RADV PHOENIX (780M, Mesa 26.0.4) the AV1 stream now decodes as coded 1920x1080 / render
1920x1080 — genuinely unpadded, where before the alignment rows were encoded and cropped
back out. The CSC path still reports coded 1920x1088 / render 1920x1080, which is correct for
it. All four `vulkan_smoke*` pass at 256x256 and 1920x1080.

Two validation errors remain on the RGB-direct path and are NOT ours, now with evidence
rather than assumption: `VUID-VkImageViewCreateInfo-image-08336` uses the PROFILE-BLIND format
query, so it cannot see that RGB conversion legalises BGRA as an encode source — the
profile-aware query used by -06811 accepts the very same image; and
`VUID-VkQueryPoolCreateInfo-pNext-pNext` rejects
`VkVideoEncodeProfileRgbConversionInfoVALVE`, which the VALVE extension REQUIRES for profile
identity, and the layer diagnoses itself as "a struct from an extension added to a later
version of the Vulkan header".

Verified: canonical Linux gate (docker linux/amd64) L1-L4 green; on-glass on RADV PHOENIX
under `VK_LOADER_LAYERS_ENABLE='*validation*'`, with the bitstreams read back through
libdav1d/trace_headers.
2026-07-25 15:12:08 +02:00
enricobuehler 87863c12a9 fix(encode/windows): clear the Windows Phase 4 backlog — NVENC sticky state, QSV HDR + ABR
arch / build-publish (push) Failing after 5m39s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m3s
android / android (push) Successful in 11m50s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
ci / bench (push) Successful in 6m25s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / rust-arm64 (push) Successful in 10m5s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m22s
deb / build-publish-client-arm64 (push) Successful in 9m34s
deb / build-publish (push) Successful in 10m58s
docker / deploy-docs (push) Successful in 12s
deb / build-publish-host (push) Successful in 11m51s
docker / build-push-arm64cross (push) Successful in 6m20s
apple / swift (push) Successful in 5m22s
ci / rust (push) Successful in 22m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m2s
windows-host / package (push) Successful in 10m48s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m13s
apple / screenshots (push) Canceled after 16m50s
Four items, gated on the RTX box (.173, all 7 Windows legs) and the Linux docker gate, with
the QSV HDR one proven by an A/B on real Intel silicon (.42, UHD 750).

WP4.1 — Windows NVENC sticky state. Both halves were the same mistake: a single field doing
duty as both "what the session negotiated" and "what this session can actually do", so the
first downgrade destroyed the negotiation.
  - `query_caps` clears `self.hdr` on a GPU without 10-bit encode, but `submit` re-derives the
    requested HDR state from every frame's pixel format and compared it against that cleared
    value. On such a GPU a P010 capturer therefore reported "HDR changed" on EVERY FRAME and
    tore down and rebuilt the whole encode session each time. The trigger now compares against
    `hdr_requested`, and the downgrade is latched in `hdr_unsupported` so it is remembered
    (and warned once) instead of rediscovered per init.
  - One subsampled-YUV frame cleared `chroma_444` permanently, so 4:4:4 never returned when
    the capturer went back to RGB. The negotiation now lives in `chroma_444_requested` and the
    effective value is recomputed at every init; the per-session downgrade still happens and
    is still reported honestly through `caps()`, it just is not destructive any more.

WP4.3(a) — QSV HDR mastering luminance was divided by 10,000. The old comment justified it as
"VPL wants whole cd/m²", which is true of the VIDEO-PROCESSING unit but not the encode path:
the runtime passes the value straight into the ITU-T H.265 Annex D mastering-display SEI,
whose unit is 0.0001 cd/m². Verified end to end on Intel UHD 750 by dumping the bitstream and
reading SEI 137 with trace_headers, before and after:
    before: max_display_mastering_luminance = 1000      (0.1 nits)
    after:  max_display_mastering_luminance = 10000000  (1000 nits)
with `min` (500 = 0.05 nits) and both content-light-level fields unchanged. That asymmetry —
a correct min beside a 10,000x-low max — was the original tell.

WP4.3(b) — `reconfigure_bitrate` never re-read `BufferSizeInKB`, so an ABR step-up could hand
the runtime an AU buffer sized for the old, lower bitrate. Fixed in both places it was broken:
re-read the driver's answer via `GetVideoParam` after the `Reset` (mirroring `init_encode`),
AND stop `take_bs` recycling pooled buffers without checking their capacity — the pool would
otherwise keep serving short buffers even once `bs_bytes` was correct.

WP7.8 — the `PUNKTFUNK_QSV_FFMPEG` half, finally landable now that a Windows gate exists (it
sits behind `#[cfg(feature = "qsv")]`, so nothing in a Linux-only matrix compiles it — it was
deliberately held back for exactly that reason). Trimmed like its siblings, and kept
case-insensitive: the knob already accepted `TRUE`, and the bare house `matches!` would have
silently dropped that spelling.

Phase 4 of the audit is now complete, and WP7.8 with it.

Verified: Windows gate on .173 — clippy -D warnings at nvenc,amf-qsv,qsv (host + pf-encode
--all-targets), amf-qsv without qsv, qsv alone, no-features, `cargo test --features qsv`
(34 passed), rustfmt. Linux docker gate L1-L4 green. On-glass: the QSV A/B above on .42.
Owed: NVENC sticky-state on-glass on .173 (needs a live 10-bit-capable capture toggle).
2026-07-25 14:37:49 +02:00
enricobuehler bab309017d fix(encode/vulkan): five mode-specific correctness bugs, three caught on real silicon
WP4.2 of the pf-encode audit, validated on RADV PHOENIX (780M, Mesa 26.0.4) under the Vulkan
validation layers rather than by inspection alone. Three of the five reproduce with the
driver naming the violation; one turned out milder than filed and is recorded as such.

1. `ensure_cpu_rgb` cached the staging image on FORMAT ONLY while sizing it to the SOURCE
   frame, and the copy uses the CURRENT frame's extent — so a same-format source-size
   increase copied past the allocation. Now keyed on (format, width, height). This is the
   memory-safety one: on hardware it was 8 x
   VUID-vkCmdCopyBufferToImage-imageSubresource-07971 ("extent.width (512) exceeds
   imageSubresource width extent (128)") and `submit` returned Ok every single time, so
   nothing upstream could notice. Zero after the fix. Note the trigger for anyone re-testing:
   the image is cached PER RING SLOT, so the ring must wrap before a slot sees a larger frame
   — two differently-sized submits in a row land on different slots and look fine.

2. The RGB-direct CPU padding loop only ever grows a source into the aligned extent; a source
   LARGER than the encode extent panicked on the row `copy_from_slice`. Now refused by name,
   matching what the DMA-BUF arms already do, so a bad frame takes the encoder-rebuild path
   instead of unwinding out of the encode thread with an index message.

3. AV1 wrote `render_width/height_minus_1` but never set
   `render_and_frame_size_different`, and AV1 ignores the render size unless that flag says it
   differs — so every mode needing 64x16 alignment presented the padding. libdav1d on the
   encoded stream now reports `size 1920x1088 ... render 1920x1080`; before, the flag being
   unset meant render defaulted to the coded 1088, i.e. 8 rows of duplicated edge pixels
   shipped to the client.

4. `read_slot` read the PERF timestamp pool whenever the pool merely EXISTED, but the
   padded-RGB path's CPU-upload arm records its own command buffer and writes no timestamps —
   reading an unreset query with WAIT is undefined. Now gated on a per-slot `ts_written`,
   because "a pool exists" was being used as proof of "the pool was written". SEVERITY
   CORRECTION vs the audit: this is not a hang in practice. It needs PUNKTFUNK_PERF *and*
   RGB-direct *and* the non-default PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 *and* an unaligned mode
   *and* CPU capture, and RADV then fails the read rather than blocking. Another driver may
   block, hence the fix.

5. `reset()` re-arms `first_frame`, which was also what decided whether begin-video-coding
   declares the rate-control state. But a reset does not rebuild the session, so the CBR
   installed earlier is still current when the next frame opens its coding scope: the
   declaration was omitted exactly when it was required. Split into `rc_installed`, which
   survives reset. On hardware this was
   VUID-vkCmdBeginVideoCodingKHR-pBeginInfo-08253 ("no VkVideoEncodeRateControlInfoKHR ... but
   the currently set mode is CBR"); zero after.

Also fixes two pre-existing AV1 spec violations that were firing on EVERY frame of a shipped
path and are not in the audit at all — found only because the validation layers were on, and
confirmed untouched by this branch before being attributed as pre-existing:
`constantQIndex` must be 0 unless rate control is DISABLED (VUID-...-10320; this session is
always CBR, so the driver owns Q), and `pStdPictureInfo->pSegmentation` must be NULL
(VUID-...-10350; it pointed at a zeroed struct, and leaving it unset means the same
"segmentation disabled" while being spec-correct).

Items 1 and 5 keep their reproductions as `#[ignore]`d hardware tests, documented as
meaningful only under `VK_LOADER_LAYERS_ENABLE='*validation*'` since the violations are
invisible to the API's return values.

STILL OPEN, filed not fixed (three more pre-existing validation errors the cleanup exposed):
VUID-VkQueryPoolCreateInfo-pNext-pNext on the AV1 and RGB paths, and 2 x
VUID-VkImageViewCreateInfo-image-08336 (the RGB-direct encode-src view's format features lack
VIDEO_ENCODE_INPUT) — the latter may be a validation-layer gap around the VALVE extension
rather than our bug, and wants real analysis rather than a quick patch.

Verified: canonical Linux gate (docker linux/amd64) fmt + clippy --all-targets at default and
at nvenc,vulkan-encode,pyrowave + both test legs; and on RADV PHOENIX all four `vulkan_smoke*`
tests pass at 256x256 and at 1920x1080 with zero validation errors on the paths touched here.
2026-07-25 14:37:49 +02:00
enricobuehlerandClaude Opus 5 e5e68f1d24 feat(presenter): DRM card selection + a usable kmsdrm swapchain error
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m15s
apple / swift (push) Successful in 5m25s
ci / bench (push) Successful in 7m44s
deb / build-publish-host (push) Successful in 10m30s
decky / build-publish (push) Successful in 30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
android / android (push) Successful in 12m52s
deb / build-publish (push) Successful in 11m46s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
arch / build-publish (push) Successful in 13m6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 45s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
ci / rust-arm64 (push) Successful in 13m32s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m1s
deb / build-publish-client-arm64 (push) Successful in 9m12s
windows-host / package (push) Successful in 18m4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7m1s
flatpak / build-publish (push) Successful in 7m25s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m46s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 9m7s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 9m24s
ci / rust (push) Successful in 23m48s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m28s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m46s
docker / deploy-docs (push) Successful in 22s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m42s
release / apple (push) Successful in 31m30s
docker / build-push-arm64cross (push) Successful in 7m6s
apple / screenshots (push) Successful in 23m4s
Both come straight out of the first compositor-less bring-up (P3), on a
two-GPU box:

PUNKTFUNK_DRM_CARD=<n> pins SDL's KMSDRM device index. SDL enumerates
/dev/dri/card* and takes the first it can open, which is regularly the
wrong one: it chose the card a live compositor already held DRM master on
and died at swapchain creation, while the idle card with the connected
display sat unused. Kept an explicit operator choice rather than
auto-detection — deciding "is this card already mastered" needs the very
ioctl that taking master IS, so any in-process guess would be fragile.

The swapchain error now carries what to actually check. "vkCreateSwapchainKHR:
Initialization of an object has failed" is useless to someone bringing up a
kiosk; on the kmsdrm backend it now names the pinned card and lists the
three real causes in order (no connected connector / another DRM master /
NVIDIA). Empty on every other backend, where it would be noise.

The NVIDIA note is measured, not guessed: on NVIDIA proprietary + kmsdrm,
Vulkan enumerates the GPU AND the display (VK_KHR_display reports the
connected HDMI connector) and still fails — as root, with
nvidia_drm.modeset=Y, on a card no compositor was using. Not permissions,
not DRM master; their direct-display path wants the display leased via
vkAcquireDrmDisplayEXT and SDL's kmsdrm surface path does not do that.

Plan: punktfunk-planning design/embedded-arm64-client.md §P3

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehlerandClaude Opus 5 aff169e131 build(flatpak): make the manifest architecture-generic
Two things were x86-specific, both now expressed properly instead of
hardcoded:

  * PKG_CONFIG_PATH carries the runtime's multiarch directory.
    flatpak-builder does NOT shell-expand `env` values, so ${FLATPAK_ARCH}
    would be taken literally — a build-options.arch override supplies the
    aarch64 string and inherits the rest.

  * The prebuilt Skia archive is per-target and pinned by sha256. Two
    `type: file` sources discriminated by only-arches now share one
    dest-filename, so SKIA_BINARIES_URL stays a single literal path.
    Upstream publishes the aarch64 archive under the same skia commit hash
    and the same resolved-feature key, so the two stay in lockstep on a
    skia-safe bump.

build-flatpak.sh gains ARCH (default: this machine's), passes --arch to
both flatpak-builder and build-bundle, and arch-suffixes the bundle name so
an x86_64 and an aarch64 build can coexist in dist/ instead of the second
silently overwriting the first. CI composes its own published filename, so
nothing downstream changes.

The aarch64 Skia sha256 was verified by downloading the published archive,
not taken from the API. No aarch64 flatpak has been built — flatpak-builder
builds in a sandbox for the target arch, which needs an arm64 machine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehlerandClaude Opus 5 4dc078692d build(arch): build the client for aarch64 (Arch Linux ARM)
arch=('x86_64' 'aarch64'), and on aarch64 `pkgname` drops punktfunk-host
so makepkg never enters the host's build or package path; build() takes a
client-only cargo invocation without the NVENC/Vulkan-encode features.
Dropping the host from pkgname rather than giving package_punktfunk-host a
per-package arch is what makes the skip unambiguous, and it mirrors how
PF_WITH_WEB already extends that array.

The host stays x86-only because its encode stack (NVENC/QSV/AMF) is.

Verified by sourcing the PKGBUILD under both CARCH values: x86_64 yields
`punktfunk-host punktfunk-client`, aarch64 yields `punktfunk-client`. NOT
verified by a real build — there is no official arm64 Arch container, and
the README says so rather than implying it is tested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehlerandClaude Opus 5 1c76b8c7b4 build(rpm): let the spec build a client-only aarch64 RPM
The blocker was never ExclusiveArch — it was that %build builds host +
tray + client together and %install lays down the host unconditionally,
with no way to express "client only". `%bcond_without host` (default ON,
so an x86_64 build is unchanged) now gates the host binary, the tray, the
headless-session data, the firewalld services, the main %post and the bare
%files section. Omitting %files for the MAIN package is the load-bearing
part: it is what stops rpm emitting an empty `punktfunk` alongside
punktfunk-client.

build-rpm.sh exposes it as PF_WITHOUT_HOST=1 and skips the libcuda stub
regeneration and the libcuda leak check, neither of which means anything
without the host.

Verified two ways. `rpmspec -P` shows the default expansion still carrying
3 host install lines, the tray build and one bare %files, while
--without host carries none of those and one %files client. And a real
rpmbuild in a native aarch64 Fedora 43 container produced
punktfunk-client-*.aarch64.rpm with correct aarch64 sonames
(libc.so.6(GLIBC_2.17), libSDL3.so.0, libavcodec.so.61) and no main
package.

Not a cross-compile: %build runs cargo for the build machine's arch, so
this wants an arm64 builder. No CI leg yet — deliberate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehlerandClaude Opus 5 cac82d13df ci: check the client for aarch64 on every push
Adds a `rust-arm64` job: clippy -D warnings for the client crates at
--target aarch64-unknown-linux-gnu, plus a --no-default-features session
build so the minimal embedded configuration keeps compiling in its own
right rather than only as a subset of the default one. Runs in the cross
image on the ordinary amd64 runner; client crates are listed explicitly
because --workspace would drag in the x86-only host encode stack.

This is a regression guard, not an artifact leg (deb.yml ships those).
c_char signedness cuts both ways and neither direction is visible from an
x86-only CI: hardcoding i8 fails to COMPILE on ARM, while casting to u8
fails the LINT on ARM. Two such defects were already found this way.

Also documents the arm64 client .deb recipe in packaging/debian/README.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehlerandClaude Opus 5 ab3884c763 fix(core/abi): cast identity PEM pointers with .cast(), not as *mut u8
`c_char` is i8 on x86_64 and u8 on aarch64, so `cert_pem_out as *mut u8`
is a REQUIRED conversion on one target and a no-op on the other — where
clippy::unnecessary_cast then denies it. `.cast::<u8>()` is correct and
lint-clean on both.

Caught by the new aarch64 clippy leg on its first run. A sweep of the
remaining `as *mut u8` / `as *const u8` sites in the client crates found no
other c_char-derived casts; the rest convert from c_void or u16.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehlerandClaude Opus 5 000f8b85b0 build(ci): cross-compile and publish an aarch64 Linux client
Adds punktfunk-rust-ci-arm64cross (the rust-ci toolchain + an Ubuntu ports
arm64 multiarch sysroot) and a deb.yml job that cross-builds the client
package on the ordinary amd64 runner — no arm64 runner in the fleet and
none needed. Client only: the Linux host encodes with NVENC/QSV/AMF, all
x86. The apt registry keys pool entries by the .deb's own arch, so an arm64
box picks the package up with no client-side configuration.

Three things this had to get right, each of which cost a failed build:

  * rustup target add must run against the toolchain rust-toolchain.toml
    PINS, not the image's default `stable` — otherwise the pinned toolchain
    has no aarch64 std and the build dies ~230 crates in on "can't find
    crate for core". Copying the pin file in also pre-downloads the
    toolchain every CI job currently re-fetches on first use.

  * ffmpeg-sys-next compiles a probe it intends to RUN, so it forces
    .target(HOST) while still passing the TARGET's include paths; the arm64
    dir then shadows the host's own libc headers and the x86 compiler dies
    in bits/math-vector.h on NEON/SVE types. Prepending the host dir does
    NOT help — GCC drops a -I that duplicates a system directory, keeping
    its original later position — so ci/pf-host-cc strips target include
    dirs from host compiles instead.

  * the job hard-fails on a non-AArch64 session binary, rather than
    shipping an amd64 payload under an arm64 package name.

skia needs no special handling: the aarch64-linux-gnu textlayout+vulkan
prebuilt resolves, so arm64 ships the FULL client, OSD included — unlike
Windows ARM64, which still builds --no-default-features.

The cross image builds in its own docker.yml job (needs: build-push) since
it is FROM punktfunk-rust-ci:latest and must not race the entry that
publishes that base. rpm/Arch/Flatpak stay x86_64 for now — their builders
have no arm64 sysroot, which is its own piece of work.

Plan: punktfunk-planning design/embedded-arm64-client.md

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehlerandClaude Opus 5 915fc3ef9e feat(session): headless --pair, so a box with only SSH can enrol
`punktfunk-session --pair <PIN> --connect host[:port]` runs the SPAKE2
ceremony with no window and no toolkit, prints the same
`paired <addr>:<port> fp=<hex>` line as `punktfunk-client --pair`, and
exits. Until now the PIN ceremony lived only in the GTK shell or the Skia
console, so enrolling an embedded/kiosk client meant installing a desktop
on it — or copying the identity store by hand.

Dispatches above every graphics call (the machine may have no display) and
is in the `--no-default-features` build: enrolling must never be the reason
a minimal image pulls in Skia. `--name` defaults to the hostname instead of
the desktop path's hardcoded "Steam Deck".

`forget_placeholder` and `device_name` move into pf_client_core::trust so
the two binaries share one implementation rather than the session
re-deriving them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehlerandClaude Opus 5 b7cea71bbd fix(presenter): type the Vulkan extension pointers as c_char, not i8
`char` is signed on x86_64 but UNSIGNED on aarch64, so the hardcoded
`Vec<*const i8>` compiles on every desktop target and then fails to match
ash's `&[*const c_char]` when cross-compiling the client for ARM.

Found by the aarch64 cross-build spike (punktfunk-planning:
embedded-arm64-client.md) — the only portability defect in the client
stack; nothing else in the client crates assumes an architecture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:32:45 +02:00
enricobuehler 07f8e37c85 fix(encode/sw): refuse a mode openh264 cannot encode at open, not at every submit
ci / docs-site (push) Successful in 1m2s
ci / web (push) Successful in 1m10s
decky / build-publish (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
apple / swift (push) Successful in 4m12s
ci / bench (push) Successful in 7m0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m3s
android / android (push) Successful in 12m29s
docker / deploy-docs (push) Successful in 28s
deb / build-publish-host (push) Successful in 13m15s
deb / build-publish (push) Successful in 11m37s
arch / build-publish (push) Successful in 15m59s
windows-host / package (push) Successful in 18m2s
ci / rust (push) Successful in 22m20s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m44s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m53s
apple / screenshots (push) Successful in 24m23s
openh264 tops out at level 5.2 — 3840x2160 landscape or 2160x3840 portrait — and enforces
that ceiling inside `reinit`, which the crate calls on the FIRST ENCODE rather than at
encoder construction. So an oversized mode built a perfectly healthy-looking encoder and
then failed every single submit: the session connects, negotiates, and never delivers a
frame, with the real reason buried in a per-frame error rather than at the open.

`validate_dimensions` does not cover this. It is keyed on the codec, and H.264 legitimately
reaches 4096 on every hardware backend — this ceiling belongs to the software backend alone,
which is exactly the path a GPU-less host falls back to.

Rejects at open instead, mirroring the rule from the openh264 version we actually ship
(0.9.3) rather than from its docs — including the orientation-aware shape, since a naive
per-axis `w <= 3840 && h <= 2160` would wrongly refuse a legal 2160x3840 portrait session.

Three tests: the accepted modes in both orientations, the modes `validate_dimensions` lets
through but openh264 cannot serve, and that `open` itself refuses rather than deferring to
submit. They cost nothing to run — the guard fires before openh264 is initialised at all.

Verified with the canonical Linux gate (docker linux/amd64): fmt, clippy --all-targets at
default and at nvenc,vulkan-encode,pyrowave, and the pf-encode test leg (37 passed).
2026-07-25 13:01:01 +02:00
enricobuehler ed525c4c73 fix(encode/vulkan): PUNKTFUNK_VULKAN_RGB_DIRECT read "0 " as force-ON
The knob tested `v == "0"` for the disable case and treated everything else that was set as
a force-enable. So a trailing space — the kind a `.env` line or a shell script leaves behind
— made `PUNKTFUNK_VULKAN_RGB_DIRECT=0 ` mean the exact opposite of what the operator wrote,
forcing the RGB-direct encode source on. On a cursor-blend session that is not a subtle
difference: the EFC front-end cannot blend, so the pointer disappears from the stream. An
empty value and any typo did the same thing.

Now parsed like every sibling knob in the crate — `matches!(v.trim(), "1"|"true"|"yes"|"on")`
and the matching false spellings — so unrecognised input falls back to the default instead
of being read as a force-on. `=0` and `=1` keep working exactly as documented.

Splits the parse into a pure function so the accepted spellings are covered by tests. That
matters more than it looks: the env-var read itself is untestable in a parallel test binary
(the process environment is shared), which is precisely why this knob's behaviour had no
coverage and the inverted case survived.

Verified with the canonical Linux gate (docker linux/amd64): fmt, clippy --all-targets at
default and at nvenc,vulkan-encode,pyrowave, and the pf-encode test leg (37 passed).
2026-07-25 13:00:50 +02:00
enricobuehler 3fac1a6da1 fix(encode/linux): stop the capability probes racing each other's libav log level
Every capability probe drops libav's log level to AV_LOG_FATAL around an encoder open it
expects to fail, then restores what was there before. That level is one process-global int
and the probes genuinely overlap — they are reached both from `/serverinfo` and from session
bring-up. Two overlapping save/restore pairs interleave as get(INFO) → get(FATAL) →
set(INFO) → set(FATAL), and the process is then pinned at AV_LOG_FATAL permanently: every
later libav diagnostic silently dropped, including the ones you want when a stream fails to
open later on.

Replaces the four hand-rolled save/restore pairs with one RAII guard over a single shared
mutex. The audit filed two sites; there are four — the NVENC 4:4:4 and 10-bit probes in
`linux/mod.rs` and both VAAPI probes, which is why the lock has to be shared across the two
modules rather than kept local to either.

Being RAII also closes a smaller hole the old shape had: the restore was a statement after
the open, so any early return added to those functions later would have leaked the quiet
level. Now it cannot.

The guard is poison-tolerant — a probe that panicked mid-window has already restored the
level through `Drop`, so refusing the lock forever afterwards would be strictly worse than
proceeding. It is not re-entrant, which is safe here because no probe body reaches another
probe (they only call encoder-open helpers); that invariant is written down at the type.

Serialising the probes costs nothing measurable: they are process-once behind their caches
and each already pays for a real encoder open.

Verified with the canonical Linux gate (docker linux/amd64): fmt, clippy --all-targets at
default and at nvenc,vulkan-encode,pyrowave, and the pf-encode test leg (37 passed).
2026-07-25 13:00:39 +02:00
enricobuehler a17413327b fix(encode/vulkan): give the vendored Vulkan ABI the layout guard it never had
`vk_av1_encode.rs` and `vk_valve_rgb.rs` are hand-copied `#[repr(C)]` structs handed to the
driver through raw `p_next` chains. Nothing in the type system relates them to the C
definitions any more, so an edit that inserts, drops, widens or re-pads a field is not a
compile error — it is the driver reading our bytes at the wrong offsets, silently. The
sibling vendored ABI in `amf.rs` has carried assertions for exactly this reason; these two
had none at all.

Adds size, alignment and per-field offset assertions for all 19 vendored structs. They are
`const` rather than `#[cfg(test)]` (the shape `amf.rs` uses) so they hold in every build
including the shipped one, and on any target the modules compile for.

Assertions cannot catch a swap of two same-typed fields — the offsets are unchanged — so
the field order was diffed field-by-field against the authoritative headers while writing
them: `vulkan_core.h` and `vk_video/vulkan_video_codec_av1std_encode.h` from Vulkan-Headers
`main` as of 2026-07-25. That diff covered every struct, every `ST_*` structure-type value,
every flag bit and both enum groups, and found no drift — the vendored copies are faithful.

The one remaining hand-copied table the compiler still cannot see is the bitfield member
order inside the three `*Flags` words, where a wrong index means the driver reads
`use_superres` where we meant `render_and_frame_size_different`. Three tests pin those to
the header by listing the members in C declaration order and asserting the Nth setter
writes bit N, with the trailing `reserved` field checked too — a dropped member shifts
`reserved` down and fails.

Verified with the canonical Linux gate (docker linux/amd64): fmt, clippy `--all-targets`
default and `nvenc,vulkan-encode,pyrowave`, and both test legs.
2026-07-25 12:46:24 +02:00
enricobuehler 6c2183ceec fix(nvenc): stop telling operators to reboot when the driver version is fine
`NV_ENC_ERR_INVALID_VERSION` is how the driver reports two opposite failures, and we only
ever explained one of them. The message told the operator to update the NVIDIA driver or
reboot — correct for a genuine header/kernel-module skew, and actively misleading for the
field case behind it: a host that streams once per boot and then fails every later session
at the caps probe, until the *process* restarts. A version skew is static; it cannot come
and go inside one process, so that advice cost the reporter a reboot per stream.

Split the two on the only fact that tells them apart: whether a session has already opened
in this process. `nvenc_status` gains a `SESSION_OPENED` latch set right after every
successful `open_encode_session_ex` — both backends' caps probe and real open, plus the
Windows availability probe. No session yet, the version word really is in question and the
existing skew advice stands. A session already opened, and the kernel module demonstrably
accepted this build's version word, so the message now names per-process driver state and
points at the cheap fix (restart the host service, no reboot) plus a request for the log.

The load-time gate cannot serve as this discriminator: `NvEncodeAPIGetMaxSupportedVersion`
is a pure userspace query, so the classic "updated the driver, didn't reboot" skew sails
through it and only fails later at the open. Only a session that actually opened proves the
kernel module agreed.

The split lives in a pure `invalid_version(bool)` so both halves are unit-tested without
touching the process-wide latch. This is a diagnosis change only — it does not fix the
underlying field bug, whose root cause is still open.

Verified on Linux (192.168.1.25): clippy `-p pf-encode --features nvenc` clean, `cargo test
-p pf-encode --features nvenc --lib` 15 passed, rustfmt clean.
2026-07-25 12:40:51 +02:00
enricobuehlerandClaude Opus 5 ead37d066f perf(encode/nvenc-linux): drop the bring-up probe left on the per-frame blend path
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 58s
decky / build-publish (push) Successful in 17s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
apple / swift (push) Successful in 5m27s
ci / bench (push) Successful in 6m10s
deb / build-publish (push) Successful in 9m22s
docker / deploy-docs (push) Successful in 24s
windows-host / package (push) Successful in 10m0s
deb / build-publish-host (push) Successful in 13m54s
android / android (push) Successful in 16m12s
arch / build-publish (push) Successful in 16m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m44s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m6s
ci / rust (push) Successful in 21m41s
apple / screenshots (push) Successful in 23m17s
`33121ece` added a block explicitly labelled "TEMP KWin composite probe (DROP BEFORE
MERGE)" to prove the cursor blend was dispatching. It merged, and has been running on
every blended frame since: an atomic fetch_add per frame plus a `tracing::info!` with
seven computed fields every 512 frames, on the submit hot path of the default-on
Linux direct-NVENC backend.

The signal it existed for is already covered — the failure arm above it warns once
with the dispatch error, and the `else` arm warns when an overlay arrives with no
blend at all. What is deleted is only the success-path telemetry.

Verified `-D warnings --all-targets` + tests on Linux (default; shipped
nvenc,vulkan-encode,pyrowave), and `clippy --features vulkan-encode,pyrowave` on real
AMD RDNA3 hardware.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 03:48:59 +02:00
enricobuehlerandClaude Opus 5 310b85f155 fix(encode): a forced-Vulkan pref must not advertise codecs that arm will refuse
ci / docs-site (push) Successful in 54s
ci / web (push) Successful in 56s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 59s
apple / swift (push) Successful in 5m10s
ci / bench (push) Successful in 5m58s
windows-host / package (push) Successful in 10m3s
deb / build-publish (push) Successful in 12m10s
docker / deploy-docs (push) Successful in 27s
deb / build-publish-host (push) Successful in 12m52s
android / android (push) Successful in 15m8s
arch / build-publish (push) Successful in 15m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m27s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m51s
ci / rust (push) Successful in 22m34s
apple / screenshots (push) Successful in 24m6s
With `PUNKTFUNK_ENCODER=vulkan`, `open_video_backend`'s vulkan arm bails outright for
anything that is not HEVC/AV1 ("the Vulkan Video encoder supports HEVC + AV1; the
session negotiated {codec:?}"). But `host_wire_caps` for that same pref fell through
to the VAAPI probe / static superset, which includes H.264 — so the host advertised
H.264, a client could negotiate it, and the session died at encoder open.

The pref now contributes a CEILING that is intersected with the device probe, never a
replacement for it. That distinction is the whole fix: pinning a static HEVC|AV1
would have ADDED AV1 on the AMD/Intel hosts whose probe currently withholds it
(pre-RDNA3, pre-Arc), re-creating this very bug for a different codec. Intersecting
can only ever narrow.

Without the `vulkan-encode` feature the pref cannot open anything at all — that arm
bails with "requires a build with --features vulkan-encode" — so the ceiling is
empty there rather than optimistic.

Costs nothing on the handshake path: a `&str` match plus a const-folded `cfg!`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 03:17:45 +02:00
enricobuehlerandClaude Opus 5 087a3e358a fix(encode/vaapi): key the entrypoint cache on the device, not just the codec
`LP_MODE` cached the resolved VAAPI entrypoint (full-feature `EncSlice` vs low-power
`EncSliceLP`/VDEnc) in a process-global array indexed by codec alone. That is not a
staleness bug, it is a session-killer, because the cache is load-bearing: once a mode
is latched the open tries exactly ONE mode and the `[false, true]` fallback is gone.

Latch low-power on an Intel Arc (Gen12+ removed the full-feature entrypoints, so it
is the only one that works there), then switch the web-console GPU preference to an
AMD dGPU. `render_node()` follows that preference, so every VAAPI open now goes to
radeonsi passing `low_power=1`, which it rejects — with no full-feature retry, for
the process lifetime. `probe_can_encode` reports all-false (so the advertisement
silently falls back to the static superset) AND the session's own encoder open fails.

Now keyed on (render node, codec, bit depth):

  - The render node because the entrypoint is a property of the DEVICE libva opens,
    and it is literally what `render_node()` hands libva — so key and device cannot
    describe different GPUs. Deliberately not `pf_gpu::selection_key()`, which can
    name a different adapter than the node actually opened.
  - The bit depth because Main10 and 8-bit can resolve to different entrypoints on
    the same device; one shared slot let an 8-bit answer pin the 10-bit open, i.e.
    HDR under-advertisement.

Verified `-D warnings --all-targets` + tests on Linux (default; shipped
nvenc,vulkan-encode,pyrowave). The multi-GPU switch itself needs a box with two
VAAPI-capable adapters — owed on-glass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 03:17:44 +02:00
enricobuehlerandClaude Opus 5 7536f7319a fix(gamestream): a software-encode host must advertise H.264 only
ci / web (push) Successful in 1m0s
ci / docs-site (push) Successful in 1m7s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
ci / bench (push) Successful in 7m9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5m31s
deb / build-publish-host (push) Successful in 9m36s
docker / deploy-docs (push) Successful in 26s
deb / build-publish (push) Successful in 11m22s
arch / build-publish (push) Successful in 11m56s
android / android (push) Successful in 15m17s
windows-host / package (push) Successful in 17m7s
apple / swift (push) Failing after 17m50s
apple / screenshots (push) Skipped
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m41s
ci / rust (push) Successful in 27m13s
`base_codec_mode_support` fell through to the static superset (H.264|HEVC|AV1) on a
GPU-less host, so Moonlight negotiated HEVC or AV1 and the session then died at
encoder open with "the software encoder emits H.264 only" — openh264 encodes nothing
else. The native plane's twin of this function, `pf_encode::Codec::host_wire_caps`,
has gated on exactly this since it was written; this one never did.

Deliberately a local gate rather than delegating wholesale to `host_wire_caps()`.
Delegation is the drift-proof shape and was the first design, but on Windows it
re-runs the DXGI adapter enumeration several times per `/serverinfo` GET — the probe
helpers each sample it uncached — and this endpoint is polled by every client. The
software case is a plain config read, so it costs nothing here.

Recorded in a comment rather than fixed: the static `MaxLumaPixelsHEVC` in the
serverinfo XML still advertises an HEVC limit even when the mask now drops HEVC.
Harmless (Moonlight gates capability on the mask, not the limit) but it is a second,
now-inconsistent advertisement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 02:41:57 +02:00
enricobuehlerandClaude Opus 5 4d9f94e0a4 docs(encode): record why the production loops must not call Encoder::flush
`flush` looked dead — the only caller in the host is the `spike` dev subcommand — so
an audit sweep filed it as "wire it in or delete it". Both are wrong, and without
this note the next sweep will re-file it.

Wiring it in is refuted by control flow: both encode loops reach their exit only
AFTER the transport is gone (client disconnected, or the session stopped), so the AUs
a flush would recover have nowhere to go. And flush is the one call on this trait
that can BLOCK on a wedged encoder, on exactly the teardown path a stopped session
needs to finish promptly — the Linux direct-SDK NVENC retrieve-thread join is untimed,
so flushing there could hang a session that is already ending.

Deleting it is refuted by real consumers: `spike` encodes a FINITE clip and wants the
tail, and the `#[ignore]`d hardware smoke tests across the backends assert the drain
contract on real GPUs. Those are finite-stream users; a live session is not one.

Doc only, no behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 02:41:57 +02:00
enricobuehlerandClaude Opus 5 ef239691df feat(encode): make cursor blending a queryable capability, not an assumption
`open_video`'s `cursor_blend` argument was a request with no answer: lib.rs did
`let _ = cursor_blend;` and only three backends ever read `CapturedFrame::cursor`.
So a session could ask for a composited pointer, get a backend that silently
discards it, and stream with no mouse cursor and nothing in the logs. Two
separately-confirmed audit findings — the VAAPI dmabuf path and the libav-NVENC CUDA
path — are symptoms of that one hole.

`EncoderCaps::blends_cursor` makes it a fact each backend states. The four exhaustive
`EncoderCaps { .. }` constructors mean adding the field is a compile error until every
backend answers, which is the enforcement mechanism for future backends rather than a
side effect. Vulkan Video answers from its ACTUAL configured source rather than
statically: only the CSC path composites (`prep_cursor` feeds the compute shader),
while the RGB-direct/EFC front-end and the native-NV12 source have no compositing
stage at all and merely warn once that the pointer is being dropped.

`open_video` warns when a session asked for blending and the opened backend cannot
deliver it. A warning is deliberately all it does: `open_video` cannot re-plan
capture, so refusing would trade a missing pointer for a dead session. The host owns
`plan.cursor_blend` and is the only layer that can fall back to capturer-side
compositing — this gives it something to base that on.

Enforcement is NOT included. The reviewed design proposed refusing the client's
host-composite flip to keep the client drawing its own pointer, but `CursorRenderMode`
is client->host only: there is no host->client counterpart, so refusing yields no
pointer at all — the same failure it claimed to prevent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 02:41:57 +02:00
enricobuehlerandClaude Opus 5 ab63e0dad3 fix(encode/nvenc-windows): fail safe when nobody sets the input ring depth
`set_input_ring_depth` is a DEFAULTED trait method, so a caller that forgets it
fails silently — and one did. The GameStream loop opens an encoder
(gamestream/stream.rs:671 and the rebuild at :831) and never called it, while the
Windows IDD-push capturer declares a ring of 2 and `async_inflight_cap()` defaults
to 4. With PUNKTFUNK_NVENC_ASYNC=1 a Moonlight session therefore pipelined four
encodes against a two-texture ring, letting the capturer rotate a texture out from
under a live encode: torn or mixed frames, never an error — precisely the corruption
the cap exists to prevent, and precisely what the trait doc warns "fails silently and
intermittently".

Guarded at the single point of CONSUMPTION rather than by plumbing the setter into
every loop: `input_ring_depth` has exactly one reader in the workspace, so one
fail-safe covers every caller including ones not yet written, whereas fixing N call
sites only fixes the N someone remembered. An unconfigured ring is now treated as
the shallowest any capturer here declares, so the unconfigured path degrades to less
pipelining — a latency cost, not corruption.

The two GameStream sites also pass the REAL depth, because the fail-safe is a floor,
not a substitute: `idd_depth` is configurable and a deeper ring is free pipelining
the fallback would forfeit.

Default (sync) sessions are unaffected — the cap is only read while the async
retrieve thread exists, and that is opt-in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 02:41:57 +02:00
enricobuehlerandClaude Fable 5 78fe77b049 feat(host/encode): de-escalate the latency escalation once cadence holds clean
ci / web (push) Successful in 46s
ci / docs-site (push) Successful in 1m7s
decky / build-publish (push) Successful in 32s
ci / bench (push) Successful in 6m12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m44s
deb / build-publish (push) Successful in 12m39s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
deb / build-publish-host (push) Successful in 13m29s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6m35s
android / android (push) Successful in 15m57s
arch / build-publish (push) Successful in 16m18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9m21s
docker / deploy-docs (push) Successful in 23s
windows-host / package (push) Successful in 17m51s
apple / swift (push) Successful in 5m59s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m32s
ci / rust (push) Successful in 21m53s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m20s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m40s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m45s
flatpak / build-publish (push) Successful in 6m24s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m2s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m35s
release / apple (push) Successful in 30m44s
apple / screenshots (push) Successful in 24m33s
Escalate-and-hold's missing half. The contention escalation (capture depth,
then the NVENC pipelined retrieve) was permanent: one sustained overrun — even
one CAUSED by the ABR overdrive's rebuild storms — cost the session its
depth-1 latency and its sub-frame streaming forever, and `encode_us` reported
queue depth instead of ASIC time for the rest of the session.

- The leaky bucket now keeps scoring after escalation. A sustained
  every-frame-on-cadence run (~5 s at 120 fps) winds back one stage in reverse
  order: pipelined retrieve first (its rebuild restores the IO-stream binding
  and sub-frame chunked streaming), then capture depth back to 1. Attempts are
  paced by an exponential backoff (1 → 5 → 25 min, capped) — a workload that
  truly needs the escalation converges to keeping it, but never a permanent
  latch.

- NVENC (Linux) implements `set_pipelined(false)`: a `want_sync` latch handled
  at the same drained safe point as the engage side (`maybe_disengage_async`
  mirrors `maybe_engage_async`); the lazy sync re-init re-arms everything and
  opens on an IDR. The stream loop polls until the switch lands, then re-runs
  the escalation warmup so the wind-back's own stall can't re-escalate it.
  `PUNKTFUNK_NVENC_ASYNC=1` (operator-pinned async) refuses the wind-back;
  the trait doc now specifies the two-way contract.

- While escalated, `cadence_degraded` stays latched (bitrate climbs refused)
  even with the bucket drained: the headroom is spent, and climbs resuming
  mid-escalation would saw against it and starve the clean run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 01:44:54 +02:00
enricobuehlerandClaude Fable 5 1b27706a9b feat(host/native): truthful bitrate state — applied-rate adoption, ceiling pre-clamp, climb refusal
Host half of the §ABR-overdrive fix. The stream loop now reads
`Encoder::applied_bitrate_bps()` after every bitrate apply and stores THAT into
`bitrate_kbps`/`live_bitrate` — the send pacer, web console, mgmt registry and
control-task acks all track what the ASIC really targets instead of the
requested rate (the pre-fix ack promised 1.01 Gbps while the encoder ran
794 Mbps, and the client controller climbed from the phantom base forever).

- A short apply teaches `encoder_ceiling_kbps` (shared atomic): the stream loop
  pre-clamps incoming requests to it and SKIPS the apply when nothing would
  change — ending the reconfigure-reject → full-rebuild(~0.6 s + IDR) storm —
  and the control task resolves future SetBitrate acks against it, so the
  client learns the ceiling through the existing ack path (no wire change; old
  clients converge too).

- `cadence_degraded` (shared flag, leaky-bucket level ≥ 10 or an escalated
  session): while set, the control task resolves climbs to the current applied
  rate — on a fat LAN no network signal ever stops a climb the encoder can't
  serve, and past the compute knee more bits only deepen the cadence miss.
  Descents always pass; they're the cure.

- Escalation-warmup hygiene: an ABR rebuild stall (~70 missed deadlines at
  120 fps, 3.5× the escalate threshold) and the backlog scored against a
  heavier rate no longer feed the latency escalation — rebuilds reset the
  leaky bucket and re-run the warmup; in-place down-steps clear the bucket.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 01:44:54 +02:00
enricobuehlerandClaude Fable 5 f3c3a9427b feat(client/abr): learn the host's rate cap from short acks, and read host encode time as a signal
Two client-side halves of the §ABR-overdrive fix (the 4K120 sessions that
climbed to 1 Gbps against a 794 Mbps encoder and a 8.33 ms budget):

- Short-ack cap learning: the host now resolves a climb it can't serve to what
  the encoder actually runs at (its codec-level ceiling, or the current rate
  while encode is behind cadence). Two consecutive identical short acks latch
  that value as a host cap the climb logic folds into its ceiling — one short
  ack stays a transient (a failed rebuild also acks short once). The cap is
  mode-scoped (cleared on an accepted mode switch, tracked via a mode
  generation counter the control task bumps) and re-probes one step after ~60 s
  parked clean, so a heavy-scene refusal can't quietly cap the whole session.

- Host-encode-latency down-driver: the per-AU 0xCF `encode_us` the host
  already ships (and the overlay already draws) now feeds the controller
  through its own window accumulator — the overlay channel is lossy and
  embedder-drained, so the ABR gets a dedicated mirror of the decode-latency
  path. Baseline-relative like the decode signal (an escalated host reports
  encode_us inflated by ~a frame of queue depth; an absolute budget threshold
  would read permanently red), with the baseline rebased after our own
  decreases so one backoff doesn't train-fire into the floor. This is the only
  signal that can push an already-too-high rate back under the encoder's
  compute knee — host climb refusal stops the climb, but nothing else descends
  on a clean LAN.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 01:44:54 +02:00
enricobuehlerandClaude Fable 5 c1d54b835b fix(encode/nvenc): cache the level ceiling, expose the applied bitrate, force 2-way split at 4K120
Three encode-time fixes from the 4K120 field analysis (sessions pinned ~107 of
120 fps, 14-15 ms reported encode):

- Ceiling truth: the codec-level bitrate clamp (binary search at session open)
  now records its result in a process-lifetime advisory cache keyed by
  GPU/config, so an ABR overshoot opens — or in-place reconfigures — straight
  AT the ceiling instead of re-running the ~6-open search (and a full rebuild
  + IDR) on every overshoot. New `Encoder::applied_bitrate_bps()` exposes the
  post-clamp rate so the session loop can stop pacing/acking a phantom
  requested rate (consumed host-side in a follow-up).

- Clamp-search hygiene: only NVENC parameter/caps rejections steer the search
  now (`NvCallError` keeps the raw status downcastable); a transient failure
  (busy engine, session limit, OOM, driver skew) propagates instead of
  shrinking into — and now caching — a bogus ceiling. The floor fallback also
  records the split mode it actually opened with, so a later reconfigure
  re-presents the real session params.

- Split-frame selection: one shared `resolve_split_mode` replaces the two
  byte-identical direct-SDK copies; the force-2-way threshold moves to
  `SPLIT_FORCE_PIXEL_RATE` (950 Mpix/s, shared with the libav path) because
  4K120 = 995,328,000 px/s missed the old `> 1e9` gate by 0.47% and stayed on
  AUTO — which never engages at 2160 px height, leaving the second NVENC
  engine idle in exactly the mode the threshold existed for. The Linux
  session-ready info! line now carries the final split mode (journals are
  INFO+; this was undiagnosable from user logs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 01:44:54 +02:00
enricobuehlerandClaude Opus 5 42e5f5ad1e fix(encode/windows): two dead items the new lint leg exposed in the shipped combo
windows-host / package (push) Successful in 11m35s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 1m0s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m3s
arch / build-publish (push) Successful in 12m26s
android / android (push) Successful in 15m11s
docker / deploy-docs (push) Successful in 11s
deb / build-publish (push) Successful in 11m16s
deb / build-publish-host (push) Successful in 12m13s
apple / swift (push) Successful in 5m48s
ci / rust (push) Successful in 21m49s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m43s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m52s
apple / screenshots (push) Successful in 24m41s
09aa2db3 turned on `-p pf-encode --all-targets -D warnings` for Windows and removed
the crate-wide `allow(dead_code)`; together those surfaced two genuinely-dead items
in `nvenc,amf-qsv,qsv` — the combination the installer ships — and CI went red on my
own step.

  - `WinVendor::Amf`: native AMF replaced the libavcodec AMF path in production, so
    the only remaining CONSTRUCTOR is the `#[cfg(feature = "amf-qsv")]` latency A/B
    in amf.rs — test code, so the lib target constructs it nowhere. The module
    header already says this machinery is kept deliberately for that measurement, so
    it gets a targeted `#[allow(dead_code)]` naming the reason rather than deletion.
  - `probe_can_encode`: `lib.rs`'s only caller is under
    `cfg(all(not(feature = "qsv"), feature = "amf-qsv"))`, because with the native
    VPL backend compiled in `qsv::probe_can_encode` answers instead. Gated
    `#[cfg(not(feature = "qsv"))]` to match its call site.

Both are real findings the blanket allow had been hiding; neither is a behaviour
change.

WHY THIS ESCAPED MY PRE-PUSH CHECKS, since that is the useful part: the Windows
runner I verify on has no FFmpeg dev tree, so `amf-qsv` could not build there and
`ffmpeg_win.rs` was compiled by NOTHING in my matrix — the same class of blind spot
this whole audit is about, reproduced by me. CI's own FFmpeg turns out to be cached
on that box at C:\Users\Public\ffmpeg, so the verification recipe now sets
FFMPEG_DIR and covers `nvenc,amf-qsv,qsv` (the CI command and my new step) plus
`amf-qsv` without `qsv` — the combo that still uses `probe_can_encode`. All three
clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 01:04:04 +02:00
enricobuehlerandClaude Opus 5 d49f1bba49 fix(tray): count native sessions as streaming, and stop hiding "Open web console"
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 1m1s
windows-host / package (push) Failing after 4m40s
ci / docs-site (push) Successful in 1m8s
ci / bench (push) Successful in 5m31s
decky / build-publish (push) Successful in 29s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
arch / build-publish (push) Successful in 11m30s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 41s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
android / android (push) Successful in 15m55s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m36s
deb / build-publish (push) Successful in 9m54s
deb / build-publish-host (push) Successful in 9m42s
ci / rust (push) Failing after 21m48s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 7m45s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 9m57s
apple / screenshots (push) Successful in 22m35s
docker / deploy-docs (push) Successful in 25s
The tray sat at "Idle" through an entire stream, and its Windows context menu
could come up without its main action. Two separate causes.

`GET /api/v1/local/summary` reported `video_streaming`/`audio_streaming` straight
from `AppState.streaming` — flags only the GameStream media pipeline raises. The
native punktfunk/1 plane is the DEFAULT (GameStream is opt-in) and never touches
them, so every native session read as idle: idle icon, and a tooltip that printed
that session's own resolution next to the word "idle" (the `session` field was
already native-aware). This is the blind spot `/status` was fixed for — see the
`session_status` module doc — which the summary kept. Both flags now OR in a live
native session, and the tray additionally treats a reported session as streaming,
so a new tray reads an older host correctly too.

The menu built "Open web console" (and the pairing entry) only while a live
loopback probe of the console had just succeeded. A console still starting, or an
SSR render slower than the 2 s probe timeout, therefore deleted the tray's
most-wanted action outright — indistinguishable from a tray that never had one,
with left-clicking the icon as the only, undiscoverable, way in. The entry is now
always present and the default item; a failed probe changes its label to
"(not responding)" instead of hiding it, and takes two consecutive misses to say
so, since one timeout is not "down".

While here: a "Release kept display…" entry when displays are held (the summary
field's own doc promised a one-click release that did not exist), and entries
deep-link to /pairing and /displays instead of all landing on the dashboard. The
Linux SNI menu mirrors all of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:46:30 +02:00
enricobuehlerandClaude Opus 5 09aa2db37c fix(encode): probe Vulkan encode before committing capture to producer-native NV12
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m8s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 45s
ci / bench (push) Successful in 6m59s
windows-host / package (push) Failing after 10m45s
deb / build-publish (push) Successful in 11m9s
android / android (push) Successful in 12m59s
deb / build-publish-host (push) Successful in 11m59s
arch / build-publish (push) Successful in 13m21s
apple / screenshots (push) Successful in 16m48s
ci / rust (push) Successful in 22m37s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m39s
docker / deploy-docs (push) Successful in 28s
`linux_native_nv12_ok` is the verdict the host threads into capture negotiation
(session_plan -> OutputFormat::nv12_native -> ZeroCopyPolicy::native_nv12_session
-> pf-capture's prefer_native_nv12). Its doc claimed "the backend must be eligible
to open", but it asked only three static questions — codec is H265/AV1, an env
default, and a pref denylist — and never whether this GPU has an encode queue at
all. It could not: vulkan_video.rs exposed no probe.

That verdict is uniquely load-bearing. Once the producer has been asked for
two-plane NV12 there is NO fallback: open_video deliberately makes a failed Vulkan
open FATAL for an NV12 capture rather than degrading to libav VAAPI, because VAAPI
would import that buffer as packed RGB and stream silent garbage. So on a gamescope
host whose Mesa lacks Vulkan HEVC encode the session died at its first frame, while
the same host with PUNKTFUNK_PIPEWIRE_NV12=0 streamed fine — and the comment
promising such a device "degrades gracefully to the old backend rather than
breaking the stream" was stale on every gamescope host.

Two changes:

1. The gate. The third conjunct was a denylist of the EXPLICIT prefs that skip
   Vulkan Video ("nvenc"|"nvidia"|"cuda"|"pyrowave"), which silently missed the one
   that matters: the DEFAULT encoder_pref is "", and "" resolves to auto, which on
   an NVIDIA box opens NVENC. A stock NVIDIA host passed this gate. It now consults
   `linux_zero_copy_is_vaapi`, which layers the pref on top of the same auto
   decision open_video makes — what the note on `linux_auto_is_vaapi` says a
   capability probe must use, and what the downstream consumer of this very verdict
   already uses. This alone was worth fixing; a probe added behind the old gate
   would have opened a Vulkan instance on every NVIDIA handshake.

2. The probe. `vulkan_video::probe_encode_support` runs the FIRST check open_inner
   performs and hard-fails on — the physical-device + encode-queue-family scan for
   this codec's op — and nothing more, so it is provably no stricter than the open
   and can never talk a working host out of the fast path. The scan is now a shared
   `find_encode_device` used by BOTH, because a probe that mirrors a dispatch goes
   stale the first time the dispatch grows a case (the failure open_video's
   backend-label note already records). Cost: one instance plus physical-device
   queries — no logical device, no video session, no VRAM — cached per (selected
   GPU, codec) in the can_encode_10bit idiom, with the probe run outside the lock.
   Per codec, not once: codec_op_for selects a different queue-family bit for AV1,
   and HEVC-encode-without-AV1-encode is the common VCN/ANV configuration.

Later stages (create_device, create_video_session, the capability query) can still
fail for reasons the probe does not model. Those are harmless: the capture format
is only committed once the probe says yes, and everything after keeps the ordinary
packed-RGB negotiation.

Verified `-D warnings --all-targets` + tests on Linux (default; shipped
nvenc+vulkan-encode+pyrowave). The behaviour needs an on-glass check on the bazzite
RADV box — including the negative case, which `RADV_DEBUG=novideo` reproduces.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:32:37 +02:00
enricobuehlerandClaude Opus 5 4063ddc93d perf(encode/nvenc-linux): swscale the packed 3-bpp expand instead of a per-pixel loop
`submit_cpu`'s RGB24/BGR24 → rgb0/bgr0 expand ran `w*h` iterations per frame, each
building two bounds-checked sub-slices for a 3-byte copy plus a pad-byte store — a
shape LLVM will not vectorise into the byte shuffle it actually is. At 3840x2160
that is 8.3M iterations on the encode thread, per frame.

It is not an edge case. The module header says the portal commonly negotiates
packed 24-bit RGB, and pf-capture offers `VideoFormat::RGB` first because wlroots
commonly fixates it — so this was the mainstream CPU path, running roughly an order
of magnitude slower than the 4-bpp sibling branch (a plain row memcpy) purely
because of how it was written.

swscale's packed-RGB expanders are SIMD, the sibling VAAPI backend already routes
RGB24/BGR24 through them, and this file already owned the `sws_getContext` /
`sws_freeContext` lifecycle — so this is net subtractive rather than new machinery:
the existing CSC condition widens to include `expand`, and the hand-written loop
goes away. No new field, no second context, nothing added to `Drop`: `expand` is
false whenever `want_444` (see the `nvenc_pixel`/`expand` binding), so the three
users are mutually exclusive and one context serves whichever applies. The `expand`
field itself is gone with its only reader.

`sws_setColorspaceDetails` is now applied ONLY for the 4:4:4/HDR users. The expand
is a pure byte shuffle — NVENC does the RGB→YUV itself downstream — and handing it
a matrix and range would have silently range-converted every packed-RGB session,
which is exactly what the module header promises does not happen.

Verified `-D warnings --all-targets` + tests on Linux (default; shipped
nvenc+vulkan-encode+pyrowave). The pixel path itself is unverifiable without an
NVIDIA box: `nvenc_hdr10_smoke` and friends are #[ignore]d, so the channel order
and the pad byte want an on-glass check on the CachyOS 5070 Ti.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:32:37 +02:00
enricobuehlerandClaude Opus 5 c7081b2a82 fix(encode/nvenc-linux): stop leaking the SwsContext on open's early returns
`sws_getContext` hands back a raw pointer whose only `sws_freeContext` lives in
`Drop for NvencEncoder` — and `Drop` needs a CONSTRUCTED `Self`, which does not
exist on either of `open`'s early returns: the intra-refresh-unsupported path
(which recurses into `Self::open`) and the plain error return. Both sit between
the context's creation and the `Ok(NvencEncoder { … })` that would give it an
owner, so every failed open leaked one.

That is not a once-per-process wart. `open_nvenc_probed` walks an EINVAL bitrate
ladder — requested rate, then the codec-level cap, then ×3/4 down to 50 Mb/s —
re-entering `open` at each step, so a GPU that refuses the requested bitrate leaks
a context per step (up to ~10). The intra-refresh retry adds one more.

Fixed structurally rather than with a guard: the block depends only on values known
before the encoder open (`want_444`, `want_hdr10`, `cuda`, `format`, `nvenc_pixel`,
the dimensions, `full_range_444`), so moving it BELOW `video.open_with(opts)` — to
just above the struct construction, with nothing fallible in between — makes the
leak unrepresentable instead of merely handled. No behaviour change: same context,
same colourspace details, same field.

Found while designing the WP1.4 swscale expand, which would have promoted this from
a 4:4:4/HDR-only leak (the only sessions that build a context today) to one on every
packed-RGB session. Fixing it first is the prerequisite for that change.

Verified `-D warnings --all-targets` + tests on Linux (default; shipped
nvenc+vulkan-encode+pyrowave).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:32:37 +02:00
enricobuehlerandClaude Opus 5 6be174dc9c fix(encode/windows): the encode bit depth must follow the pixels, not the negotiation
All three Windows backends derived it the same wrong way —
`bit_depth >= 10 || matches!(format, P010 | Rgb10a2)` at ffmpeg_win.rs:154,
amf.rs:1275 and qsv.rs:777 — so the NEGOTIATED depth could force a 10-bit encoder
over an 8-bit capture.

That combination is not hypothetical. A client advertises 10-bit, the handshake
negotiates bit_depth=10, and then enabling advanced colour on the IDD virtual
display fails — at which point the capturer says exactly that and delivers 8-bit
NV12 anyway ("10-bit HDR was negotiated but enabling advanced color on the virtual
display FAILED — encoding 8-bit SDR"). Every vendor then lost the session, each in
its own way:

  - native AMF and native QSV derived `expected = P010`, saw Nv12, and bail!'d at
    open;
  - the libavcodec path accepted the open, built a P010 encoder, and then failed
    EVERY submit forever, because its per-frame check recomputes the depth from the
    frame and never matches. reset() could not help: the rebuild re-derived the
    same wrong depth from the same stored bit_depth. The host burned
    MAX_ENCODER_RESETS and ended the session.

The last one is the worst of the three because it IS the fallback: native QSV
correctly refuses this input at open, and lib.rs then falls back to the ffmpeg path
"for robustness" — which accepted it and died per frame instead.

Since the duplication was the bug, the fix is one shared `ten_bit_input()` in
codec.rs that all three call, and it follows the delivered pixels. That also keeps
the stream HONEST rather than merely alive: the depth selects the colour signalling
(BT.2020 PQ vs BT.709) and the staging surface format, so an 8-bit capture now
produces an 8-bit stream that says it is SDR — which is what the capturer already
reported it was sending. It warns when the negotiated depth is discarded.

The negotiated depth stays an upper bound. The session LABEL may still claim HDR;
that mismatch lives in the negotiation, not in the encoder, and is not addressed
here.

`is_10bit_format` keeps its (now format-only) definition and is used by
submit_d3d11's per-frame check, so the predicate the encoder was built from and the
one it re-checks per frame cannot drift. That check can now only fire on a genuine
mid-stream depth change.

Verified `--all-targets -D warnings` on Windows (no features / pyrowave / qsv /
nvenc,qsv; 31 tests) and Linux (default; shipped nvenc+vulkan-encode+pyrowave).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:32:37 +02:00
enricobuehlerandClaude Opus 5 6c97c00add fix(encode/pyrowave): the RDO block-index cap must cover 4:2:0, not just 4:4:4
The vendored rate controller packs the 32x32 block index into the low 16 bits of
`RDOperation::block_offset_saving` (pyrowave-sys patches/0002). Past u16::MAX the
index collides with the `saving` field, the resolve over-credits, and the emitted
payload can overshoot the buffer `pyrowave_encoder_packetize` writes into — whose
only bounds check is an `assert` that the Release (NDEBUG) vendored build compiles
out. There is no Rust-side guard behind it.

All three callers of `pyrowave_mode_fits_rdo` hardcoded 4:4:4, leaving 4:2:0
unchecked — but 4:2:0 overflows too, just later: 8192x6144 is 73728 blocks and
8192x8192 is 98304, while `Codec::PyroWave.max_dimension()` permits 8192 per axis,
so both are reachable from a client-requested `mode=WxHxFPS`. Worse, the host's
only use of the helper (native/handshake.rs) is a 4:4:4 -> 4:2:0 downgrade, so an
oversized mode was actively routed INTO the unguarded branch.

The cap now lives in `validate_dimensions`, checked against 4:2:0 — the most
permissive chroma, so a mode that cannot fit there fits no PyroWave session at any
chroma. That is the single chokepoint both the negotiator (handshake.rs:180) and
`open_video_backend` already run. Both encoder opens additionally check the chroma
actually being opened: the 4:4:4 half, plus defence in depth for the
`PUNKTFUNK_ENCODER=pyrowave` lab override.

Not dormant: punktfunk-host has `default = ["pyrowave"]` and no build passes
`--no-default-features`, so these backends ship in every host binary.

Tests pin the arithmetic (8192x6144 = 73728, 8192x8192 = 98304, 7680x4320 still
fits) and assert the cap does not leak to H.265/AV1, which have no such controller.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:32:37 +02:00
enricobuehlerandClaude Opus 5 16fa43da40 ci(encode): lint and test the feature-gated encode backends
pf-encode's GPU backends are off by default, so almost none of them were under
`-D warnings`:

  - ci.yml lints/tests with default features, which do cover VAAPI, libav-NVENC
    and — via punktfunk-host's `default = ["pyrowave"]` — the PyroWave backends,
    but not `nvenc` or `vulkan-encode`.
  - deb.yml builds those two, but with `cargo build`, where warnings are not errors.

That left enc/linux/nvenc_cuda.rs, enc/linux/vulkan_video.rs and the vendored
vk_av1_encode / vk_valve_rgb bindings — ~8,150 lines carrying ~70 `unsafe` blocks —
never linted anywhere, so the crate's own
`#![deny(clippy::undocumented_unsafe_blocks)]`, its stated unsafe-proof gate, was
never actually enforced on them.

Linux gains a clippy+test leg at the SHIPPED feature set: deb.yml builds
`punktfunk-host/nvenc,punktfunk-host/vulkan-encode` WITHOUT
`--no-default-features`, so the .deb carries pyrowave too and that combination is
what deserves the lint. GPU-free — the hardware tests are `#[ignore]`d and
NVENC/CUDA dlopen their entry points, so the test binary links with no driver.

Windows gains a separate `-p pf-encode --all-targets` lint. The existing lint is
`-p punktfunk-host`, which never builds pf-encode's test targets — the blind spot
that let the Linux twin's tests rot. It must be clippy rather than `cargo test`:
on MSVC nvidia-video-codec-sdk link-imports NvEncodeAPICreateInstance /
NvEncodeAPIGetMaxSupportedVersion, so a test binary cannot link without the
driver's import lib. clippy type-checks without linking; ci.yml runs the tests.

`--all-targets` is load-bearing, not decoration: without it the feature-gated
`#[cfg(test)]` modules are never compiled at all.

Also widens windows-host.yml's paths filter, which listed `crates/pf-encode/**`
but none of the crates it compiles against (pf-frame, pf-gpu, pf-zerocopy,
pf-host-config, pf-capture, ...), so a change reaching the Windows host through
one of those triggered no Windows build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:32:37 +02:00
enricobuehlerandClaude Opus 5 07348c0175 refactor(encode): drop the crate-wide allow(dead_code)
Inherited from the pre-extraction host crate root as scaffolding for backend
paths defined ahead of the build that used them. A census across every feature
combination on both platforms (flip it to `warn`, rebuild) found it was hiding
exactly two items — so it bought nothing while blinding the crate to future rot:

  - `vaapi::fourcc` — superseded by `pf_frame::drm_fourcc`; no call sites left.
  - `vulkan_video::open_opts` — test-only (the smoke tests use it to pass the
    RGB-direct request explicitly rather than through the env), now `#[cfg(test)]`.

Removing it surfaced a real latent defect the Linux census could not see:
`amf.rs`'s `percentile` / `drive_and_measure` helpers are used only by the
`#[cfg(feature = "amf-qsv")]` latency A/B benchmark, so a `--features nvenc,qsv`
build compiled the helpers with their caller gated out. They now carry the same
gate as their caller.

Verified `--all-targets -D warnings` on Linux (no features; shipped
nvenc+vulkan-encode+pyrowave) and Windows (no features; pyrowave; qsv; nvenc,qsv).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:32:37 +02:00
enricobuehlerandClaude Opus 5 e30551b1e2 fix(encode/nvenc-linux): unrot the direct-NVENC test module
All ten `NvencCudaEncoder::open` call sites in the `#[cfg(test)]` module passed 9
arguments to a 10-parameter function: `cursor_blend` was added to `open` and the
tests were never updated. The module has been uncompilable ever since —
`cargo clippy -p pf-encode --features nvenc --all-targets` fails with 10x E0061.

Nothing in CI noticed because no job compiles this crate's feature-gated test
targets: ci.yml lints/tests with default features (which do not include `nvenc`),
and windows-host.yml lints `-p punktfunk-host`, which builds pf-encode as a
dependency and so never builds its test targets. The CI commit below closes that
gap; this has to land first or that leg starts red.

`false` is what these tests exercised before `cursor_blend` existed: every frame
they build sets `cursor: None`, and there is no cursor-blend test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 00:32:37 +02:00
enricobuehlerandClaude Fable 5 8d5a9f66c9 fix(gamescope): ship a packaged privilege path for the DM-stop takeover
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 55s
android / android (push) Successful in 12m5s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
ci / bench (push) Successful in 6m29s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
docker / deploy-docs (push) Successful in 27s
deb / build-publish (push) Successful in 8m40s
arch / build-publish (push) Successful in 14m51s
deb / build-publish-host (push) Successful in 9m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 15m32s
apple / swift (push) Successful in 23m28s
ci / rust (push) Successful in 26m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m37s
apple / screenshots (push) Successful in 2h32m51s
Field report (Nobara, 0.19.2): entering Game Mode mid-stream left the monitor
on and the stream mirroring at the desktop's resolution. Root cause: the
DM-flavor takeover needs privilege to stop plasmalogin, the polkit rule is a
docs-only manual step nobody installs, so every managed entry silently
degraded to ATTACH — and with a physical display connected the attach path
mirrors the box's own session at its own mode.

Ship the privilege with the packages instead: a root helper
(libexec/punktfunk/pf-dm-helper, verbs stop|restore) behind its own polkit
action (io.unom.punktfunk.dm-helper, allow_any — the same mechanism Nobara's
os-session-select uses, and the helper derives the DM unit from the
display-manager.service symlink itself so callers never name a unit across
the privilege boundary). The host tries the plain system-bus verbs first
(root / operator rule still take precedence), then pkexec's the helper; the
restore paths (idle restore + in-stream desktop-switch honor) use the same
ladder so a takeover that needed the helper can always be undone by it.

Packaged in rpm/deb/arch (Arch under /usr/lib/punktfunk with the policy's
exec.path annotation rewritten; the host probes both layouts). Nix is left
out deliberately: store paths can't match the probe, NixOS keeps the manual
rule. Docs updated — the rule snippet stays as the scoped alternative.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 23:41:29 +02:00
enricobuehlerandClaude Opus 5 5c7a9407ff fix(windows/web): start the console once its inputs exist, and verify it started
arch / build-publish (push) Successful in 17m1s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 58s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 47s
ci / bench (push) Successful in 6m19s
deb / build-publish (push) Successful in 9m35s
docker / deploy-docs (push) Successful in 29s
apple / screenshots (push) Successful in 10m56s
deb / build-publish-host (push) Successful in 13m40s
android / android (push) Successful in 16m19s
windows-host / package (push) Successful in 17m13s
ci / rust (push) Successful in 19m39s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m44s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m18s
A fresh install left PunktfunkWeb registered but not running: `web setup` waited
only for the mgmt token before firing `schtasks /run`, while `web-run.cmd` also
requires the host identity cert — and the host writes the token during argument
parsing but `cert.pem` only after the pure-Rust RSA-2048 keygen inside `serve`.
The launcher lost that race, exited 1, and since the task carried no trigger but
boot (Task Scheduler does not reliably restart on a non-zero exit code) the
console stayed down until the next reboot, with the installer still reporting
"web console set up + started".

- `web setup` gates on cert.pem (written last) as well as the token, 90 s budget.
- After `schtasks /run`, poll for the :47992 listener and retry before giving up;
  warn honestly instead of claiming a start that did not happen.
- `web-run.cmd` (installed + dev) waits in-process for the token + cert (~5 min)
  rather than exiting 1 and hoping restart-on-failure retries.
- Register the task with a logon trigger alongside boot, falling back to the
  boot-only XML if a Task Scheduler build rejects it.
- Linux had the same defect: punktfunk-web.service's Restart=on-failure gave up
  permanently after systemd's default 5-starts-in-10 s limit. StartLimitIntervalSec=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-24 23:22:09 +02:00
enricobuehler 7781d09e26 feat(linux): log if unsupported / too low gamescope version discovered
apple / swift (push) Successful in 2m29s
android / android (push) Successful in 12m52s
arch / build-publish (push) Successful in 12m59s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m6s
ci / bench (push) Successful in 5m30s
ci / rust (push) Successful in 19m51s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 17s
deb / build-publish (push) Successful in 8m43s
apple / screenshots (push) Successful in 20m59s
deb / build-publish-host (push) Successful in 9m25s
docker / deploy-docs (push) Successful in 24s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m22s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m22s
2026-07-24 20:57:44 +02:00
enricobuehlerandClaude Opus 4.8 f4fe4d0792 chore(release): bump workspace version to 0.19.2
android-screenshots / screenshots (push) Successful in 2m59s
windows-host / package (push) Successful in 9m45s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m48s
android / android (push) Successful in 12m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m20s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m5s
arch / build-publish (push) Successful in 12m2s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m45s
audit / cargo-audit (push) Successful in 2m10s
audit / bun-audit (push) Successful in 14s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m6s
apple / swift (push) Successful in 1m57s
ci / bench (push) Successful in 5m41s
ci / rust (push) Successful in 18m52s
decky / build-publish (push) Successful in 20s
deb / build-publish (push) Successful in 8m50s
deb / build-publish-host (push) Successful in 9m24s
web-screenshots / screenshots (push) Successful in 3m7s
flatpak / build-publish (push) Successful in 6m27s
docker / deploy-docs (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
release / apple (push) Successful in 1h4m50s
apple / screenshots (push) Successful in 23m21s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m40s
linux-client-screenshots / screenshots (push) Successful in 7m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m0s
An all-fixes patch on top of 0.19.1, all host-side:

- KDE Plasma display takeover moved off kscreen-doctor to in-process
  kde_output_management_v2 (86d5d666), with the >60 Hz custom-mode install
  ported too (64511af4) — a KWin host whose kscreen-doctor wedges now disables
  its physicals and hands over the streamed output in ~2.4 ms instead of hanging
  ~26 s and never taking over.
- The gamescope managed takeover is display-manager-flavor-aware (e35dad52) so
  switching a non-SDDM Linux desktop (Nobara / plasmalogin) into Game Mode no
  longer start-limits the display manager into a permanent black screen, and
  in-stream 'Switch to Desktop' is honored under it (a3e21d92).
- Windows display isolation anchors the kept sources at the desktop origin
  (9b241d9d) so an origin-less desktop no longer fails 0x57, with restore
  guaranteed never to leave the desktop all-dark.
- Windows pen/touch/absolute-mouse map over the streamed output's rect rather
  than the whole virtual desktop (087e7d04), so absolute input lands correctly
  when a physical monitor is kept on beside the streamed display.

Nothing moved on the wire (protocol 2) or at the C ABI (13), so no embedder
rebuild and 0.18/0.19 peers keep mixing freely. The Windows virtual-display
driver protocol is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 17:56:08 +02:00
enricobuehlerandClaude Fable 5 087e7d0405 fix(inject/windows): map absolute input over the streamed output's rect, not the whole desktop
audit / cargo-audit (push) Successful in 2m13s
audit / bun-audit (push) Successful in 15s
arch / build-publish (push) Successful in 12m46s
windows-host / package (push) Successful in 12m53s
android / android (push) Successful in 13m48s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m9s
apple / swift (push) Successful in 15m13s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m9s
ci / bench (push) Successful in 5m28s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m10s
ci / rust (push) Successful in 19m38s
decky / build-publish (push) Successful in 27s
deb / build-publish (push) Successful in 8m37s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 1m36s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 1m39s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 1m15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m54s
docker / deploy-docs (push) Skipped
deb / build-publish-host (push) Successful in 10m7s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m26s
flatpak / build-publish (push) Successful in 6m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m41s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m42s
release / apple (push) Successful in 56m19s
apple / screenshots (push) Successful in 23m11s
Pen, touch, and absolute mouse arrive normalized to the STREAMED display's
frame, but pointer_windows::to_screen and sendinput's MouseMoveAbs mapped
them over the whole virtual desktop — correct only when the virtual display
is the sole active display (Exclusive topology). In Extend — a physical
monitor kept on beside the virtual output, or an Exclusive isolate degraded
to the 0x57 keep-physicals fallback — the streamed output sits at a non-zero
origin, so every sample landed shifted and mis-scaled. The pen exposed it
first (field report 2026-07-24): a stylus is strictly absolute, with no
closed-loop correction onto the target like a cursor.

New pf-inject::stream_target (Windows): the host publishes the streamed
output's CCD target id at capture bring-up (one central site —
capture_virtual_output covers the native and GameStream planes); mapping
sites resolve its current desktop rect through pf-win-display's
source_desktop_rect — the same resolver the cursor-readback poller uses, so
inject and readback always agree — TTL-cached (250 ms) because a
group-layout re-arrange moves a live output's origin mid-session. No target
set / never resolved falls back to the whole virtual desktop (the historical
mapping, still right for Exclusive topology and devtest).

One change in to_screen covers pen + touch; MouseMoveAbs converts the same
desktop pixel into the 0..65535 MOUSEEVENTF_VIRTUALDESK coordinate, closing
the identical latent absolute-mouse bug (apollo-comparison open item #14/#30).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 17:38:36 +02:00
enricobuehlerandClaude Opus 4.8 64511af4d3 fix(kwin): install >60Hz custom modes in-process too — KWin path is now kscreen-free
audit / bun-audit (push) Successful in 35s
audit / cargo-audit (push) Successful in 2m34s
decky / build-publish (push) Successful in 17s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 47s
deb / build-publish-host (push) Successful in 9m56s
deb / build-publish (push) Successful in 11m17s
android / android (push) Successful in 12m31s
docker / deploy-docs (push) Successful in 29s
arch / build-publish (push) Successful in 15m14s
apple / swift (push) Successful in 1m21s
apple / screenshots (push) Successful in 4m50s
release / apple (push) Successful in 12m45s
flatpak / build-publish (push) Successful in 7m14s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m58s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m18s
windows-host / package (push) Successful in 16m32s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m19s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m14s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m28s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m25s
ci / rust (push) Successful in 17m55s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 59s
ci / bench (push) Successful in 5m45s
Follow-up to the previous commit: the only KWin topology op still shelling out to
kscreen-doctor was the >60 Hz custom-mode INSTALL (`set_custom_refresh`). Port it
to `kde_output_management_v2` so the whole KWin path is kscreen-free on modern KWin.

`kwin_output_mgmt::set_custom_mode` builds a one-entry `kde_mode_list_v2`
(set_resolution / set_refresh_rate / set_reduced_blanking=full / add_mode), applies
it via `kde_output_configuration_v2.set_custom_modes` (since 18), waits for KWin to
generate the mode (its CVT generator may align the width down — matched with the
same height-exact / width-within-8 / refresh-within-1Hz gate as before), then
selects it — which changes the output size and drives the sacrificial-birth stream
renegotiation. `kwin::create`'s want_high branch tries this first and only resolves
a kscreen id + calls `set_custom_refresh` if it returns None (pre-6.6 KWin without
set_custom_modes, or the compositor not answering) — so a >60 Hz session on a box
where kscreen-doctor wedges no longer eats a 5 s resolve timeout either.

Bonus: `set_custom_modes` REPLACES the custom list, so reconnects are idempotent —
no more one-custom-mode-per-connect growth of the user's display list.

Verified: cargo test -p pf-vdisplay (73 pass), clippy -D warnings clean, fmt clean.
The set_custom_modes round-trip is proven live on KWin 6.6.4: apply=applied, and a
1648x928@75 request generates 1648x928@74.901 (CVT fractional refresh, caught by the
1 Hz tolerance). The mode SELECT + full >60 Hz stream is still owed on-glass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 12:47:34 +02:00
enricobuehlerandClaude Opus 4.8 86d5d66660 fix(kwin): drive display topology over kde_output_management_v2, not kscreen-doctor
A KDE host on Nobara stopped disabling its physical screens and creating the
virtual output the moment it updated: streaming still came up, but bring-up took
~26 s and the streamed output never became the desktop (`also_disabled=[]`).

Root cause is not our topology logic — it's that every `kscreen-doctor` call on
the reporter's session wedges. kscreen-doctor drives libkscreen, which (per
setup) waits on the kscreen KDED module over D-Bus; when that layer is stuck it
blocks in its own connect and never returns, so all five topology queries hit
their 5 s budget and got killed (host log). Reproduced live on a Nobara / KWin
6.6.4 box: `kscreen-doctor -j` there times out at 8 s, every time.

But the compositor's OWN Wayland is fully responsive on that same session — the
host just created a virtual output over it via zkde_screencast. So drive the
topology (resolve our output, take primary, disable the physical/bootstrap
outputs, capture their modes, re-enable on teardown, position) directly over
`kde_output_management_v2` + `kde_output_device_v2` instead of shelling out.
On that same Nobara box the in-process path binds management (v19) and
enumerates the outputs in 2.4 ms — ~3400x faster than the 8 s hang, and immune
to whatever wedges the standalone tool.

- vendor kde-output-management-v2 / kde-output-device-v2 (KWin advertises mgmt
  v19, device v20); generate client bindings inline (the interdependent-protocol
  module split from the wayland-protocols crate; needs the `bitflags` dep for the
  device protocol's bitfield enums).
- new `kwin_output_mgmt`: bounded enumerate-then-apply over one Wayland
  connection; every wait is time-bounded so a genuinely wedged compositor
  degrades to `handled = false` and the old kscreen-doctor path still runs.
- `kwin::create` topology + `apply_position` prefer the in-process path (address
  our output by its stable device UUID, supersede-robust) with kscreen-doctor as
  the fallback. The 60 Hz path now makes ZERO kscreen-doctor calls; only the
  >60 Hz custom-mode install still shells out (its in-process port is a follow-up).

Verified on Linux: cargo test -p pf-vdisplay (73 pass), clippy --all-targets
-D warnings clean, fmt clean, punktfunk-host checks clean. In-process enumeration
proven live against KWin 6.6.4 (the reporter's env); the disable/apply path is
not yet on-glass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 12:47:33 +02:00
enricobuehlerandClaude Fable 5 a3e21d9227 feat(vdisplay/gamescope): honor in-stream 'Switch to Desktop' under the managed DM takeover
ci / web (push) Successful in 59s
ci / docs-site (push) Successful in 1m7s
apple / swift (push) Successful in 1m21s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
ci / bench (push) Successful in 6m2s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m32s
apple / screenshots (push) Successful in 6m21s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m27s
deb / build-publish-host (push) Successful in 9m39s
docker / deploy-docs (push) Successful in 29s
deb / build-publish (push) Successful in 13m6s
android / android (push) Successful in 17m28s
arch / build-publish (push) Successful in 18m8s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m9s
windows-host / package (push) Successful in 24m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m12s
ci / rust (push) Successful in 26m49s
First on-glass test of the DM-flavor takeover confirmed the black-screen
fix, but exposed that the managed model could not switch back: Steam's
in-stream session-select is a silent no-op while the display manager is
stopped (every config-write branch in os-session-select requires the DM
to be running), so the capture loss it causes just relaunched game mode.

The user pass of the switch script does leave one durable trace — the
~/.config/steamos-session-select sentinel, written before any DM check.
The managed launch now baselines its mtime; a capture loss with the
sentinel advanced reads as the switch request and is honored by
replaying the switch with the DM up: restore the display manager, run
the distro's own os-session-select (its internal pkexec is authorized
allow_any by the distro policy, so it works from the host's sessionless
context), then stop the autologin game-mode unit so Relogin lands in
the newly selected desktop. The capture-loss re-detection then follows
KWin as it comes up; a 120 s post-honor grace stops the rebuild loop
from racing the booting desktop back into game mode (superseded early
if the box's own game-mode unit reappears — the desktop→game leg stays
fast). The baseline is re-recorded on crash-restore so a pre-existing
sentinel never reads as a fresh request. Every verb in the sequence was
live-validated on the Nobara repro VM.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 11:49:24 +02:00
enricobuehlerandClaude Fable 5 e35dad529b fix(vdisplay/gamescope): DM-flavor-aware session takeover — stop masking Nobara's plasmalogin to death
Field report (Nobara 43 HTPC): switching the host to Steam Game Mode
mid-stream permanently black-screened the box. Live-proven root cause on
a Nobara repro VM: our takeover masks the box's gamescope-session-plus
unit, plasmalogin's Relogin=true then fails its session Exec repeatedly
and trips systemd's start limit within ~1 s — the display manager dies,
and our restore verb (unmask + user start) cannot bring a seatless
gamescope back. Only 'reset-failed + restart' of the DM recovers.

The takeover is now display-manager-flavor-aware:
- SDDM (Bazzite/SteamOS) keeps the proven mask+SIGKILL path unchanged.
- plasmalogin/unknown DMs never mask: with privilege (root or an
  operator polkit rule scoped to the DM unit — documented) the host
  stops the DM for the stream and restores it with reset-failed +
  restart (PUNKTFUNK_RECOVER_SESSION_CMD as fallback), recorded in the
  persisted takeover state so a host crash still restores; without
  privilege the managed takeover degrades to ATTACH and mirrors the
  box's live Game Mode instead of destabilizing the seat. Both legs of
  the privileged cycle live-verified on the repro VM (headless managed
  session works with zero login sessions; render nodes are 0666).
  A loaded-but-inactive leftover instance never triggers the DM stop.

Companion fixes from the same triage:
- ensure_box_gamescope_mode gains the attach-only rebuild-probe guard
  both managed paths already had (stale post-capture-loss detection
  restarted the box's unit), and no longer re-modes a box that drives a
  physical display — attach mirrors on-glass; re-mode is the
  headless-box model.
- Capture-loss rebuilds targeting gamescope get a 100 s budget: the
  40 s budget expired inside the first 45 s Steam-cold-start launch
  attempt, a guaranteed single-shot failure.
- A PUNKTFUNK_COMPOSITOR pin now WARNs once per capture loss when the
  live session no longer matches it (the pin disables
  session-following — the reporter's original stream-death trigger).
- A managed session that took nothing over (client gamescope pin beside
  a live desktop) is stopped on disconnect instead of being orphaned
  forever.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 11:49:24 +02:00
enricobuehlerandClaude Fable 5 9b241d9d7b fix(pf-win-display): anchor the kept sources at the desktop origin when isolating
ci / docs-site (push) Successful in 1m0s
ci / web (push) Successful in 1m4s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 34s
ci / bench (push) Successful in 6m36s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 46s
apple / screenshots (push) Successful in 6m35s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m51s
deb / build-publish (push) Successful in 12m42s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
deb / build-publish-host (push) Successful in 13m31s
android / android (push) Successful in 15m54s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7m18s
arch / build-publish (push) Successful in 16m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9m34s
docker / deploy-docs (push) Successful in 29s
ci / rust (push) Successful in 20m38s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m34s
Round 3 of the field-reported exclusive-topology 0x57, with the reporter's
retest logs finally separating the variable: the SAME isolate call converged
rc=0 whenever a kept member already sat at (0,0), and failed 0x57 on every
shape — doomed-path-carried AND keep-only escalation — whenever the doomed
physical held the origin. A committable config must still contain a primary
(a source pinned exactly at (0,0)); deactivating the display that held it
while the kept virtual stays at its EXTEND offset supplies an origin-less
desktop, which Windows rejects wholesale. Translate the kept sources rigidly
so the top-left-most lands on the origin; sets already covering (0,0) are
untouched so plain re-commits stay byte-identical.

Also from the same logs:
- restore_displays_ccd now guarantees the desk is never left all-dark: the
  saved snapshot can be unappliable (0x64a — it pinned a virtual target
  incarnation removed before teardown) or apply cleanly yet re-light nothing
  (snapshotted while an earlier failed teardown had the physicals off). If no
  external physical is active after the apply while one is connected, fall
  back to the database EXTEND preset. Internal panels don't count — a closed
  lid must not be forced back on.
- the final isolate failure now names the surviving targets instead of
  asserting 'a non-virtual display stayed active' — the logs showed a sibling
  VIRTUAL display as the survivor (linger-expiry shrink), and the wording sent
  triage the wrong way.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 10:21:47 +02:00
enricobuehlerandClaude Opus 4.8 ebddffa4c8 chore(release): bump workspace version to 0.19.1
apple / swift (push) Successful in 1m30s
android-screenshots / screenshots (push) Successful in 3m1s
audit / cargo-audit (push) Successful in 2m14s
audit / bun-audit (push) Successful in 15s
ci / web (push) Successful in 1m8s
ci / docs-site (push) Successful in 1m11s
release / apple (push) Successful in 9m36s
android / android (push) Successful in 14m52s
ci / bench (push) Successful in 6m31s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 14m40s
apple / screenshots (push) Successful in 6m40s
ci / rust (push) Successful in 19m21s
deb / build-publish-host (push) Successful in 10m36s
deb / build-publish (push) Successful in 9m18s
linux-client-screenshots / screenshots (push) Successful in 7m40s
docker / deploy-docs (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
web-screenshots / screenshots (push) Successful in 2m59s
windows-host / package (push) Successful in 9m59s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m8s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m58s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m17s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m43s
flatpak / build-publish (push) Failing after 8m0s
An all-fixes patch on top of 0.19.0. The headline is session teardown — the
native stop flag made enforceable and bounded (ac3dc432) and GameStream ending
on the control peer's ENet disconnect (41fa25c4), so a session can no longer
outlive the client that started it on either plane.

Also: the Windows input-desktop pair (display writes b5fa878b + pen/touch
injection 24a24734 both surviving a UAC prompt), the Apple display-sleep guard
(bc5f6a38), the gamescope corner-parked cursor (4b2d2d1e), the KWin custom-mode
selection (3a33a694), the tvOS glob warnings (cc8cccbe), and the Linux
starter/env docs revamp (66172753).

Nothing moved on the wire (protocol 2) or at the C ABI (13), so no embedder
rebuild and 0.18/0.19 peers keep mixing freely.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 02:15:40 +02:00
enricobuehlerandClaude Fable 5 cc8cccbee6 fix(core/client): drop two redundant glob imports that warned on tvOS
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 35s
release / apple (push) Successful in 9m19s
ci / web (push) Successful in 1m12s
ci / docs-site (push) Successful in 1m21s
android / android (push) Successful in 12m19s
arch / build-publish (push) Successful in 12m19s
ci / bench (push) Successful in 6m36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
apple / screenshots (push) Successful in 6m35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
windows-host / package (push) Successful in 9m44s
deb / build-publish (push) Failing after 4m47s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m52s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m25s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m14s
deb / build-publish-host (push) Successful in 9m39s
flatpak / build-publish (push) Failing after 8m25s
ci / rust (push) Successful in 23m58s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m37s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m13s
docker / deploy-docs (push) Successful in 12s
`punktfunk-core` emitted `unused import: super::super::*` for
client/pump/handshake.rs and client/pump/input_task.rs — but only when
built for a tvOS target, where the cfg set leaves the glob contributing
nothing. Nobody saw them because CI never builds tvOS: that needs
BUILD_TVOS=1 and the Tier-3 build-std nightly path, and the Apple
workflow only runs swift build/test (macOS).

Both files also `use super::*`, i.e. the parent `pump` module, which
itself globs `super::*` (`client`) unconditionally — so every name stays
reachable by that path and the inner glob was pure duplication. Their
sibling datagram_task.rs already imports only `use super::*`, so this
also makes the module consistent. control_task.rs and data.rs keep
theirs: those genuinely resolve names through it and never warned.

Verified warning-free AND compiling on every slice the xcframework
ships — aarch64/x86_64-apple-darwin, aarch64-apple-ios,
aarch64-apple-ios-sim, aarch64-apple-tvos, aarch64-apple-tvos-sim,
x86_64-apple-tvos — plus Linux with and without the `quic` feature. A
full xcframework rebuild now reports 0 warnings (was 6: two per tvOS
slice), and swift build, swift test (170 tests), and xcodebuild for the
macOS, iOS and tvOS schemes all pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:11:49 +02:00
enricobuehlerandClaude Fable 5 ac3dc4323f fix(native/session): make the stop flag enforceable so a session can't outlive its client
Native sessions could survive long after the client was gone, in two
independent ways.

HOST: `stop` was only advisory. The one thing that ends a session is the
stream thread returning — every teardown (conn.close, the joins, and the
RAII drops of the session permit, admission entry and stream marker)
sits after that await, and nothing forced the issue. The encode loop
checks `stop` between iterations, so any unbounded call INSIDE one never
reaches the check, and one stuck syscall became a permanent zombie: it
held its semaphore slot (four of those and the host stops accepting
QUIC entirely), its admission entry (a later client gets "host busy"
forever), and not even the console's Stop button could clear it — that
button sets this same flag.

  * Bound the wait: once the session has been told to stop, the thread
    gets STREAM_STOP_GRACE (90 s, well past the 40 s capture-rebuild
    budget) to return, then teardown runs anyway. The thread is detached,
    not killed — Rust can't cancel a blocking thread — so it keeps its
    capturer/encoder until the stuck call returns, but the session's slot
    and admission entry come back and the host keeps serving. It logs at
    ERROR as the host wedge it is.
  * Bound the audio/input joins too — the last unbounded await in
    teardown.
  * Take the session permit AFTER the QUIC handshake instead of before
    `accept()`, so a host at its concurrency cap still accepts and the
    waiting client sees a live path instead of a silent dial timeout.
  * Bound the compositor helpers that caused the wedge in the first
    place: new pf-vdisplay `proc::{status_within, output_within}` kill a
    child that outlives its budget. `kscreen-doctor` is a Wayland client
    of the very compositor it configures, so against a wedged KWin it
    never returned; same for systemctl/dbus against a stuck session bus.

CLIENTS: the connection was never closed, so the host was right to keep
the session — it still had a live, keep-alive-answering peer.

  * Android: backgrounding did no teardown at all, and Android doesn't
    suspend the process, so the worker kept answering keep-alives until
    the OS reclaimed it (on a TV box, never). End the session on ON_STOP,
    via the existing onDispose path; a plain close, not a quit, so the
    host lingers the display for a fast return.
  * Apple: the .background arm was iOS-only AND gated on an opt-in that
    defaults off, so backgrounding did nothing — while the `audio`
    background mode kept the app (and its connection) alive indefinitely.
    Act unconditionally, and cover tvOS.
  * Core: `conn.close()` only queues the frame, and run_pump is the body
    of a block_on whose runtime is dropped the instant it returns, so the
    driver could never put it on the wire — a deliberate quit reached the
    host as silence (8 s idle timeout, no quit code, and the linger meant
    for an unwanted disconnect). Carry the endpoint out of the handshake
    and flush with wait_idle(), the same discipline the pairing and probe
    paths already use.

Linux check/clippy/tests green: 262 host, 71 pf-vdisplay (incl. new
bounded-process tests), 231 core.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:11:49 +02:00
enricobuehlerandClaude Fable 5 41fa25c440 fix(gamestream/session): end the session when the client disconnects or vanishes
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m6s
apple / swift (push) Successful in 1m24s
decky / build-publish (push) Successful in 31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 18s
ci / bench (push) Successful in 7m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m8s
deb / build-publish (push) Successful in 9m35s
arch / build-publish (push) Successful in 12m19s
deb / build-publish-host (push) Successful in 12m46s
android / android (push) Successful in 16m5s
windows-host / package (push) Successful in 16m34s
apple / screenshots (push) Successful in 6m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m11s
ci / rust (push) Successful in 27m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m19s
docker / deploy-docs (push) Successful in 27s
GameStream sessions outlived their client: the only complete teardowns
were the explicit ones (RTSP TEARDOWN, nvhttp /cancel, mgmt DELETE
/session), and the only automatic detector was a media-UDP send error —
which needs an ICMP port-unreachable, so a true vanish (Wi-Fi drop,
sleep, power off, crash) left video+audio encoding into the void
forever, and even a plain Moonlight quit (which sends neither TEARDOWN
nor /cancel) leaked the session. The stale state then cascaded: a
lingering launch 503-blocked a different client under
mode_conflict=reject, and streaming=true made a reconnect's PLAY take
its "stream already running" branch — no new threads, old threads still
aimed at the dead endpoint, the reconnect got no media.

ENet already detects all of this — the control peer's reliable-ping
timeout (or clean disconnect) fires Event::Disconnect within ~5-30 s —
but the handler only reset input state. Wire the real teardown into it:

* AppState::end_session — THE compat-plane session teardown: stops both
  media threads (their flags), clears launch + negotiated stream config;
  idempotent. /cancel and mgmt stop_session now share it.
* control.rs Disconnect → end_session. Gated on the TRACKED session
  peer, and Connect only tracks a peer from the /launch owner's IP (the
  same source-IP bind the RTSP/media plane uses), so an unauthenticated
  LAN peer connect+disconnect can't end a live session, and a fast
  reconnect's stale-peer timeout can't kill its successor.
* Client-unreachable UDP send errors now end the whole session via an
  OnSessionLost callback (built at PLAY) instead of stopping only the
  plane that noticed — audio no longer keeps streaming after video
  detects the dead client, and vice versa.

Linux check/clippy/tests green (53 gamestream tests incl. the new
end_session regression test).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:10:23 +02:00
enricobuehlerandClaude Opus 4.8 3a33a69401 fix(kwin): select the custom mode KWin actually built, not the one we asked for
apple / swift (push) Successful in 1m19s
apple / screenshots (push) Successful in 6m41s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m3s
ci / bench (push) Successful in 5m33s
android / android (push) Successful in 11m38s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
deb / build-publish (push) Successful in 9m6s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m43s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
deb / build-publish-host (push) Successful in 9m50s
arch / build-publish (push) Successful in 21m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m27s
docker / deploy-docs (push) Failing after 15s
ci / rust (push) Successful in 28m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m21s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m49s
A KDE host streaming to a phone-shaped client fell back to 60 Hz even though the
120 Hz mode was sitting right there in KDE's display list, one manual click away.

KWin generates every custom mode's timing with libxcvt, whose first step is
`hdisplay_rnd = hdisplay - (hdisplay % 8)`. A 2868x1320@120 request (iPhone 16
Pro Max) therefore becomes 2864x1320@119.92 — 4 px narrower, fractional refresh.
We then asked kscreen-doctor for `mode.2868x1320@120`; its `findMode` matches a
mode's id or its own `WxH@qRound(Hz)` name, so that string matched nothing, the
select silently no-op'd, the output stayed on its sacrificial birth mode, and
`size_applied` came back false → "KWin rejected the custom mode" → 60 Hz.
Widths like 1920/2560/3840 are all multiples of 8, which is why only clients with
phone-shaped panels ever hit this.

Resolve the mode out of the output's OWN list instead and address it by kscreen
mode id: exact height, width at most one cell narrower than asked, refresh within
1 Hz (which excludes the native 60 Hz entry). `set_custom_refresh` now returns the
whole achieved mode, and `create` reports that as the output's `preferred_mode` —
so the capturer's renegotiation gate waits for the size KWin will actually deliver
and the encoder opens against it, rather than starving on the requested one.

Also skip `addCustomMode` when a usable mode is already installed: kscreen-doctor
APPENDS to the list and KWin persists it per output name, so the old code grew the
user's display list by one entry per connect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:55:36 +02:00
enricobuehlerandClaude Opus 4.8 24a24734eb fix(inject/windows): follow the input desktop for pen + touch injection
apple / swift (push) Successful in 1m16s
apple / screenshots (push) Successful in 6m43s
android / android (push) Failing after 7m44s
ci / web (push) Successful in 2m3s
ci / docs-site (push) Successful in 1m14s
arch / build-publish (push) Successful in 12m50s
ci / bench (push) Successful in 6m30s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 16s
deb / build-publish (push) Successful in 9m13s
ci / rust (push) Successful in 19m44s
deb / build-publish-host (push) Successful in 9m30s
docker / deploy-docs (push) Successful in 26s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 25m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 25m10s
Same field report as the display-write fix, other half of the symptom: with a
UAC consent prompt up — one the user could SEE in the stream, because capture
already renders the secure desktop (326d6e17) — pen and touch did nothing,
while mouse and keyboard kept working.

The split was exactly which paths knew the input desktop can move. sendinput.rs
reattaches, so mouse/keyboard reached the prompt. pointer_windows.rs had no
desktop handling at all, so every InjectSyntheticPointerInput came back
ERROR_ACCESS_DENIED:

    22:20:31  virtual pen created (PT_PEN)
    22:20:31  pen inject failed    error=Zugriff verweigert (0x80070005)
    22:20:42  touch inject failed  error=Zugriff verweigert (0x80070005) contacts=1

Measured on glass before fixing, to find out what it actually takes:

    device on Default, thread on Default  -> 0x80070005   (the field failure)
    device on Default, thread on INPUT    -> OK
    device on INPUT,   thread on INPUT    -> OK

The middle row is load-bearing: the synthetic pointer device is NOT
desktop-affine, so rebinding the thread suffices and the device is never
destroyed and recreated across a desktop switch — which would have dropped
in-flight contacts and the pen's in-range state mid-stroke.

Injection now retries once bound to the input desktop. The binding is scoped,
not persistent like sendinput's: inject_pen/inject_touch_frame run on TWO
threads (the caller's apply_batch and the refresh threads), and the batch caller
is a shared task thread that must not be left parked on a Winlogon desktop that
disappears when the prompt is dismissed.

The first-failure WARN now carries the rejected sample (flags, pen flags/mask,
pressure, rotation, tilt, position). A 0x80070057 INVALID_PARAMETER was seen
once BEFORE any prompt existed and is still unexplained; the ranges all look
sound on inspection (coordinates clamped to the virtual screen, roll/azimuth
u16 so the modulo cannot go negative, tilt bounded 0..=90), so catching the
actual offending sample is the way to find it.

Verified on glass: with a consent prompt up, pen and touch now reach it — zero
inject failures where every prior session failed immediately. A UAC prompt can
now be dismissed from an iPad with the Pencil.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:36:51 +02:00
enricobuehlerandClaude Opus 4.8 b5fa878bc6 fix(pf-win-display): follow the input desktop so a UAC prompt can't refuse display writes
Field-reported 2026-07-23: a consent prompt left up on an unattended host made
the box unreachable. Every connect black-screened and failed after ~60 s, and
the host had to be reached over SSH to clear the prompt.

Windows refuses ChangeDisplaySettingsEx/SetDisplayConfig issued from a thread
that is not on the desktop currently receiving input. While UAC (or the lock /
logon screen) is up that desktop is Winlogon, and the host — which the service
launches explicitly onto WinSta0\Default — got DISP_CHANGE_FAILED /
ERROR_ACCESS_DENIED for every write. A session starting in that window could
never set its virtual display's mode, so the capturer sized its ring to a stale
mode, every composed frame was dropped for a size mismatch, and bring-up burned
8 retries before giving up.

Measured on glass (RTX box, SYSTEM host in console session 2, real consent
prompt up, virtual display active):

    INPUT desktop = Winlogon
    UNBOUND  CDS_TEST      -> -1 (DISP_CHANGE_FAILED)
    UNBOUND  SDC_VALIDATE  -> 0x5 (ERROR_ACCESS_DENIED)
    BOUND    CDS_TEST      ->  0 (DISP_CHANGE_SUCCESSFUL)
    BOUND    SDC_VALIDATE  -> 0x0 (ERROR_SUCCESS)

New input_desktop.rs mirrors pf-inject's sendinput.rs retry model: issue the
write, and only when it fails the way a wrong-desktop write fails, rebind this
thread to the current input desktop and retry once. A working write is never
touched, so the normal path is unchanged. The retry predicate stays narrow
(ERROR_ACCESS_DENIED only) so the unrelated 0x57 exclusive-mode topology bug is
not re-issued and mis-attributed. Unlike sendinput's dedicated injector thread,
the binding here is SCOPED — a shared display-write thread left on a Winlogon
desktop that is later destroyed would fail every subsequent write, which is the
wedge this removes.

Applied to all eight SetDisplayConfig sites and both ChangeDisplaySettingsExW
calls in set_active_mode. The isolate site now decides its supplied config once,
outside the write, so a retry re-issues the identical config rather than logging
its escalation twice.

A save is logged, because a silent one is indistinguishable in a field log from
a write that never needed saving — which made the first on-glass verification of
this change inconclusive.

Both ACCESS_DENIED diagnostics now ask which cause applies instead of naming
only the disconnected-RDP one: that phrasing sent this investigation chasing a
phantom RDP session on a host that was already service-launched in the console
session, while a consent prompt was the actual cause.

Verified on glass: connect with a consent prompt up now reaches first frame in
3.0 s (was a black screen then "Connection failed" after ~60 s), with six
"retried bound to it and it applied desktop=Winlogon" lines and rc=0x0
throughout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:36:51 +02:00
enricobuehler 4b2d2d1e14 fix(gamescope/cursor): follow gamescope's own cursor verdict — the corner-parked pointer
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 58s
apple / swift (push) Successful in 1m23s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m36s
apple / screenshots (push) Successful in 6m34s
deb / build-publish (push) Successful in 9m41s
deb / build-publish-host (push) Successful in 10m13s
arch / build-publish (push) Successful in 12m43s
android / android (push) Successful in 16m41s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m41s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m4s
ci / rust (push) Successful in 29m44s
gamescope hides its pointer by WARPING the X pointer to the root's bottom-right corner pixel;
it does not swap in a transparent X cursor, so XFixesGetCursorImage keeps handing back the last
opaque arrow. The XFixes source picked its display by "whichever pointer moved", and a parked
pointer never moves again — so it froze on the parked Xwayland and composited that arrow at
(w-1, h-1): a sliver of cursor welded to the corner of the stream for the rest of the session
while the real pointer went undrawn. Reported on-glass in every game (a game grabs the pointer,
so the hide is permanent): "part of cursor shows up on bottom right ... isn't where it really is".

Follow GAMESCOPE_CURSOR_VISIBLE_FEEDBACK instead — gamescope publishes it on every nested
Xwayland root: 1 on the server whose pointer it is drawing, 0 on the others and 0 on all of them
once the pointer is hidden. It answers both questions correctly for a STATIC pointer, which
motion cannot: which Xwayland owns it, and whether to draw it at all. Read at connect, re-read
on its root PropertyNotify (event drain now discriminates CursorNotify vs PropertyNotify) with a
250 ms resync as the self-heal. A pointer gamescope draws nowhere is published visible:false,
not dropped — the encode loop overwrites the frame's overlay from this slot and strips invisible
ones, so a None would leave the last visible overlay standing on repeat frames. Honouring the
atom also gives the stream gamescope's own idle auto-hide, which this source never had.

The pointer-motion heuristic stays as the fallback for a gamescope that publishes no verdict
(logged at session start via cursor_feedback=false), so nothing regresses to a cursorless stream.

Measured on a live 1920x1080 Steam Gaming Mode session (RTX 5070 Ti, gamescope c31743d+,
--xwayland-count 2 --hide-cursor-delay 3000): idle 3 s => pointer (1919,1079) + feedback 0;
real evdev motion => pointer live + feedback 1. End-to-end against that session the source now
publishes visible=false while parked and tracks the live pointer within one 250 ms sample of
motion. cargo clippy -p pf-capture --lib -D warnings and the 5 new pick_active tests are green
at cargo 1.96.1 (CI parity); cargo fmt --all --check clean.
2026-07-24 00:31:55 +02:00
enricobuehlerandClaude Opus 4.8 bc5f6a3881 fix(apple/session): keep the display awake for the length of a session
ci / web (push) Successful in 50s
apple / swift (push) Successful in 1m19s
ci / docs-site (push) Successful in 1m58s
decky / build-publish (push) Successful in 33s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 54s
deb / build-publish (push) Failing after 4m47s
ci / bench (push) Successful in 6m1s
docker / deploy-docs (push) Successful in 33s
release / apple (push) Successful in 10m4s
android / android (push) Successful in 13m5s
deb / build-publish-host (push) Successful in 12m52s
arch / build-publish (push) Successful in 16m56s
apple / screenshots (push) Successful in 6m33s
ci / rust (push) Successful in 19m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m27s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m15s
A stream is not user activity to the OS, and controller input does not feed
the HID idle timer on any Apple platform — so a gamepad-only session reliably
idles the panel out from under the user mid-play. A keyboard/mouse capture
session masks the bug because those events are real local HID. The Android
client has held FLAG_KEEP_SCREEN_ON while streaming all along; the Apple
clients held nothing.

DisplaySleepGuard is acquired in beginStreaming and released at the top of
disconnect, so it is scoped to the session — every teardown path (user quit,
sessionEnded, the backgrounded keep-alive timeout) funnels through disconnect.

  iOS/iPadOS/tvOS: UIApplication.isIdleTimerDisabled. App-wide and ignored
  while backgrounded, which is what the audio-only keep-alive wants.

  macOS: ProcessInfo.beginActivity(.userInitiated, .idleDisplaySleepDisabled),
  the Foundation wrapper over IOKit power assertions. That defers display
  sleep but NOT the screen saver, which runs off the HID idle timer
  independently — so a 30 s IOPMAssertionDeclareUserActivity heartbeat runs
  alongside it. Intended side effect: an idle-lock that follows the screen
  saver is deferred for the session too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:00:55 +02:00
enricobuehlerandClaude Fable 5 6617275387 docs(env): stop teaching the compositor pin + uid-1000 anchors in starters
ci / docs-site (push) Successful in 54s
ci / web (push) Successful in 59s
apple / swift (push) Successful in 1m21s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 25s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
ci / bench (push) Successful in 6m23s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m43s
apple / screenshots (push) Successful in 6m18s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m9s
arch / build-publish (push) Successful in 13m44s
deb / build-publish (push) Successful in 13m48s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7m39s
deb / build-publish-host (push) Successful in 13m11s
docker / deploy-docs (push) Successful in 32s
android / android (push) Successful in 16m23s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m15s
ci / rust (push) Successful in 24m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m55s
Field triage (Nobara, Discord): the kde.md starter host.env told desktop
users to set PUNKTFUNK_COMPOSITOR=kwin, which PINS the backend — detect()
short-circuits and the capture-loss rebuild never re-detects — so a
mid-stream switch to Game Mode killed the stream instead of following it.
A follow-up hardcoded XDG_RUNTIME_DIR=/run/user/1000 anchor broke PipeWire
for any non-1000 uid (pw audio connect: Creation failed).

Revamp across every starter/example/reference:
- Desktop starters (kde/gnome/hyprland/sway) shrink to
  PUNKTFUNK_VIDEO_SOURCE=virtual + an explicit warning that pinning
  disables session-following; forcing a backend is CI/appliance-only.
- host.env.example: rewritten around auto-detection; anchors demoted to a
  commented ssh/cron-only block with the uid trap spelled out; the
  gamescope ATTACH/MANAGED knobs documented (previously missing);
  case-sensitivity called out.
- packaging/bazzite/host.env + README: drop the uid-1000 anchors (a
  systemctl --user service inherits/derives them); README's stale
  PUNKTFUNK_COMPOSITOR=gamescope-era template synced to the real one.
- packaging/kde/host.env: loud APPLIANCE-ONLY header (it pins on purpose).
- configuration.md: session-anchors section inverted to "leave unset",
  compositor row states the pin consequence, case-sensitivity note.
- troubleshooting.md: new "session fails right after editing host.env"
  section (case, wrong-uid anchors, stale pin, restart-to-apply).
- gamescope.md/bazzite.md: attach/managed descriptions match current
  behavior (managed is the infra-detected default; attach re-modes a
  box-owned session to the client's resolution).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 22:42:59 +02:00
enricobuehlerandClaude Opus 4.8 bda015b101 docs(release): rewrite v0.19.0 notes for end users + set end-user notes voice
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m8s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 15s
apple / screenshots (push) Successful in 6m29s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m1s
ci / bench (push) Successful in 8m1s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 18s
deb / build-publish-host (push) Successful in 10m19s
deb / build-publish (push) Successful in 12m13s
android / android (push) Successful in 13m20s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m26s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 16m53s
ci / rust (push) Successful in 25m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m52s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m8s
Release notes were written for the people who build Punktfunk, not the people who use it —
dense with protocol/ABI/type names that confused even technical readers. Rewrite v0.19.0 in a
benefit-first, plain-language voice (New/Improved/Fixed) with all internal terms removed from the
body and every protocol/ABI/embedder detail moved to a single bottom "Under the hood" section.
docs/releases/README.md now codifies this voice as the format spec for all future notes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:15:08 +02:00
enricobuehlerandClaude Opus 4.8 ec219763a6 fix(ci/release): make _release_notes_path POSIX-sh safe (deb/decky attach runs under dash)
ci / web (push) Successful in 55s
apple / swift (push) Successful in 1m26s
ci / docs-site (push) Successful in 56s
android-screenshots / screenshots (push) Successful in 3m20s
ci / bench (push) Successful in 6m47s
release / apple (push) Successful in 10m49s
android / android (push) Successful in 15m8s
decky / build-publish (push) Successful in 26s
apple / screenshots (push) Successful in 6m40s
deb / build-publish (push) Successful in 9m36s
deb / build-publish-host (push) Successful in 9m39s
flatpak / build-publish (push) Successful in 6m33s
arch / build-publish (push) Successful in 15m57s
ci / rust (push) Successful in 26m4s
docker / deploy-docs (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
web-screenshots / screenshots (push) Successful in 3m10s
windows-host / package (push) Successful in 10m33s
linux-client-screenshots / screenshots (push) Successful in 7m51s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m12s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m57s
The v0.19.0 deb + decky release-attach steps source gitea-release.sh under `sh` (dash) and
died with "Bad substitution" at _release_notes_path's `${BASH_SOURCE[0]:-$0}` — a bash array
subscript dash rejects. So those legs never attached their assets to the release (the bash
legs — apple, android, rpm — were fine, which is why the body still seeded and the DMG/.ipa
attached). CI always sources this from the repo root, so resolve the notes as
docs/releases/<tag>.md relative to CWD and drop $BASH_SOURCE entirely. Verified under dash.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 19:03:01 +02:00
enricobuehlerandClaude Opus 4.8 0dce1cd116 chore(release): bump workspace version to 0.19.0
audit / bun-audit (push) Successful in 11s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
apple / swift (push) Successful in 1m20s
audit / cargo-audit (push) Successful in 2m13s
ci / bench (push) Successful in 6m52s
android-screenshots / screenshots (push) Successful in 3m58s
release / apple (push) Successful in 11m11s
deb / build-publish (push) Failing after 8m56s
decky / build-publish (push) Failing after 24s
deb / build-publish-host (push) Failing after 9m15s
android / android (push) Successful in 13m6s
ci / rust (push) Successful in 25m44s
arch / build-publish (push) Successful in 13m37s
docker / deploy-docs (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
apple / screenshots (push) Successful in 6m32s
web-screenshots / screenshots (push) Successful in 3m2s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 2m43s
flatpak / build-publish (push) Failing after 8m20s
linux-client-screenshots / screenshots (push) Successful in 7m46s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m45s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m4s
windows-host / package (push) Successful in 15m54s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m26s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m49s
Headline: the whole pen/tablet input stack (P0–P5) — a pressure-sensitive stylus plane
across core wire, Linux, GameStream/Moonlight, Windows, iPad Pencil and Android. Plus
touch-injection fixes, cursor-DPI scaling, three Windows-host display/cursor fixes, a
gamepad same-PID conflict fix, and the Android mouse/keyboard (TV) regressions.

Release notes authored in-repo at docs/releases/v0.19.0.md (first release on the new flow;
CI seeds the release body from it). cargo update --workspace syncs the lock (versions only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:26:49 +02:00
enricobuehlerandClaude Opus 4.8 c3c0115a82 fix(core/abi): bump ABI_VERSION to 13 for punktfunk_connection_send_pen
The pen P0 work added a new embeddable C entry point (punktfunk_connection_send_pen)
but left ABI_VERSION at 12 — the value 0.18.0 shipped. Per the project's convention
(0.18.0 bumped to 12 for the additive cursor channel), additive C-surface growth bumps
the ABI so embedders can gate on the new symbol. Regenerated include/punktfunk_core.h
via cbindgen (cargo build -p punktfunk-core); WIRE_VERSION is unchanged (pen is
capability-gated on HOST_CAP_PEN).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:23:38 +02:00
enricobuehlerandClaude Opus 4.8 00c8c12a0f feat(ci/apple): export the iOS .ipa to the release + run artifacts
The iOS leg archives with development signing and re-signs for App Store distribution at
export time, but the existing export uses destination=upload (straight to TestFlight) and
leaves no .ipa on disk. Add an export step that re-exports the SAME archive with
destination=export to produce an App Store distribution-signed .ipa, then:
  * attach it as the `punktfunk-ios-ipa` run artifact on every build (upload-artifact@v3,
    since Gitea's backend rejects @v4), and
  * attach it to the unified Gitea release on vX.Y.Z tags (alongside the DMG), via the
    shared ensure_release/upsert_asset helpers.

Same gate as the archive; warn+skip (never fails the best-effort iOS leg) if the archive is
absent. Note: an App Store-signed .ipa installs only via TestFlight/App Store, not by direct
sideload — it is a release/archival artifact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:18:45 +02:00
enricobuehlerandClaude Fable 5 988b5742ec fix(gamestream): tolerate NaN pressureOrDistance — VoidLink's finger touches were dropped whole
apple / swift (push) Successful in 1m18s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m6s
apple / screenshots (push) Successful in 6m11s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
windows-host / package (push) Successful in 16m57s
android / android (push) Successful in 15m58s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 17m3s
deb / build-publish (push) Successful in 9m26s
deb / build-publish-host (push) Successful in 10m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m39s
ci / rust (push) Successful in 25m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m31s
The hex dumps from the on-glass session show textbook SS_TOUCH packets with
pressureOrDistance = 0x7fc00000 (NaN): VoidLink encodes 'unknown finger
pressure' as NaN (iPad fingers have no force sensor), and the anti-forgery
finiteness gate rejected the entire packet — silently disabling the touch
plane while pen (real Pencil force, always finite) worked. Coordinates keep
the strict gate (they feed injector scaling); pressureOrDistance now
sanitizes non-finite to 0.0, the spec's own 'unknown', which the pen path
already maps to full-scale ink on contact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 f938174d86 debug(gamestream): hex-dump the first few undecodable pointer packets
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 40611cd54d debug(gamestream): surface the pointer plane's silent failure modes
An on-glass 'touch does nothing' currently leaves NO trace: log the first
SS_TOUCH per session at info (proves the client sends + we decode), and warn
when a packet carrying a pointer magic fails the body parse (layout mismatch)
instead of vanishing into the unknown-magic drop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 98b97d7d76 fix(android/input): stop mouse back/forward leaking to nav + IME popping on hardware typing
apple / swift (push) Successful in 1m25s
apple / screenshots (push) Successful in 7m4s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m14s
android / android (push) Successful in 11m54s
arch / build-publish (push) Successful in 11m58s
ci / bench (push) Successful in 5m36s
decky / build-publish (push) Successful in 24s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
deb / build-publish (push) Successful in 11m52s
docker / deploy-docs (push) Successful in 11s
deb / build-publish-host (push) Successful in 10m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m15s
ci / rust (push) Successful in 26m1s
Two Android TV M&K regressions from the c90343c2 overhaul, both user-reported:

1. Mouse back/forward buttons doubled as Android navigation while streaming.
   The BUTTON_BACK/FORWARD motion edges already go over the wire as X1/X2, but
   Android's input reader ALSO synthesizes KEYCODE_BACK/FORWARD key events from
   those buttons unconditionally (stamped SOURCE_MOUSE, no FLAG_FALLBACK) — and
   the guard only swallowed the FLAG_FALLBACK variant, so the duplicate reached
   the BackHandler and yanked the user out of the stream. Swallow mouse-sourced
   BACK/FORWARD too; a remote/keyboard BACK is never mouse-sourced and still
   exits.

2. The soft keyboard popped up on physical-keyboard typing. KeyCaptureView (the
   always-focused IME/capture anchor) unconditionally claimed to be a text
   editor with a live editable InputConnection, so the TV IME counted input as
   active and showed its UI the moment hardware keys arrived. Gate the editor
   identity on imeShown (the user actually summoning the keyboard via gesture /
   remote toggle), with restartInput on both edges so the gate is re-read, and
   an onKeyPreIme BACK hook so an IME self-dismissal can't leave the gate
   stale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:07:18 +02:00
enricobuehlerandClaude Fable 5 51792536d1 fix(pf-win-display): escalate CCD isolate to a keep-only supplied config on 0x57
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 1m8s
apple / screenshots (push) Successful in 6m59s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 14m35s
arch / build-publish (push) Successful in 14m20s
decky / build-publish (push) Successful in 21s
ci / bench (push) Successful in 6m4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
deb / build-publish (push) Successful in 9m41s
ci / rust (push) Successful in 20m14s
deb / build-publish-host (push) Successful in 11m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m42s
docker / deploy-docs (push) Successful in 12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m2s
Field bug (Steam Deck client, exclusive topology): every isolate retry failed
SetDisplayConfig 0x57 with the physical panel left lit, while iPhone sessions
worked — their virtual monitor's connected-set already had a sole-display
topology persisted, so Windows deactivated the physical on hotplug and the
isolate was a no-op re-commit (deactivated 0). The Deck's monitor identity had
no such entry, forcing the live-deactivate path: supplying the doomed path
(inactive, modes unpinned) plus its orphaned mode entries is rejected 0x57 by
some driver/OS validation combos, so the retry loop could never converge — the
prior mode-index unpin (5e088af7) was necessary but not sufficient on this box.

Attempt 1 keeps the shipped shape; once a survivor is seen, later attempts
supply ONLY the keep paths with the mode table rebuilt to just their referenced
entries (indexes remapped, clone-shared entries deduped). The same host applies
exactly this keep-only shape rc=0 whenever the DB has pre-isolated the virtual
display, so the escalated shape is field-validated on the failing machine. The
final attempt also drops SDC_FORCE_MODE_ENUMERATION — a real path removal
drives COMMIT_MODES on its own.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:50:44 +02:00
enricobuehlerandClaude Fable 5 bf2c4456bb fix(inject/windows): compact wire touch ids into slots — Moonlight native touch injected nothing
apple / swift (push) Successful in 1m28s
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m12s
release / apple (push) Successful in 9m24s
ci / bench (push) Successful in 6m28s
android / android (push) Successful in 11m59s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 12m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
apple / screenshots (push) Successful in 6m53s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 19s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 17s
deb / build-publish (push) Successful in 9m36s
deb / build-publish-host (push) Successful in 11m24s
ci / rust (push) Successful in 25m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m53s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m59s
docker / deploy-docs (push) Successful in 25s
On-glass report (iPad Moonlight vs the Windows host): pen inked, native touch
did nothing. Cause: POINTER_INFO.pointerId was the client's raw wire id —
Moonlight sends arbitrary large pointerIds, and synthetic-pointer injection
rejects them (Apollo compacts ids into dense slots for exactly this reason);
the failure was logged at trace, i.e. invisibly. Wire ids now map to the
lowest free slot for the contact's lifetime, and the FIRST injection failure
of a device logs at WARN (pen too) so an inert input plane is never silent
again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Fable 5 c0f792ee8d feat(android): pen P5 — stylus capture onto the pen plane
The Android leg of design/pen-tablet-input.md §7: against a HOST_CAP_PEN host,
stylus/eraser pointers split out of BOTH touch models (passthrough + gesture)
into StylusStream — state-full samples with pressure, AXIS_TILT, azimuth from
AXIS_ORIENTATION (Android's 0 = away-from-user IS wire north), AXIS_DISTANCE
hover, both stylus barrel buttons, the eraser tool, and historical (coalesced)
samples batched oldest-first. Kotlin heartbeats ≤100ms per the wire contract.
JNI: nativeHostSupportsPen + nativeSendPen (flat 10-float stride, sentinels
<0). No barrel-roll axis exists on Android — roll stays unknown here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Fable 5 c8aa3ed48b feat(apple/ios): pen P4 — Apple Pencil (Pro) capture onto the stylus plane
The flagship leg of design/pen-tablet-input.md §7: against a HOST_CAP_PEN host
the Pencil splits out of the finger path (independent of touch-input mode) into
PencilStream — state-full samples with force→pressure, altitude→polar tilt,
azimuth (+90° to the wire's north-clockwise convention), Pencil Pro rollAngle→
barrel roll (17.5+), coalesced touches batched ≤8/send for full 240Hz fidelity,
and Pencil hover (zOffset>0 distinguishes it from trackpad hover) as in-range
samples with distance. Squeeze holds barrel 1, double-tap clicks barrel 2 (the
Pencil has no hardware buttons/eraser — these are how host apps get those
affordances). Implements the ≤100ms heartbeat wire contract (80ms timer) so a
stationary held stroke survives the host's 200ms dead-client failsafe. Toward a
pen-less host nothing changes (Pencil stays a finger).

Also fixes hostSupportsCursor testing the STALE 0x04 bit — HOST_CAP_CURSOR
moved to 0x08 when TEXT_INPUT claimed 0x04; the old test would mistake a
text-input host (e.g. Windows) for a cursor grant and double-render the pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Opus 4.8 15d09a8c10 feat(ci/release): author release notes in-repo + announce stable releases to Discord
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m25s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 6m55s
apple / screenshots (push) Successful in 6m47s
deb / build-publish (push) Successful in 11m23s
android / android (push) Successful in 12m36s
arch / build-publish (push) Successful in 12m46s
docker / deploy-docs (push) Successful in 23s
deb / build-publish-host (push) Successful in 12m32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m9s
ci / rust (push) Successful in 26m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m30s
Release notes now live in the repo at docs/releases/vX.Y.Z.md, authored during the
version bump (before the tag). ensure_release (gitea-release.sh + .ps1 twin) seeds the
Gitea release body from that file in the create POST, so the release is born WITH its
notes instead of being created empty and PATCHed afterward. canary/rc have no file ->
empty body, unchanged.

New manual `announce` workflow (workflow_dispatch, tag input) re-syncs the notes file
over the live release and posts a violet embed (title, notes lead-in, release-page link)
to the Discord #releases channel via the DISCORD_RELEASE_WEBHOOK secret. Stable-only:
a -rc tag is refused unless allow_prerelease=true. Pressing "go" is the quality gate, so
a half-built or failed release is never announced.

docs/releases/README.md documents the ritual; TEMPLATE.md is the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:43:18 +02:00
enricobuehlerandClaude Opus 4.8 6cbc3eec8b fix(sdl/cursor): scale the forwarded pointer to the video fit
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m3s
android / android (push) Canceled after 1m27s
apple / screenshots (push) Canceled after 1m39s
arch / build-publish (push) Canceled after 1m29s
ci / rust (push) Canceled after 1m29s
ci / bench (push) Canceled after 1m17s
deb / build-publish (push) Canceled after 1m15s
deb / build-publish-host (push) Canceled after 27s
decky / build-publish (push) Canceled after 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 15s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 1m18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 18s
flatpak / build-publish (push) Canceled after 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / deploy-docs (push) Canceled after 0s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 17s
Parity with the Apple client (35c2c145): the SDL/Linux presenter built the
host-forwarded cursor as an OS cursor at its native bitmap size, so the
on-screen pointer tracked the HOST's display scaling — a 4K/high-DPI virtual
display forwards a 96 px bitmap that then rendered huge on the client.

SDL cursors are fixed-size from their surface (no draw-time scaling), so
cache the shapes RAW and resample each by the live video-fit factor
(min(window_px / mode) — the same aspect-fit `finger_to_content` uses) at
install time, rebuilding when the serial OR the fit changes (a resize). The
resample area-averages on premultiplied alpha so a transparent pixel's
colour can't bleed into the fringe. The pointer now stays sized to the
streamed desktop at any host scaling.

Not yet eyeballed on a Deck/Linux box — SDL's HiDPI cursor sizing is
version-dependent; validate on glass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:28:34 +02:00
enricobuehlerandClaude Fable 5 45c9799aa6 feat(inject/windows): pen P3 — PT_PEN + PT_TOUCH synthetic pointer injection
apple / swift (push) Successful in 1m26s
ci / web (push) Successful in 59s
ci / docs-site (push) Successful in 54s
release / apple (push) Successful in 9m40s
arch / build-publish (push) Successful in 12m4s
ci / bench (push) Successful in 7m5s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 22s
apple / screenshots (push) Successful in 6m50s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
android / android (push) Successful in 14m47s
ci / rust (push) Canceled after 13m52s
deb / build-publish (push) Canceled after 7m26s
deb / build-publish-host (push) Canceled after 5m38s
flatpak / build-publish (push) Canceled after 1m55s
docker / deploy-docs (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 1m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 1m59s
windows-host / package (push) Canceled after 8m32s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
The Windows leg of design/pen-tablet-input.md §6, following Apollo's recipe:
a per-session PT_PEN device (pressure rescaled 0..1024, polar tilt→tiltX/Y,
barrel roll on rotation — Windows Ink renders Pencil Pro roll natively, barrel
button, eraser via INVERTED/ERASER flags, hover) with a 40ms refresh thread
against the ~100ms synthetic-pointer staleness auto-lift, plus a PT_TOUCH
device closing the historical SendInput wire-touch no-op (full active-contact
frames, per-id DOWN/UP edges, self-healing lost-DOWN synthesis). pen_supported
now probes PT_PEN creation on Windows (1809+), which lights up HOST_CAP_PEN
and the GameStream featureFlags there — Moonlight iPad + Pencil can ink on a
Windows host. Frame grouping mirrors the Linux uinput backend.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 b192825869 feat(gamestream): pen P2 — SS_PEN/SS_TOUCH ingest, featureFlags advertised on capable hosts
The Moonlight leg of design/pen-tablet-input.md §4: DESCRIBE now advertises
SS_FF_PEN_TOUCH_EVENTS wherever pen injection exists (Linux+uinput, same gate
as HOST_CAP_PEN — elsewhere the flag stays 0 and Moonlight keeps client-side
mouse emulation exactly as today). SS_PEN packets merge over last-known state
into state-full PenSamples (BUTTON_ONLY per spec carries no position; unknown
contact pressure 0.0 inks at full scale; UP keeps proximity only for clients
that demonstrated hover) and drive the same PenTracker→VirtualPen chain as the
native plane — Moonlight iOS + Apple Pencil now exercises the full injection
path end to end. SS_TOUCH forwards as wire touch on a synthetic 65535² surface
with CANCEL_ALL replayed per tracked contact. No stroke timeout on this plane:
ENet is ordered/reliable and Moonlight doesn't heartbeat a stationary pen.
Packet layouts pinned against moonlight-common-c Input.h/Limelight.h upstream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 aca5aa9993 feat(host+inject): pen P1 — per-session uinput virtual tablet, HOST_CAP_PEN live
The Linux injection leg of design/pen-tablet-input.md: 0xCC/0x05 pen batches now
route through the per-session PenTracker into a lazily-created 'Punktfunk Pen'
uinput tablet (BTN_TOOL_PEN/RUBBER, pressure, tilt-from-polar, ABS_Z barrel
roll, hover distance, INPUT_PROP_DIRECT) — compositors pick it up via libinput
and hand apps zwp_tablet_v2 with full fidelity. The host now advertises
HOST_CAP_PEN when /dev/uinput is accessible (PUNKTFUNK_PEN=0 kill-switch);
transitions group into SYN frames so proximity-enter carries its position.
Stroke failsafe: clients heartbeat ≤100ms while in range (documented wire
contract — capture APIs are silent for a stationary pen); 200ms of silence
force-releases. 'punktfunk-host pen-test' draws a pressure-ramped sine stroke
through the real tracker→uinput chain, no client needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 248e1cbf08 fix(pf-vdisplay/driver): report cursor_excluded adapter-wide — the declare's exclusion is not per-target
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m20s
windows-drivers / probe-and-proto (push) Successful in 1m2s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
windows-drivers / driver-build (push) Successful in 2m23s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
ci / bench (push) Successful in 6m3s
apple / screenshots (push) Successful in 6m41s
deb / build-publish (push) Successful in 12m20s
arch / build-publish (push) Successful in 12m47s
deb / build-publish-host (push) Successful in 13m7s
android / android (push) Successful in 16m14s
windows-host / package (push) Successful in 15m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m8s
ci / rust (push) Successful in 26m6s
docker / deploy-docs (push) Canceled after 25s
Field report (iPad via GameStream, 2026-07-23): cursor missing from the
stream. The session's fresh target 257 reported cursor_excluded=false
(DECLARED_TARGETS is keyed per target and only 259 had declared), so the
host skipped forced-composite — but DWM's pointer exclusion after an
IddCx hardware-cursor declare reaches EVERY later monitor of the adapter,
not just the declaring target (proven on-glass: declare on 259 under
WUDFHost 2192, then 257's frames streamed pointer-free with CURSOR_SHOWING
set and SM_MOUSEPRESENT true). GameStream/Moonlight clients cannot draw a
forwarded cursor, so those sessions were silently cursor-less.

ADD replies now report any-declare-this-WUDFHost-life (the state's real
scope — it dies with the adapter reset, same as before); per-target ids
stay only as the dbglog audit trail. Channel-less sessions on a declared
adapter then hit the existing forced-composite gate (GDI poller + blend)
and carry the pointer in-video. Host code unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 15:58:54 +02:00
enricobuehlerandClaude Opus 4.8 35c2c1450e fix(apple/cursor): scale the forwarded pointer to the video fit
apple / swift (push) Successful in 1m24s
release / apple (push) Successful in 10m11s
apple / screenshots (push) Successful in 6m42s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m9s
ci / bench (push) Successful in 5m48s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 20s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 18s
android / android (push) Successful in 16m24s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
deb / build-publish (push) Successful in 10m11s
arch / build-publish (push) Successful in 12m18s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7m5s
docker / deploy-docs (push) Successful in 11s
deb / build-publish-host (push) Successful in 11m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m4s
ci / rust (push) Successful in 18m39s
The Mac client drew the host-forwarded cursor bitmap at its native pixel
size (pixel-as-points), so the on-screen pointer tracked the HOST's display
scaling: a 4K/high-DPI virtual display renders a 96 px pointer, which then
appeared huge on the client — and on retina every pointer was already
2x-inflated and upscaled-blurry, since the backing scale was ignored.

Cache the shape raw and rebuild the NSCursor at the live video-fit factor
(fit.width / mode.width — the same AVMakeRect fit the input mapping uses)
on each apply, re-fitting on resize / retina moves. The pointer now renders
at its true size relative to the streamed desktop at any host scaling:
crisp and 1:1 with the video at 100%, and proportional (not ballooning)
when the host sits at 200-300% — including a deliberately raised scale.

iOS/tvOS (UIPointerInteraction) and Android (no forwarded-bitmap path)
render the pointer differently and need no change; the SDL/Linux client has
the same native-size issue and will follow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:14:41 +02:00
enricobuehlerandClaude Fable 5 326d6e17c8 fix(windows/capture): stand the IddCx hardware cursor down while the secure desktop is up
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 6m41s
ci / web (push) Successful in 1m7s
ci / docs-site (push) Successful in 1m18s
android / android (push) Successful in 12m44s
arch / build-publish (push) Successful in 12m36s
decky / build-publish (push) Successful in 26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 20s
ci / bench (push) Successful in 6m14s
windows-host / package (push) Successful in 9m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
deb / build-publish (push) Successful in 9m38s
deb / build-publish-host (push) Successful in 9m58s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7m0s
ci / rust (push) Successful in 24m1s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m55s
docker / deploy-docs (push) Successful in 12s
0.18.0 regression: UAC consent and Winlogon (lock/logon) stopped appearing
in streams. The cursor channel's IddCx hardware-cursor declare — re-issued
by the driver on every swap-chain assign — keeps the path out of the OS's
software-cursor mode, which is the only mode the secure desktop renders
through; DWM then never presents UAC/Winlogon into our swap-chain and the
stream repeats the last normal-desktop frame for the whole interaction.

The GDI cursor poller now classifies the input desktop on its reattach
cadence (UOI_NAME != Default = secure, 250 ms cadence instead of 2 s), and
the capturer edge-triggers the existing-but-unused proto-v6
IOCTL_SET_CURSOR_FORWARD flip: OFF at secure entry (the driver stops its
per-assign re-declare; the host facade forces the same-mode re-commit that
actualises the software-cursor default, under the vdisplay manager lock via
the new force_recommit()), back ON at dismissal for channel sessions.
Channel-session open resets the driver's persisted desired state to ON so
a session that died mid-secure-desktop can't leave the next one adopting
UNdeclared (the §8.6 cross-session composite trap); orderly teardown does
the same. The flip closure is built for every Windows session — a
channel-less session can reuse a driver monitor whose earlier-session
cursor worker still re-declares, and NOT_FOUND from never-declared targets
is ignored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 14:13:15 +02:00
enricobuehlerandClaude Opus 4.8 588a7077e8 fix(host/gamepad): only degrade a virtual Steam pad on a same-PID conflict
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 1m0s
apple / swift (push) Successful in 1m26s
decky / build-publish (push) Successful in 32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 23s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 26s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
ci / bench (push) Successful in 6m10s
apple / screenshots (push) Successful in 6m38s
deb / build-publish (push) Successful in 9m26s
android / android (push) Successful in 12m27s
deb / build-publish-host (push) Successful in 13m55s
windows-host / package (push) Successful in 16m20s
arch / build-publish (push) Successful in 17m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m2s
ci / rust (push) Successful in 27m20s
docker / deploy-docs (push) Successful in 26s
The Steam-conflict gate matched any physical Valve `28DE` HID device, so a
Steam Machine with a physical Steam Controller 2 (`28DE:1302`) plugged in
degraded a client's requested virtual Steam Deck (`28DE:1205`) to a DualSense
— the reporter's controller passthrough came up as the wrong pad.

Steam Input drives *distinct* Steam controllers side by side; only two
*identical* `28DE` devices confuse it (the Deck-as-host case: built-in
`28DE:1205` + a second virtual Deck, observed live on Bazzite). Narrow the
gate to key on the exact VID+PID instead of the `28DE` vendor alone:

- `steam_backend_product()` maps each virtual Steam backend to the PID it
  enumerates as (Deck 1205, classic SC 1102, SC2 1302, SC2 Puck 1304).
- `physical_steam_controller_present()` -> `physical_steam_product_present(pid)`
  matches `:28DE:{PID}`, keeping the FVPF-serial / `vhci_hcd` / `/virtual/`
  exclusions so our own virtual pads never count.
- `degrade_steam_on_conflict` degrades only on a same-PID duplicate and logs
  the conflicting identity. `PUNKTFUNK_STEAM_FORCE=1` override unchanged.

A physical SC2 no longer blocks a virtual Deck (and vice versa); a genuine
same-identity duplicate (physical Deck + virtual Deck, or physical SC2 +
SC2 passthrough) still degrades. Adds a unit test for the PID mapping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:11:44 +02:00
enricobuehlerandClaude Fable 5 4714235fe6 feat(core): stylus wire P0 — state-full RICH_PEN batches, PenTracker, HOST_CAP_PEN
apple / swift (push) Successful in 1m24s
release / apple (push) Successful in 10m3s
android / android (push) Successful in 13m31s
arch / build-publish (push) Successful in 13m39s
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m12s
windows-host / package (push) Successful in 16m40s
apple / screenshots (push) Successful in 6m38s
ci / bench (push) Successful in 5m26s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m9s
ci / rust (push) Successful in 20m46s
decky / build-publish (push) Successful in 30s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m15s
deb / build-publish (push) Successful in 8m50s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 45s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish-host (push) Successful in 9m52s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m44s
docker / deploy-docs (push) Successful in 28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 7m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m45s
flatpak / build-publish (push) Failing after 8m6s
The pen plane from design/pen-tablet-input.md, protocol side only (the P1 uinput
tablet injector will consume it): 0xCC kind 0x05 carries batches of state-full
PenSamples (pressure, polar tilt + azimuth, barrel roll, hover distance, eraser
tool, barrel buttons); PenTracker diffs samples into injector transitions so a
lost datagram self-heals; HOST_CAP_PEN (0x10) gates sending and stays
unadvertised until a real backend exists. C ABI: PunktfunkPenSample +
punktfunk_connection_send_pen, documented in docs/embedding-the-c-abi.md §8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 12:58:31 +02:00
178 changed files with 18876 additions and 3811 deletions
+6 -3
View File
@@ -1,9 +1,12 @@
# Root build context is used only by web/Dockerfile, which needs web/ and
# api/openapi.json. Allowlist those; keep everything else (target/, .git, crates)
# out of the context upload.
# The root build context is used by web/Dockerfile (which needs web/ and
# api/openapi.json) and by ci/rust-ci-arm64cross.Dockerfile (which needs the toolchain
# pin). Allowlist those; keep everything else (target/, .git, crates) out of the
# context upload.
*
!web
!api/openapi.json
!rust-toolchain.toml
!ci/pf-host-cc
web/node_modules
web/.output
web/dist
+39
View File
@@ -0,0 +1,39 @@
# Announce a stable release to the Discord #releases channel.
#
# This is the deliberate "go" step for a release. Release notes live in the repo at
# docs/releases/<tag>.md and are seeded into the Gitea release body at creation by the build
# workflows (scripts/ci/gitea-release.sh), so the release is never noteless. Once every
# platform's CI is green for a tag, dispatch this workflow with that tag: it re-asserts the notes
# file over the live release and posts a formatted embed to #releases.
#
# Manual on purpose — pressing "go" is the quality gate that says "all platforms built, notes are
# final, tell the community." It is NOT wired to the tag push, so a half-built or failed release
# is never announced. Stable-only: a -rc/pre-release tag is refused unless allow_prerelease=true.
#
# Requires the repo secret DISCORD_RELEASE_WEBHOOK (the #releases channel webhook URL); GITEA auth
# reuses REGISTRY_TOKEN like the other release workflows.
name: announce
on:
workflow_dispatch:
inputs:
tag:
description: "Release tag to announce (e.g. v0.18.0)"
required: true
allow_prerelease:
description: "Announce even if the tag is a pre-release (-rc)"
required: false
default: "false"
jobs:
announce:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Post release announcement to Discord
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
DISCORD_RELEASE_WEBHOOK: ${{ secrets.DISCORD_RELEASE_WEBHOOK }}
ALLOW_PRERELEASE: ${{ inputs.allow_prerelease }}
run: bash scripts/ci/discord-announce.sh "${{ inputs.tag }}"
+84
View File
@@ -61,6 +61,43 @@ jobs:
- name: Test (unit + loopback + proptest + C ABI harness)
run: cargo test --workspace --locked
# The GPU encode backends are OFF by default, so every step above compiles ~none of them:
# `nvenc` gates enc/linux/nvenc_cuda.rs (+ nvenc_core/nvenc_status) and `vulkan-encode` gates
# enc/linux/vulkan_video.rs (+ the vendored vk_av1_encode/vk_valve_rgb bindings) — ~8,150
# lines carrying ~70 `unsafe` blocks. Their ONLY prior CI coverage was deb.yml's
# `cargo build`, where warnings are not errors, so pf-encode's own
# `#![deny(clippy::undocumented_unsafe_blocks)]` — the crate's stated unsafe-proof gate —
# was never actually enforced on them. (`pyrowave` needs no extra step: punktfunk-host has
# `default = ["pyrowave"]`, so the steps above already cover it.)
#
# `--all-targets` is load-bearing, not decoration: without it the feature-gated
# `#[cfg(test)]` modules are never compiled, which is exactly how all ten
# `NvencCudaEncoder::open` call sites in nvenc_cuda.rs's tests drifted to the wrong arity
# (E0061 x10) without any job noticing.
#
# GPU-free: every test needing real hardware is `#[ignore]`d, and NVENC/CUDA resolve their
# entry points at RUNTIME (dlopen), so the test binary links without a driver present.
# (On MSVC the same crate link-imports those symbols instead, which is why windows-host.yml
# can only type-check these tests via clippy — see the note there.)
#
# Scoped to `-p pf-encode` with ITS OWN feature names: punktfunk-host has no code gated on
# `nvenc`/`vulkan-encode` (its only `cfg(feature)` sites are the two `pyrowave` ones in
# capture.rs, and pyrowave is default-on, so the steps above already cover them). Going
# through `--features punktfunk-host/...` would force punktfunk-host into the selection and
# re-run its entire test suite a second time for no extra coverage.
#
# `pyrowave` is listed explicitly even though it is punktfunk-host's default: selecting only
# `-p pf-encode` takes the host out of the resolution, and pf-encode's own default is empty.
# Naming it keeps this the SHIPPED Linux feature set — deb.yml builds
# `--features punktfunk-host/nvenc,punktfunk-host/vulkan-encode` WITHOUT
# `--no-default-features`, so the .deb carries nvenc + vulkan-encode + pyrowave together, and
# that combination is what deserves the lint.
- name: Clippy + test the feature-gated Linux encode backends
run: |
cargo clippy -p pf-encode --all-targets --locked \
--features nvenc,vulkan-encode,pyrowave -- -D warnings
cargo test -p pf-encode --locked --features nvenc,vulkan-encode,pyrowave
- name: C ABI harness (standalone link proof)
run: bash crates/punktfunk-core/tests/c/run.sh
@@ -71,6 +108,53 @@ jobs:
git diff --exit-code include/punktfunk_core.h \
|| (echo "include/punktfunk_core.h is stale — commit the regenerated header" && exit 1)
# The client stack cross-checked for aarch64. NOT an artifact job — deb.yml ships those —
# this exists so a portability defect fails here instead of surfacing in a release build or
# on a user's board. It earns its runtime: the bug that motivated it (a Vulkan extension
# array typed `*const i8`, where `c_char` is signed on x86_64 and UNSIGNED on aarch64)
# compiled cleanly on every target CI built at the time.
#
# Client crates only, listed explicitly: the host's encode stack is x86 (NVENC/QSV/AMF) and
# `--workspace` would drag it in. Runs in the cross image (amd64 toolchain + arm64 sysroot,
# ci/rust-ci-arm64cross.Dockerfile) on the ordinary runner — no arm64 runner involved.
rust-arm64:
runs-on: ubuntu-24.04
container:
image: git.unom.io/unom/punktfunk-rust-ci-arm64cross:latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- name: Cache keys
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
- uses: actions/cache@v4
with:
path: |
/usr/local/cargo/registry
/usr/local/cargo/git
key: cargo-home-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-home-
- uses: actions/cache@v4
with:
path: target
# Its OWN prefix: aarch64 artifacts must never share the amd64 jobs' target cache.
key: cargo-target-arm64-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-target-arm64-v1-${{ env.rustc }}-
- name: Clippy for aarch64 (deny warnings)
run: |
cargo clippy --target aarch64-unknown-linux-gnu --all-targets --locked \
-p punktfunk-core -p pf-client-core -p pf-presenter -p pf-console-ui \
-p punktfunk-client-session -p punktfunk-client-linux \
-- -D warnings
# The minimal embedded build — no Skia, no PyroWave — is what a small image installs, so
# it has to keep compiling on its own, not just as a subset of the default features.
- name: Build the session binary, minimal features
run: |
cargo build --release --target aarch64-unknown-linux-gnu --locked \
-p punktfunk-client-session --no-default-features
web:
runs-on: ubuntu-24.04
container:
+96 -1
View File
@@ -1,9 +1,13 @@
# Build the punktfunk .debs and publish them to Gitea's Debian package registry, so Ubuntu
# boxes get new builds via `apt update && apt upgrade`. Two jobs, both publishing to the same
# boxes get new builds via `apt update && apt upgrade`. Three jobs, all publishing to the same
# apt distribution/component:
#
# build-publish — client + web + scripting, on the Ubuntu 26.04 rust-ci image (the client
# needs 24.04-absent libs: SDL3, GTK4 ≥ 4.20).
# build-publish-client-arm64
# — the same client package for arm64, CROSS-compiled on the same amd64
# runner in the rust-ci-arm64cross image. No host counterpart: the Linux
# host's encode stack is x86 (NVENC/QSV/AMF).
# build-publish-host — the HOST, on the Ubuntu 24.04 rust-ci-noble image with a from-source
# FFmpeg 8 BUNDLED into the .deb. This lowers the host's glibc floor to 2.39
# and removes the hard `Depends: libavcodec62`, so the ONE host .deb installs
@@ -269,3 +273,94 @@ jobs:
for DEB in dist/*.deb; do
upsert_asset "$RID" "$DEB"
done
# ---------------------------------------------------------------------------------------------
# The aarch64 CLIENT .deb. Cross-compiled on the ordinary amd64 runner in the
# punktfunk-rust-ci-arm64cross image (the rust-ci toolchain + an arm64 multiarch sysroot — see
# ci/rust-ci-arm64cross.Dockerfile); there is no arm64 runner in the fleet and none is needed.
# Client only, by decision: the Linux host encodes with NVENC/QSV/AMF, all x86.
# Publishes to the same distribution/component as the amd64 jobs — the apt registry keys pool
# entries by arch, so `apt` on an arm64 box picks this one up with no client-side configuration.
build-publish-client-arm64:
runs-on: ubuntu-24.04
container:
image: git.unom.io/unom/punktfunk-rust-ci-arm64cross:latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
# Byte-identical to build-publish's version step (pf-version.sh is deterministic per
# commit), so the arm64 package always shares the amd64 version line.
- name: Version + channel
run: |
eval "$(bash scripts/ci/pf-version.sh)"
SHORT=$(echo "$GITHUB_SHA" | cut -c1-8)
case "$GITHUB_REF" in
refs/tags/v*) V="${GITHUB_REF_NAME#v}"; DIST=stable ;;
*) V="${PF_BASE}~ci${GITHUB_RUN_NUMBER}.g${SHORT}"; DIST=canary ;;
esac
echo "VERSION=$V" >> "$GITHUB_ENV"
echo "DISTRIBUTION=$DIST" >> "$GITHUB_ENV"
echo "package version $V -> apt distribution '$DIST' (arm64)"
# dpkg-shlibdeps + dpkg-deb. The arm64 link deps themselves are the cross image's whole
# point and are already baked in; python3 is for scripts/ci/gitea-release.sh.
- name: dpkg-dev
run: |
apt-get update
apt-get install -y --no-install-recommends dpkg-dev python3
- name: Cache keys
run: echo "rustc=$(rustc --version | cut -d' ' -f2)" >> "$GITHUB_ENV"
- uses: actions/cache@v4
with:
path: |
/usr/local/cargo/registry
/usr/local/cargo/git
key: cargo-home-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-home-
- uses: actions/cache@v4
with:
path: target
# Its OWN key — these are aarch64 artifacts under target/aarch64-unknown-linux-gnu/
# and must never share the amd64 jobs' target cache.
key: cargo-target-arm64-v1-${{ env.rustc }}-${{ hashFiles('Cargo.lock') }}
restore-keys: cargo-target-arm64-v1-${{ env.rustc }}-
- name: Build the arm64 client .deb
env:
PUNKTFUNK_BUILD_VERSION: ${{ env.VERSION }} # stamped into the binaries (build.rs)
run: |
git config --global --add safe.directory "$PWD"
ARCH=arm64 TARGET=aarch64-unknown-linux-gnu \
bash packaging/debian/build-client-deb.sh
# Fail here rather than shipping an amd64 binary under an arm64 package name.
readelf -h target/aarch64-unknown-linux-gnu/release/punktfunk-session \
| grep -q AArch64 || { echo "ERROR: session binary is not AArch64"; exit 1; }
- name: Publish to the Gitea apt registry
env:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
for DEB in dist/*.deb; do
echo "uploading $DEB"
NAME=$(dpkg-deb -f "$DEB" Package)
VER=$(dpkg-deb -f "$DEB" Version)
ARCH=$(dpkg-deb -f "$DEB" Architecture)
curl -fsS -o /dev/null --user "enricobuehler:$TOKEN" -X DELETE \
"https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/$NAME/$VER/$ARCH" || true
curl -fsS --user "enricobuehler:$TOKEN" --upload-file "$DEB" \
"https://$REGISTRY/api/packages/$OWNER/debian/pool/$DISTRIBUTION/$COMPONENT/upload"
done
echo "published arm64 client to $OWNER/debian $DISTRIBUTION/$COMPONENT"
- name: Attach the arm64 .deb to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v')
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
. scripts/ci/gitea-release.sh
RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
for DEB in dist/*.deb; do
upsert_asset "$RID" "$DEB"
done
+36
View File
@@ -2,6 +2,7 @@
# punktfunk-web — management console (web/Dockerfile, repo-root context)
# punktfunk-docs — documentation site (docs-site/Dockerfile)
# punktfunk-rust-ci — Rust CI builder image consumed by ci.yml
# punktfunk-rust-ci-arm64cross — the above + an arm64 sysroot, for the aarch64 client legs
# punktfunk-fedora-rpm — Fedora 43 builder image consumed by rpm.yml (Bazzite RPM)
# Host and clients are intentionally NOT containerized (see CLAUDE.md "What's left").
#
@@ -80,6 +81,41 @@ jobs:
docker push "$REGISTRY/$OWNER/${{ matrix.image }}:latest"
case "$GITHUB_REF" in refs/tags/v*) docker push "$REGISTRY/$OWNER/${{ matrix.image }}:${GITHUB_REF_NAME}" ;; esac
# The aarch64 CROSS builder — a SEPARATE job because it is `FROM punktfunk-rust-ci:latest`
# and so must not race the matrix entry that publishes that base. Consumed by the arm64
# client legs in deb.yml/rpm.yml/arch.yml. Root context: it needs rust-toolchain.toml to
# install the target against the toolchain the workspace actually pins.
build-push-arm64cross:
runs-on: ubuntu-24.04
needs: build-push
timeout-minutes: 45
env:
IMAGE: punktfunk-rust-ci-arm64cross
steps:
- uses: actions/checkout@v4
- name: Login to registry
run: |
echo "${{ secrets.REGISTRY_TOKEN }}" \
| docker login "$REGISTRY" -u enricobuehler --password-stdin
- name: Build
run: |
EXTRA=""
case "$GITHUB_REF" in refs/tags/v*) EXTRA="-t $REGISTRY/$OWNER/$IMAGE:${GITHUB_REF_NAME}" ;; esac
docker build --pull \
-f ci/rust-ci-arm64cross.Dockerfile \
-t "$REGISTRY/$OWNER/$IMAGE:latest" \
-t "$REGISTRY/$OWNER/$IMAGE:sha-${GITHUB_SHA::8}" \
$EXTRA \
.
- name: Push
run: |
docker push "$REGISTRY/$OWNER/$IMAGE:sha-${GITHUB_SHA::8}"
docker push "$REGISTRY/$OWNER/$IMAGE:latest"
case "$GITHUB_REF" in refs/tags/v*) docker push "$REGISTRY/$OWNER/$IMAGE:${GITHUB_REF_NAME}" ;; esac
# Deploy the docs site to unom-1, the DMZ services VM website/cms also deploy to
# (docs.punktfunk.unom.io via Caddy on home-reverse-proxy-1 -> :3220). Same secret set
# as unom/website's deploy: DEPLOY_HOST/DEPLOY_USER/DEPLOY_PORT/DEPLOY_SSH_KEY (the
+70
View File
@@ -393,6 +393,76 @@ jobs:
-authenticationKeyID "${{ secrets.ASC_API_KEY_ID }}" \
-authenticationKeyIssuerID "${{ secrets.ASC_API_ISSUER_ID }}"
- name: iOS — export .ipa (Gitea release + run artifact)
# The TestFlight step above uploads straight to App Store Connect (destination=upload) and
# leaves NO .ipa on disk. Re-export the SAME archive with destination=export to get an
# App Store distribution-signed .ipa for the Gitea release + the run artifacts. Same gate as
# that archive; a warn+skip (never fails the best-effort iOS leg) if the archive is absent,
# e.g. a workflow_dispatch with testflight=false. NOTE: an App Store-signed .ipa installs
# only via TestFlight/App Store, not by direct sideload — it's a release/archival artifact.
if: gitea.event_name != 'workflow_dispatch' || inputs.testflight == 'true'
id: ios_ipa
run: |
ARCHIVE="$RUNNER_TEMP/Punktfunk-ios.xcarchive"
if [ ! -d "$ARCHIVE" ]; then
echo "::warning::iOS archive not found — skipping .ipa export"
exit 0
fi
PROFILE="Punktfunk iOS App Store Distribution"
WIDGET_PROFILE="Punktfunk iOS Widgets App Store Distribution"
# destination=export writes the .ipa to -exportPath; otherwise identical manual signing to
# the upload plist (both profiles, Apple Distribution). No ASC key needed — no network.
cat > "$RUNNER_TEMP/export-appstore-ipa.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>method</key><string>app-store-connect</string>
<key>destination</key><string>export</string>
<key>teamID</key><string>$TEAM_ID</string>
<key>signingStyle</key><string>manual</string>
<key>signingCertificate</key><string>Apple Distribution</string>
<key>provisioningProfiles</key>
<dict>
<key>io.unom.punktfunk</key><string>$PROFILE</string>
<key>io.unom.punktfunk.widgets</key><string>$WIDGET_PROFILE</string>
</dict>
</dict>
</plist>
EOF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild -exportArchive \
-archivePath "$ARCHIVE" \
-exportOptionsPlist "$RUNNER_TEMP/export-appstore-ipa.plist" \
-exportPath "$RUNNER_TEMP/export-ipa"
SRC=$(ls "$RUNNER_TEMP/export-ipa/"*.ipa 2>/dev/null | head -1)
[ -n "$SRC" ] || { echo "::warning::no .ipa was produced by export"; exit 0; }
mkdir -p "$GITHUB_WORKSPACE/dist"
IPA="$GITHUB_WORKSPACE/dist/Punktfunk-$VERSION.ipa"
mv "$SRC" "$IPA"
echo "IPA=$IPA" >> "$GITHUB_ENV"
echo "ipa=dist/Punktfunk-$VERSION.ipa" >> "$GITHUB_OUTPUT"
echo "exported $IPA"
- name: Attach .ipa to the workflow run
if: steps.ios_ipa.outputs.ipa != ''
# v3, not v4: Gitea's artifact backend identifies as GHES, which upload-artifact@v4 refuses
# (same reason as android.yml / apple.yml). Download is a zip of the .ipa.
uses: actions/upload-artifact@v3
with:
name: punktfunk-ios-ipa
path: ${{ steps.ios_ipa.outputs.ipa }}
if-no-files-found: warn
retention-days: 30
- name: Attach .ipa to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v') && steps.ios_ipa.outputs.ipa != ''
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
. scripts/ci/gitea-release.sh
RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
upsert_asset "$RID" "$IPA" "Punktfunk-$VERSION.ipa"
- name: tvOS — archive + upload to TestFlight
# Canary + stable, the same track as iOS/macOS — the tvOS xcframework slice is now built
# on every apple push (above), so this matches the iOS step's gate exactly.
+32
View File
@@ -50,6 +50,23 @@ on:
# builds — without these, encoder changes only reached this workflow via Cargo.lock luck.
- 'crates/pf-encode/**'
- 'crates/libvpl-sys/**'
# …and the rest of the W6 subsystem crates this build compiles. pf-encode was listed while
# the crates it speaks (pf-frame's CapturedFrame/PixelFormat/dxgi vocabulary, pf-gpu's
# adapter selection, pf-zerocopy, pf-host-config) were not, so a change that broke the
# Windows host through one of THEM reached main with no Windows build at all — the same
# Cargo.lock-luck gap the two lines above were added to close.
- 'crates/pf-frame/**'
- 'crates/pf-gpu/**'
- 'crates/pf-zerocopy/**'
- 'crates/pf-host-config/**'
- 'crates/pf-capture/**'
- 'crates/pf-win-display/**'
- 'crates/pf-vdisplay/**'
- 'crates/pf-inject/**'
- 'crates/pf-paths/**'
- 'crates/pf-driver-proto/**'
- 'crates/pf-clipboard/**'
- 'crates/pyrowave-sys/**'
- 'packaging/windows/**'
- 'scripts/windows/**'
- 'web/**'
@@ -154,8 +171,23 @@ jobs:
# build minutes earlier). Linting in release reuses those native build-script artifacts (no
# openh264 rebuild), and keeps everything in one C:\t\release tree. Same reason
# pf-vkhdr-layer's clippy below runs --release.
#
# pf-encode is linted SEPARATELY with --all-targets so its Windows `#[cfg(test)]` modules
# are type-checked — the AMF C-ABI layout assertions (`variant_layout_matches_c` and
# friends, which are the only guard on a hand-mirrored vtable ABI), the QSV tests, and the
# PyroWave-Windows smoke test. The host lint above cannot cover them: `-p punktfunk-host`
# only builds pf-encode as a dependency, so its test targets are never compiled, and that
# blind spot is what let the Linux twin's tests rot to the wrong arity unnoticed.
# NOTE: clippy (a check, no link step) is deliberately the vehicle here — `cargo test`
# with `nvenc` cannot LINK on MSVC: nvidia-video-codec-sdk link-imports
# NvEncodeAPICreateInstance / NvEncodeAPIGetMaxSupportedVersion, which resolve only against
# the driver's import lib. (On Linux the same crate dlopens them, so ci.yml can and does
# run the tests there.) Running them here would need an `--features amf-qsv,qsv` build
# without `nvenc`, i.e. a third full dep tree on a runner that already trips C1069 — not
# worth it while ci.yml executes the same tests.
run: |
cargo clippy --release -p punktfunk-host --features nvenc,amf-qsv,qsv -- -D warnings; if ($LASTEXITCODE) { throw "host clippy" }
cargo clippy --release -p pf-encode --all-targets --features nvenc,amf-qsv,qsv -- -D warnings; if ($LASTEXITCODE) { throw "pf-encode clippy" }
cargo clippy --release -p punktfunk-tray -- -D warnings; if ($LASTEXITCODE) { throw "tray clippy" }
- name: Build + lint the HDR Vulkan layer (pf-vkhdr-layer)
Generated
+30 -27
View File
@@ -2194,7 +2194,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.18.0"
version = "0.19.2"
[[package]]
name = "lazy_static"
@@ -2299,7 +2299,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"bindgen",
"cmake",
@@ -2334,7 +2334,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"punktfunk-core",
]
@@ -2823,7 +2823,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ashpd",
@@ -2844,7 +2844,7 @@ dependencies = [
[[package]]
name = "pf-client-core"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ash",
@@ -2868,7 +2868,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ashpd",
@@ -2886,7 +2886,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ash",
@@ -2907,7 +2907,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ash",
@@ -2931,7 +2931,7 @@ dependencies = [
[[package]]
name = "pf-ffvk"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"ash",
"bindgen",
@@ -2940,7 +2940,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"libc",
@@ -2952,7 +2952,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"pf-host-config",
@@ -2966,11 +2966,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.18.0"
version = "0.19.2"
[[package]]
name = "pf-inject"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ashpd",
@@ -2981,6 +2981,7 @@ dependencies = [
"pf-driver-proto",
"pf-host-config",
"pf-paths",
"pf-win-display",
"punktfunk-core",
"reis",
"tokio",
@@ -2998,14 +2999,14 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ash",
@@ -3020,10 +3021,11 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ashpd",
"bitflags",
"bytemuck",
"futures-util",
"hex",
@@ -3050,7 +3052,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"pf-paths",
@@ -3062,7 +3064,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ash",
@@ -3071,6 +3073,7 @@ dependencies = [
"libloading",
"serde",
"serde_json",
"tempfile",
"tracing",
]
@@ -3269,7 +3272,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"android_logger",
"jni",
@@ -3285,7 +3288,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"async-channel",
@@ -3301,7 +3304,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3316,7 +3319,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3335,7 +3338,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"aes-gcm",
"bytes",
@@ -3367,7 +3370,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"aes",
"aes-gcm",
@@ -3451,7 +3454,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3465,7 +3468,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"anyhow",
"ksni",
@@ -3488,7 +3491,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.18.0"
version = "0.19.2"
dependencies = [
"bindgen",
"cmake",
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.18.0"
version = "0.19.2"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+3 -3
View File
@@ -5106,7 +5106,7 @@
"properties": {
"audio_streaming": {
"type": "boolean",
"description": "True while the audio stream thread is running."
"description": "True while audio is streaming on either plane (same rule as `video_streaming`)."
},
"conflicts": {
"type": "array",
@@ -5150,7 +5150,7 @@
},
{
"$ref": "#/components/schemas/SessionInfo",
"description": "The active launch session (set by Moonlight's `/launch`, cleared on cancel/stop)."
"description": "The active session: GameStream's launch (Moonlight `/launch`) when present, else the first\nlive native session. `null` when nothing is streaming."
}
]
},
@@ -5160,7 +5160,7 @@
},
"video_streaming": {
"type": "boolean",
"description": "True while the video stream thread is running."
"description": "True while video is streaming on EITHER plane: the GameStream media pipeline, or a live\nnative (punktfunk/1) session — the default plane, invisible in the GameStream flag alone."
}
}
},
+45
View File
@@ -0,0 +1,45 @@
#!/bin/bash
# Host-side C compiler wrapper for the aarch64 cross image (ci/rust-ci-arm64cross.Dockerfile).
#
# Why this exists: ffmpeg-sys-next's build script compiles a probe it intends to RUN — it
# executes the binary to read the libav* version macros — so it forces `.target(HOST)` with
# the comment "don't cross-compile this", but still hands that host compile the TARGET's
# pkg-config include paths. `-I/usr/include/aarch64-linux-gnu` then shadows the host's own
# multiarch libc headers and the x86 compiler dies inside bits/math-vector.h on NEON/SVE
# types it has never heard of.
#
# Prepending the host's multiarch dir does NOT fix it: GCC drops a `-I` that duplicates a
# directory already on its system include path (keeping it in the original, later position),
# so the arm64 dir stays in front. The reliable fix is to remove the target include dirs from
# the host compile entirely — the probe only wants FFmpeg's version macros, and the amd64
# libav*-dev headers are installed and on the default search path, at the same version (both
# come from this Ubuntu release).
#
# Scope: only ever invoked as CC for the HOST triple (CC_x86_64_unknown_linux_gnu). Target
# compiles go to aarch64-linux-gnu-gcc and never pass through here.
set -euo pipefail
declare -a out=()
while (($#)); do
case "$1" in
# `-I dir` as two arguments — the form cc's Command building and ffmpeg-sys both emit.
-I)
if [[ ${2-} == *aarch64-linux-gnu* ]]; then
shift 2
continue
fi
out+=("$1" "${2-}")
shift 2
;;
# `-Idir` glued into one argument.
-I*aarch64-linux-gnu*)
shift
;;
*)
out+=("$1")
shift
;;
esac
done
exec /usr/bin/cc "${out[@]}"
+81
View File
@@ -0,0 +1,81 @@
# Cross-compiling CI builder: amd64 host toolchain + an arm64 multiarch sysroot, for the
# aarch64 Linux CLIENT artifacts (punktfunk-client + punktfunk-session).
#
# docker build -f ci/rust-ci-arm64cross.Dockerfile -t punktfunk-rust-ci-arm64cross .
#
# Derived from punktfunk-rust-ci so the Rust toolchain, clang, and CMake are byte-identical
# to the amd64 legs — this image only adds the target side. Kept as a SEPARATE image rather
# than folded into the base because the :arm64 dev libs are ~1 GB that every other CI job
# would otherwise pull for nothing.
#
# Client only: the Linux HOST stays amd64 (its encode stack is NVENC/QSV/AMF), so none of the
# host's CUDA/GBM link deps are mirrored here.
#
# Ubuntu splits archives by architecture: amd64 lives on archive.ubuntu.com, every port
# (arm64 included) on ports.ubuntu.com. Both stanzas therefore have to be pinned with an
# explicit `Architectures:` or apt tries to fetch arm64 from the amd64 mirror and 404s.
#
# Built from the REPO ROOT context (not ci/) — see the rust-toolchain.toml copy below.
FROM git.unom.io/unom/punktfunk-rust-ci:latest
ENV DEBIAN_FRONTEND=noninteractive
# 1. Pin the stock sources to amd64, add ports.ubuntu.com for arm64.
RUN sed -i 's|^Types: deb$|Types: deb\nArchitectures: amd64|' /etc/apt/sources.list.d/ubuntu.sources \
&& . /etc/os-release \
&& printf 'Types: deb\nArchitectures: arm64\nURIs: http://ports.ubuntu.com/ubuntu-ports/\nSuites: %s %s-updates %s-backports %s-security\nComponents: main universe restricted multiverse\nSigned-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg\n' \
"$VERSION_CODENAME" "$VERSION_CODENAME" "$VERSION_CODENAME" "$VERSION_CODENAME" \
> /etc/apt/sources.list.d/ubuntu-ports-arm64.sources \
&& dpkg --add-architecture arm64
# 2. The cross toolchain + every arm64 dev lib the client links. Mirrors the client half of
# rust-ci.Dockerfile's list (FFmpeg, PipeWire, Opus, SDL3, GTK4/libadwaita, xkbcommon,
# Vulkan headers for pf-ffvk's bindgen over hwcontext_vulkan.h).
RUN apt-get update && apt-get install -y --no-install-recommends \
crossbuild-essential-arm64 \
libavcodec-dev:arm64 libavformat-dev:arm64 libavutil-dev:arm64 libswscale-dev:arm64 \
libavfilter-dev:arm64 libavdevice-dev:arm64 \
libpipewire-0.3-dev:arm64 libopus-dev:arm64 \
libsdl3-dev:arm64 libgtk-4-dev:arm64 libadwaita-1-dev:arm64 \
libwayland-dev:arm64 libxkbcommon-dev:arm64 libvulkan-dev:arm64 \
&& rm -rf /var/lib/apt/lists/*
# 3. The Rust target — installed against the toolchain the WORKSPACE pins, not the image's
# default. The base image bakes whatever `stable` was at its build time, while every build
# in the repo switches to the exact channel in rust-toolchain.toml; adding the target to
# the default toolchain instead leaves the pinned one without an aarch64 std, and the build
# dies on `can't find crate for core` a few hundred crates in. Running rustup from a
# directory that contains the pin file resolves the right toolchain (and pre-downloads it,
# which every workspace job would otherwise pay for on first use).
COPY rust-toolchain.toml /opt/pf-toolchain/
WORKDIR /opt/pf-toolchain
RUN rustup target add aarch64-unknown-linux-gnu && rustup show
WORKDIR /
# 4. Cross wiring. Everything in this image is a cross build, so the plain (un-suffixed)
# variables are safe and cover the crates that roll their own pkg-config/bindgen calls
# instead of going through the target-scoped lookups.
# * PKG_CONFIG uses Debian's multiarch wrapper, which resolves the arm64 .pc files and
# rewrites -I/-L into the sysroot without per-crate cooperation.
# * BINDGEN_EXTRA_CLANG_ARGS: clang defaults to the host triple, so bindgen would parse
# arm64 headers with amd64 type layouts (silently wrong, not a build error) — the
# explicit --target plus the multiarch include dir is what keeps the layouts honest.
# * CC_x86_64_unknown_linux_gnu routes HOST-targeted compiles through a wrapper that
# strips the arm64 include dirs — see ci/pf-host-cc for the ffmpeg-sys-next probe it
# exists for.
COPY ci/pf-host-cc /usr/local/bin/pf-host-cc
RUN chmod 0755 /usr/local/bin/pf-host-cc
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++ \
AR_aarch64_unknown_linux_gnu=aarch64-linux-gnu-ar \
CC_x86_64_unknown_linux_gnu=/usr/local/bin/pf-host-cc \
PKG_CONFIG=aarch64-linux-gnu-pkg-config \
PKG_CONFIG_ALLOW_CROSS=1 \
BINDGEN_EXTRA_CLANG_ARGS="--target=aarch64-unknown-linux-gnu -I/usr/include/aarch64-linux-gnu"
# Fail the BUILD, not some later CI job, if the wrapper or a sysroot .pc is missing.
RUN command -v aarch64-linux-gnu-pkg-config \
&& aarch64-linux-gnu-pkg-config --cflags libavcodec sdl3 gtk4 libpipewire-0.3 \
&& aarch64-linux-gnu-gcc -dumpmachine | grep -q aarch64
@@ -355,11 +355,19 @@ class MainActivity : ComponentActivity() {
return true
}
when (event.keyCode) {
// A mouse back/forward button whose BUTTON_* press went unconsumed makes the
// framework synthesize a FALLBACK BACK — the button already went over the wire
// as X1/X2, and it must never yank the user out of the stream.
KeyEvent.KEYCODE_BACK ->
if (event.flags and KeyEvent.FLAG_FALLBACK != 0) return true
// A mouse's back/forward buttons already go over the wire as X1/X2 via their
// BUTTON_* motion edges — but Android ALSO delivers them as key events: the input
// reader synthesizes KEYCODE_BACK/FORWARD (stamped SOURCE_MOUSE) unconditionally,
// and a view-level FALLBACK BACK appears when the BUTTON_* press goes unconsumed.
// Swallow every such duplicate or it doubles as Android navigation and yanks the
// user out of the stream. A remote/keyboard BACK is never mouse-sourced, so it
// still falls through to the BackHandler and exits.
KeyEvent.KEYCODE_BACK, KeyEvent.KEYCODE_FORWARD ->
if (event.isFromSource(InputDevice.SOURCE_MOUSE) ||
event.flags and KeyEvent.FLAG_FALLBACK != 0
) {
return true
}
// Leave these to the system even while streaming.
// (BACK above → BackHandler leaves the stream.)
KeyEvent.KEYCODE_VOLUME_UP,
@@ -13,6 +13,7 @@ import android.net.wifi.WifiManager
import android.os.Build
import android.text.InputType
import android.util.Log
import android.view.KeyEvent
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.View
@@ -49,6 +50,9 @@ import androidx.core.content.ContextCompat
import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.LifecycleOwner
import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.GamepadRouter
import io.unom.punktfunk.kit.deviceBodyVibrator
@@ -383,6 +387,26 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Leaving the app (Home, task switch, screen off) MUST end the session. Android does not
// suspend a process for going to background, so without this the native worker kept running and
// its QUIC connection kept answering the host's keep-alives — the user was long gone but the
// host still saw a live client and held the session (and its display + encoder) open until the
// OS eventually reclaimed the process, which on a TV box is effectively never.
//
// Route it through `onDisconnect()` so the composable's `onDispose` above runs the one real
// teardown path. Deliberately NOT a `nativeDisconnectQuit`: backgrounding isn't a user "quit",
// so the host should linger the display and make coming straight back a fast reconnect.
DisposableEffect(handle) {
val lifecycle = (context as? LifecycleOwner)?.lifecycle
val obs = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_STOP) {
onDisconnect()
}
}
lifecycle?.addObserver(obs)
onDispose { lifecycle?.removeObserver(obs) }
}
// Auto-engage pointer capture at stream start (setting on + a mouse actually present).
// Delayed a beat: the grab needs window focus and the capture view attached.
LaunchedEffect(handle) {
@@ -471,12 +495,22 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
// keyboard gesture: its fingers belong to the host verbatim (a swipe there may BE a
// host-OS gesture), so intercepting three fingers would corrupt real multi-touch.
// Stylus lane (design/pen-tablet-input.md §7): against a HOST_CAP_PEN host a stylus
// splits out of BOTH touch models onto the pen plane; its heartbeat coroutine keeps a
// stationary held stroke alive (and its cancellation lifts everything on teardown).
val stylus = remember(handle) {
if (NativeBridge.nativeHostSupportsPen(handle)) StylusStream(handle) else null
}
if (stylus != null) {
LaunchedEffect(stylus) { stylus.heartbeatLoop() }
}
Box(
Modifier.fillMaxSize().pointerInput(handle, touchMode) {
when (touchMode) {
TouchMode.TOUCH -> streamTouchPassthrough(handle)
TouchMode.TOUCH -> streamTouchPassthrough(handle, stylus)
else -> streamTouchInput(
handle,
stylus,
trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() },
@@ -549,9 +583,15 @@ private class KeyCaptureView(context: Context) : View(context) {
var imeShown = false
private set
override fun onCheckIsTextEditor(): Boolean = true
override fun onCheckIsTextEditor(): Boolean = imeShown
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection? {
// Only an editor while the user has SUMMONED the keyboard (gesture / remote toggle).
// This view holds focus for the whole stream (it's the capture anchor), and with an
// always-live editable connection the IME counts input as active on it — TV IMEs then
// pop their UI the moment a PHYSICAL keyboard key arrives. With no connection, hardware
// typing stays on the raw dispatchKeyEvent → Keymap → wire path and no keyboard appears.
if (!imeShown) return null
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return if (textHandle != 0L) {
@@ -570,11 +610,29 @@ private class KeyCaptureView(context: Context) : View(context) {
imeShown = show
if (show) {
requestFocus()
// The view may already be focused from a null-connection state — restart so the
// framework re-queries onCreateInputConnection with the gate now open.
imm.restartInput(this)
imm.showSoftInput(this, 0)
} else {
imm.hideSoftInputFromWindow(windowToken, 0)
imm.restartInput(this) // gate closed — drop the editable connection
}
}
/**
* BACK while the summoned keyboard is up: the IME consumes it pre-IME to dismiss itself, so
* [setImeVisible] never hears about it — sync the gate here or a stale `imeShown` leaves the
* editable connection live and physical typing re-pops the keyboard.
*/
override fun onKeyPreIme(keyCode: Int, event: KeyEvent): Boolean {
if (keyCode == KeyEvent.KEYCODE_BACK && imeShown && event.action == KeyEvent.ACTION_UP) {
imeShown = false
(context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager)
?.restartInput(this)
}
return super.onKeyPreIme(keyCode, event)
}
}
/**
@@ -0,0 +1,195 @@
package io.unom.punktfunk
import android.view.MotionEvent
import androidx.compose.ui.ExperimentalComposeUiApi
import androidx.compose.ui.input.pointer.PointerEvent
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.unit.IntSize
import io.unom.punktfunk.kit.NativeBridge
import kotlinx.coroutines.delay
// Wire PEN_* state bits (punktfunk_core::quic::pen; mirrored, asserted by the Rust shim's docs).
private const val PEN_IN_RANGE = 1f
private const val PEN_TOUCHING = 2f
private const val PEN_BARREL1 = 4f
private const val PEN_BARREL2 = 8f
private const val STRIDE = 10
private const val MAX_SAMPLES = 8
/**
* Android stylus → the state-full pen plane (design/pen-tablet-input.md §7): pressure, tilt
* (`AXIS_TILT`, radians from the surface normal), azimuth (`AXIS_ORIENTATION` — Android's 0 =
* "pointed away from the user" IS the wire's north, no offset needed), hover with
* `AXIS_DISTANCE`, the eraser tool, both stylus barrel buttons, and historical (coalesced)
* samples batched oldest-first for full capture-rate fidelity. Android has no barrel-roll
* axis — roll stays unknown on this client.
*
* Both touch loops call [intercept] first; stylus/eraser pointers are consumed here (against a
* pen-capable host) and never reach the finger paths, independent of the touch-input mode.
* [heartbeatLoop] implements the ≤100 ms keepalive wire contract: a stationary held stylus is
* silent in Android's input pipeline, and the host force-releases a stroke after 200 ms
* without samples.
*/
internal class StylusStream(private val handle: Long) {
private var inRange = false
private var touching = false
private var sawHover = false
private val last = FloatArray(STRIDE)
private val batch = FloatArray(MAX_SAMPLES * STRIDE)
init {
idle(last)
}
/**
* Consume the event's stylus pointers into pen samples. Returns true when this event
* carried any (the caller's finger/gesture handling must then skip those changes).
*/
@OptIn(ExperimentalComposeUiApi::class)
fun intercept(ev: PointerEvent, size: IntSize): Boolean {
val stylusChanges = ev.changes.filter {
it.type == PointerType.Stylus || it.type == PointerType.Eraser
}
if (stylusChanges.isEmpty()) return false
stylusChanges.forEach { it.consume() }
val me = ev.motionEvent ?: return true
if (size.width <= 0 || size.height <= 0) return true
// At most one stylus exists — find its pointer index by tool type.
val idx = (0 until me.pointerCount).firstOrNull {
me.getToolType(it) == MotionEvent.TOOL_TYPE_STYLUS ||
me.getToolType(it) == MotionEvent.TOOL_TYPE_ERASER
} ?: return true
when (me.actionMasked) {
MotionEvent.ACTION_DOWN, MotionEvent.ACTION_POINTER_DOWN,
MotionEvent.ACTION_MOVE,
-> {
touching = true
inRange = true
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_MOVE -> {
sawHover = true
inRange = true
touching = false
emitSamples(me, idx, size)
}
MotionEvent.ACTION_UP, MotionEvent.ACTION_POINTER_UP -> {
touching = false
// Hover-capable hardware keeps proximity (HOVER_EXIT owns the leave);
// anything else leaves range on lift — the host never parks a phantom pen.
inRange = sawHover
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_EXIT, MotionEvent.ACTION_CANCEL -> release()
else -> {}
}
return true
}
/** Session/composition teardown: leave range so the host lifts anything still inked. */
fun reset() {
if (inRange || touching) release()
sawHover = false
}
/** The ≤100 ms keepalive (80 ms leaves headroom for one lost datagram). Runs until
* cancelled; resends the last state-full sample while the pen is in range. */
suspend fun heartbeatLoop() {
try {
while (true) {
delay(80)
if (inRange || touching) {
last[9] = 0f // dt
NativeBridge.nativeSendPen(handle, last, 1)
}
}
} finally {
reset()
}
}
private fun release() {
touching = false
inRange = false
last[0] = 0f // state: out of range
last[4] = 0f // pressure
NativeBridge.nativeSendPen(handle, last, 1)
}
/** Historical (coalesced) samples oldest-first, then the current one — a single batch. */
private fun emitSamples(me: MotionEvent, idx: Int, size: IntSize) {
val history = minOf(me.historySize, MAX_SAMPLES - 1)
var count = 0
var prevT = if (history > 0) me.getHistoricalEventTime(0) else me.eventTime
for (h in (me.historySize - history) until me.historySize) {
val t = me.getHistoricalEventTime(h)
fill(
batch, count * STRIDE, size,
x = me.getHistoricalX(idx, h), y = me.getHistoricalY(idx, h),
pressure = me.getHistoricalPressure(idx, h),
tiltRad = me.getHistoricalAxisValue(MotionEvent.AXIS_TILT, idx, h),
orientRad = me.getHistoricalAxisValue(MotionEvent.AXIS_ORIENTATION, idx, h),
distance = me.getHistoricalAxisValue(MotionEvent.AXIS_DISTANCE, idx, h),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((t - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
prevT = t
count++
}
fill(
batch, count * STRIDE, size,
x = me.getX(idx), y = me.getY(idx), pressure = me.getPressure(idx),
tiltRad = me.getAxisValue(MotionEvent.AXIS_TILT, idx),
orientRad = me.getAxisValue(MotionEvent.AXIS_ORIENTATION, idx),
distance = me.getAxisValue(MotionEvent.AXIS_DISTANCE, idx),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((me.eventTime - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
count++
batch.copyInto(last, 0, (count - 1) * STRIDE, count * STRIDE)
NativeBridge.nativeSendPen(handle, batch, count)
}
private fun fill(
out: FloatArray,
off: Int,
size: IntSize,
x: Float,
y: Float,
pressure: Float,
tiltRad: Float,
orientRad: Float,
distance: Float,
buttons: Int,
tool: Int,
dtUs: Float,
) {
var state = 0f
if (inRange || touching) state += PEN_IN_RANGE
if (touching) state += PEN_TOUCHING
if (buttons and MotionEvent.BUTTON_STYLUS_PRIMARY != 0) state += PEN_BARREL1
if (buttons and MotionEvent.BUTTON_STYLUS_SECONDARY != 0) state += PEN_BARREL2
out[off + 0] = state
out[off + 1] = if (tool == MotionEvent.TOOL_TYPE_ERASER) 1f else 0f
out[off + 2] = (x / (size.width - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 3] = (y / (size.height - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 4] = if (touching) pressure.coerceIn(0f, 1f) else 0f
// AXIS_DISTANCE units are device-arbitrary; 0..1 covers real hardware, and 0 while
// hovering legitimately means "at the hover floor".
out[off + 5] = if (touching) 0f else distance.coerceIn(0f, 1f)
out[off + 6] = Math.toDegrees(tiltRad.toDouble()).toFloat().coerceIn(0f, 90f)
// AXIS_ORIENTATION: 0 = pointed away from the user (= wire north), clockwise, −π..π.
out[off + 7] = ((Math.toDegrees(orientRad.toDouble()) + 360.0) % 360.0).toFloat()
out[off + 8] = -1f // no barrel-roll axis on Android
out[off + 9] = dtUs
}
private fun idle(out: FloatArray) {
out.fill(0f)
out[5] = -1f // distance unknown
out[6] = -1f // tilt unknown
out[7] = -1f // azimuth unknown
out[8] = -1f // roll unknown
}
}
@@ -1,9 +1,11 @@
package io.unom.punktfunk
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.ui.input.pointer.AwaitPointerEventScope
import androidx.compose.ui.input.pointer.PointerId
import androidx.compose.ui.input.pointer.PointerInputChange
import androidx.compose.ui.input.pointer.PointerInputScope
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.input.pointer.changedToDownIgnoreConsumed
import androidx.compose.ui.input.pointer.changedToUpIgnoreConsumed
import androidx.compose.ui.input.pointer.positionChanged
@@ -56,7 +58,26 @@ private const val ACCEL_MAX = 3.0f
* normalizes and maps into the output). On teardown (stream leaves composition) every still-held
* contact is lifted so nothing stays stuck on the host.
*/
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
/** Whether this change belongs to the stylus lane (only when a pen-capable host is live). */
private fun isStylus(c: PointerInputChange, stylus: StylusStream?): Boolean =
stylus != null && (c.type == PointerType.Stylus || c.type == PointerType.Eraser)
/** [awaitFirstDown] with the stylus lane split out: pen events feed [stylus] and never start a
* mouse/touch gesture. Toward a pen-less host ([stylus] == null) a stylus stays a finger. */
private suspend fun AwaitPointerEventScope.awaitFirstFingerDown(
stylus: StylusStream?,
): PointerInputChange {
while (true) {
val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val down = ev.changes.firstOrNull {
it.changedToDownIgnoreConsumed() && !isStylus(it, stylus)
}
if (down != null) return down
}
}
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, stylus: StylusStream?) {
val ids = mutableMapOf<PointerId, Int>()
fun alloc(p: PointerId): Int {
var id = 0
@@ -68,10 +89,12 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
awaitPointerEventScope {
while (true) {
val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val sw = size.width
val sh = size.height
if (sw <= 0 || sh <= 0) continue
for (c in ev.changes) {
if (isStylus(c, stylus)) continue // the pen plane owns it
val x = c.position.x.roundToInt().coerceIn(0, sw - 1)
val y = c.position.y.roundToInt().coerceIn(0, sh - 1)
when {
@@ -98,6 +121,7 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput(
handle: Long,
stylus: StylusStream?,
trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit,
@@ -120,7 +144,7 @@ internal suspend fun PointerInputScope.streamTouchInput(
)
}
awaitEachGesture {
val down = awaitFirstDown(requireUnconsumed = false)
val down = awaitFirstFingerDown(stylus)
val startX = down.position.x
val startY = down.position.y
// A touch landing just after a quick tap nearby = tap-and-drag: hold the left
@@ -157,7 +181,8 @@ internal suspend fun PointerInputScope.streamTouchInput(
while (true) {
val ev = awaitPointerEvent()
val pressed = ev.changes.filter { it.pressed }
stylus?.intercept(ev, size)
val pressed = ev.changes.filter { it.pressed && !isStylus(it, stylus) }
if (pressed.isEmpty()) {
upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime
break
@@ -287,6 +287,22 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised full-fidelity stylus injection (`HOST_CAP_PEN`) — the gate
* for splitting stylus pointers out of the touch path onto the pen plane. False on `0`.
*/
external fun nativeHostSupportsPen(handle: Long): Boolean
/**
* One stylus batch of STATE-FULL samples (the pen plane; design/pen-tablet-input.md §7):
* [count] × 10 floats, oldest first — `[state, tool, x, y, pressure, distance, tilt_deg,
* azimuth_deg, roll_deg, dt_us]`. `state` = the wire in-range/touching/barrel bits; `tool`
* 0=pen 1=eraser; x/y/pressure/distance normalized 0..1; distance/tilt/azimuth/roll < 0 =
* unknown. Send only when [nativeHostSupportsPen]; repeat the last sample ≤100 ms while the
* pen is in range (the host force-releases a silent stroke after 200 ms).
*/
external fun nativeSendPen(handle: Long, samples: FloatArray, count: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) — its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
+92 -2
View File
@@ -6,11 +6,14 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject, JString};
use jni::objects::{JByteBuffer, JFloatArray, JObject, JString};
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX, HOST_CAP_TEXT_INPUT};
use punktfunk_core::quic::{
PenSample, PenTool, RichInput, HID_REPORT_MAX, HOST_CAP_PEN, HOST_CAP_TEXT_INPUT,
PEN_ANGLE_UNKNOWN, PEN_BATCH_MAX, PEN_DISTANCE_UNKNOWN, PEN_TILT_UNKNOWN,
};
use super::SessionHandle;
@@ -162,6 +165,93 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSu
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeHostSupportsPen(handle)` — the host advertised `HOST_CAP_PEN`, so the
/// Kotlin side splits stylus pointers out of the touch path onto the pen plane
/// (design/pen-tablet-input.md §7). `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostSupportsPen(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_PEN != 0)
}
/// Floats per sample in the `nativeSendPen` flat array.
const PEN_JNI_STRIDE: usize = 10;
/// `NativeBridge.nativeSendPen(handle, samples, count)` — one stylus batch of STATE-FULL
/// samples, `count` × [`PEN_JNI_STRIDE`] floats, oldest first:
/// `[state, tool, x, y, pressure, distance, tilt_deg, azimuth_deg, roll_deg, dt_us]`.
/// `state` = the wire `PEN_*` bits; `tool` 0=pen 1=eraser; `x`/`y`/`pressure`/`distance`
/// normalized 0..1; `distance`/`tilt_deg`/`azimuth_deg`/`roll_deg` < 0 = unknown. Call only
/// against a [`nativeHostSupportsPen`] host; the client heartbeats the last sample ≤100 ms
/// while in range (Kotlin side — see `StylusStream`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPen(
env: JNIEnv,
_this: JObject,
handle: jlong,
samples: JFloatArray,
count: jint,
) {
if handle == 0 || count <= 0 {
return;
}
let count = (count as usize).min(PEN_BATCH_MAX);
let mut buf = [0f32; PEN_BATCH_MAX * PEN_JNI_STRIDE];
let flat = &mut buf[..count * PEN_JNI_STRIDE];
if env.get_float_array_region(&samples, 0, flat).is_err() {
return; // short array — a bridge bug, never worth a crash on the input path
}
let mut batch = [PenSample::default(); PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(flat.chunks_exact(PEN_JNI_STRIDE)) {
if !s[2].is_finite() || !s[3].is_finite() {
return; // never forward a NaN coordinate
}
*slot = PenSample {
state: s[0] as u8,
tool: if s[1] as u8 == 1 {
PenTool::Eraser
} else {
PenTool::Pen
},
x: s[2].clamp(0.0, 1.0),
y: s[3].clamp(0.0, 1.0),
pressure: (s[4].clamp(0.0, 1.0) * 65535.0) as u16,
distance: if s[5] < 0.0 {
PEN_DISTANCE_UNKNOWN
} else {
(s[5].clamp(0.0, 1.0) * 65534.0) as u16
},
tilt_deg: if s[6] < 0.0 {
PEN_TILT_UNKNOWN
} else {
(s[6].clamp(0.0, 90.0)) as u8
},
azimuth_deg: if s[7] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[7] as u16) % 360
},
roll_deg: if s[8] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[8] as u16) % 360
},
dt_us: s[9].clamp(0.0, 65535.0) as u16,
};
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_pen is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_pen(&batch[..count]);
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
@@ -144,14 +144,26 @@ struct ContentView: View {
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
// parallel session this drives the one `model` ContentView owns.
.onOpenURL { handleDeepLink($0) }
#if os(iOS)
// Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a
// transient peek) is ignored so the disconnect timer never starts for a Control-Center pull.
#if os(iOS) || os(tvOS)
// Backgrounding driver. Only .background/.active matter; .inactive (a transient peek) is
// ignored so neither branch fires for a Control-Center pull.
//
// Backgrounding MUST end the session one way or the other: the app keeps running while
// streaming (the `audio` background mode plus a live audio session), so its QUIC connection
// keeps answering the host's keep-alives with the user long gone the host has no way to
// tell that apart from someone watching, and the session survived indefinitely. Either hold
// it under the opt-in keep-alive (bounded by that path's own auto-disconnect timer) or end
// it here.
.onChange(of: scenePhase) { _, phase in
switch phase {
case .background:
if backgroundKeepAlive, model.phase == .streaming {
guard model.phase == .streaming else { break }
if backgroundKeepAlive {
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
} else {
// Not deliberate: the user may come straight back, so let the host linger the
// display for a fast reconnect instead of tearing it down.
model.disconnect(deliberate: false)
}
case .active:
model.exitBackground()
@@ -159,7 +171,11 @@ struct ContentView: View {
break
}
}
// Live Activity lifecycle, driven from the model's published state.
#endif
#if os(iOS)
// Live Activity lifecycle, driven from the model's published state. iPhone/iPad only
// ActivityKit (and so `liveActivity`) does not exist on tvOS, which is why this stays in its
// own os(iOS) block rather than riding the backgrounding driver's.
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
@@ -0,0 +1,97 @@
// Keeps the local display awake for the duration of a streaming session.
//
// A stream is not "user activity" to the OS: the pixels arrive over the network and the input that
// drives them is often a game controller, which does NOT feed the HID idle timer on any Apple
// platform. So a controller-only session reliably idles the panel out from under the user the
// same reason the Android client holds FLAG_KEEP_SCREEN_ON while streaming (StreamScreen.kt).
//
// Held by SessionModel from `beginStreaming` to `disconnect`, so it is scoped to the session and
// never leaks past it (including a host-ended or timed-out background session, which both land in
// `disconnect`).
import Foundation
#if os(macOS)
import IOKit.pwr_mgt
#else
import UIKit
#endif
@MainActor
final class DisplaySleepGuard {
#if os(macOS)
/// The `beginActivity` token; non-nil exactly while held.
private var activity: NSObjectProtocol?
/// Re-used across heartbeats so the whole session shares one assertion instead of
/// accumulating one per tick.
private var userActivityAssertion: IOPMAssertionID = IOPMAssertionID(0)
private var heartbeat: Timer?
/// The power assertion defers DISPLAY SLEEP but not the screen saver that runs off the
/// HID idle timer, which a controller-only session never touches. Declaring user activity
/// on an interval well under the shortest selectable screen-saver delay (1 minute) keeps
/// that timer from ever reaching it. Side effect, and the intended one: an idle-lock
/// configured to follow the screen saver is deferred too, for the session only.
private static let heartbeatInterval: TimeInterval = 30
#endif
private(set) var isHeld = false
/// Idempotent a second acquire while held is a no-op.
func acquire() {
guard !isHeld else { return }
isHeld = true
#if os(macOS)
// The high-level Foundation API over IOKit power assertions: `.idleDisplaySleepDisabled`
// is the panel, `.userInitiated` also holds off idle SYSTEM sleep and sudden termination
// for a session the user is watching in real time.
activity = ProcessInfo.processInfo.beginActivity(
options: [.userInitiated, .idleDisplaySleepDisabled],
reason: "Punktfunk streaming session")
declareUserActivity()
let timer = Timer.scheduledTimer(withTimeInterval: Self.heartbeatInterval, repeats: true) {
[weak self] _ in
MainActor.assumeIsolated { self?.declareUserActivity() }
}
// The stream runs under a tracking run-loop mode while a menu or a window resize is up;
// .common keeps the heartbeat ticking through those.
RunLoop.main.add(timer, forMode: .common)
heartbeat = timer
#else
// iOS/iPadOS/tvOS: app-wide, and ignored while backgrounded the background keep-alive
// (audio-only, video dropped) correctly lets the device sleep without touching this.
UIApplication.shared.isIdleTimerDisabled = true
#endif
}
/// Idempotent safe to call when not held (`disconnect` runs on paths that never streamed).
func release() {
guard isHeld else { return }
isHeld = false
#if os(macOS)
heartbeat?.invalidate()
heartbeat = nil
if let activity {
ProcessInfo.processInfo.endActivity(activity)
self.activity = nil
}
if userActivityAssertion != IOPMAssertionID(0) {
IOPMAssertionRelease(userActivityAssertion)
userActivityAssertion = IOPMAssertionID(0)
}
#else
UIApplication.shared.isIdleTimerDisabled = false
#endif
}
#if os(macOS)
/// Resets the HID idle timer (see `heartbeatInterval`). `kIOPMUserActiveLocal` = activity at
/// this Mac's own display, which is what a stream being watched here is.
private func declareUserActivity() {
IOPMAssertionDeclareUserActivity(
"Punktfunk streaming session" as CFString,
kIOPMUserActiveLocal,
&userActivityAssertion)
}
#endif
}
@@ -196,6 +196,11 @@ final class SessionModel: ObservableObject {
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
private var backgroundTimer: DispatchSourceTimer?
/// Holds off display sleep (and, on macOS, the screen saver) for the life of a session
/// nothing about watching a stream looks like user activity to the OS, least of all a
/// controller-only session. Acquired in `beginStreaming`, released in `disconnect`.
private let displaySleepGuard = DisplaySleepGuard()
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
/// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
@@ -455,6 +460,8 @@ final class SessionModel: ObservableObject {
func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate()
statsTimer = nil
// No-op when this session never reached `.streaming` (a refused/aborted connect).
displaySleepGuard.release()
// Drop any armed background keep-alive (incl. the timeout that just fired us).
backgroundTimer?.cancel()
backgroundTimer = nil
@@ -550,6 +557,7 @@ final class SessionModel: ObservableObject {
// Input capture itself is owned by StreamView (engaged by the captureEnabled
// flip this phase change causes, released/re-engaged by the user from there).
phase = .streaming
displaySleepGuard.acquire()
// Audio starts with streaming, not during the trust prompt no host sound (or
// mic uplink!) before the user trusted the host. Devices come from Settings;
// "" = system default.
@@ -387,8 +387,16 @@ public final class PunktfunkConnection {
/// The host answered `HOST_CAP_CURSOR`: it stopped compositing the pointer and forwards
/// shape/state on the cursor planes the client MUST draw the cursor locally.
/// `0x08` the bit moved when `HOST_CAP_TEXT_INPUT` claimed `0x04` on main; testing the
/// old bit would mistake a text-input-capable host (e.g. Windows) for a cursor grant.
public var hostSupportsCursor: Bool {
hostCaps & 0x04 != 0
hostCaps & 0x08 != 0
}
/// The host injects full-fidelity stylus input (`HOST_CAP_PEN`) the gate for splitting
/// Apple Pencil out of the touch path onto the pen plane (``sendPen(_:)``).
public var hostSupportsPen: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_PEN) != 0
}
/// One forwarded host-cursor shape (the cursor channel, ABI v11): straight-alpha RGBA,
@@ -1133,6 +1141,19 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_input(h, &ev)
}
/// Send one stylus sample batch ( `PUNKTFUNK_PEN_BATCH_MAX`, oldest first) on the pen
/// plane. Gate on ``hostSupportsPen`` the core refuses toward a host without the cap.
/// Thread-safe; silently dropped after close (input is lossy by design).
public func sendPen(_ samples: [PunktfunkPenSample]) {
guard !samples.isEmpty else { return }
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return }
samples.withUnsafeBufferPointer { buf in
_ = punktfunk_connection_send_pen(h, buf.baseAddress, UInt32(buf.count))
}
}
/// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with
/// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the
/// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a
@@ -0,0 +1,268 @@
// Apple Pencil state-full wire pen samples (design/pen-tablet-input.md §7).
//
// Every sample carries the COMPLETE pen state (in-range/touching/buttons + all axes) the
// host diffs consecutive samples and synthesizes down/up/button transitions itself, so a lost
// datagram self-heals and this file never sends edge events. Three sources feed one stream:
// UITouch contacts (with coalesced samples for full 240 Hz fidelity), the hover gesture
// (zOffset > 0 distinguishes a hovering Pencil from a trackpad pointer), and
// UIPencilInteraction (squeeze held barrel 1, double-tap a momentary barrel 2
// Apple Pencil has no hardware eraser end or barrel buttons, so these mappings are how
// host-side apps get their stylus button/eraser affordances).
//
// HEARTBEAT (wire contract see `PunktfunkPenSample` in punktfunk_core.h): while the pen is
// in range or touching, the last sample repeats every 100 ms even when nothing changed.
// UIKit is silent for a stationary Pencil, and the host force-releases the stroke after
// 200 ms without samples (its dead-client failsafe) the timer keeps a held stroke alive.
#if os(iOS)
import PunktfunkCore
import UIKit
final class PencilStream: NSObject, UIPencilInteractionDelegate {
enum Phase { case down, move, up, cancel }
/// One assembled batch ( `PUNKTFUNK_PEN_BATCH_MAX` samples) ready for the connection.
var send: (([PunktfunkPenSample]) -> Void)?
/// View-space point normalized [0,1] video coordinates (the letterbox mapping the
/// touch path already uses). nil until a mode is negotiated samples are dropped then.
var videoNorm: ((CGPoint) -> (Float, Float)?)?
private var inRange = false
private var touching = false
/// Squeeze held (mapped to wire BARREL1).
private var squeezeHeld = false
/// Whether this device/Pencil pair has demonstrated hover decides what a lift means:
/// hover-capable hardware keeps proximity (the hover recognizer owns the exit), anything
/// else leaves range on lift so the host never parks a phantom hovering pen.
private var sawHover = false
/// A hover gesture is live right now (routes ended-state hover callbacks to us even when
/// the recognizer's final zOffset reads 0).
private(set) var hoverActive = false
private var last = PencilStream.idleSample()
private var heartbeat: Timer?
// MARK: - Contact path (UITouch, `.pencil` only)
func touches(_ touches: Set<UITouch>, event: UIEvent?, phase: Phase, in view: UIView) {
// At most one Pencil exists; a set with several is UIKit batching phases of the same
// stylus the last one carries the freshest state.
guard let touch = touches.max(by: { $0.timestamp < $1.timestamp }) else { return }
switch phase {
case .down, .move:
touching = true
inRange = true
// Coalesced samples restore the Pencil's full capture rate (UIKit delivers at
// display cadence); oldest first, `dt_us` preserving their spacing.
let raw = event?.coalescedTouches(for: touch) ?? [touch]
var batch: [PunktfunkPenSample] = []
var prevTs: TimeInterval?
for t in raw.suffix(Int(PUNKTFUNK_PEN_BATCH_MAX)) {
guard let s = contactSample(t, in: view, prevTs: prevTs) else { continue }
prevTs = t.timestamp
batch.append(s)
}
emit(batch)
case .up:
touching = false
// Hover-capable hardware: lift back to hover, the recognizer exits range later.
// Otherwise a lift IS the range exit (mirror of the host's GameStream heuristic).
inRange = sawHover
var s = last
s.pressure = 0
s.state = stateBits()
if let posSample = contactSample(touch, in: view, prevTs: nil) {
s.x = posSample.x
s.y = posSample.y
}
emit([s])
case .cancel:
release()
}
}
// MARK: - Hover path (forwarded from the view's hover recognizer)
/// Returns whether the event was consumed as Pencil hover; `false` hands it back to the
/// pointer path. A hovering Pencil reports `zOffset > 0`; trackpad/mouse hover is 0.
func maybeHover(_ r: UIHoverGestureRecognizer, in view: UIView) -> Bool {
switch r.state {
case .began, .changed:
guard r.zOffset > 0 || hoverActive else { return false }
hoverActive = true
sawHover = true
inRange = true
touching = false
guard let (x, y) = videoNorm?(r.location(in: view)) else { return true }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
s.distance = UInt16((r.zOffset.clamped(to: 0...1) * 65534).rounded())
s.tilt_deg = Self.tiltDeg(altitude: r.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(r.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(r.rollAngle) }
emit([s])
return true
case .ended, .cancelled, .failed:
guard hoverActive else { return false }
hoverActive = false
if !touching { release() }
return true
default:
return hoverActive
}
}
// MARK: - UIPencilInteractionDelegate (squeeze barrel 1 held, tap barrel 2 click)
@available(iOS 17.5, *)
func pencilInteraction(
_ interaction: UIPencilInteraction, didReceiveSqueeze squeeze: UIPencilInteraction.Squeeze
) {
switch squeeze.phase {
case .began:
squeezeHeld = true
case .ended, .cancelled:
squeezeHeld = false
default:
return
}
guard inRange || touching else { return }
var s = last
s.state = stateBits()
emit([s])
}
func pencilInteractionDidTap(_ interaction: UIPencilInteraction) {
guard inRange || touching else { return }
// A momentary barrel-2 click: press + release as two state-full samples in ONE batch
// the host's tracker emits the button press and release in order.
var press = last
press.state = stateBits() | UInt8(PUNKTFUNK_PEN_BARREL2)
var releaseS = last
releaseS.state = stateBits()
emit([press, releaseS])
}
// MARK: - Lifecycle
/// Session stop / view teardown: leave range so the host lifts anything held.
func reset() {
if inRange || touching { release() }
sawHover = false
hoverActive = false
squeezeHeld = false
}
private func release() {
touching = false
inRange = false
var s = last
s.pressure = 0
s.state = 0
emit([s])
}
// MARK: - Sample assembly
private func contactSample(
_ t: UITouch, in view: UIView, prevTs: TimeInterval?
) -> PunktfunkPenSample? {
guard let (x, y) = videoNorm?(t.location(in: view)) else { return nil }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
// maximumPossibleForce is 0 until the system knows the stylus full force then
// (binary-stylus semantics, matching the host's unknown-pressure rule).
let maxForce = t.maximumPossibleForce
s.pressure =
maxForce > 0
? UInt16((Double(t.force / maxForce).clamped(to: 0...1) * 65535).rounded())
: UInt16.max
s.distance = 0
s.tilt_deg = Self.tiltDeg(altitude: t.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(t.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(t.rollAngle) }
if let prevTs {
s.dt_us = UInt16(((t.timestamp - prevTs) * 1_000_000).clamped(to: 0...65535))
}
return s
}
private func stateBits() -> UInt8 {
var bits: UInt8 = 0
if inRange || touching { bits |= UInt8(PUNKTFUNK_PEN_IN_RANGE) }
if touching { bits |= UInt8(PUNKTFUNK_PEN_TOUCHING) }
if squeezeHeld { bits |= UInt8(PUNKTFUNK_PEN_BARREL1) }
return bits
}
private func emit(_ batch: [PunktfunkPenSample]) {
guard !batch.isEmpty else { return }
last = batch[batch.count - 1]
last.dt_us = 0
send?(batch)
armHeartbeat()
}
/// The 100 ms keepalive while in range (see the file header). 80 ms leaves headroom
/// under the host's 200 ms failsafe even with one lost datagram.
private func armHeartbeat() {
heartbeat?.invalidate()
guard inRange || touching else {
heartbeat = nil
return
}
heartbeat = Timer.scheduledTimer(withTimeInterval: 0.08, repeats: true) {
[weak self] _ in
guard let self, self.inRange || self.touching else {
self?.heartbeat?.invalidate()
self?.heartbeat = nil
return
}
self.send?([self.last])
}
}
// MARK: - Angle conversions
/// Altitude (π/2 = perpendicular) wire tilt-from-normal in degrees, 0...90.
private static func tiltDeg(altitude: CGFloat) -> UInt8 {
UInt8((90 - altitude * 180 / .pi).rounded().clamped(to: 0...90))
}
/// Apple azimuth (0 along the view's +x axis, clockwise, y-down) wire azimuth
/// (0 = north/up on screen, clockwise): +90° offset.
private static func azimuthDeg(_ apple: CGFloat) -> UInt16 {
let deg = (apple * 180 / .pi + 90).truncatingRemainder(dividingBy: 360)
return UInt16((deg + 360).truncatingRemainder(dividingBy: 360).rounded()) % 360
}
/// Pencil Pro roll (radians, π...π) wire barrel roll 0...359°.
private static func rollDeg(_ roll: CGFloat) -> UInt16 {
let deg = (roll * 180 / .pi).truncatingRemainder(dividingBy: 360)
return UInt16(((deg + 360).truncatingRemainder(dividingBy: 360)).rounded()) % 360
}
/// All-unknown baseline: sentinel angles/distance, tool = pen (the eraser is host-side
/// state driven by the squeeze/tap mappings, not a hardware end).
private static func idleSample() -> PunktfunkPenSample {
PunktfunkPenSample(
x: 0, y: 0, pressure: 0,
distance: UInt16(PUNKTFUNK_PEN_DISTANCE_UNKNOWN),
azimuth_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
roll_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
dt_us: 0, state: 0,
tool: UInt8(PUNKTFUNK_PEN_TOOL_PEN),
tilt_deg: UInt8(PUNKTFUNK_PEN_TILT_UNKNOWN),
_reserved: (0, 0, 0))
}
}
extension Comparable {
fileprivate func clamped(to range: ClosedRange<Self>) -> Self {
min(max(self, range.lowerBound), range.upperBound)
}
}
#endif
@@ -223,7 +223,19 @@ public final class StreamLayerView: NSView {
/// when the Welcome carried `HOST_CAP_CURSOR` (only sessions that advertised the client
/// cap get it). Shapes cache by serial; state is latest-wins. Main-thread only.
private var cursorChannelActive = false
private var hostCursors: [UInt32: NSCursor] = [:]
/// A forwarded host cursor shape, cached RAW (not as a finished `NSCursor`) so the pointer can be
/// (re)built at the CURRENT video-fit scale see `scaledCursor`. The host forwards the bitmap in
/// host FRAMEBUFFER pixels, whose size tracks the host's display scaling (32 px at 100%, 96 px at
/// 300% DPI); scaling by the video fit keeps the pointer sized to the streamed desktop at any host
/// scaling instead of ballooning on a high-DPI host.
private struct HostCursorShape {
let cg: CGImage
let width: Int
let height: Int
let hotX: Int
let hotY: Int
}
private var hostCursors: [UInt32: HostCursorShape] = [:]
private var cursorState: PunktfunkConnection.CursorStateEvent?
/// Last `CursorRenderMode.clientDraws` told to the host (the §8 mid-stream render flip);
/// nil = nothing sent yet. Edge-detected by [`reconcileCursorRender`] from the live mouse
@@ -500,8 +512,8 @@ public final class StreamLayerView: NSView {
// video); hidden host pointer (or no shape yet) = invisible. Without the channel,
// M1 behavior: invisible local cursor, the composited host cursor is the visible one.
if cursorChannelActive, let st = cursorState, st.visible,
let host = hostCursors[st.serial] {
addCursorRect(bounds, cursor: host)
let shape = hostCursors[st.serial] {
addCursorRect(bounds, cursor: scaledCursor(shape))
} else {
addCursorRect(bounds, cursor: Self.invisibleCursor)
}
@@ -570,12 +582,12 @@ public final class StreamLayerView: NSView {
}
private func applyCursorShape(_ ev: PunktfunkConnection.CursorShapeEvent) {
guard let cursor = Self.makeCursor(ev) else {
guard let shape = Self.makeShape(ev) else {
streamInputLog.warning("cursor shape rejected (\(ev.width)x\(ev.height)) — keeping the previous cursor")
return
}
if hostCursors.count >= 64 { hostCursors.removeAll() } // degenerate host: reset
hostCursors[ev.serial] = cursor
hostCursors[ev.serial] = shape
if cursorState?.serial == ev.serial {
window?.invalidateCursorRects(for: self)
}
@@ -597,8 +609,10 @@ public final class StreamLayerView: NSView {
_ = (lastHint, hintOverride)
}
/// Build an `NSCursor` from a forwarded straight-alpha RGBA shape.
private static func makeCursor(_ ev: PunktfunkConnection.CursorShapeEvent) -> NSCursor? {
/// Decode a forwarded straight-alpha RGBA shape into a CGImage + hotspot. The on-screen SIZE is
/// NOT baked in here it is applied per-use in `scaledCursor` from the live video-fit scale, so
/// the same shape re-fits across window resizes / retina moves without a re-forward.
private static func makeShape(_ ev: PunktfunkConnection.CursorShapeEvent) -> HostCursorShape? {
let (w, h) = (ev.width, ev.height)
guard w > 0, h > 0, ev.rgba.count >= w * h * 4,
let provider = CGDataProvider(data: ev.rgba as CFData),
@@ -609,10 +623,40 @@ public final class StreamLayerView: NSView {
provider: provider, decode: nil, shouldInterpolate: false,
intent: .defaultIntent)
else { return nil }
let image = NSImage(cgImage: cg, size: NSSize(width: w, height: h))
return NSCursor(
image: image,
hotSpot: NSPoint(x: min(ev.hotX, w - 1), y: min(ev.hotY, h - 1)))
return HostCursorShape(
cg: cg, width: w, height: h,
hotX: min(ev.hotX, w - 1), hotY: min(ev.hotY, h - 1))
}
/// Points-per-host-pixel: the exact factor the video frame is aspect-fit into the view (the same
/// `AVMakeRect` fit `hostPoint`/`cgScreenPoint` use). The host forwards the pointer bitmap in host
/// framebuffer pixels the mode we drive is in the client's BACKING pixels, so on retina this is
/// ~1/backingScale and the pointer lands at its TRUE size relative to the streamed desktop
/// (crisp, 1:1 with the video) rather than the 2×-inflated pixel-as-points it used to be. Because
/// the bitmap grows with the host's display scaling (96 px at 300% DPI), scaling by this is what
/// keeps a high-DPI host from forwarding a giant pointer. Falls back to 1 before the first
/// mode/layout.
private func cursorFitScale() -> CGFloat {
guard let connection else { return 1 }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0, bounds.width > 0, bounds.height > 0 else { return 1 }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)), insideRect: bounds)
guard fit.width > 0 else { return 1 }
return fit.width / CGFloat(mode.width)
}
/// Build the `NSCursor` for a cached shape at the CURRENT video-fit scale (see `cursorFitScale`).
/// Both the image size and the hotspot scale together so the click point stays true.
private func scaledCursor(_ shape: HostCursorShape) -> NSCursor {
let scale = cursorFitScale()
let sw = max(1, (CGFloat(shape.width) * scale).rounded())
let sh = max(1, (CGFloat(shape.height) * scale).rounded())
let image = NSImage(cgImage: shape.cg, size: NSSize(width: sw, height: sh))
let hot = NSPoint(
x: min(CGFloat(shape.hotX) * scale, sw - 1),
y: min(CGFloat(shape.hotY) * scale, sh - 1))
return NSCursor(image: image, hotSpot: hot)
}
/// Host video px CG GLOBAL screen coordinates (top-left origin, the
@@ -908,6 +952,11 @@ public final class StreamLayerView: NSView {
matchFollower?.noteSize(
widthPx: Int(px.width.rounded()), heightPx: Int(px.height.rounded()))
}
// The video-fit scale just changed (resize / retina move); rebuild the worn host pointer at
// the new scale so it tracks the video instead of freezing at its build-time size.
if captured, desktopMouse, cursorChannelActive {
window?.invalidateCursorRects(for: self)
}
}
public override func viewDidChangeBackingProperties() {
@@ -333,6 +333,13 @@ public final class StreamViewController: StreamViewControllerBase {
guard self?.captureEnabled == true else { return }
connection?.send(event)
}
// Apple Pencil the stylus plane, only against a pen-capable host (elsewhere the
// Pencil stays a finger, exactly as before). Same trust gate as touch.
streamView.penEnabled = connection.hostSupportsPen
streamView.onPenBatch = { [weak self, weak connection] batch in
guard self?.captureEnabled == true else { return }
connection?.sendPen(batch)
}
// Indirect pointer (mouse/trackpad) WITHOUT a lock absolute cursor + buttons + scroll.
// While the scene is pointer-LOCKED the GCMouse path owns motion AND buttons AND scroll, so
// the whole UIKit indirect path is gated off here (`gcMouseForwarding`). The trackpad and a
@@ -499,6 +506,8 @@ public final class StreamViewController: StreamViewControllerBase {
// onTouchEvent can still deliver the button-up.
streamView.resetTouchInput()
streamView.onTouchEvent = nil
streamView.onPenBatch = nil // after reset the pen's leave-range sample rides it
streamView.penEnabled = false
streamView.onPointerMoveAbs = nil
streamView.onPointerButton = nil
streamView.onScroll = nil
@@ -692,6 +701,12 @@ final class StreamLayerUIView: UIView {
/// Direct fingers / Pencil wire events: real touches in passthrough mode, or the
/// touch-driven mouse events (`TouchMouse`) in the trackpad/pointer modes.
var onTouchEvent: ((PunktfunkInputEvent) -> Void)?
/// Apple Pencil state-full pen sample batches (the stylus plane). Active only while
/// `penEnabled`; without it the Pencil stays on the finger path exactly as before.
var onPenBatch: (([PunktfunkPenSample]) -> Void)?
/// The host advertised `HOST_CAP_PEN`, so Pencil input splits out of the finger path onto
/// the pen plane independent of the touch-input mode (drawing must not depend on it).
var penEnabled = false
/// Indirect pointer (mouse/trackpad with no lock) absolute cursor moves.
var onPointerMoveAbs: ((HostPoint) -> Void)?
/// Indirect-pointer buttons (GameStream ids: 1=left 3=right); `down` = press.
@@ -715,10 +730,21 @@ final class StreamLayerUIView: UIView {
/// The finger route latched at gesture start a Settings change mid-gesture applies to
/// the NEXT touch, so one gesture never splits across input models.
private var fingerRoute: TouchInputMode?
/// The Apple Pencil pipeline (contacts + hover + squeeze/tap pen samples).
private lazy var pencil: PencilStream = {
let stream = PencilStream()
stream.send = { [weak self] batch in self?.onPenBatch?(batch) }
stream.videoNorm = { [weak self] point in
guard let h = self?.hostPoint(from: point) else { return nil }
return (Float(h.x) / Float(max(h.w - 1, 1)), Float(h.y) / Float(max(h.h - 1, 1)))
}
return stream
}()
/// Release anything the touch-driven mouse holds and forget gesture state session stop.
func resetTouchInput() {
touchMouse.reset()
pencil.reset() // leaves range the host lifts anything still inked
fingerRoute = nil
setSoftKeyboardVisible(false) // a stream that's gone takes its keyboard with it
}
@@ -755,6 +781,11 @@ final class StreamLayerUIView: UIView {
scrollPan.allowedScrollTypesMask = .all
scrollPan.allowedTouchTypes = []
addGestureRecognizer(scrollPan)
// Pencil squeeze / double-tap the pen plane's barrel buttons (no-op while
// `penEnabled` is false PencilStream ignores interactions out of range).
let pencilInteraction = UIPencilInteraction()
pencilInteraction.delegate = pencil
addInteraction(pencilInteraction)
#endif
backgroundColor = .black
}
@@ -779,17 +810,32 @@ final class StreamLayerUIView: UIView {
private enum TouchKind { case down, move, up, cancel }
/// Split a touch batch by kind: an INDIRECT POINTER (mouse/trackpad with no lock) drives
/// the host cursor as an absolute mouse; everything else (direct finger, Pencil) is a host
/// touch. Mixed batches are possible, so partition rather than branch on the first touch.
/// the host cursor as an absolute mouse; a Pencil goes to the pen plane when the host
/// supports it; everything else (direct finger and the Pencil toward a pen-less host)
/// is a host touch. Mixed batches are possible, so partition rather than branch on the
/// first touch.
private func route(_ touches: Set<UITouch>, event: UIEvent?, kind: TouchKind) {
var fingers: Set<UITouch> = []
var pencilTouches: Set<UITouch> = []
for touch in touches {
if touch.type == .indirectPointer {
handleIndirectPointer(touch, event: event, kind: kind)
} else if penEnabled, touch.type == .pencil {
pencilTouches.insert(touch)
} else {
fingers.insert(touch)
}
}
if !pencilTouches.isEmpty {
let phase: PencilStream.Phase =
switch kind {
case .down: .down
case .move: .move
case .up: .up
case .cancel: .cancel
}
pencil.touches(pencilTouches, event: event, phase: phase, in: self)
}
if !fingers.isEmpty { forwardFingers(fingers, kind: kind) }
}
@@ -861,8 +907,11 @@ final class StreamLayerUIView: UIView {
}
}
/// Button-less mouse/trackpad movement (no lock) absolute cursor move.
/// Button-less mouse/trackpad movement (no lock) absolute cursor move unless it is a
/// hovering PENCIL (`zOffset > 0`) on a pen-capable host, which becomes in-range pen
/// samples (hover preview with distance/tilt/azimuth) instead of a cursor move.
@objc private func handleHover(_ recognizer: UIHoverGestureRecognizer) {
if penEnabled, pencil.maybeHover(recognizer, in: self) { return }
switch recognizer.state {
case .began, .changed:
if let h = hostPoint(from: recognizer.location(in: self)) { onPointerMoveAbs?(h) }
+1 -13
View File
@@ -3,7 +3,7 @@
//! scenes.
use crate::app::AppModel;
use crate::trust::{KnownHost, KnownHosts};
use crate::trust::{forget_placeholder, KnownHost, KnownHosts};
use crate::ui_hosts::{ConnectRequest, HostsMsg};
use gtk::glib;
use gtk::prelude::*;
@@ -252,18 +252,6 @@ fn probe_all(hosts: &[KnownHost]) -> Vec<bool> {
)
}
/// Drop an fp-less placeholder for `addr:port` (see `headless_pair`). No-op when none exists.
fn forget_placeholder(addr: &str, port: u16) {
let mut known = KnownHosts::load();
let before = known.hosts.len();
known
.hosts
.retain(|h| !(h.fp_hex.is_empty() && h.addr == addr && h.port == port));
if known.hosts.len() != before {
let _ = known.save();
}
}
/// `--list-hosts [--probe]` — the saved known-hosts store as JSON (the store the Decky plugin
/// renders). With `--probe`, each host carries an `online` bool from a live reachability probe
/// (mDNS-independent); without it, `online` is `null` (unknown — the caller falls back to its
+9 -1
View File
@@ -8,6 +8,7 @@ presenter of the Linux client re-architecture (punktfunk-planning:
```
punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen] [--stats]
punktfunk-session --browse host[:port] [--mgmt PORT] [--fullscreen]
punktfunk-session --pair <PIN> --connect host[:port] [--name LABEL]
```
`--browse` opens the console game library (the Skia coverflow over the animated aurora)
@@ -17,9 +18,16 @@ pairing is the desktop client / Decky plugin's job. `PUNKTFUNK_FAKE_LIBRARY=<fil
feeds canned entries with no host (portrait paths starting with `/` load from disk).
Reads the same identity / known-hosts / settings stores as the desktop client
(`punktfunk-client`) — pair there (or via its headless `--pair`) first; this binary never
(`punktfunk-client`), so enrolling on either side makes the other work; this binary never
connects to a host it has no pinned fingerprint for (`--fp HEX` overrides the store).
`--pair <PIN> --connect host[:port]` runs the SPAKE2 ceremony with no window and no
toolkit, prints `paired <addr>:<port> fp=<hex>`, and exits — the route for a machine that
has only SSH (an embedded/kiosk client, an image being provisioned). `--name` sets the
label the host files this client under, defaulting to the hostname. It is in the
`--no-default-features` build too: enrolling must never be the reason a minimal image has
to pull in Skia.
Stdout is the machine interface: `{"ready":true}` after the first presented frame,
`stats: …` once per second while the overlay tier isn't Off (always the full detailed
text, whatever the OSD shows; `--stats` forces the overlay on), one
+1 -17
View File
@@ -105,7 +105,7 @@ pub fn run(target: Option<&str>) -> u8 {
};
let opts = ConsoleOptions {
device_name: device_name(),
device_name: trust::device_name(),
deck: is_steam_deck(),
};
let (overlay, handles) = match SkiaOverlay::console(opts, entry) {
@@ -251,22 +251,6 @@ pub fn run(target: Option<&str>) -> u8 {
}
}
/// The machine's name — what the host lists this client as after pairing.
fn device_name() -> String {
#[cfg(target_os = "linux")]
if let Ok(s) = std::fs::read_to_string("/etc/hostname") {
let s = s.trim();
if !s.is_empty() {
return s.to_string();
}
}
std::env::var("COMPUTERNAME")
.or_else(|_| std::env::var("HOSTNAME"))
.ok()
.filter(|s| !s.trim().is_empty())
.unwrap_or_else(|| "This device".into())
}
fn host_display_name(name: &str, addr: &str) -> String {
if name.trim().is_empty() {
addr.to_string()
+62 -4
View File
@@ -5,8 +5,9 @@
//! One stream session per invocation: `--connect host[:port]` (+ `--fp HEX`,
//! `--launch id`, `--fullscreen`), exits when the session ends. Reads the same identity
//! / known-hosts / settings stores as the desktop shell on each OS — the GTK client
//! (`punktfunk-client`) on Linux, the WinUI client on Windows — so pairing there (or
//! via the shell's headless `--pair`) makes this binary connect silently.
//! (`punktfunk-client`) on Linux, the WinUI client on Windows — so pairing on either side
//! makes the other connect silently. `--pair <PIN> --connect host` runs the ceremony here,
//! with no window and no toolkit, for machines that have only a shell.
//!
//! Stdout is the machine interface (the shell↔session contract): `{"ready":true}` after
//! the first presented frame, `stats:` lines per 1 s window, one `{"error": …}` /
@@ -61,6 +62,54 @@ mod session_main {
Some((x.trim().parse().ok()?, y.trim().parse().ok()?))
}
/// `--pair <PIN> --connect host[:port]` — the SPAKE2 PIN ceremony with no window, no GTK
/// and no console UI, so a machine that has only SSH can be enrolled: an embedded/kiosk
/// client, a headless box, an image being provisioned. Writes the verified host into the
/// same known-hosts store `--connect` reads, so pairing here is exactly what makes the
/// later stream connect silently.
///
/// Deliberately identical in shape and output to `punktfunk-client --pair` (which stays
/// the desktop route) — the difference is only that this binary carries no toolkit, so it
/// is the one a minimal image installs. Present in the `--no-default-features` build too:
/// enrolment must not be the reason an embedded image has to pull in Skia.
fn headless_pair(pin: &str) -> u8 {
let Some(target) = arg_value("--connect") else {
eprintln!("--pair requires --connect host[:port]");
return EXIT_CONNECT_FAILED;
};
let (addr, port) = parse_host_port(&target);
// The label the HOST files this client under. A headless box has nobody to ask, so
// the hostname is the only name that will mean anything in the paired-devices list.
let name = arg_value("--name").unwrap_or_else(trust::device_name);
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
eprintln!("client identity: {e:#}");
return EXIT_CONNECT_FAILED;
}
};
match trust::pair_with_host(&addr, port, &identity, pin, &name) {
Ok(fp) => {
let fp_hex = trust::hex(&fp);
trust::persist_host(
&arg_value("--host-label").unwrap_or_else(|| addr.clone()),
&addr,
port,
&fp_hex,
true,
);
trust::forget_placeholder(&addr, port);
println!("paired {addr}:{port} fp={fp_hex}");
0
}
Err(e) => {
eprintln!("pairing failed: {} ({e:?})", trust::pair_error_message(&e));
EXIT_TRUST_REJECTED
}
}
}
/// `host[:port]`, port defaulting to the native 9777.
pub(crate) fn parse_host_port(target: &str) -> (String, u16) {
match target.rsplit_once(':') {
@@ -312,6 +361,13 @@ mod session_main {
};
}
// `--pair <PIN>`: enrol this machine against a host and exit. Sits with the other
// non-streaming subcommands, above every graphics call — the box doing this may have
// no display at all.
if let Some(pin) = arg_value("--pair") {
return headless_pair(&pin);
}
// Before any Vulkan call: make RADV expose its video-decode queue + extensions so the
// decoder's `auto` path prefers Vulkan Video over VAAPI (Steam Deck, and any gated RADV).
// Windows drivers (NVIDIA/AMD Adrenalin) expose theirs unconditionally.
@@ -369,12 +425,14 @@ mod session_main {
eprintln!(
"usage: punktfunk-session --connect host[:port] [--fp HEX] [--launch id] [--fullscreen]\n\
\x20 punktfunk-session --browse [host[:port]] [--mgmt PORT] [--fullscreen] [--json-status]\n\
\x20 punktfunk-session --pair <PIN> --connect host[:port] [--name LABEL]\n\
\n\
Streams from a paired punktfunk host in a Vulkan window. --browse opens the\n\
gamepad console instead: bare --browse is the host list (discovery, PIN\n\
pairing, settings, wake-on-LAN); with a target it opens that host's game\n\
library. --connect never dials a host it has no pinned fingerprint for \n\
pair in the console or via `punktfunk-client --pair <PIN> --connect `."
enrol with --pair (no display needed), in the console, or from the desktop\n\
client."
);
return EXIT_CONNECT_FAILED;
};
@@ -406,7 +464,7 @@ mod session_main {
"error",
&format!(
"no pinned fingerprint for {addr}:{port} — pair first \
(punktfunk-client --pair <PIN> --connect {addr}:{port}) or pass --fp HEX"
(punktfunk-session --pair <PIN> --connect {addr}:{port}) or pass --fp HEX"
),
Some(true),
);
+12
View File
@@ -405,6 +405,16 @@ pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
/// The mid-stream cursor-render flip (`IOCTL_SET_CURSOR_FORWARD`, proto v6) as a host-facade
/// closure — same contract as [`CursorChannelSender`]. `bool` = declare the IddCx hardware
/// cursor (`true`) or stand it down (`false`; the host facade additionally forces the same-mode
/// re-commit that actualises the OS's software-cursor default). The capturer drives this from
/// its secure-desktop watch: UAC/Winlogon render only through the software-cursor path, so a
/// path pinned to the hardware cursor never presents them (the 0.18.0 secure-desktop
/// regression).
#[cfg(target_os = "windows")]
pub type CursorForwardSender = std::sync::Arc<dyn Fn(bool) -> Result<()> + Send + Sync>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")]
pub mod pwinit;
@@ -491,6 +501,7 @@ pub fn open_idd_push(
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
cursor_forward: Option<CursorForwardSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(
target,
@@ -501,6 +512,7 @@ pub fn open_idd_push(
keepalive,
sender,
cursor_sender,
cursor_forward,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
+27 -1
View File
@@ -2191,7 +2191,15 @@ mod pipewire {
// consumer imports raw dmabufs itself — the VAAPI backend (libva import + GPU CSC) or a
// PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw
// dmabuf straight to the encoder.
let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough;
//
// ...unless the encoder already proved it cannot import them here. A driver that refuses
// the compositor's buffers refuses them identically on every retry, so without this the
// session died on its first frame and every reconnect repeated it. The latch (set by the
// encode side after consecutive import failures) is what turns that into one bad session
// followed by a working, if slower, host.
let raw_dmabuf_off = raw_passthrough && pf_zerocopy::raw_dmabuf_import_disabled();
let vaapi_passthrough =
zerocopy && !force_shm && importer.is_none() && raw_passthrough && !raw_dmabuf_off;
// Producer-side NV12 (default-on; PUNKTFUNK_PIPEWIRE_NV12=0 escapes): gamescope offers a
// one-fd LINEAR NV12 image when the consumer asks — its compositor pass does the RGB→YUV,
// and the Vulkan Video encoder imports the buffer as its encode source directly (no host
@@ -2249,6 +2257,12 @@ mod pipewire {
tracing::info!(
"capture: PUNKTFUNK_FORCE_SHM — race-free SHM download path (no dmabuf, no zero-copy)"
);
} else if raw_dmabuf_off {
tracing::warn!(
"zero-copy raw-dmabuf passthrough disabled after repeated encoder import failures \
capturing CPU frames instead (this host's GPU driver would not import the \
compositor's buffers)"
);
} else if zerocopy && !want_dmabuf {
tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path");
} else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() {
@@ -2652,6 +2666,18 @@ mod pipewire {
build_default_format_obj(preferred)
};
// gamescope trap — the Steam overlay's presence in the stream is decided HERE by omission:
// gamescope's `paint_pipewire()` composites the overlay (Shift+Tab / Quick Access Menu) into
// the node it hands us ONLY when the consumer-negotiated `gamescope_focus_appid` is 0 — the
// default, and the "mirror the focused window + overlay" branch (gamescope ≥ 3.16.23; see
// `MIN_GAMESCOPE_OVERLAY`). None of the EnumFormat pods below advertise the
// `SPA_FORMAT_VIDEO_gamescope_focus_appid` property, so gamescope reads 0 and paints the
// overlay for us for free. DO NOT add a non-zero focus-appid (e.g. to "isolate the game" in a
// dedicated session) — that flips gamescope into the Remote-Play branch that deliberately
// drops the overlay (and all host chrome) back out of the capture. The cursor, external
// overlay (MangoHUD), and notifications are excluded from the node on EVERY gamescope
// version and are composited host-side instead (see `xfixes_cursor.rs`).
//
// When zero-copy is on, offer ONLY a BGRx dmabuf format with our EGL-importable modifiers
// (offering shm too makes the compositor pick shm). The modifier list is advertised with
// DONT_FIXATE so the compositor's allocator chooses one; we re-emit the fixated format in
+234 -33
View File
@@ -10,20 +10,31 @@
//!
//! **Multiple Xwaylands.** gamescope runs one Xwayland per `--xwayland-count` (Steam Gaming Mode
//! uses 2: one for Big Picture, one for the game). The pointer lives on whichever is FOCUSED — an
//! inactive display's pointer is frozen. So the source connects to ALL of them and each tick
//! follows the one whose pointer actually moved (gamescope routes input to the focused surface, so
//! exactly one moves at a time). It reads that display's shape too, since each Xwayland has its own
//! current cursor. This is why a single-display read froze the pointer the moment a game on the
//! OTHER Xwayland took focus.
//! inactive display's pointer is frozen. So the source connects to ALL of them and publishes from
//! the one gamescope is actually drawing the pointer on; it reads that display's shape too, since
//! each Xwayland has its own current cursor. This is why a single-display read froze the pointer
//! the moment a game on the OTHER Xwayland took focus.
//!
//! Two X sources per display, split by cost (Sunshine's split):
//! Three X sources per display, split by cost (Sunshine's split, plus gamescope's own verdict):
//! * **Position** — core `QueryPointer` on the root, polled fast. Cheap (a few-byte reply, no
//! bitmap), so it can out-pace the stream fps and keep the composited pointer smooth. It also
//! doubles as the focus signal (the display whose pointer moves is the active one).
//! * **Shape / hotspot / visibility** — `XFixesGetCursorImage`, refreshed only after an XFixes
//! `CursorNotify` (a real cursor change). A game hiding the pointer IS a cursor change → the
//! image comes back fully transparent → `visible: false`, which the encode loop strips before
//! any blend path draws it (so a grabbed pointer shows nothing, matching native gamescope).
//! bitmap), so it can out-pace the stream fps and keep the composited pointer smooth.
//! * **Shape / hotspot** — `XFixesGetCursorImage`, refreshed only after an XFixes `CursorNotify`
//! (a real cursor change). A fully-transparent image reads as hidden.
//! * **Visibility + focus** — [`GAMESCOPE_CURSOR_VISIBLE_FEEDBACK`](GS_CURSOR_FEEDBACK) on the
//! root, read at connect and re-read on its `PropertyNotify`.
//!
//! **Why the feedback atom and not pointer motion.** gamescope hides its pointer by WARPING the X
//! pointer to the root's bottom-right corner pixel — it does NOT swap in a transparent X cursor, so
//! `XFixesGetCursorImage` keeps handing back the last opaque arrow. The original "follow whichever
//! display's pointer moved" heuristic therefore stuck to the parked display (a parked pointer never
//! moves again) and composited that arrow at `(w-1, h-1)`: a sliver of cursor welded to the corner
//! of the stream for the rest of the session, while the real pointer went undrawn. Reported
//! on-glass as "part of cursor shows up on bottom right … isn't where it really is", in every game
//! (a game grabs the pointer, so the hide is permanent). Measured on a live 1920x1080 Gaming Mode
//! session: real motion ⇒ pointer live + feedback `1`; 3 s idle (`--hide-cursor-delay 3000`) ⇒
//! pointer `(1919, 1079)` + feedback `0`. The atom answers BOTH questions correctly for a static
//! pointer, which motion cannot: which Xwayland owns it, and whether to draw it at all — so
//! honouring it also gives the stream gamescope's own idle auto-hide, which this source never had.
use std::sync::{
atomic::{AtomicBool, Ordering},
@@ -35,7 +46,11 @@ use pf_frame::CursorOverlay;
use x11rb::connection::Connection;
use x11rb::errors::ReplyError;
use x11rb::protocol::xfixes::{self, ConnectionExt as _, GetCursorImageReply};
use x11rb::protocol::xproto::{ConnectionExt as _, QueryPointerReply, Window};
use x11rb::protocol::xproto::{
Atom, AtomEnum, ChangeWindowAttributesAux, ConnectionExt as _, EventMask, QueryPointerReply,
Window,
};
use x11rb::protocol::Event;
use x11rb::rust_connection::RustConnection;
/// Serializes the brief `XAUTHORITY` env swap around a connect (the var is process-global). Only
@@ -47,6 +62,17 @@ static XAUTH_LOCK: Mutex<()> = Mutex::new(());
/// and must out-run a 240 fps session or the pointer stutters.
const POLL: Duration = Duration::from_millis(4);
/// gamescope's own pointer verdict, published on EVERY nested Xwayland's root: `1` on the server
/// whose pointer gamescope is currently drawing, `0` on the others — and `0` on all of them once
/// the pointer is hidden (a game grabbed it, or `--hide-cursor-delay` fired). See the module docs
/// for why this, and not pointer motion, is the signal this source follows.
const GS_CURSOR_FEEDBACK: &str = "GAMESCOPE_CURSOR_VISIBLE_FEEDBACK";
/// Self-heal cadence for the feedback re-read: `PropertyNotify` drives it, this only covers a
/// missed/coalesced event (and a gamescope that publishes the atom after we connected). One
/// `GetProperty` per display per interval is nothing next to the 250 Hz pointer poll.
const FEEDBACK_RESYNC: Duration = Duration::from_millis(250);
/// A running XFixes cursor reader. Dropping it stops the worker thread and joins it, releasing the
/// X connections — so it lives exactly as long as the capturer that owns it.
pub(super) struct XFixesCursorSource {
@@ -68,7 +94,9 @@ impl XFixesCursorSource {
let mut displays = Vec::new();
for (dpy, xauth) in targets {
match connect(&dpy, xauth.as_deref()) {
Ok((conn, root)) => displays.push(XDisplay::new(dpy, conn, root)),
Ok((conn, root, feedback)) => {
displays.push(XDisplay::new(dpy, conn, root, feedback))
}
Err(e) => tracing::warn!(
dpy = %dpy,
error = %e,
@@ -84,9 +112,13 @@ impl XFixesCursorSource {
return None;
}
let names: Vec<&str> = displays.iter().map(|d| d.name.as_str()).collect();
let feedback = displays.iter().any(|d| d.gs_visible.is_some());
tracing::info!(
displays = ?names,
"gamescope cursor: XFixes source live — following the focused Xwayland's pointer"
cursor_feedback = feedback,
"gamescope cursor: XFixes source live — following the Xwayland gamescope draws the \
pointer on (cursor_feedback=false this gamescope publishes no \
GAMESCOPE_CURSOR_VISIBLE_FEEDBACK, degrading to the pointer-motion heuristic)"
);
let stop = Arc::new(AtomicBool::new(false));
@@ -111,10 +143,15 @@ impl Drop for XFixesCursorSource {
}
}
/// Open the X connection, negotiate XFixes, and select cursor-change events — returning the
/// connection + root window. `RustConnection` reads `XAUTHORITY` from the env at connect time only,
/// so set it under the lock (the host isn't a gamescope child), connect, then restore.
fn connect(dpy: &str, xauthority: Option<&str>) -> Result<(RustConnection, Window), String> {
/// Open the X connection, negotiate XFixes, and select cursor-change + root-property events —
/// returning the connection, root window and this display's initial
/// [`GAMESCOPE_CURSOR_FEEDBACK`](GS_CURSOR_FEEDBACK) reading (`(atom, value)`; the value is `None`
/// when gamescope publishes no such property here). `RustConnection` reads `XAUTHORITY` from the
/// env at connect time only, so set it under the lock (the host isn't a gamescope child), connect,
/// then restore.
type Connected = (RustConnection, Window, (Atom, Option<bool>));
fn connect(dpy: &str, xauthority: Option<&str>) -> Result<Connected, String> {
let (conn, screen_num) = {
let _g = XAUTH_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let prev = std::env::var_os("XAUTHORITY");
@@ -148,8 +185,41 @@ fn connect(dpy: &str, xauthority: Option<&str>) -> Result<(RustConnection, Windo
.map_err(ReplyError::from)
.and_then(|c| c.check())
.map_err(|e| format!("SelectCursorInput: {e}"))?;
// …and whenever gamescope republishes its cursor verdict. Interned with `only_if_exists=false`
// so we hold a matchable atom id even on a gamescope that sets the property later; a failure to
// select PROPERTY_CHANGE is NOT fatal — the resync re-read still tracks it, just at 250 ms.
let feedback_atom = conn
.intern_atom(false, GS_CURSOR_FEEDBACK.as_bytes())
.map_err(ReplyError::from)
.and_then(|c| c.reply())
.map(|r| r.atom)
.unwrap_or(0);
if let Ok(c) = conn.change_window_attributes(
root,
&ChangeWindowAttributesAux::new().event_mask(EventMask::PROPERTY_CHANGE),
) {
let _ = c.check();
}
let _ = conn.flush();
Ok((conn, root))
let feedback = read_cursor_feedback(&conn, root, feedback_atom);
Ok((conn, root, (feedback_atom, feedback)))
}
/// Read `GAMESCOPE_CURSOR_VISIBLE_FEEDBACK` off `root`. `None` = the property is absent or
/// unreadable (not a gamescope that publishes it) — the caller then falls back to the pointer-motion
/// heuristic rather than blanking the cursor, so an older gamescope keeps today's behaviour.
fn read_cursor_feedback(conn: &RustConnection, root: Window, atom: Atom) -> Option<bool> {
if atom == 0 {
return None;
}
let reply = conn
.get_property(false, root, atom, AtomEnum::CARDINAL, 0, 1)
.ok()?
.reply()
.ok()?;
let value = reply.value32()?.next()?;
Some(value != 0)
}
/// One gamescope Xwayland the source tracks.
@@ -163,12 +233,22 @@ struct XDisplay {
shape: Shape,
/// A `CursorNotify` (or first read) is pending — fetch the shape when this display is active.
need_shape: bool,
/// Interned [`GS_CURSOR_FEEDBACK`] atom (`0` = intern failed — treated as absent).
feedback_atom: Atom,
/// gamescope's verdict for THIS display: `Some(true)` = it is drawing the pointer here,
/// `Some(false)` = it is not, `None` = this gamescope publishes no verdict at all.
gs_visible: Option<bool>,
/// The X connection died (game/Xwayland exited) — skip it.
dead: bool,
}
impl XDisplay {
fn new(name: String, conn: RustConnection, root: Window) -> Self {
fn new(
name: String,
conn: RustConnection,
root: Window,
(feedback_atom, gs_visible): (Atom, Option<bool>),
) -> Self {
XDisplay {
name,
conn,
@@ -176,9 +256,20 @@ impl XDisplay {
last_pos: None,
shape: Shape::default(),
need_shape: true,
feedback_atom,
gs_visible,
dead: false,
}
}
/// Re-read gamescope's verdict, keeping a previously-seen one if the read fails (a transient
/// failure must not look like "this gamescope has no feedback" and re-arm the motion heuristic).
fn resync_feedback(&mut self) {
let fresh = read_cursor_feedback(&self.conn, self.root, self.feedback_atom);
if fresh.is_some() || self.gs_visible.is_none() {
self.gs_visible = fresh;
}
}
}
/// Cached cursor shape for one display.
@@ -209,20 +300,35 @@ fn run(
let mut out_serial = 0u64;
let mut last_key = (usize::MAX, u64::MAX);
let mut warned_image = false;
let mut last_resync = std::time::Instant::now();
while !stop.load(Ordering::Relaxed) {
// 1) Poll every display's pointer; note which moved since last tick (the focus signal).
// A missed/coalesced PropertyNotify would otherwise strand the verdict — re-read on a slow
// cadence so the source always converges (also picks the atom up if gamescope adds it late).
let resync = last_resync.elapsed() >= FEEDBACK_RESYNC;
if resync {
last_resync = std::time::Instant::now();
}
// 1) Poll every display's pointer; note which moved since last tick (the fallback focus
// signal, used only when this gamescope publishes no cursor verdict).
let mut active_moved = false;
let mut other_moved: Option<usize> = None;
for (i, d) in displays.iter_mut().enumerate() {
if d.dead {
continue;
}
// Drain pending events; the only ones selected are CursorNotify, so ANY event means
// "re-read this display's shape". poll_for_event never blocks.
// Drain pending events. Two kinds are selected: XFixes CursorNotify (the shape
// changed) and root PropertyNotify (gamescope republished its cursor verdict, among
// the many other properties it keeps on the root — hence the atom match).
let mut need_feedback = resync;
loop {
match d.conn.poll_for_event() {
Ok(Some(_)) => d.need_shape = true,
Ok(Some(Event::XfixesCursorNotify(_))) => d.need_shape = true,
Ok(Some(Event::PropertyNotify(ev))) => {
need_feedback |= d.feedback_atom != 0 && ev.atom == d.feedback_atom;
}
Ok(Some(_)) => {}
Ok(None) => break,
Err(_) => {
d.dead = true;
@@ -230,6 +336,9 @@ fn run(
}
}
}
if need_feedback && !d.dead {
d.resync_feedback();
}
match fetch_pointer(&d.conn, d.root) {
Ok(p) if p.same_screen => {
let pos = (i32::from(p.root_x), i32::from(p.root_y));
@@ -248,13 +357,11 @@ fn run(
}
}
// 2) Switch focus: sticky to the active display while it moves (no flapping); otherwise
// follow another display that moved. If the active one died, fall to any live display.
if !active_moved {
if let Some(j) = other_moved {
active = j;
}
}
// 2) Pick the display to publish from, and decide whether a pointer should be drawn at all.
let states: Vec<(bool, Option<bool>)> =
displays.iter().map(|d| (d.dead, d.gs_visible)).collect();
let hidden_by_gamescope;
(active, hidden_by_gamescope) = pick_active(&states, active, active_moved, other_moved);
if displays.get(active).is_none_or(|d| d.dead) {
match displays.iter().position(|d| !d.dead) {
Some(k) => active = k,
@@ -283,7 +390,11 @@ fn run(
// 4) Publish the ACTIVE display's pointer + shape (or clear the slot when it has no cursor
// of its own — so a focus switch never leaves the other display's stale pointer showing).
// A pointer gamescope is not drawing is published `visible: false`, NOT dropped: the
// encode loop overwrites the frame's overlay from this slot and strips invisible ones, so
// a `None` here would leave the last visible overlay standing on repeat frames.
let d = &displays[active];
let drawn = d.shape.visible && !hidden_by_gamescope;
let overlay = match (d.last_pos, d.shape.rgba.is_empty()) {
(Some((px, py)), false) => {
let key = (active, d.shape.serial);
@@ -301,7 +412,7 @@ fn run(
serial: out_serial,
hot_x: d.shape.hot_x,
hot_y: d.shape.hot_y,
visible: d.shape.visible,
visible: drawn,
})
}
_ => None,
@@ -314,6 +425,39 @@ fn run(
}
}
/// Which display to publish from, and whether gamescope is drawing NO pointer right now —
/// `(active, hidden)`. `states` is one `(dead, gs_visible)` per display, in `displays` order.
///
/// PREFERRED: gamescope's own verdict. It is authoritative for a STATIC pointer, which is exactly
/// the case motion cannot read — a game grabs the pointer, gamescope parks it in the bottom-right
/// corner, and "follow whichever moved" then never switches again (see the module docs).
///
/// FALLBACK, only when NO live display publishes the verdict: the original motion heuristic —
/// sticky to the active display while its pointer moves (no flapping), else follow another that
/// moved. `hidden` is never asserted on this path, so an older gamescope keeps today's behaviour.
fn pick_active(
states: &[(bool, Option<bool>)],
active: usize,
active_moved: bool,
other_moved: Option<usize>,
) -> (usize, bool) {
let live = |&(dead, _): &(bool, Option<bool>)| !dead;
if states.iter().filter(|s| live(s)).any(|(_, v)| v.is_some()) {
return match states.iter().position(|s| live(s) && s.1 == Some(true)) {
// gamescope is drawing the pointer here — publish from it.
Some(i) => (i, false),
// Drawing none of them (a game grabbed it, or the idle auto-hide fired). Keep `active`
// so the shape cache and last position stay warm for the re-show; the caller publishes
// the overlay `visible: false` instead of dropping it.
None => (active, true),
};
}
match other_moved {
Some(j) if !active_moved => (j, false),
_ => (active, false),
}
}
/// Update `shape` from a fresh `GetCursorImage` reply. A hidden pointer (all-transparent) keeps the
/// last bitmap (instant re-show) but flips visibility; the serial still bumps so the change shows.
fn update_shape(shape: &mut Shape, img: &GetCursorImageReply) {
@@ -364,3 +508,60 @@ fn argb_premul_to_straight_rgba(argb: &[u32]) -> Vec<u8> {
}
out
}
#[cfg(test)]
mod tests {
use super::pick_active;
/// Steam Gaming Mode shape: display 0 = Big Picture's Xwayland, 1 = the game's.
const BPM: usize = 0;
const GAME: usize = 1;
#[test]
fn follows_the_display_gamescope_draws_on() {
// Verdict beats motion: gamescope says the game's Xwayland owns the pointer, so we publish
// from it even though only BPM's (parked) pointer looks like it moved.
let states = [(false, Some(false)), (false, Some(true))];
assert_eq!(pick_active(&states, BPM, false, Some(BPM)), (GAME, false));
}
/// THE REGRESSION: gamescope hides the pointer by warping it to the root's bottom-right corner
/// and leaves the opaque arrow as the X cursor. Nothing moves ever again, so the motion
/// heuristic stayed on the parked display and composited that arrow at (w-1, h-1) — a sliver of
/// cursor welded to the corner of the stream for the whole session, in every game.
#[test]
fn a_pointer_gamescope_draws_nowhere_is_hidden_not_parked() {
let states = [(false, Some(false)), (false, Some(false))];
// `active` is kept (shape cache + last position stay warm for the re-show) but hidden.
assert_eq!(pick_active(&states, GAME, false, None), (GAME, true));
}
#[test]
fn re_show_returns_to_the_drawing_display() {
let states = [(false, Some(true)), (false, Some(false))];
assert_eq!(pick_active(&states, GAME, false, None), (BPM, false));
}
#[test]
fn a_dead_displays_verdict_is_ignored() {
// The game's Xwayland exited mid-session with a stale `Some(true)`; BPM is the live one.
let states = [(false, Some(true)), (true, Some(true))];
assert_eq!(pick_active(&states, GAME, false, None), (BPM, false));
// …and a dead display's `Some` must not count as "this gamescope publishes a verdict",
// which would blank the cursor forever on a gamescope that publishes none.
let states = [(false, None), (true, Some(false))];
assert_eq!(pick_active(&states, BPM, false, Some(GAME)), (GAME, false));
}
#[test]
fn no_verdict_falls_back_to_the_motion_heuristic() {
let none = [(false, None), (false, None)];
// Sticky while the active display's own pointer moves…
assert_eq!(pick_active(&none, BPM, true, Some(GAME)), (BPM, false));
// …otherwise follow the one that moved…
assert_eq!(pick_active(&none, BPM, false, Some(GAME)), (GAME, false));
// …and never assert `hidden` on this path (that would regress an older gamescope to a
// cursorless stream).
assert_eq!(pick_active(&none, BPM, false, None), (BPM, false));
}
}
+102 -1
View File
@@ -408,6 +408,16 @@ pub struct IddPushCapturer {
/// shared slot) destroys the driver's cursor worker — the section here survives, so the
/// channel is re-delivered on ring recreates.
cursor_sender: Option<crate::CursorChannelSender>,
/// The cursor-render flip sender (`IOCTL_SET_CURSOR_FORWARD`) — the secure-desktop guard's
/// actuator. UAC/Winlogon render only through the OS's software-cursor path (its default on
/// every mode commit); with our hardware cursor declared (and re-declared on every
/// swap-chain assign) that path never comes back, and the secure desktop never presents —
/// the stream freezes on the last normal-desktop frame for the whole UAC/lock interaction.
/// [`Self::poll_secure_desktop`] flips the declare off/on at the secure-desktop edges.
cursor_forward: Option<crate::CursorForwardSender>,
/// The secure-desktop guard's edge state: `true` = the poller reports a secure input
/// desktop and the declare is currently stood down.
secure_active: bool,
/// The CAPTURE mouse model is active — the HOST composites the pointer into the frame
/// (see cursor_blend.rs for why DWM cannot: a declared IddCx hardware cursor is forever).
composite_cursor: bool,
@@ -657,7 +667,6 @@ impl IddPushCapturer {
/// instead of tearing the display down (audit §5.1 — no more 20 s black bail). "Failure" includes the
/// driver not attaching to the ring within a few seconds (e.g. a hybrid-GPU render mismatch).
#[allow(clippy::too_many_arguments)]
#[allow(clippy::too_many_arguments)]
pub fn open(
target: WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
@@ -667,6 +676,7 @@ impl IddPushCapturer {
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
@@ -679,6 +689,7 @@ impl IddPushCapturer {
pyrowave,
sender,
cursor_sender,
cursor_forward,
) {
Ok(mut me) => {
me._keepalive = keepalive;
@@ -697,6 +708,7 @@ impl IddPushCapturer {
pyrowave: bool,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -720,6 +732,7 @@ impl IddPushCapturer {
luid,
sender.clone(),
cursor_sender.clone(),
cursor_forward.clone(),
) {
Ok(me) => Ok(me),
Err(e) => {
@@ -754,6 +767,7 @@ impl IddPushCapturer {
drv,
sender,
cursor_sender,
cursor_forward,
)
.context("IDD-push rebind to the driver's reported render adapter")
}
@@ -770,6 +784,7 @@ impl IddPushCapturer {
luid: LUID,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> {
let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -1046,6 +1061,17 @@ impl IddPushCapturer {
.unwrap_or((0, 0, i32::MAX, i32::MAX));
cursor_poll::CursorPoller::spawn(target.target_id, rect)
});
// Heal the driver's persisted cursor-forward state: a session that died on the
// secure desktop (client drops at the lock screen — the common case) leaves the
// per-target desired state `false`, and the NEXT session's channel delivery would
// adopt UNdeclared (the exact cross-session composite trap of §8.6). A fresh
// session always starts declared; the secure-desktop guard re-disables if the
// secure desktop is (still) up, via its first `poll_secure_desktop` edge.
if let (Some(_), Some(fwd)) = (cursor_shared.as_ref(), cursor_forward.as_ref()) {
if let Err(e) = fwd(true) {
tracing::debug!("cursor-forward reset at open failed (pre-v6 driver?): {e:#}");
}
}
tracing::info!(
target_id = target.target_id,
@@ -1108,6 +1134,8 @@ impl IddPushCapturer {
cursor_shared,
cursor_poll,
cursor_sender,
cursor_forward,
secure_active: false,
composite_cursor: composite_forced,
composite_forced,
cursor_blend: None,
@@ -1883,8 +1911,71 @@ impl IddPushCapturer {
Some((tex, srv))
}
/// The secure-desktop guard (the 0.18.0 UAC/Winlogon regression). UAC consent and Winlogon
/// live on the SECURE desktop, which the OS renders through the software-cursor path — its
/// per-mode-commit default. With this session's IddCx hardware cursor declared (and
/// re-declared by the driver on every swap-chain assign), that path never materialises, the
/// secure desktop never presents into our swap-chain, and the stream freezes on the last
/// normal-desktop frame for the entire UAC/lock interaction. On the poller's secure edge:
/// stand the declare down (`SET_CURSOR_FORWARD` off — the driver stops its per-assign
/// re-declare — plus the host facade's forced same-mode re-commit that actualises the
/// software cursor); on dismissal, re-declare. Runs on the capture/encode thread every tick
/// (it must keep running while frames are stalled — that is exactly the state it exits).
fn poll_secure_desktop(&mut self) {
let Some(fwd) = self.cursor_forward.as_ref() else {
return;
};
// Sessions with a declare possibly in play: the channel session that declared it, and
// the forced-composite session whose (reused) driver monitor may still run an earlier
// session's cursor worker. A plain session on a clean target has no poller — no guard.
if self.cursor_shared.is_none() && !self.composite_forced {
return;
}
let secure = self
.cursor_poll
.as_ref()
.is_some_and(|p| p.secure_desktop());
if secure == self.secure_active {
return;
}
self.secure_active = secure;
if secure {
tracing::info!(
target_id = self.target_id,
"secure desktop (UAC/Winlogon) active — standing the IddCx hardware-cursor \
declare down so the OS software-cursor path can render it"
);
if let Err(e) = fwd(false) {
tracing::warn!(
"secure-desktop cursor-forward stand-down failed (secure content may stay \
invisible this session): {e:#}"
);
}
} else {
tracing::info!(
target_id = self.target_id,
"secure desktop dismissed — restoring the cursor render model"
);
// Re-declare only for the session that RUNS the cursor channel; a forced-composite
// session never wanted the declare (leaving the driver's desired state off also
// stops a reused worker's per-assign re-declares for good — the next channel
// session's open-time reset re-arms it).
if self.cursor_shared.is_some() {
if let Err(e) = fwd(true) {
tracing::warn!(
"secure-desktop cursor-forward re-enable failed (client-drawn cursor \
may double with a composited one): {e:#}"
);
}
}
}
}
fn try_consume(&mut self) -> Result<Option<CapturedFrame>> {
self.log_driver_status_once();
// The secure-desktop guard first: while UAC/Winlogon is up there may be NO fresh frames
// at all — this edge is what brings them back.
self.poll_secure_desktop();
// Follow the display: a "Use HDR" flip recreates the ring at the matching format.
self.poll_display_hdr();
// Recover-or-drop (GB1): if a descriptor change triggered a recreate but no fresh frame has resumed
@@ -2486,6 +2577,16 @@ fn warn_444_hdr_downgrade_once() {
impl Drop for IddPushCapturer {
fn drop(&mut self) {
// A channel session ending while the secure-desktop guard is engaged must not leave the
// driver's per-target desired state off — the next session's channel delivery would
// adopt UNdeclared and silently run the composite model (§8.6's cross-session trap).
// The open-time reset also covers this (host-crash case); this is the orderly-teardown
// belt.
if self.secure_active && self.cursor_shared.is_some() {
if let Some(fwd) = self.cursor_forward.as_ref() {
let _ = fwd(true);
}
}
self.slots.clear();
// The shared header section (`MappedSection`), the frame-ready `event` (`OwnedHandle`) and the
// broker's WUDFHost process handle free themselves via RAII (unmap view, then close handle) —
@@ -29,8 +29,8 @@ use windows::Win32::Graphics::Gdi::{
BI_RGB, DIB_RGB_COLORS, HBITMAP, HDC,
};
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, OpenInputDesktop, SetThreadDesktop, DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS,
HDESK,
CloseDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, HDESK, UOI_NAME,
};
use windows::Win32::UI::HiDpi::{
SetThreadDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
@@ -63,6 +63,11 @@ struct Shape {
pub(super) struct CursorPoller {
slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>>,
stop: Arc<AtomicBool>,
/// The input desktop is a SECURE desktop (Winlogon — UAC consent / lock / logon). Classified
/// on every reattach; the capturer polls it to stand the IddCx hardware-cursor declare down
/// while the secure desktop needs the software-cursor path to render (see
/// `IddPushCapturer::poll_secure_desktop`).
secure: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
@@ -75,7 +80,11 @@ impl CursorPoller {
const INTERVAL: Duration = Duration::from_millis(4);
/// Unconditional input-desktop reattach cadence — catches secure-desktop (UAC/lock) switches
/// without a failure signal (`GetCursorInfo` on a stale desktop *succeeds* with stale data).
const REATTACH: Duration = Duration::from_secs(2);
/// 250 ms, not the original 2 s: the reattach now also feeds [`Self::secure_desktop`], which
/// gates when the secure desktop becomes VISIBLE in the stream (the hardware-cursor
/// stand-down) — a 2 s freeze at every UAC prompt is user-visible, ~4 `OpenInputDesktop`
/// syscalls/s are not.
const REATTACH: Duration = Duration::from_millis(250);
/// Spawn the poller for the virtual display `target_id`. `rect` = the target's desktop rect
/// (`source_desktop_rect` order: x, y, w, h) — cursor positions are desktop-global; the
@@ -84,15 +93,21 @@ impl CursorPoller {
pub(super) fn spawn(target_id: u32, rect: (i32, i32, i32, i32)) -> Self {
let slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>> = Arc::new(Mutex::new(None));
let stop = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t) = (slot.clone(), stop.clone());
let secure = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t, secure_t) = (slot.clone(), stop.clone(), secure.clone());
let thread = std::thread::Builder::new()
.name("pf-cursor-poll".into())
.spawn(move || run(target_id, rect, &slot_t, &stop_t))
.spawn(move || run(target_id, rect, &slot_t, &stop_t, &secure_t))
.ok();
if thread.is_none() {
tracing::warn!("cursor poller thread spawn failed — cursor falls back to driver shm");
}
Self { slot, stop, thread }
Self {
slot,
stop,
secure,
thread,
}
}
/// The latest overlay snapshot (`None` until the first successful shape rasterisation).
@@ -100,6 +115,12 @@ impl CursorPoller {
self.slot.lock().unwrap_or_else(|p| p.into_inner()).clone()
}
/// Whether the input desktop is currently a SECURE desktop (UAC consent / Winlogon lock or
/// logon). Latched by the poll thread on its reattach cadence (≤ [`Self::REATTACH`] stale).
pub(super) fn secure_desktop(&self) -> bool {
self.secure.load(Ordering::Relaxed)
}
/// Whether the worker thread is (still) alive — `false` degrades the capturer to the shm read.
pub(super) fn alive(&self) -> bool {
self.thread.as_ref().is_some_and(|t| !t.is_finished())
@@ -121,6 +142,7 @@ fn run(
rect: (i32, i32, i32, i32),
slot: &Mutex<Option<pf_frame::CursorOverlay>>,
stop: &AtomicBool,
secure: &AtomicBool,
) {
// Physical-pixel coordinates on this thread regardless of the process's DPI awareness:
// `rect` comes from CCD (always physical), and a DPI-virtualized `GetCursorInfo` position
@@ -130,7 +152,8 @@ fn run(
let _ = unsafe { SetThreadDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) };
let mut desktop = DesktopBinding::default();
desktop.reattach(); // best-effort: already on winsta0\default if this fails
// best-effort: already on winsta0\default if this fails
publish_secure(secure, desktop.reattach());
let mut last_attach = Instant::now();
let mut shape: Option<Shape> = None;
@@ -143,7 +166,7 @@ fn run(
std::thread::sleep(CursorPoller::INTERVAL);
if last_attach.elapsed() >= CursorPoller::REATTACH {
last_attach = Instant::now();
desktop.reattach();
publish_secure(secure, desktop.reattach());
}
let mut ci = CURSORINFO {
@@ -155,7 +178,7 @@ fn run(
if unsafe { GetCursorInfo(&mut ci) }.is_err() {
// Desktop went away under us (secure-desktop switch mid-call) — rebind and retry
// next tick; the slot keeps its last snapshot meanwhile.
desktop.reattach();
publish_secure(secure, desktop.reattach());
last_attach = Instant::now();
continue;
}
@@ -218,13 +241,25 @@ fn run(
}
}
/// Store a reattach's secure-desktop verdict (`None` = classification unavailable — keep the
/// previous state rather than flapping the capturer's hardware-cursor stand-down).
fn publish_secure(secure: &AtomicBool, verdict: Option<bool>) {
if let Some(s) = verdict {
secure.store(s, Ordering::Relaxed);
}
}
/// The thread's owned input-desktop handle — the [`SendInputInjector`] reattach model
/// (`pf-inject` sendinput.rs): keep the current binding, swap on demand, close exactly once.
#[derive(Default)]
struct DesktopBinding(Option<HDESK>);
impl DesktopBinding {
fn reattach(&mut self) {
/// Rebind to the CURRENT input desktop. Returns whether that desktop is a SECURE one
/// (`UOI_NAME` != "Default": "Winlogon" during UAC consent / lock / logon) — `None` when the
/// input desktop could not be opened, in which case the binding (and the caller's secure
/// state) stays put.
fn reattach(&mut self) -> Option<bool> {
const GENERIC_ALL: u32 = 0x1000_0000;
// SAFETY: `OpenInputDesktop`/`SetThreadDesktop`/`CloseDesktop` take only by-value args.
// `OpenInputDesktop` yields an owned `HDESK` only on `Ok`; it is either installed (and the
@@ -238,6 +273,7 @@ impl DesktopBinding {
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
) {
Ok(h) => {
let secure = desktop_is_secure(h);
if SetThreadDesktop(h).is_ok() {
if let Some(old) = self.0.replace(h) {
let _ = CloseDesktop(old);
@@ -245,13 +281,40 @@ impl DesktopBinding {
} else {
let _ = CloseDesktop(h);
}
Some(secure)
}
Err(_) => { /* not privileged for this desktop; stay put */ }
Err(_) => None, // not privileged for this desktop; stay put
}
}
}
}
/// `UOI_NAME` of `h` != "Default" — i.e. the input desktop is Winlogon (UAC consent / lock /
/// logon) or a screen-saver desktop, both of which need the OS's software-cursor render path.
/// Unnameable desktops read as NOT secure: the only in-contract failure is a too-small buffer,
/// and misreading secure-as-normal merely keeps today's behavior for a beat.
fn desktop_is_secure(h: HDESK) -> bool {
let mut name = [0u16; 64]; // "Default"/"Winlogon"/"Screen-saver" all fit with room to spare
let mut needed = 0u32;
// SAFETY: `h` is the live desktop handle the caller just opened; `name`/`needed` are live
// out-params sized exactly as passed; the call writes at most `nlength` bytes.
let ok = unsafe {
GetUserObjectInformationW(
windows::Win32::Foundation::HANDLE(h.0),
UOI_NAME,
Some(name.as_mut_ptr().cast()),
(name.len() * 2) as u32,
Some(&mut needed),
)
};
if ok.is_err() {
return false;
}
let len = name.iter().position(|&c| c == 0).unwrap_or(name.len());
let name = String::from_utf16_lossy(&name[..len]);
!name.eq_ignore_ascii_case("Default")
}
impl Drop for DesktopBinding {
fn drop(&mut self) {
if let Some(h) = self.0.take() {
+34
View File
@@ -212,6 +212,40 @@ pub fn persist_host(name: &str, addr: &str, port: u16, fp_hex: &str, paired: boo
let _ = known.save();
}
/// This machine's name — the label a host files this client under in its paired-devices list.
/// `/etc/hostname` first (the answer on any Linux box, and the only one available in a minimal
/// build with no GTK to ask), then the usual environment fallbacks.
pub fn device_name() -> String {
#[cfg(target_os = "linux")]
if let Ok(s) = std::fs::read_to_string("/etc/hostname") {
let s = s.trim();
if !s.is_empty() {
return s.to_string();
}
}
std::env::var("COMPUTERNAME")
.or_else(|_| std::env::var("HOSTNAME"))
.ok()
.filter(|s| !s.trim().is_empty())
.unwrap_or_else(|| "This device".into())
}
/// Drop an fp-less placeholder entry for `addr:port`. A host added by address before any
/// ceremony (`--add-host` with no `--fp`) is stored keyed by address with an empty fingerprint;
/// once pairing yields the real one, [`persist_host`] writes a second, fp-keyed entry — so the
/// placeholder has to go or the host list shows the same box twice. No-op (and no disk write)
/// when there is none, which is the usual case.
pub fn forget_placeholder(addr: &str, port: u16) {
let mut known = KnownHosts::load();
let before = known.hosts.len();
known
.hosts
.retain(|h| !(h.fp_hex.is_empty() && h.addr == addr && h.port == port));
if known.hosts.len() != before {
let _ = known.save();
}
}
/// Learn/refresh a saved host's Wake-on-LAN MAC(s) from its live advert (called while the host
/// is online, matched by fingerprint or address). No-op — and no disk write — when unchanged, so
/// the hosts page can call it on every discovery tick without churning the store.
+37 -14
View File
@@ -6,11 +6,12 @@
//! queue under the device's [`QueueLock`], fence-waited (sub-ms — Phase-0 measured
//! 0.067 ms GPU at 1080p on the RTX 5070 Ti).
//!
//! Output: three separate R8 planes (Y full-res, Cb/Cr half-res) — the decode path
//! requires STORAGE usage and IDENTITY/R swizzles, so the encoder's two-component
//! RG8 trick is not allowed here (pyrowave.h validation). The presenter samples them
//! with its planar CSC variant (BT.709 limited — the codec's fixed colour contract,
//! there is no VUI). A small ring of plane-sets keeps a decode from overwriting the set
//! Output: three separate single-component planes (Y full-res; Cb/Cr half-res, or
//! full-res on a 4:4:4 session) — R8, or R16 UNORM on a 10-bit session — the decode
//! path requires STORAGE usage and IDENTITY/R swizzles, so the encoder's two-component
//! RG trick is not allowed here (pyrowave.h validation). The presenter samples them
//! with its planar CSC variant (colour per the negotiated `ColorInfo` — the wavelet
//! bitstream carries no VUI). A small ring of plane-sets keeps a decode from overwriting the set
//! the presenter is still sampling; the synchronous fence bounds decode-side reuse and
//! the ring depth covers present-side latency (≤ 12 frames in this pipeline).
//!
@@ -222,9 +223,10 @@ unsafe extern "C" fn queue_unlock_cb(ud: *mut c_void) {
unsafe { (*(ud as *const crate::video::QueueLock)).unlock() }
}
/// One decoded PyroWave frame: three R8 plane images on the presenter's device, GENERAL
/// layout, decode-complete (the decoder fence-waits before handing it over). `slot`
/// identifies the ring entry; the images/views live as long as the decoder.
/// One decoded PyroWave frame: three single-component plane images (R8, or R16 on a
/// 10-bit session) on the presenter's device, GENERAL layout, decode-complete (the
/// decoder fence-waits before handing it over). `slot` identifies the ring entry; the
/// images/views live as long as the decoder.
pub struct PyroWavePlanarFrame {
/// Raw `VkImageView`s (Y, Cb, Cr) for the presenter's planar CSC sampling.
pub views: [u64; 3],
@@ -252,7 +254,8 @@ struct RetiredRing {
retired_at: Instant,
}
/// One decode-output plane: R8, storage (decode writes) + sampled (presenter CSC).
/// One decode-output plane (`fmt` = R8, or R16 on a 10-bit session): storage (decode
/// writes) + sampled (presenter CSC).
unsafe fn make_plane(
device: &ash::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
@@ -539,7 +542,8 @@ impl PyroWaveDecoder {
return Err(e);
}
// Plane-set ring: 3 × R8, storage (decode writes) + sampled (presenter CSC).
// Plane-set ring: 3 single-component planes (R8; R16 on a 10-bit session),
// storage (decode writes) + sampled (presenter CSC).
let mem_props = instance.get_physical_device_memory_properties(
vk::PhysicalDevice::from_raw(vkd.physical_device as u64),
);
@@ -873,12 +877,26 @@ impl PyroWaveDecoder {
&vk::DependencyInfo::default().image_memory_barriers(&pre),
);
// The declared format/extent MUST equal the ring image's real ones: pyrowave wraps
// our VkImage under `image_format` and vkCreateImageView's its storage view with
// `view_format` (pyrowave_c.cpp `WrappedViewBuffers::wrap` — its own
// `pyrowave_image_get_image_view` helper fills both from the image itself).
// Declaring R8 over a 10-bit session's R16_UNORM planes is an invalid view the
// driver executes anyway: its addressing covers half the surface, so decoded 8-bit
// codes fuse pairwise into 16-bit texels (structured garbage) and the never-written
// remainder samples as all-plane zeros (saturated green) — the 2026-07 AMD-client
// field report. Same discipline for chroma extents: a 4:4:4 ring is full-res.
let fmt = if self.hdr16 {
pw::VkFormat_VK_FORMAT_R16_UNORM
} else {
pw::VkFormat_VK_FORMAT_R8_UNORM
};
let plane = |img: vk::Image, w: u32, h: u32| pw::pyrowave_image_view {
image: img.as_raw() as usize as pw::VkImage,
width: w,
height: h,
image_format: pw::VkFormat_VK_FORMAT_R8_UNORM,
view_format: pw::VkFormat_VK_FORMAT_R8_UNORM,
image_format: fmt,
view_format: fmt,
mip_level: 0,
layer: 0,
aspect: pw::VkImageAspectFlagBits_VK_IMAGE_ASPECT_COLOR_BIT,
@@ -886,11 +904,16 @@ impl PyroWaveDecoder {
layout: pw::VkImageLayout_VK_IMAGE_LAYOUT_GENERAL,
};
let (w, h) = (self.width, self.height);
let (cw, ch) = if self.chroma444 {
(w, h)
} else {
(w / 2, h / 2)
};
let buffers = pw::pyrowave_gpu_buffers {
planes: [
plane(self.ring[slot].imgs[0], w, h),
plane(self.ring[slot].imgs[1], w / 2, h / 2),
plane(self.ring[slot].imgs[2], w / 2, h / 2),
plane(self.ring[slot].imgs[1], cw, ch),
plane(self.ring[slot].imgs[2], cw, ch),
],
};
pw::pyrowave_device_set_command_buffer(
+11 -9
View File
@@ -80,12 +80,13 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// and the desktop model gets exclusion + forwarding. Nothing existing changed; against a v5
/// driver the unknown IOCTL fails and the host logs + keeps the declared-at-ADD behavior.
/// v6 tail ext (no bump, the `AddRequest` luminance-tail discipline):
/// [`control::AddReply::cursor_excluded`] — the driver reports whether the ADDed monitor's OS
/// target already carries a hardware-cursor declare from an earlier session. A declare is
/// IRREVOCABLE for the target's life (remote-desktop-sweep §8.6, proven on-glass): DWM never
/// composites the software cursor back into that target's frames, and the sticky state survives
/// monitor REMOVE→ADD because the host hands every client a STABLE target id. The host uses the
/// flag to self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// [`control::AddReply::cursor_excluded`] — the driver reports whether its ADAPTER already
/// carries a hardware-cursor declare from an earlier session. A declare is IRREVOCABLE
/// (remote-desktop-sweep §8.6, proven on-glass) and its exclusion reaches EVERY later monitor of
/// the adapter, not just the declaring target (on-glass 2026-07-23: a declare on one target left
/// a different client's fresh target cursor-less): DWM never composites the software cursor back
/// into any of the adapter's frames until the adapter resets. The host uses the flag to
/// self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// cursor channel — without it those sessions are silently cursor-less. Both skews degrade
/// cleanly: an old driver writes only the 20-byte reply prefix (host reads `0` = unknown/clean),
/// an old host retrieves a 20-byte buffer (driver writes just the prefix).
@@ -217,9 +218,10 @@ pub mod control {
/// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a
/// WUDFHost restart between sessions can never leave the host duplicating into a dead process.
pub wudf_pid: u32,
/// Non-zero = this monitor's OS target already carries an IRREVOCABLE hardware-cursor
/// declare from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer
/// from every frame on this target, forever, and a session without the cursor channel must
/// Non-zero = the ADAPTER already carries an IRREVOCABLE hardware-cursor declare from an
/// earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23): DWM excludes the pointer from every frame on every monitor until
/// the adapter resets, and a session without the cursor channel must
/// self-composite (GDI poller + blend) or stream a cursor-less desktop. Appended after
/// [`ADD_REPLY_LEGACY_SIZE`] under the same dual-size discipline as the `AddRequest`
/// luminance tail: an un-upgraded driver writes only the legacy prefix (the host's
+191 -3
View File
@@ -7,6 +7,45 @@
use anyhow::Result;
use pf_frame::CapturedFrame;
/// Whether an encoder fed `format` must be built 10-bit — decided by **the pixels that actually
/// arrive**, never by the negotiated `bit_depth`.
///
/// The three Windows backends each derived this as `bit_depth >= 10 || matches!(format, P010 |
/// Rgb10a2)`, i.e. the *negotiated* depth could force a 10-bit encoder over an 8-bit capture. That
/// combination is not hypothetical: a client advertises 10-bit, the handshake negotiates
/// `bit_depth = 10`, and then enabling advanced colour on the IDD virtual display fails — at which
/// point the capturer says so and delivers 8-bit NV12 anyway (`pf-capture`'s idd_push logs "10-bit
/// HDR was negotiated but enabling advanced color on the virtual display FAILED — encoding 8-bit
/// SDR"). Every backend then lost the session, each in its own way: native AMF and native QSV
/// `bail!` at open because the format does not match the P010 they derived, and the libavcodec
/// path accepted the open and then failed EVERY submit forever (its per-frame depth check
/// recomputes from the frame, which never matches), where `reset()` could not help because the
/// rebuild re-derived the same wrong depth.
///
/// Following the pixels keeps the stream alive and, more importantly, keeps it HONEST: the depth
/// also selects the colour signalling (BT.2020 PQ vs BT.709) and the staging surface format, so an
/// 8-bit capture now yields an 8-bit stream that says it is SDR — which is what the capturer
/// already reported it is sending. The negotiated depth remains an upper bound; the session label
/// may still claim HDR, and that mismatch belongs to the negotiation, not to the encoder.
/// Windows-only: the three backends that derive an encoder depth from a capture live there
/// (native AMF, native QSV, libavcodec AMF/QSV). The Linux backends take the depth from the
/// negotiated `bit_depth` alone because their capture formats carry it unambiguously.
#[cfg(target_os = "windows")]
pub(crate) fn ten_bit_input(format: pf_frame::PixelFormat, negotiated_depth: u8) -> bool {
use pf_frame::PixelFormat;
let ten = matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2);
if negotiated_depth >= 10 && !ten {
tracing::warn!(
?format,
negotiated_depth,
"session negotiated 10-bit but the capturer delivers an 8-bit format — encoding 8-bit \
SDR (the stream's colour signalling follows the pixels; check whether advanced colour \
failed to enable on the virtual display)"
);
}
ten
}
/// An encoded access unit (one NAL/AU) to hand to `punktfunk_core` for FEC + packetization.
/// `data` is in-band Annex-B (the encoder is opened without a global header), so each
/// keyframe carries its own VPS/SPS/PPS — the bytes are both a playable elementary
@@ -73,7 +112,7 @@ impl AuChunk {
}
/// Codec selection negotiated with the client.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub enum Codec {
H264,
H265,
@@ -255,6 +294,19 @@ pub struct EncoderCaps {
/// `USER_FLAG_RECOVERY_POINT` on every Nth emitted AU, re-phased at each IDR). 0 when intra-refresh
/// is off. Only consulted when [`intra_refresh_recovery`](Self::intra_refresh_recovery) is set.
pub intra_refresh_period: u32,
/// The encoder composites [`CapturedFrame::cursor`] into the picture it encodes.
///
/// `open_video`'s `cursor_blend` argument is a REQUEST, and for most of this crate's life it was
/// nothing else: `lib.rs` literally did `let _ = cursor_blend;` and only three backends ever read
/// `frame.cursor`. So a session could ask for a composited pointer, get a backend that silently
/// discards it, and stream with no mouse cursor at all — the confirmed symptom on the VAAPI
/// dmabuf path and the libav-NVENC CUDA path.
///
/// This makes the answer queryable instead of assumed. It is deliberately a plain fact about the
/// encoder, not a policy: what to DO when a session wants blending and the backend cannot is the
/// host's call, since only the host can re-plan capture (fall back to capturer-side compositing).
/// `open_video` can only warn, which it does.
pub blends_cursor: bool,
}
/// A hardware encoder. One per session; runs on the encode thread.
@@ -313,8 +365,15 @@ pub trait Encoder: Send {
/// encoder analog of the capturer depth escalation: AUs ride ~one loop tick behind (`poll`
/// may return `None` while an encode is in flight) in exchange for capture/submit no longer
/// serializing on the encode wait. Returns whether pipelined retrieve is (now) active; the
/// switch may be deferred to the next safe point internally. `false` from the default impl =
/// unsupported — the session loop stops asking. De-escalation is a v2 item everywhere.
/// switch may be deferred to the next safe point internally. `set_pipelined(true)` returning
/// `false` (the default impl) = unsupported — the session loop stops asking.
///
/// `set_pipelined(false)` requests the wind-back (de-escalation, latency recovery): the
/// backend restores its sync-retrieve mode — and the latency features that mode carries
/// (IO-stream binding, sub-frame chunking) — at its next safe point, usually via a session
/// rebuild whose first frame is an IDR. The return is still "is pipelined retrieve active":
/// the caller polls until it reads `false`. Backends that never escalate return `false`
/// trivially. An operator pin (`PUNKTFUNK_NVENC_ASYNC=1`) refuses the wind-back.
fn set_pipelined(&mut self, _on: bool) -> bool {
false
}
@@ -360,6 +419,16 @@ pub trait Encoder: Send {
fn reconfigure_bitrate(&mut self, _bps: u64) -> bool {
false
}
/// The bitrate (bps) the encoder is ACTUALLY running at (or will open at, for a lazily-opened
/// backend) — the encoder-side truth after any internal clamp, e.g. the direct-NVENC
/// codec-level ceiling search. The session loop reads this after every open/reconfigure and
/// stores IT, not the requested rate, as the live bitrate — so the send pacer, the console
/// and the client controller's ack all track what the ASIC really targets (a controller fed
/// the requested rate keeps climbing from a phantom base, §ABR overdrive). `None` (the
/// default) = the backend doesn't track an applied rate; the caller keeps the requested one.
fn applied_bitrate_bps(&self) -> Option<u64> {
None
}
/// Wire-chunk the encoder's AUs at the session's shard payload size (the PyroWave
/// datagram-aligned mode, plan §4.4): every `shard_payload` window of the emitted AU
/// starts a fresh self-delimiting codec packet, zero-padded to the window — so a lost
@@ -379,6 +448,20 @@ pub trait Encoder: Send {
fn set_input_ring_depth(&mut self, _depth: usize) {}
/// Signal end-of-stream. After this, drain the remaining AUs with [`poll`](Self::poll)
/// until it returns `None` — NVENC buffers frames internally even at `delay=0`.
///
/// **The two production encode loops deliberately do not call this**, and that is not an
/// oversight to be "fixed" by a later sweep. Both reach their exit only after the transport is
/// already gone (the client disconnected, or the session was stopped), so the AUs a flush would
/// recover have nowhere to go — while flushing is the one call on this trait that can BLOCK on a
/// wedged encoder, on precisely the teardown path a stopped session needs to complete promptly.
/// The Linux direct-SDK NVENC backend makes that concrete: its retrieve-thread join is untimed
/// (see the note in `enc/linux/nvenc_cuda.rs`), so a flush there could hang a session that is
/// already ending.
///
/// It is kept rather than deleted because it does have real consumers: the `spike` dev
/// subcommand, which encodes a FINITE clip and genuinely wants the tail, and the `#[ignore]`d
/// hardware smoke tests across the backends, which assert the drain contract on real GPUs.
/// Those are finite-stream users; a live session is not one.
fn flush(&mut self) -> Result<()>;
}
@@ -413,6 +496,17 @@ impl Codec {
}
}
/// Pixel rate (luma samples/s) at or above which NVENC split-frame encoding is FORCED 2-way —
/// one number shared by the direct-SDK selector (`nvenc_core::resolve_split_mode`) and the libav
/// `split_encode_mode` option author (`linux::NvencEncoder`), so the two paths can never disagree
/// about which modes split. A single NVENC engine tops out ~1 Gpix/s on HEVC, and AUTO doesn't
/// engage below ~2112 px height, so the sessions that need the second engine must be forced. Set
/// BELOW 1 Gpix/s deliberately: 4K120 — the mode this threshold exists for — is 3840×2160×120 =
/// 995,328,000, which a `> 1_000_000_000` gate missed by 0.47% and left on AUTO (pinned ~107 fps
/// on a 4090). 950 M keeps margin for fractional refresh rates while leaving 1440p240 (884.7 M,
/// comfortably single-engine) on AUTO.
pub const SPLIT_FORCE_PIXEL_RATE: u64 = 950_000_000;
/// `PUNKTFUNK_VBV_FRAMES` — HRD/VBV size in frame intervals (default 1.0, the strict low-latency
/// shape every backend ships: each frame must fit its rate share, keeping frame sizes uniform for
/// the pacer). The AMF/VAAPI/QSV paths parse the same variable locally; this helper brings the
@@ -426,6 +520,35 @@ pub(crate) fn vbv_frames_env() -> f64 {
.unwrap_or(1.0)
}
/// The same HRD/VBV window as [`vbv_frames_env`], expressed the way the Vulkan Video encode API
/// wants it: `(virtualBufferSizeInMs, initialVirtualBufferSizeInMs)`.
///
/// Every other backend states the window in **bits** (`bitrate / fps × frames`); Vulkan states it
/// in **milliseconds**. `vulkan_video.rs` consumes this ONLY when the driver advertises VBR
/// (WP6.3): a tight window under CBR makes the driver stuff underspent frames with filler NALs up
/// to the exact rate share — measured 97 % filler on the 780M — because CBR must keep the CPB from
/// overflowing and Vulkan exposes no filler-suppression control. VBR permits the underspend, so
/// the tight window only ever *bounds* a complex frame.
///
/// The initial fill stays at half the window, preserving the RATIO the hardcoded (1000, 500)
/// pair had — the direct-NVENC house shape uses a FULL-window initial fill instead; measured on
/// RADV the difference is inert (the firmware showed no window sensitivity at all). Both
/// VUIDs on `VkVideoEncodeRateControlInfoKHR`'s window fields are satisfied by construction: the
/// window clamps to `>= 1` so it is non-zero, and `window / 2 <= window` always
/// (`VUID-...-08358` is `<=`, relaxed in Vulkan 1.3.299).
///
/// Carries its only caller's gate: `vulkan_video.rs` is the sole ms-form consumer, and with the
/// crate-wide `allow(dead_code)` gone (WP0.3) an item unused in ANY feature combination is a hard
/// error — this is dead on every Windows leg.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
pub(crate) fn vbv_window_ms(fps: u32) -> (u32, u32) {
let frames = vbv_frames_env();
let ms = (frames * 1000.0 / fps.max(1) as f64).round();
// `f64 as u32` saturates at the bounds in Rust, so an absurd `PUNKTFUNK_VBV_FRAMES` cannot wrap.
let window = (ms as u32).max(1);
(window, window / 2)
}
/// Validate a requested encode resolution before we allocate buffers or open NVENC. Rejects
/// zero/odd-sized and out-of-range modes with a clear error instead of letting buffer math
/// overflow or the encoder open fail with an opaque NVENC code. A client can request any
@@ -454,6 +577,26 @@ pub fn validate_dimensions(codec: Codec, width: u32, height: u32) -> Result<()>
(use HEVC/AV1 above 4096, or lower the client resolution)"
);
}
// PyroWave's vendored rate controller packs the 32×32 block index into the low 16 bits of
// `RDOperation::block_offset_saving` (pyrowave-sys `patches/0002-rdo-saving-clamp.patch`).
// Past `u16::MAX` blocks the index collides with the `saving` field, the resolve over-credits,
// and the emitted payload can overshoot the buffer `pyrowave_encoder_packetize` writes into —
// whose only bounds check is an `assert` that the Release (NDEBUG) vendored build compiles out.
// So this is a hard cap, not a quality knob.
//
// Checked against 4:2:0, the *most permissive* chroma: a mode that cannot fit even there can
// fit no PyroWave session at all, so it belongs at this single chokepoint (which both the
// negotiator and `open_video_backend` run) rather than only in the per-backend opens. 4:4:4
// has twice the block count and is checked again at open, where the real chroma is known —
// and the negotiator's 4:4:4 → 4:2:0 downgrade means an oversized mode arrives at the encoder
// as 4:2:0, which is exactly the case the old open-time guard skipped.
#[cfg(feature = "pyrowave")]
if codec == Codec::PyroWave && !crate::pyrowave_mode_fits_rdo(width, height, false) {
anyhow::bail!(
"invalid PyroWave resolution {width}x{height}: exceeds the rate controller's 16-bit \
block index (pyrowave-sys patches/0002) lower the client resolution"
);
}
Ok(())
}
@@ -461,6 +604,31 @@ pub fn validate_dimensions(codec: Codec, width: u32, height: u32) -> Result<()>
mod tests {
use super::*;
/// WP6.3. The window VUIDs on `VkVideoEncodeRateControlInfoKHR` are the whole contract of
/// this helper, and both are edge cases: the window must be non-zero (a high-refresh mode
/// rounds a sub-1 ms window down to nothing) and the initial fill must be at most the window
/// (`<=` — `VUID-...-08358` was relaxed in 1.3.299). Env-free so it pins the default shape —
/// the scaled cases belong to whoever sets `PUNKTFUNK_VBV_FRAMES`. Carries the helper's own
/// cfg gate (see its note), so it runs on the Linux `vulkan-encode` leg.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
#[test]
fn vbv_window_is_about_one_frame_and_always_legal() {
// The house default is ~1 frame interval, not the 1000 ms the Vulkan backend hardwired.
assert_eq!(vbv_window_ms(60).0, 17); // 16.67 ms
assert_eq!(vbv_window_ms(30).0, 33);
assert_eq!(vbv_window_ms(240).0, 4);
for fps in [1, 24, 30, 60, 120, 144, 240, 480, 1000, 4000, u32::MAX] {
let (window, initial) = vbv_window_ms(fps);
assert!(window > 0, "virtualBufferSizeInMs must be > 0 (fps {fps})");
assert!(
initial <= window,
"initialVirtualBufferSizeInMs must be <= virtualBufferSizeInMs (fps {fps})"
);
}
// fps 0 must not divide by zero — `open` clamps, but the helper is called directly too.
assert!(vbv_window_ms(0).0 > 0);
}
#[test]
fn rejects_zero_and_odd_dimensions() {
assert!(validate_dimensions(Codec::H265, 0, 1080).is_err());
@@ -477,6 +645,26 @@ mod tests {
assert!(validate_dimensions(Codec::H264, 3840, 4098).is_err());
}
/// PyroWave's hard cap is the rate controller's 16-bit block index, not just
/// `max_dimension()`. Checked at 4:2:0 (the most permissive chroma), because a mode that
/// cannot fit there cannot fit at any chroma — and because the negotiator's 4:4:4 → 4:2:0
/// downgrade delivers oversized modes to the encoder AS 4:2:0. HEVC/AV1 at the same
/// dimensions must stay unaffected.
#[cfg(feature = "pyrowave")]
#[test]
fn pyrowave_rejects_modes_past_the_rdo_block_index() {
// Fits: 8K 4:2:0 is 49125 blocks.
assert!(validate_dimensions(Codec::PyroWave, 7680, 4320).is_ok());
// Does not fit at 4:2:0 (73728 / 98304 blocks) — must be refused even though both are
// within `Codec::PyroWave.max_dimension()` (8192).
assert!(validate_dimensions(Codec::PyroWave, 8192, 6144).is_err());
assert!(validate_dimensions(Codec::PyroWave, 8192, 8192).is_err());
// The same modes remain legal for the H.26x/AV1 codecs, which have no such rate
// controller — the cap must not leak across codecs.
assert!(validate_dimensions(Codec::H265, 8192, 8192).is_ok());
assert!(validate_dimensions(Codec::Av1, 8192, 6144).is_ok());
}
#[test]
fn hevc_and_av1_allow_up_to_8192() {
for c in [Codec::H265, Codec::Av1] {
+165 -101
View File
@@ -180,7 +180,6 @@ pub struct NvencEncoder {
/// This session opened as full-chroma 4:4:4 (FREXT) — via either input path.
want_444: bool,
src_format: PixelFormat,
expand: bool,
width: u32,
height: u32,
fps: u32,
@@ -413,57 +412,6 @@ impl NvencEncoder {
None
};
// CPU CSC paths: build the packed-RGB → planar swscale (no rescale) into the encoder's
// input frame. Two users: 4:4:4 (RGB→YUV444P, BT.709, range per the flag) and HDR
// (X2RGB10/X2BGR10→P010, BT.2020 limited — the PQ transfer is per-channel and rides
// through the matrix untouched). Skipped on the zero-copy path (`cuda`): the worker's GPU
// convert already delivers ready CUDA frames — no CPU pixels exist to scale.
let sws_csc = if (want_444 || want_hdr10) && !cuda {
let src_av = pixel_to_av(sws_src_pixel(format)?);
let dst_av = pixel_to_av(nvenc_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated packed-RGB source), the dst is
// YUV444P (4:4:4) or P010LE (HDR). The trailing filter/param pointers are null = "use
// defaults" (documented as accepted). No Rust memory is borrowed; the returned pointer is
// null-checked below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
height as c_int,
src_av,
width as c_int,
height as c_int,
dst_av,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
ptr::null(),
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→{nvenc_pixel:?}) failed");
}
// SAFETY: `sws` is the non-null context from the call above (null-checked). The
// coefficient tables from `sws_getCoefficients` (ITU-709 for 4:4:4, BT.2020 NCL for HDR
// — matching the VUI written above) are process-lifetime libswscale statics, reused for
// src+dst matrices; `sws_setColorspaceDetails` only reads them and writes scalar CSC
// settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
// PUNKTFUNK_444_FULLRANGE experiment; HDR is always limited). No Rust memory is passed.
unsafe {
let cs = ffi::sws_getCoefficients(if want_hdr10 {
super::libav::SWS_CS_BT2020
} else {
SWS_CS_ITU709
});
let dst_range = i32::from(full_range_444);
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, dst_range, 0, 1 << 16, 1 << 16);
}
Some(sws)
} else {
None
};
// Low-latency NVENC tuning (plan §7 / linux-setup doc).
let mut opts = Dictionary::new();
opts.set("preset", "p1"); // fastest
@@ -491,15 +439,19 @@ impl NvencEncoder {
}
// Split-frame encode across both NVENC engines (GB203 has 2) when the pixel rate exceeds
// a single engine's HEVC capacity (~1 Gpix/s); e.g. 5120x1440@240 = 1.77 Gpix/s needs it,
// @120 = 0.88 Gpix/s does not. HEVC/AV1 only (not H.264). AUTO won't engage below ~2112px
// a single engine's HEVC capacity; e.g. 5120x1440@240 = 1.77 Gpix/s needs it, @120
// (0.88 Gpix/s) does not. HEVC/AV1 only (not H.264). AUTO won't engage below ~2112px
// height, so we force `2`; below the threshold we leave it AUTO (split costs ~2% BD-rate).
// Output is standard HEVC — transparent to the client. Override with PUNKTFUNK_SPLIT_ENCODE.
// Threshold shared with the direct-SDK selector ([`super::SPLIT_FORCE_PIXEL_RATE`] — set
// so 4K120 = 995.3 Mpix/s forces, which `> 1e9` famously missed by 0.47%). Output is
// standard HEVC — transparent to the client. Override with PUNKTFUNK_SPLIT_ENCODE.
let pix_rate = width as u64 * height as u64 * fps as u64;
let split = std::env::var("PUNKTFUNK_SPLIT_ENCODE").ok();
match split.as_deref() {
Some(mode) => opts.set("split_encode_mode", mode),
None if matches!(codec, Codec::H265 | Codec::Av1) && pix_rate > 1_000_000_000 => {
None if matches!(codec, Codec::H265 | Codec::Av1)
&& pix_rate >= super::SPLIT_FORCE_PIXEL_RATE =>
{
opts.set("split_encode_mode", "2");
tracing::info!(
pix_rate,
@@ -549,6 +501,84 @@ impl NvencEncoder {
);
}
// Built HERE, below the fallible encoder open, NOT above it. `sws_getContext` returns a raw
// pointer whose only free is `Drop for NvencEncoder` — and `Drop` needs a CONSTRUCTED
// `Self`, which does not exist on `open`'s early returns (the intra-refresh-unsupported
// retry, which recurses into `Self::open`, and the plain error return). Creating the
// context above them leaked one per failed attempt, and `open_nvenc_probed`'s EINVAL
// bitrate ladder calls `open` up to ~10 times, so a host stepping its bitrate down leaked a
// context per step. Nothing between here and the `Ok(NvencEncoder { … })` below can return,
// so this placement makes the leak unrepresentable rather than merely unlikely.
// CPU CSC paths: build the packed-RGB → planar swscale (no rescale) into the encoder's
// input frame. THREE users: 4:4:4 (RGB→YUV444P, BT.709, range per the flag), HDR
// (X2RGB10/X2BGR10→P010, BT.2020 limited — the PQ transfer is per-channel and rides
// through the matrix untouched), and the packed 3-bpp expand (RGB24/BGR24→rgb0/bgr0).
//
// The expand used to be a hand-written per-pixel loop in `submit_cpu`: `w*h` iterations,
// each building two bounds-checked sub-slices for a 3-byte copy — a shape LLVM will not
// vectorise into the byte shuffle it is, on the COMMON CPU path (the portal and wlroots
// both commonly fixate packed 24-bit RGB, and pf-capture offers it first). swscale's
// packed-RGB expanders are SIMD, the sibling VAAPI backend already routes RGB24/BGR24
// through them, and this file already owned the context lifecycle — so the change is net
// subtractive. The three are mutually exclusive by construction: `expand` is only ever
// true on the packed-RGB 4:2:0 path (see `nvenc_pixel`/`expand` above), never with
// `want_444`, so one context serves whichever applies.
//
// Skipped on the zero-copy path (`cuda`): the worker's GPU convert already delivers ready
// CUDA frames — no CPU pixels exist to scale.
let sws_csc = if (want_444 || want_hdr10 || expand) && !cuda {
let src_av = pixel_to_av(sws_src_pixel(format)?);
let dst_av = pixel_to_av(nvenc_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated packed-RGB source), the dst is
// YUV444P (4:4:4) or P010LE (HDR). The trailing filter/param pointers are null = "use
// defaults" (documented as accepted). No Rust memory is borrowed; the returned pointer is
// null-checked below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
height as c_int,
src_av,
width as c_int,
height as c_int,
dst_av,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
ptr::null(),
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→{nvenc_pixel:?}) failed");
}
// Colour math applies to the CSC users ONLY. The expand is a pure byte shuffle —
// packed 3-bpp RGB/BGR to the same channels in 4 bytes, `nvenc_pixel` being `rgb0`/
// `bgr0` — and NVENC does the RGB→YUV itself downstream. Handing it a matrix + range
// here would silently range-convert every packed-RGB session, which is exactly what the
// module header promises does not happen ("no colour math").
if want_444 || want_hdr10 {
// SAFETY: `sws` is the non-null context from the call above (null-checked). The
// coefficient tables from `sws_getCoefficients` (ITU-709 for 4:4:4, BT.2020 NCL for
// HDR — matching the VUI written above) are process-lifetime libswscale statics,
// reused for src+dst matrices; `sws_setColorspaceDetails` only reads them and writes
// scalar CSC settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
// PUNKTFUNK_444_FULLRANGE experiment; HDR is always limited). No Rust memory is passed.
unsafe {
let cs = ffi::sws_getCoefficients(if want_hdr10 {
super::libav::SWS_CS_BT2020
} else {
SWS_CS_ITU709
});
let dst_range = i32::from(full_range_444);
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, dst_range, 0, 1 << 16, 1 << 16);
}
}
Some(sws)
} else {
None
};
let frame = if cuda {
None
} else {
@@ -561,7 +591,6 @@ impl NvencEncoder {
sws_csc,
want_444,
src_format: format,
expand,
width,
height,
fps,
@@ -591,6 +620,9 @@ impl NvencEncoder {
impl Encoder for NvencEncoder {
fn caps(&self) -> super::EncoderCaps {
super::EncoderCaps {
// libav NVENC hands the frame straight to the encoder — `frame.cursor` is never read,
// so a cursor-as-metadata session loses its pointer on this backend (audit finding).
blends_cursor: false,
// 4:4:4 iff this session opened FREXT — the CPU swscale path or the zero-copy GPU
// convert. RFI/HDR-SEI stay unsupported on libavcodec NVENC (the trait defaults).
chroma_444: self.want_444,
@@ -695,8 +727,10 @@ impl NvencEncoder {
bytes.len(),
src_row * h
);
// 4:4:4: swscale the packed RGB straight into the planar YUV444P input frame (BT.709 limited),
// then send it — no byte-expand. The 4:2:0 RGB path (below) feeds NVENC packed RGB directly.
// swscale the packed RGB straight into the encoder's input frame, then send it. Serves all
// three CSC users (see `open`): 4:4:4 → planar YUV444P, HDR → P010, and the packed 3-bpp
// expand → `rgb0`/`bgr0`. The remaining branch below is the 4-bpp source, which needs no
// conversion at all — just a row copy honouring the destination stride.
if let Some(sws) = self.sws_csc {
let frame = self
.frame
@@ -705,10 +739,13 @@ impl NvencEncoder {
// SAFETY: `format == self.src_format` and `bytes.len() >= src_row * h` (the `ensure!`s
// above), so `sws_scale` reads `h` rows of `src_row` bytes from `src_data[0] = bytes`
// (packed RGB is single-plane; the other src planes are null/0) — all in bounds. `sws` is
// the non-null context built in `open`. The dst is `frame`'s underlying `AVFrame`: its
// `data`/`linesize` in-struct arrays were sized for YUV444P by `VideoFrame::new`, and the
// 3 planes are each `width`×`height`. All pointers are live locals for this synchronous
// call; the encoder runs only on this thread (`unsafe impl Send`), so no aliasing/race.
// the non-null context built in `open`. The dst is `frame`'s underlying `AVFrame`, whose
// `data`/`linesize` in-struct arrays were sized by `VideoFrame::new` for the very
// `nvenc_pixel` this context was built to output — 3 planes of `width`×`height` for
// YUV444P, 2 for P010, 1 packed plane for `rgb0`/`bgr0` — so swscale writes exactly the
// planes it allocated, at the strides it reports. All pointers are live locals for this
// synchronous call; the encoder runs only on this thread (`unsafe impl Send`), so no
// aliasing/race.
unsafe {
let dst_av = frame.as_mut_ptr();
let src_data: [*const u8; 4] =
@@ -724,7 +761,7 @@ impl NvencEncoder {
(*dst_av).linesize.as_ptr(),
);
if r < 0 {
bail!("sws_scale(RGB→YUV444P) failed ({r})");
bail!("sws_scale(CPU CSC → encoder input) failed ({r})");
}
}
frame.set_pts(Some(pts));
@@ -733,7 +770,7 @@ impl NvencEncoder {
} else {
ffmpeg::picture::Type::None
});
self.enc.send_frame(frame).context("send_frame(444)")?;
self.enc.send_frame(frame).context("send_frame(swscale)")?;
return Ok(());
}
let frame = self
@@ -742,18 +779,9 @@ impl NvencEncoder {
.context("CPU frame missing (encoder opened in CUDA mode)")?;
let stride = frame.stride(0); // dst is 4-bpp, aligned
let dst = frame.data_mut(0);
if self.expand {
// packed 3-bpp RGB/BGR → 4-bpp *0 (copy 3 bytes, zero the pad byte)
for y in 0..h {
let s = &bytes[y * src_row..y * src_row + src_row];
let drow = &mut dst[y * stride..y * stride + w * 4];
for x in 0..w {
drow[x * 4..x * 4 + 3].copy_from_slice(&s[x * 3..x * 3 + 3]);
drow[x * 4 + 3] = 0;
}
}
} else {
// 4-bpp → 4-bpp, honoring the (possibly larger) dst stride
{
// 4-bpp → 4-bpp, honoring the (possibly larger) dst stride. The 3-bpp expand that used
// to live here as a per-pixel loop is now swscale's job (see the branch above).
for y in 0..h {
dst[y * stride..y * stride + src_row]
.copy_from_slice(&bytes[y * src_row..y * src_row + src_row]);
@@ -876,6 +904,58 @@ impl Drop for NvencEncoder {
}
}
/// Serialises the save → `AV_LOG_FATAL` → restore window that every capability probe opens around
/// an encoder open it *expects* to fail.
///
/// libav's log level is one process-global `int`, and the probes race each other for real: the
/// NVENC and VAAPI 4:4:4/10-bit probes are reached from `/serverinfo` and from session bring-up.
/// Two overlapping save/restore pairs interleave as get(INFO) → get(FATAL) → set(INFO) →
/// set(FATAL), and the process is then pinned at `AV_LOG_FATAL` for good — every later libav
/// diagnostic silently dropped, which is precisely the logging you want when a stream later fails
/// to open. The probes run process-once and already cost a real encoder open, so serialising them
/// costs nothing measurable.
static LIBAV_LOG_LEVEL: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// RAII quiet-window over libav's global log level: drops it to `AV_LOG_FATAL` on construction and
/// restores the previous level on drop, holding [`LIBAV_LOG_LEVEL`] for the whole window.
///
/// Callers must have completed `ffmpeg::init()` first. Not re-entrant — no probe may construct a
/// second guard while holding one (none do; the probe bodies only reach encoder-open helpers).
/// `pub(crate)` so the VAAPI probes share the one lock: they race the NVENC probes on the same
/// global.
pub(crate) struct QuietLibavLog {
prev: c_int,
// Held for the lifetime of the guard. `Drop for QuietLibavLog` runs before the struct's fields
// are dropped, so the restore below still happens under the lock.
_lock: std::sync::MutexGuard<'static, ()>,
}
impl QuietLibavLog {
pub(crate) fn new() -> Self {
// Poison-tolerant: a probe that panicked mid-window already restored the level via `Drop`,
// and refusing the lock forever afterwards would be a worse outcome than proceeding.
let lock = LIBAV_LOG_LEVEL
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
// SAFETY: libav is initialized by the caller; `av_log_{get,set}_level` only read/write the
// global int level (no pointer args) and are always sound post-init.
let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
Self { prev, _lock: lock }
}
}
impl Drop for QuietLibavLog {
fn drop(&mut self) {
// SAFETY: restore the saved global level (scalar arg, no pointers); libav was initialized
// before this guard was constructed.
unsafe { ffi::av_log_set_level(self.prev) };
}
}
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode HEVC **4:4:4** (Range
/// Extensions). Opens a tiny real `hevc_nvenc` 4:4:4 session — the exact path [`NvencEncoder::open`]
/// takes for a live 4:4:4 stream — and reports whether it succeeded. HEVC-only; the result is cached
@@ -889,14 +969,9 @@ pub fn probe_can_encode_444(codec: Codec) -> bool {
return false;
}
// Quiet ffmpeg's open error on a GPU that lacks 4:4:4 — the probe failing is an expected outcome.
// SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
// (no pointer args) and are always sound post-init.
let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
let ok = NvencEncoder::open(
// Held until the function returns, so the level is restored after the open either way.
let _quiet = QuietLibavLog::new();
NvencEncoder::open(
codec,
PixelFormat::Bgra,
640,
@@ -907,10 +982,7 @@ pub fn probe_can_encode_444(codec: Codec) -> bool {
8,
ChromaFormat::Yuv444,
)
.is_ok();
// SAFETY: restore the saved global log level (scalar arg, no pointers).
unsafe { ffi::av_log_set_level(prev) };
ok
.is_ok()
}
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode 10-bit (HEVC Main10 /
@@ -926,14 +998,9 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
return false;
}
// Quiet ffmpeg's open error on a GPU that lacks 10-bit — the probe failing is an expected outcome.
// SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
// (no pointer args) and are always sound post-init.
let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
let ok = NvencEncoder::open(
// Held until the function returns, so the level is restored after the open either way.
let _quiet = QuietLibavLog::new();
NvencEncoder::open(
codec,
PixelFormat::X2Rgb10,
640,
@@ -944,10 +1011,7 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
10,
ChromaFormat::Yuv420,
)
.is_ok();
// SAFETY: restore the saved global log level (scalar arg, no pointers).
unsafe { ffi::av_log_set_level(prev) };
ok
.is_ok()
}
#[cfg(test)]
+208 -103
View File
@@ -61,8 +61,9 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::nvenc_core::{
apply_low_latency_config, build_init_params, codec_guid, resolve_slices, resolve_subframe,
LowLatencyConfig, NvStatusExt, RFI_DPB,
apply_low_latency_config, build_init_params, cached_ceiling, codec_guid, plan_range_recovery,
resolve_slices, resolve_split_mode, resolve_subframe, store_ceiling, CeilingKey,
LowLatencyConfig, NvStatusExt, RangePlan,
};
use super::nvenc_status;
use super::{AuChunk, ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
@@ -268,13 +269,19 @@ fn async_retrieve_requested() -> bool {
/// Max encodes in flight in two-thread mode (`PUNKTFUNK_NVENC_ASYNC_DEPTH`, default 4, clamped
/// `2..=POOL-1` — a bitstream must never be reused mid-encode, and the input ring is the same
/// depth). Mirrors the Windows knob exactly.
/// depth). Mirrors the Windows knob exactly, memoization included: this is the backpressure
/// **loop condition** in `submit`, so an engaged two-thread session re-read the environment once
/// per spin. The default session never pays it (the condition short-circuits on `async_rt`), which
/// is why the audit's severity ranking for this site was inverted — but an escalated one did.
fn async_inflight_cap() -> usize {
std::env::var("PUNKTFUNK_NVENC_ASYNC_DEPTH")
.ok()
.and_then(|s| s.parse::<usize>().ok())
.unwrap_or(4)
.clamp(2, POOL - 1)
static CAP: std::sync::OnceLock<usize> = std::sync::OnceLock::new();
*CAP.get_or_init(|| {
std::env::var("PUNKTFUNK_NVENC_ASYNC_DEPTH")
.ok()
.and_then(|s| s.trim().parse::<usize>().ok())
.unwrap_or(4)
.clamp(2, POOL - 1)
})
}
/// Stream-ordered submit (`PUNKTFUNK_NVENC_STREAM_ORDERED`, default ON; `0` = the pre-existing
@@ -595,6 +602,12 @@ pub struct NvencCudaEncoder {
/// (escalate-and-hold, like the depth escalation); the switch itself happens at the next
/// safe point via [`maybe_engage_async`](Self::maybe_engage_async).
want_async: bool,
/// A de-escalation request ([`Encoder::set_pipelined(false)`]) waiting for its safe point:
/// the next drained moment tears the session down and lazily re-inits SYNC (IO-stream
/// binding and sub-frame chunking re-arm at that re-init). Distinct from `!want_async` —
/// an operator-forced async session (`PUNKTFUNK_NVENC_ASYNC=1`) also has `want_async`
/// false, and a de-escalation must never tear THAT down.
want_sync: bool,
/// Boxed `CUstream` the session's IO-stream binding points at (`NvEncSetIOCudaStreams` takes
/// POINTERS to `CUstream`, and this struct moves — the pointee needs a stable heap address for
/// the session's lifetime). Null when stream-ordering is off; freed in `teardown` AFTER the
@@ -702,6 +715,7 @@ impl NvencCudaEncoder {
last_rfi_range: None,
async_rt: None,
want_async: false,
want_sync: false,
io_stream: ptr::null_mut(),
stream_ordered: false,
slices: 1,
@@ -735,6 +749,29 @@ impl NvencCudaEncoder {
}
}
/// [`maybe_engage_async`](Self::maybe_engage_async)'s inverse — wind the escalated pipelined
/// retrieve back at a safe point: nothing in flight, then a clean session rebuild whose lazy
/// SYNC re-init restores the IO-stream binding and re-arms sub-frame chunking (the two
/// latency features the escalation traded away). No-op until
/// [`want_sync`](Self::want_sync) is set and `pending` drains.
fn maybe_disengage_async(&mut self) {
if !self.want_sync || self.async_rt.is_none() || !self.pending.is_empty() {
return;
}
self.want_sync = false;
if self.inited {
// SAFETY: encode thread, `pending` empty ⇒ no encode in flight (and nothing queued
// to the retrieve thread); `teardown` joins the retrieve thread and handles exactly
// this live-session state — the next submit lazily re-inits sync.
unsafe { self.teardown() };
tracing::info!(
"NVENC pipelined-retrieve de-escalation: rebuilding the session with the sync \
retrieve (IO-stream binding and sub-frame chunking restored); next frame opens \
with an IDR"
);
}
}
/// Tear down the encode session + pooled resources. Reused on a size change and at Drop.
unsafe fn teardown(&mut self) {
if self.encoder.is_null() {
@@ -839,6 +876,9 @@ impl NvencCudaEncoder {
e,
));
}
// The handshake with the kernel module just succeeded — from here on, an
// `NV_ENC_ERR_INVALID_VERSION` in this process cannot be a driver version skew.
nvenc_status::note_session_opened();
let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX);
let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX);
let yuv444 = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE);
@@ -1000,6 +1040,23 @@ impl NvencCudaEncoder {
Ok(cfg)
}
/// This session config's identity in the process-lifetime bitrate-ceiling cache
/// (`nvenc_core::{cached_ceiling, store_ceiling}`). GPU identity is the process-global shared
/// `CUcontext` pointer — one context per process, stable for its lifetime; only valid once
/// `cu_ctx` is bound (`init_session` start), which every caller is downstream of.
fn ceiling_key(&self, split_mode: u32) -> CeilingKey {
CeilingKey {
gpu: self.cu_ctx as u64,
codec: self.codec,
width: self.width,
height: self.height,
fps: self.fps,
bit_depth: self.bit_depth,
chroma_444: self.chroma_444,
split_mode,
}
}
/// Open + configure + initialize ONE NVENC CUDA session at `bitrate` (bps) and `split_mode`.
/// Returns the session handle, or destroys it and returns the error.
unsafe fn try_open_session(&self, bitrate: u64, split_mode: u32) -> Result<*mut c_void> {
@@ -1019,6 +1076,7 @@ impl NvencCudaEncoder {
}
return Err(nvenc_status::call_err("open_encode_session_ex", e));
}
nvenc_status::note_session_opened();
let mut cfg = match self.build_config(enc, bitrate) {
Ok(cfg) => cfg,
@@ -1074,58 +1132,71 @@ impl NvencCudaEncoder {
}
const FLOOR_BPS: u64 = 10_000_000;
let requested_bps = self.bitrate_bps;
// 2-way NVENC split-frame encoding (Ada dual-NVENC) above ~1 Gpix/s; env override
// PUNKTFUNK_SPLIT_ENCODE = 0/disable | 1/auto | 2 | 3. HEVC/AV1 only.
// 2-way NVENC split-frame encoding (Ada dual-NVENC) — shared selector, see
// [`resolve_split_mode`] for the precedence (env override / 10-bit / pixel rate).
let pixel_rate = self.width as u64 * self.height as u64 * self.fps.max(1) as u64;
let mut split_mode: u32 = match std::env::var("PUNKTFUNK_SPLIT_ENCODE").ok().as_deref()
{
Some("0") | Some("disable") => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32
}
Some("1") | Some("auto") => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_AUTO_FORCED_MODE as u32
}
Some("3") => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_THREE_FORCED_MODE as u32,
Some("2") => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
_ if self.bit_depth >= 10 => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32
}
_ if pixel_rate > 1_000_000_000 => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_TWO_FORCED_MODE as u32
}
_ => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_AUTO_MODE as u32,
};
let split_mode: u32 = resolve_split_mode(self.bit_depth, pixel_rate);
const CLAMP_TOL_BPS: u64 = 20_000_000;
let mut probe = self.try_open_session(requested_bps, split_mode);
// Disambiguate a forced-split rejection from a bitrate-cap rejection.
// Ceiling cache (process lifetime, `nvenc_core`): a prior clamp search already found
// this config's max accepted rate — open straight AT the ceiling instead of paying
// the ~6-open binary search (and its session churn) on every ABR overshoot.
let mut target_bps = requested_bps;
if let Some(ceiling) = cached_ceiling(&self.ceiling_key(split_mode)) {
if requested_bps > ceiling {
tracing::info!(
requested_mbps = requested_bps / 1_000_000,
ceiling_mbps = ceiling / 1_000_000,
"NVENC (Linux): requested bitrate above the cached codec-level ceiling — \
opening at the ceiling"
);
target_bps = ceiling;
}
}
let mut probe = self.try_open_session(target_bps, split_mode);
// The cache is advisory: a stale entry (driver change, identity collision) must not
// wedge the open — retry the requested rate and let the search below rediscover.
if probe.is_err() && target_bps < requested_bps {
target_bps = requested_bps;
probe = self.try_open_session(requested_bps, split_mode);
}
// Disambiguate a forced-split rejection from a bitrate-cap rejection. `used_split`
// tracks the mode sessions ACTUALLY open with from here on — it feeds
// `self.split_mode` (a reconfigure must re-present it) and the ceiling-cache key.
let mut used_split = split_mode;
let split_on =
split_mode != nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
if probe.is_err() && split_on {
let no_split = nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
if let Ok(e) = self.try_open_session(requested_bps, no_split) {
if let Ok(e) = self.try_open_session(target_bps, no_split) {
tracing::warn!(
"NVENC (Linux): split-encode rejected by codec/config — disabled"
);
split_mode = no_split;
used_split = no_split;
probe = Ok(e);
}
}
let enc = match probe {
Ok(enc) => {
self.bitrate_bps = requested_bps;
self.bitrate_bps = target_bps;
enc
}
// Only a parameter/caps rejection means "the bitrate is above the codec-level
// ceiling". A transient failure (busy engine, session limit, OOM, device loss,
// version skew) must propagate — a search steered by it would discover, and
// cache, a bogus ceiling.
Err(e) if !nvenc_status::is_param_rejection(&e) => return Err(e),
Err(_) => {
// Requested bitrate exceeds the codec-level ceiling — binary-search the max accepted.
let mut lo = FLOOR_BPS;
let mut hi = requested_bps;
let mut hi = target_bps;
let mut best: *mut c_void = ptr::null_mut();
let mut best_bps = 0u64;
while hi > lo + CLAMP_TOL_BPS {
let mid = lo + (hi - lo) / 2;
match self.try_open_session(mid, split_mode) {
match self.try_open_session(mid, used_split) {
Ok(e) => {
if !best.is_null() {
let _ = (api().destroy_encoder)(best);
@@ -1134,18 +1205,30 @@ impl NvencCudaEncoder {
best_bps = mid;
lo = mid;
}
Err(_) => hi = mid,
Err(e) if nvenc_status::is_param_rejection(&e) => hi = mid,
Err(e) => {
// Environmental mid-search failure: don't let it shrink the
// search — release the partial result and propagate.
if !best.is_null() {
let _ = (api().destroy_encoder)(best);
}
return Err(e);
}
}
}
if best.is_null() {
let no_split =
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
best = self
.try_open_session(FLOOR_BPS, split_mode)
.or_else(|_| self.try_open_session(FLOOR_BPS, no_split))
.context(
"NVENC initialize_encoder rejected even at the floor bitrate",
)?;
best = match self.try_open_session(FLOOR_BPS, used_split) {
Ok(e) => e,
Err(_) => {
let e = self.try_open_session(FLOOR_BPS, no_split).context(
"NVENC initialize_encoder rejected even at the floor bitrate",
)?;
used_split = no_split;
e
}
};
best_bps = FLOOR_BPS;
}
tracing::warn!(
@@ -1153,15 +1236,13 @@ impl NvencCudaEncoder {
clamped_mbps = best_bps / 1_000_000,
"NVENC (Linux): requested bitrate above the GPU codec-level ceiling — clamped"
);
store_ceiling(self.ceiling_key(used_split), best_bps);
self.bitrate_bps = best_bps;
best
}
};
self.encoder = enc;
// (Best effort: the floor fallback above may have succeeded split-disabled without
// updating `split_mode` — a later reconfigure then presents the forced mode, NVENC
// rejects it, and the caller's rebuild fallback covers the mismatch.)
self.split_mode = split_mode;
self.split_mode = used_split;
// Output bitstream pool.
for _ in 0..POOL {
@@ -1345,6 +1426,10 @@ impl NvencCudaEncoder {
mbps = self.bitrate_bps / 1_000_000,
codec = ?self.codec_guid,
fmt = ?self.buffer_fmt,
// The FINAL split mode (post any rejection fallback) at INFO — journals run
// INFO+, and "did 4K120 actually split across engines?" was undiagnosable from
// a user log without it (Windows only had a debug! at selection time).
split_mode = self.split_mode,
"NVENC CUDA session ready"
);
Ok(())
@@ -1426,9 +1511,10 @@ impl Encoder for NvencCudaEncoder {
"Linux direct-NVENC needs a CUDA frame (FramePayload::Cuda); got a CPU/dmabuf frame"
),
};
// A pending pipelined-retrieve escalation engages here, at the submit-side safe point
// (nothing in flight after the previous poll drained).
// A pending pipelined-retrieve escalation — or de-escalation — engages here, at the
// submit-side safe point (nothing in flight after the previous poll drained).
self.maybe_engage_async();
self.maybe_disengage_async();
// Re-init on a size change (the capturer can return at a different resolution after a mode
// switch). Format changes (NV12↔YUV444) likewise re-init.
let new_fmt = buffer_format(buf);
@@ -1560,25 +1646,6 @@ impl Encoder for NvencCudaEncoder {
}
} else {
self.cursor_blend_warned = false;
// TEMP KWin composite probe (DROP BEFORE MERGE): prove the blend dispatches
// and with what geometry.
{
use std::sync::atomic::{AtomicU64, Ordering as ProbeOrd};
static PROBE_BLEND: AtomicU64 = AtomicU64::new(0);
let n = PROBE_BLEND.fetch_add(1, ProbeOrd::Relaxed) + 1;
if n == 1 || n % 512 == 0 {
tracing::info!(
n,
fmt = ?self.buffer_fmt,
ov_x = ov.x,
ov_y = ov.y,
ov_w = ov.w,
ov_h = ov.h,
visible = ov.visible,
"cursor blend probe: vulkan dispatch submitted"
);
}
}
}
} else if !self.cursor_blend_warned {
self.cursor_blend_warned = true;
@@ -1736,12 +1803,25 @@ impl Encoder for NvencCudaEncoder {
fn set_pipelined(&mut self, on: bool) -> bool {
if !on {
// v1 is escalate-and-hold (no de-escalation), mirroring the depth escalation.
// De-escalation (the v2 of escalate-and-hold): latch the wind-back intent; the
// switch itself happens at the next drained safe point
// ([`maybe_disengage_async`](Self::maybe_disengage_async)) — the caller polls
// this same method until it reports inactive.
if async_retrieve_env() == Some(true) {
// Operator pinned async on — de-escalation must not undo an explicit choice.
return self.want_async || self.async_rt.is_some();
}
if self.want_async || self.async_rt.is_some() {
self.want_async = false;
self.want_sync = true;
self.maybe_disengage_async();
}
return self.want_async || self.async_rt.is_some();
}
if async_retrieve_env() == Some(false) {
return false; // operator veto: PUNKTFUNK_NVENC_ASYNC=0 means NEVER
}
self.want_sync = false; // latest intent wins — cancel a pending wind-back
if !self.want_async && self.async_rt.is_none() {
self.want_async = true;
self.maybe_engage_async();
@@ -1751,6 +1831,8 @@ impl Encoder for NvencCudaEncoder {
fn caps(&self) -> EncoderCaps {
EncoderCaps {
// Composites `frame.cursor` via the SPIR-V blend over the Vulkan-allocated input slot.
blends_cursor: true,
supports_rfi: self.rfi_supported,
supports_hdr_metadata: self.hdr,
chroma_444: self.chroma_444,
@@ -1765,48 +1847,46 @@ impl Encoder for NvencCudaEncoder {
}
fn invalidate_ref_frames(&mut self, first: i64, last: i64) -> bool {
if self.encoder.is_null() || !self.rfi_supported || first < 0 || first > last {
// Range validity, covering-range dedup, DPB window and clamp all live in
// `nvenc_core::plan_range_recovery` — one policy for both direct-NVENC backends; only the
// session gate and the driver loop are this backend's.
if self.encoder.is_null() || !self.rfi_supported {
return false;
}
// Already invalidated a covering range for this loss event — re-arm the anchor (the previous
// anchor AU may itself have been lost) but skip the driver calls.
if let Some((pf, pl)) = self.last_rfi_range {
if first >= pf && last <= pl {
match plan_range_recovery(first, last, self.frame_idx, self.last_rfi_range) {
// Already invalidated a covering range for this loss event — re-arm the anchor (the
// previous anchor AU may itself have been lost) but skip the driver calls.
RangePlan::Covered => {
self.pending_anchor = true;
return true;
true
}
}
// The DPB holds `[frame_idx - RFI_DPB, frame_idx - 1]`; a lost frame older than that can't be
// invalidated, so the only correct recovery is an IDR.
let oldest_in_dpb = self.frame_idx - RFI_DPB as i64;
if first < oldest_in_dpb {
return false;
}
let last = last.min(self.frame_idx - 1);
if first > last {
return false;
}
// Each input's `inputTimeStamp` is the WIRE frame index (pinned by `submit_indexed`), so the
// client's lost-frame range maps 1:1 onto the timestamps NVENC invalidates here.
// SAFETY: `invalidate_ref_frames` is a function pointer from the runtime table; `self.encoder`
// was checked non-null and is the live session; this runs on the encode thread (no concurrent
// NVENC use). Each `ts` is clamped to `[oldest_in_dpb, frame_idx - 1]`, naming a frame still
// in the DPB; the call passes only that `u64` (no struct).
unsafe {
for ts in first..=last {
if (api().invalidate_ref_frames)(self.encoder, ts as u64)
.nv_ok()
.is_err()
{
return false;
RangePlan::Decline => false,
RangePlan::Invalidate { first, last } => {
// Each input's `inputTimeStamp` is the WIRE frame index (pinned by
// `submit_indexed`), so the client's lost-frame range maps 1:1 onto the timestamps
// NVENC invalidates here.
// SAFETY: `invalidate_ref_frames` is a function pointer from the runtime table;
// `self.encoder` was checked non-null and is the live session; this runs on the
// encode thread (no concurrent NVENC use). The plan clamped each `ts` to
// `[oldest_in_dpb, frame_idx - 1]`, naming a frame still in the DPB; the call
// passes only that `u64` (no struct).
unsafe {
for ts in first..=last {
if (api().invalidate_ref_frames)(self.encoder, ts as u64)
.nv_ok()
.is_err()
{
return false;
}
}
}
self.last_rfi_range = Some((first, last));
// The next submitted frame is the clean re-anchor — arm the tag so its AU ships
// with `recovery_anchor` and the client lifts its post-loss freeze on it.
self.pending_anchor = true;
true
}
}
self.last_rfi_range = Some((first, last));
// The next submitted frame is the clean re-anchor — arm the tag so its AU ships with
// `recovery_anchor` and the client lifts its post-loss freeze on it.
self.pending_anchor = true;
true
}
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
@@ -2061,6 +2141,15 @@ impl Encoder for NvencCudaEncoder {
self.bitrate_bps = bps;
return true;
}
// Cached codec-level ceiling: clamp the target BEFORE the driver call, so a known
// overshoot retargets to the ceiling IN PLACE instead of bouncing off the driver into
// the caller's full-rebuild fallback (an IDR plus ~half a second of session churn per
// ABR overshoot on the pre-cache path). The caller reads the clamp back through
// [`Encoder::applied_bitrate_bps`].
let bps = match cached_ceiling(&self.ceiling_key(self.split_mode)) {
Some(ceiling) => bps.min(ceiling),
None => bps,
};
// SAFETY: `inited` ⟹ `self.encoder` is the live session and every call here runs on the
// encode thread with no NVENC call in flight (the session loop calls this between
// submit/poll). `build_config` only queries the preset on that session; `cfg` outlives
@@ -2108,6 +2197,12 @@ impl Encoder for NvencCudaEncoder {
}
}
fn applied_bitrate_bps(&self) -> Option<u64> {
// `bitrate_bps` is the post-clamp truth: the open path's ceiling search and the
// reconfigure path's cache clamp both write what the session ACTUALLY targets.
Some(self.bitrate_bps)
}
fn flush(&mut self) -> Result<()> {
Ok(()) // P1/ULL + frameIntervalP=1: each submit yields its AU; no internal queue to drain.
}
@@ -2165,6 +2260,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
)
.expect("open NVENC CUDA session");
@@ -2241,6 +2337,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv444,
false,
)
.expect("open NVENC CUDA 4:4:4 session");
@@ -2286,6 +2383,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
)
.expect("open NVENC CUDA session");
@@ -2343,6 +2441,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
) else {
eprintln!(
"skipping rfi_declines_impossible_ranges: NVENC unavailable (no NVIDIA driver)"
@@ -2369,6 +2468,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
)
.expect("open NVENC CUDA encoder")
}
@@ -2403,6 +2503,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
)
.expect("open");
for f in 0..4u32 {
@@ -2540,6 +2641,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
)
.expect("open NVENC CUDA session");
let frame = nv12_frame(W, H, 0);
@@ -2581,6 +2683,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
)
.expect("open NVENC CUDA session");
// Steady sync frames first (stream-ordered mode).
@@ -2660,6 +2763,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
)
.expect("open NVENC CUDA session");
@@ -2765,6 +2869,7 @@ mod tests {
true,
8,
ChromaFormat::Yuv420,
false,
)
.expect("open NVENC CUDA session");
File diff suppressed because it is too large Load Diff
+92 -53
View File
@@ -28,22 +28,18 @@ use ffmpeg::format::Pixel;
use ffmpeg::{codec, encoder, Dictionary};
use ffmpeg_next as ffmpeg;
use pf_frame::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat};
use std::collections::HashMap;
use std::ffi::{CStr, CString};
use std::os::fd::AsRawFd;
use std::os::raw::c_int;
use std::ptr;
use std::sync::atomic::{AtomicU8, Ordering};
use std::sync::{Mutex, OnceLock};
use super::libav::{
apply_low_latency_rc, pixel_to_av, poll_encoder, PollOutcome, SWS_CS_ITU709, SWS_POINT,
};
use ffmpeg::ffi; // = ffmpeg_sys_next
/// `fourcc(a,b,c,d)` — DRM FourCC packing (`a | b<<8 | c<<16 | d<<24`).
const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
(a as u32) | ((b as u32) << 8) | ((c as u32) << 16) | ((d as u32) << 24)
}
/// The render node a VAAPI/DRM device should open, from [`pf_gpu::linux_render_node`]: a
/// matched web-console GPU preference pins it, else `PUNKTFUNK_RENDER_NODE`, else the single-GPU
/// default.
@@ -71,21 +67,38 @@ fn vaapi_sws_src(format: PixelFormat) -> Result<Pixel> {
})
}
/// Which VAAPI entrypoint mode opened successfully, cached per codec (index = [`lp_idx`]):
/// 0 = unknown, 1 = default (full-feature `EncSlice`), 2 = low-power (`EncSliceLP`/VDEnc).
/// Modern Intel (Gen12+/Arc) removed the full-feature encode entrypoints, so the default open
/// fails there and only `low_power=1` works; AMD (radeonsi) is the reverse. Caching the resolved
/// mode lets later sessions/probes skip the known-failing attempt (and its libav error spew).
static LP_MODE: [AtomicU8; 3] = [AtomicU8::new(0), AtomicU8::new(0), AtomicU8::new(0)];
/// Which VAAPI entrypoint mode opened successfully: 1 = default (full-feature `EncSlice`),
/// 2 = low-power (`EncSliceLP`/VDEnc). Modern Intel (Gen12+/Arc) removed the full-feature encode
/// entrypoints, so the default open fails there and only `low_power=1` works; AMD (radeonsi) is the
/// reverse. Caching the resolved mode lets later sessions/probes skip the known-failing attempt
/// (and its libav error spew).
///
/// Keyed on **(render node, codec, bit depth)**, and every part of that key is load-bearing:
///
/// * The render node, because the entrypoint is a property of the DEVICE libva opens — and
/// `render_node()` follows the web-console GPU preference. This used to be a process-global array
/// keyed by codec alone, which made it a session-killer rather than a staleness bug: once a mode
/// is latched the open tries exactly ONE mode and the `[false, true]` fallback is gone. Latch
/// low-power on an Intel Arc, switch the preference to an AMD dGPU, and every VAAPI open there
/// passes `low_power=1` — which radeonsi rejects — with no full-feature retry, for the process
/// lifetime: the probe reports all-false AND the session's own encoder open fails.
/// Keyed on the node rather than `pf_gpu::selection_key()` on purpose — the node is literally what
/// `render_node()` hands libva, so key and device cannot describe different GPUs.
/// * The bit depth, because Main10 and 8-bit can resolve to different entrypoints on the same
/// device; one shared slot let an 8-bit answer pin the 10-bit open (HDR under-advertisement).
static LP_MODE: OnceLock<Mutex<HashMap<LpKey, u8>>> = OnceLock::new();
fn lp_idx(codec: Codec) -> usize {
match codec {
Codec::H264 => 0,
Codec::H265 => 1,
Codec::Av1 => 2,
// Guarded by the open_video dispatch: PyroWave never opens the VAAPI backend.
Codec::PyroWave => unreachable!("PyroWave has no VAAPI encoder"),
}
/// (render-node path, codec label, 10-bit) — see [`LP_MODE`].
type LpKey = (String, &'static str, bool);
/// The [`LP_MODE`] key for this device/codec/depth. `render_node()` is re-read rather than cached
/// so a GPU-preference change is picked up on the next open.
fn lp_key(codec: Codec, ten_bit: bool) -> LpKey {
(
render_node().to_string_lossy().into_owned(),
codec.label(),
ten_bit,
)
}
/// `PUNKTFUNK_VAAPI_LOW_POWER` pins the entrypoint mode (`1` = low-power only, `0` = full-feature
@@ -116,11 +129,16 @@ unsafe fn open_vaapi_encoder(
frames_ref: *mut ffi::AVBufferRef,
ten_bit: bool,
) -> Result<encoder::video::Encoder> {
let idx = lp_idx(codec);
let key = lp_key(codec, ten_bit);
let cached = LP_MODE
.get_or_init(|| Mutex::new(HashMap::new()))
.lock()
.map(|m| m.get(&key).copied().unwrap_or(0))
.unwrap_or(0);
let modes: &[bool] = match low_power_override() {
Some(true) => &[true],
Some(false) => &[false],
None => match LP_MODE[idx].load(Ordering::Relaxed) {
None => match cached {
1 => &[false],
2 => &[true],
_ => &[false, true],
@@ -140,7 +158,9 @@ unsafe fn open_vaapi_encoder(
lp,
) {
Ok(enc) => {
LP_MODE[idx].store(if lp { 2 } else { 1 }, Ordering::Relaxed);
if let Ok(mut m) = LP_MODE.get_or_init(|| Mutex::new(HashMap::new())).lock() {
m.insert(key.clone(), if lp { 2 } else { 1 });
}
if lp {
tracing::info!(
encoder = codec.vaapi_name(),
@@ -253,20 +273,20 @@ pub fn probe_can_encode(codec: Codec) -> bool {
if ffmpeg::init().is_err() {
return false;
}
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `av_log_get_level`/
// `av_log_set_level` only read/write libav's global integer log level (no pointer args) and are
// always sound to call post-init. `VaapiHw::new` (an `unsafe fn`) builds a VAAPI device + NV12
// frames pool from the literal NV12/640x480/pool=2 args and hands back a RAII handle that unrefs
// both `AVBufferRef`s on drop. `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/
// `hw.frames_ref` — the two non-null refs `VaapiHw::new` just created — and `av_buffer_ref`s them
// into the encoder; `hw` is a live local for the whole match arm, so the borrows outlive the
// synchronous call, and both `hw` and the probe encoder are dropped (RAII) when the arm ends.
// A missing VA device (non-VAAPI host, GPU-less CI) is an expected probe outcome — quiet
// ffmpeg's "No VA display found" error for the probe. Held until the function returns, so the
// level is restored after the open either way. Shares one lock with the NVENC probes, which
// race this one on the same libav global (see [`crate::linux::QuietLibavLog`]).
let _quiet = crate::linux::QuietLibavLog::new();
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `VaapiHw::new` (an
// `unsafe fn`) builds a VAAPI device + NV12 frames pool from the literal NV12/640x480/pool=2
// args and hands back a RAII handle that unrefs both `AVBufferRef`s on drop.
// `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` — the two
// non-null refs `VaapiHw::new` just created — and `av_buffer_ref`s them into the encoder; `hw`
// is a live local for the whole match arm, so the borrows outlive the synchronous call, and
// both `hw` and the probe encoder are dropped (RAII) when the arm ends.
unsafe {
// A missing VA device (non-VAAPI host, GPU-less CI) is an expected probe outcome — quiet
// ffmpeg's "No VA display found" error for the probe, then restore the level.
let prev = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, 640, 480, 2) {
match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, 640, 480, 2) {
Ok(hw) => open_vaapi_encoder(
codec,
640,
@@ -279,9 +299,7 @@ pub fn probe_can_encode(codec: Codec) -> bool {
)
.is_ok(),
Err(_) => false,
};
ffi::av_log_set_level(prev);
ok
}
}
}
@@ -297,18 +315,17 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
if ffmpeg::init().is_err() {
return false;
}
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `av_log_{get,set}_level`
// only read/write libav's global integer log level (no pointer args). `VaapiHw::new` (an
// A missing VA device / no Main10 entrypoint is an expected probe outcome — quiet ffmpeg's
// error for the probe. Held until the function returns, so the level is restored after the open
// either way, and shared with the other probes (see [`crate::linux::QuietLibavLog`]).
let _quiet = crate::linux::QuietLibavLog::new();
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `VaapiHw::new` (an
// `unsafe fn`) builds a VAAPI device + P010 frames pool from the literal args and hands back a
// RAII handle; `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` —
// the two non-null refs `VaapiHw::new` just created, live locals for the whole match arm — and
// `av_buffer_ref`s them into the probe encoder. Both `hw` and the encoder drop (RAII) at arm end.
unsafe {
// A missing VA device / no Main10 entrypoint is an expected probe outcome — quiet ffmpeg's
// error for the probe, then restore the level.
let prev = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_P010LE, 640, 480, 2) {
match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_P010LE, 640, 480, 2) {
Ok(hw) => open_vaapi_encoder(
codec,
640,
@@ -321,9 +338,7 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
)
.is_ok(),
Err(_) => false,
};
ffi::av_log_set_level(prev);
ok
}
}
}
@@ -934,7 +949,8 @@ impl DmabufInner {
// `Box` puts it on the heap with a unique owner.
// * `dmabuf.fd.as_raw_fd()` is the fd of the caller's `&DmabufFrame`, which owns it for the
// whole synchronous `submit`; we describe one object/layer/plane from its
// fourcc/modifier/offset/stride and pass `object.size = 0` (ffmpeg queries the real size).
// fourcc/modifier/offset/stride and its `lseek`-queried size. `libc::lseek` on that live
// fd only reads the description's size and returns it (or -1); it touches no Rust memory.
// * `av_frame_alloc` → `drm` (null-checked); we set its scalar fields and
// `hw_frames_ctx = av_buffer_ref(self.drm_frames)` (new ref of the live owned ctx).
// * `data[0] = Box::into_raw(desc)` transfers the box into the frame; `buf[0] =
@@ -950,7 +966,17 @@ impl DmabufInner {
let mut desc: Box<ffi::AVDRMFrameDescriptor> = Box::new(std::mem::zeroed());
desc.nb_objects = 1;
desc.objects[0].fd = dmabuf.fd.as_raw_fd();
desc.objects[0].size = 0;
// The object's REAL size, not 0. libav does not query it for us — both of its import
// paths hand this value straight to libva, as `prime_desc.objects[i].size` on the
// PRIME_2 path and `buffer_desc.data_size` on the legacy fallback — so a 0 told every
// VA driver the backing object was empty and left it to work the real size out itself.
// The drivers this has run on (radeonsi, modern Intel iHD) do; a Gen9 Intel host
// answered `vaCreateSurfaces` with VA_STATUS_ERROR_ALLOCATION_FAILED on every single
// frame. `lseek(SEEK_END)` is the standard dma-buf size query — the same one the
// Vulkan bridge already uses on these fds (`pf_zerocopy::imp::vulkan`). If a kernel
// refuses it, keep the old 0 rather than drop a frame we could still have encoded.
let obj_size = libc::lseek(dmabuf.fd.as_raw_fd(), 0, libc::SEEK_END);
desc.objects[0].size = if obj_size > 0 { obj_size as _ } else { 0 };
desc.objects[0].format_modifier = dmabuf.modifier;
desc.nb_layers = 1;
desc.layers[0].format = self.fourcc;
@@ -999,8 +1025,18 @@ impl DmabufInner {
ffi::AV_BUFFERSRC_FLAG_KEEP_REF as c_int,
);
ffi::av_frame_free(&mut drm);
// These two stages ARE the import: the push hands libav our DRM-PRIME descriptor, and
// the pull is where `hwmap` actually maps it into a VA surface (and `scale_vaapi` runs
// the CSC). A failure here means this driver would not take this compositor's dmabuf —
// which no encoder rebuild can fix — so tell the process-wide latch, and capture
// negotiates CPU frames from the next session on. `avcodec_send_frame` below is
// deliberately NOT counted: that one is the encoder stalling, which the in-place
// rebuild above us exists to recover, and disabling zero-copy over it would be a
// permanent penalty for a transient fault.
if r < 0 {
bail!("av_buffersrc_add_frame failed ({r})");
let e = format!("av_buffersrc_add_frame failed ({r})");
pf_zerocopy::note_raw_dmabuf_import_failure(&e);
bail!("{e}");
}
t_push = t0.elapsed();
let mut nv12 = ffi::av_frame_alloc();
@@ -1010,8 +1046,11 @@ impl DmabufInner {
let r = ffi::av_buffersink_get_frame(self.sink, nv12);
if r < 0 {
ffi::av_frame_free(&mut nv12);
bail!("av_buffersink_get_frame failed ({r})");
let e = format!("av_buffersink_get_frame failed ({r})");
pf_zerocopy::note_raw_dmabuf_import_failure(&e);
bail!("{e}");
}
pf_zerocopy::note_raw_dmabuf_import_ok();
t_pull = t0.elapsed() - t_push;
(*nv12).pts = pts;
(*nv12).pict_type = if idr {
@@ -47,6 +47,15 @@ pub const PRIMARY_REF_NONE: u8 = 7;
/// `VK_VIDEO_ENCODE_AV1_SUPERBLOCK_SIZE_128_BIT_KHR` (bit 1 of the superblock-size flags).
pub const SUPERBLOCK_SIZE_128: u32 = 0x2;
// `VkVideoEncodeAV1CapabilityFlagBitsKHR` — the two that decide whether the encode source may be a
// different size from the declared frame. Both absent on RADV PHOENIX.
/// Without this, the source's `codedExtent` MUST equal the sequence header's
/// `max_frame_{width,height}_minus_1 + 1` (`VUID-vkCmdEncodeVideoKHR-flags-10324`).
pub const CAPABILITY_FRAME_SIZE_OVERRIDE: u32 = 0x0000_0008;
/// Without this, EVERY reference slot's `codedExtent` MUST equal the source's
/// (`VUID-vkCmdEncodeVideoKHR-flags-10325`).
pub const CAPABILITY_MOTION_VECTOR_SCALING: u32 = 0x0000_0010;
// `VkVideoEncodeAV1PredictionModeKHR`
pub const PREDICTION_MODE_INTRA_ONLY: i32 = 0;
pub const PREDICTION_MODE_SINGLE_REFERENCE: i32 = 1;
@@ -498,3 +507,366 @@ pub struct StdVideoEncodeAV1OperatingPointInfo {
pub fn stype(raw: i32) -> vk::StructureType {
vk::StructureType::from_raw(raw)
}
// ---------- ABI layout guard ----------
//
// These structs are hand-copied and handed to the driver through raw `p_next` chains, so nothing in
// the type system relates them to the C definitions any more: an edit that inserts, drops, widens or
// re-pads a field is not a compile error, it is the driver reading our bytes at the wrong offsets.
// The assertions below are the missing compile error. They are `const` rather than `#[cfg(test)]`
// (the shape `amf.rs` uses) so they hold in every build, including the shipped one, and on any
// target this module compiles for.
//
// What they catch: a changed field width, an inserted or removed field, a changed array length, a
// padding assumption that only holds on one target. What they CANNOT catch: swapping two fields of
// the same type — offsets are unchanged. That case is only caught by reading the registry, so the
// field order here was diffed against `vulkan_core.h` and `vk_video/vulkan_video_codec_av1std_encode.h`
// (Vulkan-Headers `main`, 2026-07-25) when these assertions were written, along with every `ST_*`,
// flag-bit and enum value above; the bitfield member order is pinned by the test module below.
//
// Deliberately duplicated in `vk_valve_rgb.rs` rather than shared: both modules exist to be deleted
// wholesale once `ash` ships these bindings, and a shared helper would make deleting one break the
// other.
macro_rules! assert_abi_layout {
($t:ty { size: $size:expr, align: $align:expr $(, $field:ident @ $off:expr)* $(,)? }) => {
const _: () = {
assert!(
::core::mem::size_of::<$t>() == $size,
concat!(stringify!($t), ": size does not match the C ABI")
);
assert!(
::core::mem::align_of::<$t>() == $align,
concat!(stringify!($t), ": alignment does not match the C ABI")
);
$(assert!(
::core::mem::offset_of!($t, $field) == $off,
concat!(stringify!($t), ".", stringify!($field), ": offset does not match the C ABI")
);)*
};
};
}
// Std encode structs. The three `*Flags` types are a single C `uint32_t` of bitfields, so only their
// size and alignment are layout-checkable here; their member order is covered by `abi_tests`.
assert_abi_layout!(StdVideoEncodeAV1PictureInfoFlags { size: 4, align: 4 });
assert_abi_layout!(StdVideoEncodeAV1PictureInfo {
size: 152, align: 8,
flags @ 0,
frame_type @ 4,
frame_presentation_time @ 8,
current_frame_id @ 12,
order_hint @ 16,
primary_ref_frame @ 17,
refresh_frame_flags @ 18,
coded_denom @ 19,
render_width_minus_1 @ 20,
render_height_minus_1 @ 22,
interpolation_filter @ 24,
TxMode @ 28,
delta_q_res @ 32,
delta_lf_res @ 33,
ref_order_hint @ 34,
ref_frame_idx @ 42,
reserved1 @ 49,
delta_frame_id_minus_1 @ 52,
pTileInfo @ 80,
pQuantization @ 88,
pSegmentation @ 96,
pLoopFilter @ 104,
pCDEF @ 112,
pLoopRestoration @ 120,
pGlobalMotion @ 128,
pExtensionHeader @ 136,
pBufferRemovalTimes @ 144,
});
assert_abi_layout!(StdVideoEncodeAV1ReferenceInfoFlags { size: 4, align: 4 });
assert_abi_layout!(StdVideoEncodeAV1ReferenceInfo {
size: 24, align: 8,
flags @ 0,
RefFrameId @ 4,
frame_type @ 8,
OrderHint @ 12,
reserved1 @ 13,
pExtensionHeader @ 16,
});
assert_abi_layout!(StdVideoEncodeAV1ExtensionHeader {
size: 2, align: 1,
temporal_id @ 0,
spatial_id @ 1,
});
assert_abi_layout!(StdVideoEncodeAV1OperatingPointInfoFlags { size: 4, align: 4 });
assert_abi_layout!(StdVideoEncodeAV1OperatingPointInfo {
size: 20, align: 4,
flags @ 0,
operating_point_idc @ 4,
seq_level_idx @ 6,
seq_tier @ 7,
decoder_buffer_delay @ 8,
encoder_buffer_delay @ 12,
initial_display_delay_minus_1 @ 16,
});
// KHR extension structs.
assert_abi_layout!(VideoEncodeAV1ProfileInfoKHR {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
std_profile @ 16,
});
assert_abi_layout!(PhysicalDeviceVideoEncodeAV1FeaturesKHR {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
video_encode_av1 @ 16,
});
assert_abi_layout!(VideoEncodeAV1CapabilitiesKHR {
size: 128, align: 8,
s_type @ 0,
p_next @ 8,
flags @ 16,
max_level @ 20,
coded_picture_alignment @ 24,
max_tiles @ 32,
min_tile_size @ 40,
max_tile_size @ 48,
superblock_sizes @ 56,
max_single_reference_count @ 60,
single_reference_name_mask @ 64,
max_unidirectional_compound_reference_count @ 68,
max_unidirectional_compound_group1_reference_count @ 72,
unidirectional_compound_reference_name_mask @ 76,
max_bidirectional_compound_reference_count @ 80,
max_bidirectional_compound_group1_reference_count @ 84,
max_bidirectional_compound_group2_reference_count @ 88,
bidirectional_compound_reference_name_mask @ 92,
max_temporal_layer_count @ 96,
max_spatial_layer_count @ 100,
max_operating_points @ 104,
min_q_index @ 108,
max_q_index @ 112,
prefers_gop_remaining_frames @ 116,
requires_gop_remaining_frames @ 120,
std_syntax_flags @ 124,
});
assert_abi_layout!(VideoEncodeAV1SessionParametersCreateInfoKHR {
size: 48, align: 8,
s_type @ 0,
p_next @ 8,
p_std_sequence_header @ 16,
p_std_decoder_model_info @ 24,
std_operating_point_count @ 32,
p_std_operating_points @ 40,
});
assert_abi_layout!(VideoEncodeAV1PictureInfoKHR {
size: 80, align: 8,
s_type @ 0,
p_next @ 8,
prediction_mode @ 16,
rate_control_group @ 20,
constant_q_index @ 24,
p_std_picture_info @ 32,
reference_name_slot_indices @ 40,
primary_reference_cdf_only @ 68,
generate_obu_extension_header @ 72,
});
assert_abi_layout!(VideoEncodeAV1DpbSlotInfoKHR {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
p_std_reference_info @ 16,
});
assert_abi_layout!(VideoEncodeAV1RateControlInfoKHR {
size: 40, align: 8,
s_type @ 0,
p_next @ 8,
flags @ 16,
gop_frame_count @ 20,
key_frame_period @ 24,
consecutive_bipredictive_frame_count @ 28,
temporal_layer_count @ 32,
});
assert_abi_layout!(VideoEncodeAV1QIndexKHR {
size: 12, align: 4,
intra_q_index @ 0,
predictive_q_index @ 4,
bipredictive_q_index @ 8,
});
assert_abi_layout!(VideoEncodeAV1FrameSizeKHR {
size: 12, align: 4,
intra_frame_size @ 0,
predictive_frame_size @ 4,
bipredictive_frame_size @ 8,
});
assert_abi_layout!(VideoEncodeAV1RateControlLayerInfoKHR {
size: 64, align: 8,
s_type @ 0,
p_next @ 8,
use_min_q_index @ 16,
min_q_index @ 20,
use_max_q_index @ 32,
max_q_index @ 36,
use_max_frame_size @ 48,
max_frame_size @ 52,
});
assert_abi_layout!(VideoEncodeAV1GopRemainingFrameInfoKHR {
size: 32, align: 8,
s_type @ 0,
p_next @ 8,
use_gop_remaining_frames @ 16,
gop_remaining_intra @ 20,
gop_remaining_predictive @ 24,
gop_remaining_bipredictive @ 28,
});
assert_abi_layout!(VideoEncodeAV1SessionCreateInfoKHR {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
use_max_level @ 16,
max_level @ 20,
});
#[cfg(test)]
mod abi_tests {
use super::*;
/// Assert that `set` lights exactly one bit of the flags word, at `bit`.
fn sets_only_bit(
storage: __BindgenBitfieldUnit<[u8; 4]>,
name: &str,
bit: usize,
expected_width: usize,
) {
for probe in 0..32 {
let want = probe >= bit && probe < bit + expected_width;
assert_eq!(
storage.get_bit(probe),
want,
"`{name}` should occupy bit(s) {bit}..{}, but bit {probe} disagrees",
bit + expected_width
);
}
}
/// A C bitfield allocates its members from bit 0 upward in declaration order, so the setter for
/// the Nth member of `StdVideoEncodeAV1PictureInfoFlags` must write bit N. The array below is
/// the member list of `vulkan_video_codec_av1std_encode.h` **in declaration order** — so this
/// pins the hand-copied bit indices to the header rather than merely to themselves. A silent
/// renumbering here would make the driver read, say, `use_superres` where we meant
/// `render_and_frame_size_different`.
#[test]
fn picture_info_flag_setters_follow_the_c_declaration_order() {
#[allow(clippy::type_complexity)]
let members: [(&str, fn(&mut StdVideoEncodeAV1PictureInfoFlags)); 29] = [
("error_resilient_mode", |f| f.set_error_resilient_mode(1)),
("disable_cdf_update", |f| f.set_disable_cdf_update(1)),
("use_superres", |f| f.set_use_superres(1)),
("render_and_frame_size_different", |f| {
f.set_render_and_frame_size_different(1)
}),
("allow_screen_content_tools", |f| {
f.set_allow_screen_content_tools(1)
}),
("is_filter_switchable", |f| f.set_is_filter_switchable(1)),
("force_integer_mv", |f| f.set_force_integer_mv(1)),
("frame_size_override_flag", |f| {
f.set_frame_size_override_flag(1)
}),
("buffer_removal_time_present_flag", |f| {
f.set_buffer_removal_time_present_flag(1)
}),
("allow_intrabc", |f| f.set_allow_intrabc(1)),
("frame_refs_short_signaling", |f| {
f.set_frame_refs_short_signaling(1)
}),
("allow_high_precision_mv", |f| {
f.set_allow_high_precision_mv(1)
}),
("is_motion_mode_switchable", |f| {
f.set_is_motion_mode_switchable(1)
}),
("use_ref_frame_mvs", |f| f.set_use_ref_frame_mvs(1)),
("disable_frame_end_update_cdf", |f| {
f.set_disable_frame_end_update_cdf(1)
}),
("allow_warped_motion", |f| f.set_allow_warped_motion(1)),
("reduced_tx_set", |f| f.set_reduced_tx_set(1)),
("skip_mode_present", |f| f.set_skip_mode_present(1)),
("delta_q_present", |f| f.set_delta_q_present(1)),
("delta_lf_present", |f| f.set_delta_lf_present(1)),
("delta_lf_multi", |f| f.set_delta_lf_multi(1)),
("segmentation_enabled", |f| f.set_segmentation_enabled(1)),
("segmentation_update_map", |f| {
f.set_segmentation_update_map(1)
}),
("segmentation_temporal_update", |f| {
f.set_segmentation_temporal_update(1)
}),
("segmentation_update_data", |f| {
f.set_segmentation_update_data(1)
}),
("UsesLr", |f| f.set_UsesLr(1)),
("usesChromaLr", |f| f.set_usesChromaLr(1)),
("show_frame", |f| f.set_show_frame(1)),
("showable_frame", |f| f.set_showable_frame(1)),
];
for (bit, (name, set)) in members.into_iter().enumerate() {
let mut flags = StdVideoEncodeAV1PictureInfoFlags {
_bitfield_align_1: [],
_bitfield_1: __BindgenBitfieldUnit::new([0u8; 4]),
};
set(&mut flags);
sets_only_bit(flags._bitfield_1, name, bit, 1);
}
}
/// `reserved : 3` closes out the word — bits 29..32. Checking it is what proves the 29 members
/// above are the *whole* list: a dropped member would shift `reserved` down and fail here.
#[test]
fn picture_info_reserved_occupies_the_top_three_bits() {
let mut flags = StdVideoEncodeAV1PictureInfoFlags {
_bitfield_align_1: [],
_bitfield_1: __BindgenBitfieldUnit::new([0u8; 4]),
};
flags.set_reserved(0b111);
sets_only_bit(flags._bitfield_1, "reserved", 29, 3);
}
/// The same invariant for the two-member reference-info flags word.
#[test]
fn reference_info_flag_setters_follow_the_c_declaration_order() {
#[allow(clippy::type_complexity)]
let members: [(&str, fn(&mut StdVideoEncodeAV1ReferenceInfoFlags)); 2] = [
("disable_frame_end_update_cdf", |f| {
f.set_disable_frame_end_update_cdf(1)
}),
("segmentation_enabled", |f| f.set_segmentation_enabled(1)),
];
for (bit, (name, set)) in members.into_iter().enumerate() {
let mut flags = StdVideoEncodeAV1ReferenceInfoFlags {
_bitfield_align_1: [],
_bitfield_1: __BindgenBitfieldUnit::new([0u8; 4]),
};
set(&mut flags);
sets_only_bit(flags._bitfield_1, name, bit, 1);
}
}
}
+836
View File
@@ -0,0 +1,836 @@
//! Session/frame **construction** for the Vulkan Video encoder — the unsafe builders
//! (`make_frame*`, `make_video_image`, `probe_rgb_direct`) and the parameter-set bitstream
//! writers (`build_parameters_h265`/`_av1`, the AV1 sequence-header OBU). Split from
//! `vulkan_video.rs` (WP7.5) the way `amf_sys.rs` was split from `amf.rs`: a `#[path]` child
//! module, so this file sees the parent's private items (`Frame` and friends) with zero
//! visibility churn, and ~800 lines of construction `unsafe` get their own review surface.
//! Steady-state encode logic stays in the parent.
// The parent's whole item namespace (Frame, the consts, sibling helpers) — the point of the
// child-module shape. External imports are this file's own; `vk_util` is a crate-root sibling,
// so the path is `crate::`, not the parent-relative `super::` the parent uses.
use super::*;
use crate::vk_util::{find_mem, make_plain_image, make_view};
use anyhow::{bail, Result};
use ash::vk;
use std::ffi::c_void;
pub(super) fn align_up(v: u64, a: u64) -> u64 {
v.div_ceil(a) * a
}
/// Probe for the RGB-direct encode source (design/vulkan-rgb-direct-encode.md): can this device
/// take the captured RGB dmabuf directly, with the VCN EFC front-end doing the 709-narrow CSC,
/// via `VK_VALVE_video_encode_rgb_conversion` (RADV since Mesa 26.0, gated on EFC hardware)?
/// `Ok((x_offset, y_offset))` carries the chroma-siting bits a session must be created with
/// (the preferred available bit per axis); `Err` is the first missing requirement, logged as
/// the open-time verdict.
pub(super) unsafe fn probe_rgb_direct(
instance: &ash::Instance,
vq_inst: &ash::khr::video_queue::Instance,
pd: vk::PhysicalDevice,
codec_op: vk::VideoCodecOperationFlagsKHR,
av1: bool,
) -> Result<(u32, u32), &'static str> {
use crate::vk_av1_encode as av1b;
use crate::vk_valve_rgb as vrgb;
// 1. The device extension must exist (Mesa >= 26.0 AND the VCN has an EFC block).
let Ok(exts) = instance.enumerate_device_extension_properties(pd) else {
return Err("probe-failed(ext-enum)");
};
if !exts
.iter()
.any(|e| std::ffi::CStr::from_ptr(e.extension_name.as_ptr()) == vrgb::EXTENSION_NAME)
{
return Err("no-ext(mesa<26.0-or-no-efc)");
}
// 2. Feature bit.
let mut feat = vrgb::PhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE {
s_type: vrgb::stype(vrgb::ST_PHYSICAL_DEVICE_FEATURES),
p_next: std::ptr::null_mut(),
video_encode_rgb_conversion: vk::FALSE,
};
let mut f2 = vk::PhysicalDeviceFeatures2 {
p_next: &mut feat as *mut _ as *mut c_void,
..Default::default()
};
instance.get_physical_device_features2(pd, &mut f2);
if feat.video_encode_rgb_conversion == vk::FALSE {
return Err("no-feature");
}
// 3. Capabilities under the rgb-chained profile — the conversion must cover the compute
// CSC's colour math (rgb2yuv.comp: BT.709, narrow range; chroma siting is looser, see
// below). The profile chain is the same one every rgb-direct consumer presents.
let mut ps = RgbProfileStack::new(codec_op);
let profile = *ps.wire(av1);
let mut rgb_caps = vrgb::VideoEncodeRgbConversionCapabilitiesVALVE {
s_type: vrgb::stype(vrgb::ST_CAPABILITIES),
p_next: std::ptr::null_mut(),
rgb_models: 0,
rgb_ranges: 0,
x_chroma_offsets: 0,
y_chroma_offsets: 0,
};
let mut h265_caps = vk::VideoEncodeH265CapabilitiesKHR::default();
let mut av1_caps: av1b::VideoEncodeAV1CapabilitiesKHR = std::mem::zeroed();
av1_caps.s_type = av1b::stype(av1b::ST_CAPABILITIES);
let mut enc_caps = vk::VideoEncodeCapabilitiesKHR::default();
let mut caps = vk::VideoCapabilitiesKHR::default();
if av1 {
av1_caps.p_next = &mut rgb_caps as *mut _ as *mut c_void;
enc_caps.p_next = &mut av1_caps as *mut _ as *mut c_void;
} else {
h265_caps.p_next = &mut rgb_caps as *mut _ as *mut c_void;
enc_caps.p_next = &mut h265_caps as *mut _ as *mut c_void;
}
caps.p_next = &mut enc_caps as *mut _ as *mut c_void;
let r = (vq_inst.fp().get_physical_device_video_capabilities_khr)(pd, &profile, &mut caps);
if r != vk::Result::SUCCESS {
return Err("no-rgb-profile(caps)");
}
// Colour model + range must match the shader exactly (709 narrow). Chroma siting is looser
// BY ON-GLASS FINDING (RADV 26.0.4 / 780M): the VCN EFC advertises x=COSITED_EVEN only —
// the canonical H.26x left-cosited siting — while our 2x2-average shader is midpoint. The
// difference is a half-pel chroma-x phase, imperceptible (and EFC's is arguably the more
// correct one since nothing in our bitstream signals siting). Accept either bit per axis
// and choose the closest to the shader's math: midpoint if offered, else cosited-even.
let pick = |offered: u32| -> Option<u32> {
if offered & vrgb::CHROMA_OFFSET_MIDPOINT != 0 {
Some(vrgb::CHROMA_OFFSET_MIDPOINT)
} else if offered & vrgb::CHROMA_OFFSET_COSITED_EVEN != 0 {
Some(vrgb::CHROMA_OFFSET_COSITED_EVEN)
} else {
None
}
};
if rgb_caps.rgb_models & vrgb::MODEL_YCBCR_709 == 0
|| rgb_caps.rgb_ranges & vrgb::RANGE_NARROW == 0
{
return Err("no-709-narrow");
}
let (Some(x_offset), Some(y_offset)) = (
pick(rgb_caps.x_chroma_offsets),
pick(rgb_caps.y_chroma_offsets),
) else {
return Err("no-chroma-siting");
};
// 4. The encode-src format set under this profile must offer BGRA with DRM-modifier tiling —
// the capture hands LINEAR BGRx dmabufs (fourcc XR24), which import as B8G8R8A8_UNORM.
let profile_arr = [profile];
let plist = vk::VideoProfileListInfoKHR::default().profiles(&profile_arr);
let mut fmt_info = vk::PhysicalDeviceVideoFormatInfoKHR::default()
.image_usage(vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR);
fmt_info.p_next = &plist as *const _ as *const c_void;
let get_fmt = vq_inst.fp().get_physical_device_video_format_properties_khr;
let mut count = 0u32;
let r = get_fmt(pd, &fmt_info, &mut count, std::ptr::null_mut());
if r != vk::Result::SUCCESS || count == 0 {
return Err("no-rgb-format");
}
let mut props = vec![vk::VideoFormatPropertiesKHR::default(); count as usize];
let r = get_fmt(pd, &fmt_info, &mut count, props.as_mut_ptr());
if r != vk::Result::SUCCESS && r != vk::Result::INCOMPLETE {
return Err("no-rgb-format");
}
if !props[..count as usize].iter().any(|p| {
p.format == vk::Format::B8G8R8A8_UNORM
&& p.image_tiling == vk::ImageTiling::DRM_FORMAT_MODIFIER_EXT
}) {
return Err("no-bgra-modifier-tiling");
}
Ok((x_offset, y_offset))
}
pub(super) unsafe fn make_video_image(
device: &ash::Device,
mp: &vk::PhysicalDeviceMemoryProperties,
fmt: vk::Format,
w: u32,
h: u32,
layers: u32,
usage: vk::ImageUsageFlags,
profile_list: &mut vk::VideoProfileListInfoKHR,
concurrent: &[u32],
) -> Result<(vk::Image, vk::DeviceMemory)> {
let mut ci = vk::ImageCreateInfo::default()
.image_type(vk::ImageType::TYPE_2D)
.format(fmt)
.extent(vk::Extent3D {
width: w,
height: h,
depth: 1,
})
.mip_levels(1)
.array_layers(layers)
.samples(vk::SampleCountFlags::TYPE_1)
.tiling(vk::ImageTiling::OPTIMAL)
.usage(usage)
.initial_layout(vk::ImageLayout::UNDEFINED)
.push_next(profile_list);
if concurrent.len() >= 2 {
ci = ci
.sharing_mode(vk::SharingMode::CONCURRENT)
.queue_family_indices(concurrent);
} else {
ci = ci.sharing_mode(vk::SharingMode::EXCLUSIVE);
}
let img = device.create_image(&ci, None)?;
let req = device.get_image_memory_requirements(img);
// Unwind on failure: callers (the open path) only ever see the completed pair.
let mem = match device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(find_mem(
mp,
req.memory_type_bits,
vk::MemoryPropertyFlags::DEVICE_LOCAL,
)),
None,
) {
Ok(m) => m,
Err(e) => {
device.destroy_image(img, None);
return Err(e.into());
}
};
if let Err(e) = device.bind_image_memory(img, mem, 0) {
device.destroy_image(img, None);
device.free_memory(mem, None);
return Err(e.into());
}
Ok((img, mem))
}
/// Build one in-flight frame's private resources: NV12 encode-src, Y/UV CSC scratch, its CSC
/// descriptor set (Y/UV bound now, RGB per use), the bitstream buffer + feedback query, and the
/// per-frame command buffers + sync. `profile_list`/`profile` are borrowed only during creation.
///
/// Builds in place into `f` — a [`Frame::default`] the caller has already parked in its
/// [`VkTeardown`] guard — so every handle is owned by the unwind the moment it exists and a
/// mid-build failure leaks nothing.
pub(super) unsafe fn make_frame(
device: &ash::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
w: u32,
h: u32,
fams: &[u32],
profile: &vk::VideoProfileInfoKHR,
profile_list: &mut vk::VideoProfileListInfoKHR,
csc_dsl: vk::DescriptorSetLayout,
csc_pool: vk::DescriptorPool,
cmd_pool: vk::CommandPool,
compute_pool: vk::CommandPool,
bs_size: u64,
sampler: vk::Sampler,
with_ts: bool,
csc: bool,
pad_fmt: Option<vk::Format>,
f: &mut Frame,
) -> Result<()> {
// "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`).
f.cursor_serial = u64::MAX;
// Padded-copy staging (unaligned-mode RGB-direct or native NV12): an aligned encode-src in
// the session's picture format, filled by a transfer blit each frame — concurrent compute
// (copy) + encode (source read). TRANSFER_SRC because the width-padding pass self-copies the
// staging image's own last visible column (see `record_pad_blit`).
if let Some(fmt) = pad_fmt {
(f.pad_img, f.pad_mem) = make_video_image(
device,
mem_props,
fmt,
w,
h,
1,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR
| vk::ImageUsageFlags::TRANSFER_DST
| vk::ImageUsageFlags::TRANSFER_SRC,
profile_list,
fams,
)?;
f.pad_view = make_view(device, f.pad_img, fmt, 0)?;
}
// RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no
// cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the
// CPU staging image, built lazily). Their Frame keeps the null handles (teardown-safe).
if csc {
make_frame_csc(
device,
mem_props,
w,
h,
fams,
profile_list,
csc_dsl,
csc_pool,
sampler,
f,
)?;
}
make_frame_common(
device,
mem_props,
profile,
profile_list,
cmd_pool,
compute_pool,
bs_size,
with_ts,
f,
)
}
/// The CSC-only half of [`make_frame`]: NV12 encode-src + Y/UV scratch + cursor + descriptors.
#[allow(clippy::too_many_arguments)]
unsafe fn make_frame_csc(
device: &ash::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
w: u32,
h: u32,
fams: &[u32],
profile_list: &mut vk::VideoProfileListInfoKHR,
csc_dsl: vk::DescriptorSetLayout,
csc_pool: vk::DescriptorPool,
sampler: vk::Sampler,
f: &mut Frame,
) -> Result<()> {
// NV12 encode-src (filled by the CSC copy) — concurrent compute+encode.
(f.nv12_src, f.nv12_mem) = make_video_image(
device,
mem_props,
NV12,
w,
h,
1,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR | vk::ImageUsageFlags::TRANSFER_DST,
profile_list,
fams,
)?;
f.nv12_view = make_view(device, f.nv12_src, NV12, 0)?;
// CSC scratch (Y R8 full-res, UV RG8 half-res).
(f.y_img, f.y_mem, f.y_view) = make_plain_image(
device,
mem_props,
vk::Format::R8_UNORM,
w,
h,
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::TRANSFER_SRC,
)?;
(f.uv_img, f.uv_mem, f.uv_view) = make_plain_image(
device,
mem_props,
vk::Format::R8G8_UNORM,
w / 2,
h / 2,
vk::ImageUsageFlags::STORAGE | vk::ImageUsageFlags::TRANSFER_SRC,
)?;
// Cursor overlay: fixed CURSOR_MAX² RGBA8 sampled image + host staging (cursor-as-metadata). The
// view/descriptor is static (bound at binding 3 below); only the image *content* changes, and
// only when the pointer bitmap does — see `prep_cursor`.
(f.cursor_img, f.cursor_mem, f.cursor_view) = make_plain_image(
device,
mem_props,
vk::Format::R8G8B8A8_UNORM,
CURSOR_MAX,
CURSOR_MAX,
vk::ImageUsageFlags::SAMPLED | vk::ImageUsageFlags::TRANSFER_DST,
)?;
f.cursor_stage = device.create_buffer(
&vk::BufferCreateInfo::default()
.size((CURSOR_MAX * CURSOR_MAX * 4) as u64)
.usage(vk::BufferUsageFlags::TRANSFER_SRC),
None,
)?;
let cs_req = device.get_buffer_memory_requirements(f.cursor_stage);
f.cursor_stage_mem = device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(cs_req.size)
.memory_type_index(find_mem(
mem_props,
cs_req.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
)?;
device.bind_buffer_memory(f.cursor_stage, f.cursor_stage_mem, 0)?;
// Descriptor set — Y/UV storage bindings fixed; binding 0 (RGB) rewritten per use; binding 3
// (cursor) points at the static cursor image (its layout is SHADER_READ_ONLY once prepped).
let dsls = [csc_dsl];
f.csc_set = device.allocate_descriptor_sets(
&vk::DescriptorSetAllocateInfo::default()
.descriptor_pool(csc_pool)
.set_layouts(&dsls),
)?[0];
let y_info = [vk::DescriptorImageInfo::default()
.image_view(f.y_view)
.image_layout(vk::ImageLayout::GENERAL)];
let uv_info = [vk::DescriptorImageInfo::default()
.image_view(f.uv_view)
.image_layout(vk::ImageLayout::GENERAL)];
let cur_info = [vk::DescriptorImageInfo::default()
.sampler(sampler)
.image_view(f.cursor_view)
.image_layout(vk::ImageLayout::SHADER_READ_ONLY_OPTIMAL)];
device.update_descriptor_sets(
&[
vk::WriteDescriptorSet::default()
.dst_set(f.csc_set)
.dst_binding(1)
.descriptor_type(vk::DescriptorType::STORAGE_IMAGE)
.image_info(&y_info),
vk::WriteDescriptorSet::default()
.dst_set(f.csc_set)
.dst_binding(2)
.descriptor_type(vk::DescriptorType::STORAGE_IMAGE)
.image_info(&uv_info),
vk::WriteDescriptorSet::default()
.dst_set(f.csc_set)
.dst_binding(3)
.descriptor_type(vk::DescriptorType::COMBINED_IMAGE_SAMPLER)
.image_info(&cur_info),
],
&[],
);
Ok(())
}
/// The mode-independent half of [`make_frame`]: bitstream buffer (+ persistent map), feedback
/// query, optional timestamp pool, command buffers and sync objects.
#[allow(clippy::too_many_arguments)]
unsafe fn make_frame_common(
device: &ash::Device,
mem_props: &vk::PhysicalDeviceMemoryProperties,
profile: &vk::VideoProfileInfoKHR,
profile_list: &mut vk::VideoProfileListInfoKHR,
cmd_pool: vk::CommandPool,
compute_pool: vk::CommandPool,
bs_size: u64,
with_ts: bool,
f: &mut Frame,
) -> Result<()> {
// Bitstream buffer + feedback query.
f.bs_buf = device.create_buffer(
&vk::BufferCreateInfo::default()
.size(bs_size)
.usage(vk::BufferUsageFlags::VIDEO_ENCODE_DST_KHR)
.push_next(profile_list),
None,
)?;
let bs_req = device.get_buffer_memory_requirements(f.bs_buf);
f.bs_mem = device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(bs_req.size)
.memory_type_index(find_mem(
mem_props,
bs_req.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
)?;
device.bind_buffer_memory(f.bs_buf, f.bs_mem, 0)?;
// Map once for the slot's lifetime — read_slot copies AUs straight out of this (coherent
// memory, no per-frame map/unmap); vkFreeMemory implicitly unmaps at teardown.
f.bs_ptr = BsPtr(
device.map_memory(f.bs_mem, 0, vk::WHOLE_SIZE, vk::MemoryMapFlags::empty())? as *const u8,
);
// PUNKTFUNK_PERF: a 2-slot timestamp pool bracketing this slot's compute batch (CSC split).
if with_ts {
f.ts_pool = device.create_query_pool(
&vk::QueryPoolCreateInfo::default()
.query_type(vk::QueryType::TIMESTAMP)
.query_count(2),
None,
)?;
}
let mut fb_ci = vk::QueryPoolVideoEncodeFeedbackCreateInfoKHR::default().encode_feedback_flags(
vk::VideoEncodeFeedbackFlagsKHR::BITSTREAM_BUFFER_OFFSET
| vk::VideoEncodeFeedbackFlagsKHR::BITSTREAM_BYTES_WRITTEN,
);
fb_ci.p_next = profile as *const _ as *const c_void;
let mut query_ci = vk::QueryPoolCreateInfo::default()
.query_type(vk::QueryType::VIDEO_ENCODE_FEEDBACK_KHR)
.query_count(1);
query_ci.p_next = &fb_ci as *const _ as *const c_void;
f.query_pool = device.create_query_pool(&query_ci, None)?;
// Command buffers + per-frame sync.
f.cmd = device.allocate_command_buffers(
&vk::CommandBufferAllocateInfo::default()
.command_pool(cmd_pool)
.command_buffer_count(1),
)?[0];
f.compute_cmd = device.allocate_command_buffers(
&vk::CommandBufferAllocateInfo::default()
.command_pool(compute_pool)
.command_buffer_count(1),
)?[0];
f.csc_sem = device.create_semaphore(&vk::SemaphoreCreateInfo::default(), None)?;
f.fence = device.create_fence(&vk::FenceCreateInfo::default(), None)?;
Ok(())
}
/// Author VPS/SPS/PPS (Main, level 4.0, low-latency, conformance-window crop) and return the
/// session-parameters object + the encoded header bytes (VPS+SPS+PPS NALs) for keyframes.
pub(super) unsafe fn build_parameters_h265(
device: &ash::Device,
vq_dev: &ash::khr::video_queue::Device,
venc_dev: &ash::khr::video_encode_queue::Device,
session: vk::VideoSessionKHR,
w: u32,
h: u32,
rw: u32,
rh: u32,
quality_level: u32,
) -> Result<(vk::VideoSessionParametersKHR, Vec<u8>)> {
use ash::vk::native as hh;
let mut ptl: hh::StdVideoH265ProfileTierLevel = std::mem::zeroed();
ptl.flags.set_general_progressive_source_flag(1);
ptl.flags.set_general_frame_only_constraint_flag(1);
ptl.general_profile_idc = hh::StdVideoH265ProfileIdc_STD_VIDEO_H265_PROFILE_IDC_MAIN;
ptl.general_level_idc = hh::StdVideoH265LevelIdc_STD_VIDEO_H265_LEVEL_IDC_6_0;
let mut dpbm: hh::StdVideoH265DecPicBufMgr = std::mem::zeroed();
dpbm.max_dec_pic_buffering_minus1[0] = (DPB_SLOTS - 1) as u8;
dpbm.max_num_reorder_pics[0] = 0;
dpbm.max_latency_increase_plus1[0] = 0;
let mut vps: hh::StdVideoH265VideoParameterSet = std::mem::zeroed();
vps.flags.set_vps_temporal_id_nesting_flag(1);
vps.flags.set_vps_sub_layer_ordering_info_present_flag(1);
vps.pDecPicBufMgr = &dpbm;
vps.pProfileTierLevel = &ptl;
let mut sps: hh::StdVideoH265SequenceParameterSet = std::mem::zeroed();
sps.flags.set_sps_temporal_id_nesting_flag(1);
sps.flags.set_sps_sub_layer_ordering_info_present_flag(1);
sps.chroma_format_idc = hh::StdVideoH265ChromaFormatIdc_STD_VIDEO_H265_CHROMA_FORMAT_IDC_420;
sps.pic_width_in_luma_samples = w;
sps.pic_height_in_luma_samples = h;
sps.log2_max_pic_order_cnt_lsb_minus4 = 4;
sps.log2_diff_max_min_luma_coding_block_size = 3;
sps.log2_diff_max_min_luma_transform_block_size = 3;
sps.max_transform_hierarchy_depth_inter = 4;
sps.max_transform_hierarchy_depth_intra = 4;
sps.pProfileTierLevel = &ptl;
sps.pDecPicBufMgr = &dpbm;
if w != rw || h != rh {
sps.flags.set_conformance_window_flag(1);
sps.conf_win_right_offset = (w - rw) / 2; // 4:2:0 SubWidthC = 2
sps.conf_win_bottom_offset = (h - rh) / 2; // 4:2:0 SubHeightC = 2
}
let mut pps: hh::StdVideoH265PictureParameterSet = std::mem::zeroed();
pps.flags.set_cu_qp_delta_enabled_flag(1);
pps.flags.set_pps_loop_filter_across_slices_enabled_flag(1);
let vps_arr = [vps];
let sps_arr = [sps];
let pps_arr = [pps];
let add = vk::VideoEncodeH265SessionParametersAddInfoKHR::default()
.std_vp_ss(&vps_arr)
.std_sp_ss(&sps_arr)
.std_pp_ss(&pps_arr);
let mut h265_ci = vk::VideoEncodeH265SessionParametersCreateInfoKHR::default()
.max_std_vps_count(1)
.max_std_sps_count(1)
.max_std_pps_count(1)
.parameters_add_info(&add);
// Bake the session's quality level into the parameters object — the spec requires it to match
// the level the first frame's ENCODE_QUALITY_LEVEL control installs.
let mut q_info = vk::VideoEncodeQualityLevelInfoKHR::default().quality_level(quality_level);
let ci = vk::VideoSessionParametersCreateInfoKHR::default()
.video_session(session)
.push_next(&mut h265_ci)
.push_next(&mut q_info);
let mut params = vk::VideoSessionParametersKHR::null();
let r = (vq_dev.fp().create_video_session_parameters_khr)(
device.handle(),
&ci,
std::ptr::null(),
&mut params,
);
if r != vk::Result::SUCCESS {
bail!("create_video_session_parameters: {r:?}");
}
let mut get_h265 = vk::VideoEncodeH265SessionParametersGetInfoKHR::default()
.write_std_vps(true)
.write_std_sps(true)
.write_std_pps(true)
.std_vps_id(0)
.std_sps_id(0)
.std_pps_id(0);
let get = vk::VideoEncodeSessionParametersGetInfoKHR::default()
.video_session_parameters(params)
.push_next(&mut get_h265);
let get_fn = venc_dev.fp().get_encoded_video_session_parameters_khr;
let mut fb = vk::VideoEncodeSessionParametersFeedbackInfoKHR::default();
let mut size: usize = 0;
let r = get_fn(
device.handle(),
&get,
&mut fb,
&mut size,
std::ptr::null_mut(),
);
if r != vk::Result::SUCCESS {
// `params` is live but not yet the caller's guard's to unwind — destroy before bailing.
(vq_dev.fp().destroy_video_session_parameters_khr)(
device.handle(),
params,
std::ptr::null(),
);
bail!("get header size: {r:?}");
}
let mut buf = vec![0u8; size];
let r = get_fn(
device.handle(),
&get,
&mut fb,
&mut size,
buf.as_mut_ptr() as *mut c_void,
);
if r != vk::Result::SUCCESS {
(vq_dev.fp().destroy_video_session_parameters_khr)(
device.handle(),
params,
std::ptr::null(),
);
bail!("get header bytes: {r:?}");
}
buf.truncate(size);
Ok((params, buf))
}
/// AV1 low-overhead OBU bit-writer (MSB-first), used to hand-pack the sequence-header OBU that
/// Vulkan AV1 encode (unlike H26x) never emits itself.
struct Av1BitWriter {
buf: Vec<u8>,
cur: u8,
fill: u8,
}
impl Av1BitWriter {
fn new() -> Self {
Self {
buf: Vec::new(),
cur: 0,
fill: 0,
}
}
fn bit(&mut self, b: u32) {
self.cur = (self.cur << 1) | (b as u8 & 1);
self.fill += 1;
if self.fill == 8 {
self.buf.push(self.cur);
self.cur = 0;
self.fill = 0;
}
}
fn put(&mut self, val: u32, bits: u32) {
for i in (0..bits).rev() {
self.bit((val >> i) & 1);
}
}
/// Flush, zero-padding the final partial byte (OBU size field delimits the payload).
fn finish(mut self) -> Vec<u8> {
if self.fill > 0 {
self.cur <<= 8 - self.fill;
self.buf.push(self.cur);
}
self.buf
}
}
/// AV1 leb128 (little-endian base-128) encoding of an OBU size.
fn leb128(mut v: u64) -> Vec<u8> {
let mut out = Vec::new();
loop {
let mut byte = (v & 0x7f) as u8;
v >>= 7;
if v != 0 {
byte |= 0x80;
}
out.push(byte);
if v == 0 {
break;
}
}
out
}
/// Bit-pack a `sequence_header_obu` (AV1 spec §5.5) into a size-delimited OBU. The field values here
/// MUST mirror the `StdVideoAV1SequenceHeader` handed to the driver in `build_parameters_av1` so the
/// driver-emitted frame OBUs parse against this header. Single operating point, 8-bit 4:2:0,
/// order-hint on, CDEF+restoration+filter-intra allowed, everything exotic (compound/warp/superres)
/// disabled — the profile our single-reference P-frame encoder actually uses.
fn av1_sequence_header_obu(
sb128: bool,
fwb: u32,
fhb: u32,
max_w_m1: u32,
max_h_m1: u32,
order_hint_bits_minus_1: u32,
seq_level_idx: u32,
) -> Vec<u8> {
let mut w = Av1BitWriter::new();
w.put(0, 3); // seq_profile = MAIN
w.bit(0); // still_picture
w.bit(0); // reduced_still_picture_header
w.bit(0); // timing_info_present_flag
w.bit(0); // initial_display_delay_present_flag
w.put(0, 5); // operating_points_cnt_minus_1 = 0
w.put(0, 12); // operating_point_idc[0]
w.put(seq_level_idx, 5); // seq_level_idx[0]
if seq_level_idx > 7 {
w.bit(0); // seq_tier[0] = 0
}
w.put(fwb, 4); // frame_width_bits_minus_1
w.put(fhb, 4); // frame_height_bits_minus_1
w.put(max_w_m1, fwb + 1); // max_frame_width_minus_1
w.put(max_h_m1, fhb + 1); // max_frame_height_minus_1
w.bit(0); // frame_id_numbers_present_flag
w.bit(sb128 as u32); // use_128x128_superblock
w.bit(0); // enable_filter_intra
w.bit(0); // enable_intra_edge_filter
w.bit(0); // enable_interintra_compound
w.bit(0); // enable_masked_compound
w.bit(0); // enable_warped_motion
w.bit(0); // enable_dual_filter
w.bit(1); // enable_order_hint
w.bit(0); // enable_jnt_comp
w.bit(0); // enable_ref_frame_mvs
w.bit(1); // seq_choose_screen_content_tools -> seq_force_screen_content_tools = SELECT
w.bit(1); // seq_choose_integer_mv -> seq_force_integer_mv = SELECT
w.put(order_hint_bits_minus_1, 3); // order_hint_bits_minus_1
w.bit(0); // enable_superres
w.bit(0); // enable_cdef
w.bit(0); // enable_restoration
// color_config(): 8-bit 4:2:0, unspecified primaries/transfer/matrix, limited range
w.bit(0); // high_bitdepth
w.bit(0); // mono_chrome
w.bit(0); // color_description_present_flag
w.bit(0); // color_range (studio/limited)
w.put(0, 2); // chroma_sample_position = CSP_UNKNOWN (subsampling_x==subsampling_y==1 for profile 0)
w.bit(0); // separate_uv_delta_q
w.bit(0); // film_grain_params_present
// trailing_bits(): a stop `1` bit then zero-pad to a byte (the size field delimits the OBU, but
// the parser still requires the trailing_one_bit — dav1d/cbs reject a plain zero pad).
w.bit(1);
let payload = w.finish();
let mut obu = vec![0x0au8]; // obu_header: type=OBU_SEQUENCE_HEADER(1), has_size_field=1
obu.extend_from_slice(&leb128(payload.len() as u64));
obu.extend_from_slice(&payload);
obu
}
/// AV1 session parameters + header framing. Vulkan AV1 encode emits only the per-frame OBU, so we
/// return the app-owned prefixes: a temporal-delimiter OBU that opens every temporal unit
/// (`frame_prefix`), and TD + the bit-packed sequence-header OBU for keyframes (`header`).
#[allow(clippy::too_many_arguments)]
pub(super) unsafe fn build_parameters_av1(
device: &ash::Device,
vq_dev: &ash::khr::video_queue::Device,
session: vk::VideoSessionKHR,
w: u32,
h: u32,
_rw: u32,
_rh: u32,
max_level: ash::vk::native::StdVideoAV1Level,
sb128: bool,
quality_level: u32,
) -> Result<(vk::VideoSessionParametersKHR, Vec<u8>, Vec<u8>)> {
use crate::vk_av1_encode as av1;
use ash::vk::native as hh;
let fwb = 31 - w.leading_zeros(); // av_log2(w): enough bits for max_frame_width_minus_1 = w-1
let fhb = 31 - h.leading_zeros();
let order_hint_bits_minus_1: u32 = 7; // OrderHintBits = 8
let seq_level_idx = max_level; // StdVideoAV1Level's numeric value IS the AV1 seq_level_idx
// ---- Std sequence header (must match the OBU packed below) ----
let mut cc_flags: hh::StdVideoAV1ColorConfigFlags = std::mem::zeroed();
let _ = &mut cc_flags; // all zero: mono_chrome/color_range/description/separate_uv_delta_q = 0
let mut cc: hh::StdVideoAV1ColorConfig = std::mem::zeroed();
cc.flags = cc_flags;
cc.BitDepth = 8;
cc.subsampling_x = 1;
cc.subsampling_y = 1;
cc.color_primaries = hh::StdVideoAV1ColorPrimaries_STD_VIDEO_AV1_COLOR_PRIMARIES_BT_UNSPECIFIED;
cc.transfer_characteristics =
hh::StdVideoAV1TransferCharacteristics_STD_VIDEO_AV1_TRANSFER_CHARACTERISTICS_UNSPECIFIED;
cc.matrix_coefficients =
hh::StdVideoAV1MatrixCoefficients_STD_VIDEO_AV1_MATRIX_COEFFICIENTS_UNSPECIFIED;
cc.chroma_sample_position =
hh::StdVideoAV1ChromaSamplePosition_STD_VIDEO_AV1_CHROMA_SAMPLE_POSITION_UNKNOWN;
// Match FFmpeg's Vulkan AV1 encoder (proven on this RADV/VCN path): the ONLY coding tools
// enabled are order-hint and (per caps) 128x128 superblocks. CDEF, loop restoration, filter-
// intra, warped/compound motion, superres all OFF — enabling them made VCN emit frame-header
// sections whose bit layout our sequence header didn't match, desyncing every inter frame.
let mut sh_flags: hh::StdVideoAV1SequenceHeaderFlags = std::mem::zeroed();
if sb128 {
sh_flags.set_use_128x128_superblock(1);
}
sh_flags.set_enable_order_hint(1);
let mut sh: hh::StdVideoAV1SequenceHeader = std::mem::zeroed();
sh.flags = sh_flags;
sh.seq_profile = hh::StdVideoAV1Profile_STD_VIDEO_AV1_PROFILE_MAIN;
sh.frame_width_bits_minus_1 = fwb as u8;
sh.frame_height_bits_minus_1 = fhb as u8;
sh.max_frame_width_minus_1 = (w - 1) as u16;
sh.max_frame_height_minus_1 = (h - 1) as u16;
sh.order_hint_bits_minus_1 = order_hint_bits_minus_1 as u8;
sh.seq_force_integer_mv = 2; // SELECT
sh.seq_force_screen_content_tools = 2; // SELECT
sh.pColorConfig = &cc;
// ---- single operating point conveying the level/tier the driver targets ----
let op = av1::StdVideoEncodeAV1OperatingPointInfo {
flags: std::mem::zeroed(),
operating_point_idc: 0,
seq_level_idx: seq_level_idx as u8,
seq_tier: 0,
decoder_buffer_delay: 0,
encoder_buffer_delay: 0,
initial_display_delay_minus_1: 0,
};
let ops = [op];
let av1_spci = av1::VideoEncodeAV1SessionParametersCreateInfoKHR {
s_type: av1::stype(av1::ST_SESSION_PARAMETERS_CREATE_INFO),
p_next: std::ptr::null(),
p_std_sequence_header: &sh,
p_std_decoder_model_info: std::ptr::null(),
std_operating_point_count: 1,
p_std_operating_points: ops.as_ptr() as *const c_void,
};
// Bake the session's quality level into the parameters object (must match the level the first
// frame's ENCODE_QUALITY_LEVEL control installs); chained raw ahead of the vendored AV1 struct.
let mut q_info = vk::VideoEncodeQualityLevelInfoKHR::default().quality_level(quality_level);
q_info.p_next = &av1_spci as *const _ as *const c_void;
let mut ci = vk::VideoSessionParametersCreateInfoKHR::default().video_session(session);
ci.p_next = &q_info as *const _ as *const c_void;
let mut params = vk::VideoSessionParametersKHR::null();
let r = (vq_dev.fp().create_video_session_parameters_khr)(
device.handle(),
&ci,
std::ptr::null(),
&mut params,
);
if r != vk::Result::SUCCESS {
bail!("create_video_session_parameters (av1): {r:?}");
}
// ---- header framing: TD every temporal unit; TD + seq-header OBU on keyframes ----
let td = vec![0x12u8, 0x00]; // temporal_delimiter OBU (type=2, size=0)
let seq_obu = av1_sequence_header_obu(
sb128,
fwb,
fhb,
w - 1,
h - 1,
order_hint_bits_minus_1,
seq_level_idx,
);
let mut keyframe_prefix = td.clone();
keyframe_prefix.extend_from_slice(&seq_obu);
Ok((params, keyframe_prefix, td))
}
+223 -10
View File
@@ -54,6 +54,59 @@ pub(crate) fn pixel_to_vk(fmt: PixelFormat) -> Option<vk::Format> {
}
}
/// Normalize a CPU RGB payload for Vulkan upload. The packed 24-bpp `Rgb`/`Bgr` the PipeWire
/// capturer can negotiate are expanded 3→4 into `scratch` (kept by the caller across frames — no
/// per-frame allocation) with the pad byte = 0xFF; refusing them instead used to kill a session
/// at its first frame (WP5.4). No packed 24-bpp VkFormat is reliably uploadable/sampleable on
/// target GPUs, and this path is CPU-sourced by definition, so one cheap expand pass serves it
/// (the same call NVENC answers with its swscale 3→4 expand, WP1.4).
///
/// `bgra_target = false` (the CSC paths): channel order is preserved — the sampler reads through
/// the matching view format, so any 4-bpp order works and 4-bpp inputs pass through borrowed.
/// `bgra_target = true` (the RGB-direct encode source): the output byte order is forced to
/// B,G,R,X, because the video session's `pictureFormat` is `B8G8R8A8_UNORM` and
/// VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 requires the source image to match it — an
/// R-first source (`Rgbx`/`Rgba`/`Rgb`) is channel-swapped during the same pass. (Caught live on
/// RADV by `vulkan_smoke_rgb_cpu24`; the mismatch predates the 24-bpp support for `Rgbx` CPU
/// sources.)
///
/// Payloads are tightly packed with no row padding (`FramePayload::Cpu`'s contract), so the
/// conversion is row-agnostic; a truncated source yields a truncated output, which the upload
/// paths already bound-check exactly as they did the raw bytes.
pub(crate) fn normalize_cpu_rgb<'a>(
fmt: PixelFormat,
bytes: &'a [u8],
scratch: &'a mut Vec<u8>,
bgra_target: bool,
) -> (PixelFormat, &'a [u8]) {
// Per-pixel source layout: bytes-per-pixel + where R, G, B sit in each pixel.
let (bpp, r, g, b) = match fmt {
PixelFormat::Rgb => (3usize, 0usize, 1usize, 2usize),
PixelFormat::Bgr => (3, 2, 1, 0),
PixelFormat::Rgbx | PixelFormat::Rgba => (4, 0, 1, 2),
PixelFormat::Bgrx | PixelFormat::Bgra => (4, 2, 1, 0),
_ => return (fmt, bytes),
};
if bpp == 4 && (!bgra_target || b == 0) {
return (fmt, bytes); // 4-bpp in an acceptable order: borrow untouched
}
let px = bytes.len() / bpp;
scratch.clear();
scratch.resize(px * 4, 0xFF);
let (dr, dg, db) = if bgra_target { (2, 1, 0) } else { (r, g, b) };
for (dst, src) in scratch.chunks_exact_mut(4).zip(bytes.chunks_exact(bpp)) {
dst[dr] = src[r];
dst[dg] = src[g];
dst[db] = src[b];
}
let out_fmt = if bgra_target || b == 0 {
PixelFormat::Bgrx
} else {
PixelFormat::Rgbx
};
(out_fmt, scratch.as_slice())
}
pub(crate) unsafe fn make_view(
device: &ash::Device,
image: vk::Image,
@@ -70,6 +123,21 @@ pub(crate) unsafe fn make_view(
)?)
}
/// Whether a failed dmabuf import should count toward pf-zerocopy's raw-dmabuf degrade latch
/// (`note_raw_dmabuf_import_failure` — 3 consecutive failures flip capture to CPU delivery for
/// the process). Deterministic refusals (unsupported fourcc, the driver rejecting the buffer)
/// must count — they repeat identically forever and the latch is their only recovery. Transient
/// VRAM pressure must NOT: three tight allocation OOMs would otherwise permanently downgrade a
/// working host to CPU capture.
pub(crate) fn import_failure_feeds_latch(e: &anyhow::Error) -> bool {
match e.downcast_ref::<vk::Result>() {
Some(&r) => {
r != vk::Result::ERROR_OUT_OF_DEVICE_MEMORY && r != vk::Result::ERROR_OUT_OF_HOST_MEMORY
}
None => true,
}
}
/// Import a packed-RGB dmabuf as a SAMPLED VkImage (explicit DRM modifier). Caller destroys all
/// three returned handles. Extracted verbatim from `vulkan_video.rs`'s import path.
pub(crate) unsafe fn import_rgb_dmabuf(
@@ -108,9 +176,15 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
profile_list: Option<&mut vk::VideoProfileListInfoKHR>,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
use anyhow::Context;
use std::os::fd::IntoRawFd;
use std::os::fd::{AsRawFd, IntoRawFd};
let fmt = fourcc_to_vk(d.fourcc)
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
// Dup the fd FIRST, and keep it OWNED: ownership transfers to Vulkan only on a SUCCESSFUL
// `allocate_memory` (VK_KHR_external_memory_fd — from then on `vkFreeMemory` closes it), so
// the release below sits in exactly that arm. Every earlier failure drops the `OwnedFd` for
// a single clean close. An explicit `close` after a successful import would be a double
// close — and a recycled fd number then clobbers an unrelated descriptor in this process.
let dup = d.fd.try_clone().context("dup dmabuf fd")?;
let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
d.offset as u64 + d.stride as u64 * ch as u64,
@@ -155,14 +229,16 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
ci = ci.push_next(pl);
}
let img = device.create_image(&ci, None)?;
// dup the fd; Vulkan takes ownership of the dup on a successful import.
let dup = d.fd.try_clone().context("dup dmabuf fd")?.into_raw_fd();
// Unwind discipline below mirrors `make_plain_image`: every failure destroys what this call
// created (and ONLY that — the caller's `DmabufFrame` fd stays theirs).
let fd_props = {
let mut p = vk::MemoryFdPropertiesKHR::default();
// Borrow-only query (no ownership transfer); an error leaves memory_type_bits = 0 and
// the fallback below uses the image requirements alone.
let _ = (ext_fd.fp().get_memory_fd_properties_khr)(
device.handle(),
vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT,
dup,
dup.as_raw_fd(),
&mut p,
);
p.memory_type_bits
@@ -181,27 +257,87 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
let mut ded = vk::MemoryDedicatedAllocateInfo::default().image(img);
let mut import = vk::ImportMemoryFdInfoKHR::default()
.handle_type(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT)
.fd(dup);
let mem = device.allocate_memory(
.fd(dup.as_raw_fd());
let mem = match device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(ti)
.push_next(&mut ded)
.push_next(&mut import),
None,
)?;
device.bind_image_memory(img, mem, 0)?;
let view = device.create_image_view(
) {
Ok(mem) => {
// Success transferred fd ownership to the memory object — release, don't close.
let _ = dup.into_raw_fd();
mem
}
Err(e) => {
device.destroy_image(img, None);
return Err(e.into()); // `dup` drops here: the one close of the failed import's fd
}
};
if let Err(e) = device.bind_image_memory(img, mem, 0) {
device.destroy_image(img, None);
device.free_memory(mem, None); // closes the imported fd
return Err(e.into());
}
let view = match device.create_image_view(
&vk::ImageViewCreateInfo::default()
.image(img)
.view_type(vk::ImageViewType::TYPE_2D)
.format(fmt)
.subresource_range(color_range(0)),
None,
)?;
) {
Ok(v) => v,
Err(e) => {
device.destroy_image(img, None);
device.free_memory(mem, None);
return Err(e.into());
}
};
Ok((img, mem, view))
}
/// Create + allocate + bind a host-visible/coherent buffer with `make_plain_image`'s unwind
/// discipline: on any failure everything this call created is destroyed before returning, so
/// callers can `?` freely. Both `ensure_cpu_rgb` staging twins open-coded this sequence and
/// leaked the buffer (and then buffer+memory) on the allocate/bind failure arms.
pub(crate) unsafe fn make_host_buffer(
device: &ash::Device,
mp: &vk::PhysicalDeviceMemoryProperties,
size: u64,
usage: vk::BufferUsageFlags,
) -> Result<(vk::Buffer, vk::DeviceMemory)> {
let buf = device.create_buffer(
&vk::BufferCreateInfo::default().size(size).usage(usage),
None,
)?;
let req = device.get_buffer_memory_requirements(buf);
let mem = match device.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(req.size)
.memory_type_index(find_mem(
mp,
req.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)),
None,
) {
Ok(m) => m,
Err(e) => {
device.destroy_buffer(buf, None);
return Err(e.into());
}
};
if let Err(e) = device.bind_buffer_memory(buf, mem, 0) {
device.destroy_buffer(buf, None);
device.free_memory(mem, None);
return Err(e.into());
}
Ok((buf, mem))
}
pub(crate) unsafe fn make_plain_image(
device: &ash::Device,
mp: &vk::PhysicalDeviceMemoryProperties,
@@ -259,3 +395,80 @@ pub(crate) unsafe fn make_plain_image(
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// CSC mode (`bgra_target = false`): the 3→4 expand is a pure byte shuffle — no channel
/// reorder, pad byte 0xFF, truncated tail pixels dropped (never overrun) — and 4-bpp inputs
/// pass through borrowed untouched.
#[test]
fn normalize_cpu_rgb_expands_24bpp_and_borrows_4bpp() {
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3, 4, 5, 6], &mut scratch, false);
assert_eq!(f, PixelFormat::Rgbx);
assert_eq!(b, &[1, 2, 3, 0xFF, 4, 5, 6, 0xFF]);
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgr, &[9, 8, 7], &mut scratch, false);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[9, 8, 7, 0xFF]);
// Truncated tail: 5 bytes = one whole pixel + a 2-byte remainder that must be dropped.
let mut scratch = Vec::new();
let (_, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3, 4, 5], &mut scratch, false);
assert_eq!(b, &[1, 2, 3, 0xFF]);
// 4-bpp passthrough: borrowed, scratch untouched.
let src = [10u8, 20, 30, 40];
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgrx, &src, &mut scratch, false);
assert_eq!(f, PixelFormat::Bgrx);
assert!(std::ptr::eq(b.as_ptr(), src.as_ptr()));
assert!(scratch.is_empty());
// The 4-bpp mapping the expand lands on matches pixel_to_vk's existing table.
assert_eq!(
pixel_to_vk(PixelFormat::Rgbx),
Some(vk::Format::R8G8B8A8_UNORM)
);
assert_eq!(
pixel_to_vk(PixelFormat::Bgrx),
Some(vk::Format::B8G8R8A8_UNORM)
);
}
/// RGB-direct mode (`bgra_target = true`): everything lands in B,G,R,X order because the
/// video session's `pictureFormat` is `B8G8R8A8_UNORM` and the encode source must match it
/// (VUID-vkCmdEncodeVideoKHR-pEncodeInfo-08207 — caught live on RADV). B-first inputs pass
/// through borrowed; R-first inputs are channel-swapped, 3-bpp and 4-bpp alike.
#[test]
fn normalize_cpu_rgb_forces_bgra_for_the_encode_source() {
// Rgb (R,G,B) → B,G,R,X with the swap folded into the expand.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgb, &[1, 2, 3], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[3, 2, 1, 0xFF]);
// Bgr (B,G,R) → same order, expanded.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgr, &[9, 8, 7], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[9, 8, 7, 0xFF]);
// Rgbx: 4-bpp but R-first — swapped, alpha replaced by the 0xFF pad.
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Rgbx, &[1, 2, 3, 4], &mut scratch, true);
assert_eq!(f, PixelFormat::Bgrx);
assert_eq!(b, &[3, 2, 1, 0xFF]);
// Bgrx/Bgra already match the session order: borrowed untouched.
let src = [10u8, 20, 30, 40];
let mut scratch = Vec::new();
let (f, b) = normalize_cpu_rgb(PixelFormat::Bgra, &src, &mut scratch, true);
assert_eq!(f, PixelFormat::Bgra);
assert!(std::ptr::eq(b.as_ptr(), src.as_ptr()));
assert!(scratch.is_empty());
}
}
@@ -80,3 +80,74 @@ pub struct VideoEncodeSessionRgbConversionCreateInfoVALVE {
pub fn stype(raw: i32) -> vk::StructureType {
vk::StructureType::from_raw(raw)
}
// ---------- ABI layout guard ----------
//
// These structs are hand-copied from the registry and handed to the driver through raw `p_next`
// chains, so nothing in the type system relates them to the C definitions any more: an edit that
// inserts, drops, widens or re-pads a field is not a compile error, it is the driver reading our
// bytes at the wrong offsets. The assertions below are the missing compile error. They are `const`
// rather than `#[cfg(test)]` (the shape `amf.rs` uses) so they hold in every build, including the
// shipped one, and on any target this module compiles for.
//
// What they catch: a changed field width, an inserted or removed field, a changed array length, a
// padding assumption that only holds on one target. What they CANNOT catch: swapping two fields of
// the same type — offsets are unchanged. That case is only caught by reading the registry, so the
// field order here was diffed against `vulkan_core.h` (Vulkan-Headers `main`, 2026-07-25) when
// these assertions were written, along with every `ST_*` and flag-bit value above.
//
// Deliberately duplicated in `vk_av1_encode.rs` rather than shared: both modules exist to be
// deleted wholesale once `ash` ships these bindings, and a shared helper would make deleting one
// break the other.
macro_rules! assert_abi_layout {
($t:ty { size: $size:expr, align: $align:expr $(, $field:ident @ $off:expr)* $(,)? }) => {
const _: () = {
assert!(
::core::mem::size_of::<$t>() == $size,
concat!(stringify!($t), ": size does not match the C ABI")
);
assert!(
::core::mem::align_of::<$t>() == $align,
concat!(stringify!($t), ": alignment does not match the C ABI")
);
$(assert!(
::core::mem::offset_of!($t, $field) == $off,
concat!(stringify!($t), ".", stringify!($field), ": offset does not match the C ABI")
);)*
};
};
}
assert_abi_layout!(PhysicalDeviceVideoEncodeRgbConversionFeaturesVALVE {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
video_encode_rgb_conversion @ 16,
});
assert_abi_layout!(VideoEncodeRgbConversionCapabilitiesVALVE {
size: 32, align: 8,
s_type @ 0,
p_next @ 8,
rgb_models @ 16,
rgb_ranges @ 20,
x_chroma_offsets @ 24,
y_chroma_offsets @ 28,
});
assert_abi_layout!(VideoEncodeProfileRgbConversionInfoVALVE {
size: 24, align: 8,
s_type @ 0,
p_next @ 8,
perform_encode_rgb_conversion @ 16,
});
assert_abi_layout!(VideoEncodeSessionRgbConversionCreateInfoVALVE {
size: 32, align: 8,
s_type @ 0,
p_next @ 8,
rgb_model @ 16,
rgb_range @ 20,
x_chroma_offset @ 24,
y_chroma_offset @ 28,
});
File diff suppressed because it is too large Load Diff
+396 -3
View File
@@ -67,12 +67,385 @@ pub(super) fn resolve_subframe(default_on: bool) -> bool {
}
}
/// Resolved NVENC split-frame encode mode for a session — ONE selector shared by the Windows and
/// Linux direct-SDK backends (they had drifted into byte-identical duplicates, one of which
/// logged and one didn't). Precedence:
/// 1. `PUNKTFUNK_SPLIT_ENCODE` = `0`/`disable` | `1`/`auto` (AUTO_FORCED) | `2` | `3` — operator
/// override, always wins.
/// 2. 10-bit → DISABLE: 2-way split is measurably SLOWER on Ada for Main10 — at 5120×1440@240
/// forced-2 took 7.6 ms/frame (~131 fps) vs 2.8 ms (~357 fps) single-engine (the split/merge
/// overhead dominates), and a single engine handles 5K@240 Main10 well under budget. This was
/// the "broken animations in HDR" cap at ~131 fps.
/// 3. Pixel rate ≥ [`super::SPLIT_FORCE_PIXEL_RATE`] → force 2-way (AUTO never engages below
/// ~2112 px height, so 4K120 must be forced onto the second engine).
/// 4. Else AUTO (the ~2% BD-rate split cost isn't worth it at low pixel rates).
///
/// The caller still owns the rejection fallback (retry split-disabled) — a codec/config that
/// rejects the chosen mode downgrades at open, not here.
pub(super) fn resolve_split_mode(bit_depth: u8, pixel_rate: u64) -> u32 {
use nv::NV_ENC_SPLIT_ENCODE_MODE as M;
let mode = match std::env::var("PUNKTFUNK_SPLIT_ENCODE").ok().as_deref() {
Some("0") | Some("disable") => M::NV_ENC_SPLIT_DISABLE_MODE as u32,
Some("1") | Some("auto") => M::NV_ENC_SPLIT_AUTO_FORCED_MODE as u32,
Some("3") => M::NV_ENC_SPLIT_THREE_FORCED_MODE as u32,
Some("2") => M::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
_ if bit_depth >= 10 => M::NV_ENC_SPLIT_DISABLE_MODE as u32,
_ if pixel_rate >= super::SPLIT_FORCE_PIXEL_RATE => M::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
_ => M::NV_ENC_SPLIT_AUTO_MODE as u32,
};
tracing::debug!(
split_mode = mode,
bit_depth,
pixel_rate,
"NVENC split-encode mode selected"
);
mode
}
/// One session config's identity for the process-lifetime bitrate-ceiling cache
/// ([`cached_ceiling`]/[`store_ceiling`]). Everything the driver's codec-level validation keys
/// off: the GPU (different NVENC generations have different level ceilings), dims/fps (the luma
/// rate selects the level), depth/chroma (they select the profile) and the split mode the
/// sessions ACTUALLY opened with (a split session budgets per engine).
#[derive(Clone, Copy, PartialEq, Eq, Hash)]
pub(super) struct CeilingKey {
/// GPU identity — Linux: the process-global shared `CUcontext` pointer; Windows: the render
/// adapter LUID (0 when unresolved). Best effort: the cache is advisory (see
/// [`cached_ceiling`]), so a colliding identity costs one failed open + re-search, never a
/// wrong session.
pub gpu: u64,
pub codec: Codec,
pub width: u32,
pub height: u32,
pub fps: u32,
pub bit_depth: u8,
pub chroma_444: bool,
pub split_mode: u32,
}
fn ceilings() -> &'static std::sync::Mutex<std::collections::HashMap<CeilingKey, u64>> {
static CEILINGS: std::sync::OnceLock<
std::sync::Mutex<std::collections::HashMap<CeilingKey, u64>>,
> = std::sync::OnceLock::new();
CEILINGS.get_or_init(Default::default)
}
/// The codec-level bitrate ceiling (bps) a previous clamp search discovered for `key` this
/// process lifetime, if any. ADVISORY: the consumer must treat a failed open at the cached value
/// as a stale entry (fall back to the full search, which rewrites it via [`store_ceiling`]) —
/// that self-healing is what lets the key's GPU identity be best-effort. What this buys: an ABR
/// overshoot on a config whose ceiling is already known opens (or in-place reconfigures) straight
/// AT the ceiling instead of re-running the ~6-open binary search and its ~half-second of session
/// churn per rebuild.
pub(super) fn cached_ceiling(key: &CeilingKey) -> Option<u64> {
ceilings().lock().unwrap().get(key).copied()
}
/// Record the clamp search's discovered max accepted bitrate (bps) for `key`.
pub(super) fn store_ceiling(key: CeilingKey, bps: u64) {
ceilings().lock().unwrap().insert(key, bps);
}
#[cfg(test)]
mod tests {
use super::*;
use nv::NV_ENC_SPLIT_ENCODE_MODE as M;
// These assume PUNKTFUNK_SPLIT_ENCODE is unset (CI); an operator override deliberately wins.
/// `encodeCodecConfig` is a C union, so the HEVC 4:4:4 arm must be codec-gated or it stamps
/// `hevcConfig` bytes onto another codec's config. Before the gate this branch was reached on
/// ANY codec with `chroma_444 && full_chroma_input` and stayed non-UB only because `lib.rs`
/// degrades 4:4:4 for non-HEVC — a two-file invariant with nothing asserting it.
///
/// It also had to stop swallowing the per-codec bit-depth arm: this is an `if`/`else if`, so a
/// non-HEVC 4:4:4 session used to take the HEVC branch and get NEITHER 4:4:4 nor its own 10-bit
/// setup. AV1 asserts the depth it actually needs.
fn low_latency_cfg(codec: Codec, chroma_444: bool, bit_depth: u8) -> LowLatencyConfig {
LowLatencyConfig {
codec,
bitrate: 20_000_000,
fps: 60,
custom_vbv: false,
chroma_444,
full_chroma_input: true,
bit_depth,
av1_input_depth_minus8: 0,
hdr: false,
rfi_supported: false,
slices: 0,
}
}
#[test]
fn hevc_444_still_takes_the_frext_path() {
// `NV_ENC_CONFIG` must NOT be `mem::zeroed` — `frameFieldMode`/`mvPrecision` are C enums
// whose discriminants start at 1, so all-zero is not a valid value and Rust's own
// zero-init check aborts the process. Production seeds it the same way, from `Default`
// (then overwrites from the driver's preset).
// SAFETY: `apply_low_latency_config` only writes into the caller's config (union writes
// included) and makes no driver calls, so this is pure in-memory work.
let cfg = unsafe {
let mut cfg = nv::NV_ENC_CONFIG {
version: nv::NV_ENC_CONFIG_VER,
..Default::default()
};
apply_low_latency_config(&mut cfg, low_latency_cfg(Codec::H265, true, 10));
cfg
};
assert_eq!(cfg.profileGUID, nv::NV_ENC_HEVC_PROFILE_FREXT_GUID);
// SAFETY: an HEVC session's union arm is `hevcConfig` — the one this path wrote.
unsafe {
assert_eq!(cfg.encodeCodecConfig.hevcConfig.chromaFormatIDC(), 3);
assert_eq!(cfg.encodeCodecConfig.hevcConfig.pixelBitDepthMinus8(), 2);
}
}
#[test]
fn av1_never_takes_the_hevc_444_union_write() {
// SAFETY: as above — pure in-memory config authoring, no driver involvement.
let cfg = unsafe {
let mut cfg = nv::NV_ENC_CONFIG {
version: nv::NV_ENC_CONFIG_VER,
..Default::default()
};
apply_low_latency_config(&mut cfg, low_latency_cfg(Codec::Av1, true, 10));
cfg
};
// The HEVC FREXT profile GUID on an AV1 session is an INVALID_PARAM at open.
assert_ne!(
cfg.profileGUID,
nv::NV_ENC_HEVC_PROFILE_FREXT_GUID,
"4:4:4 on AV1 must not stamp the HEVC FREXT profile"
);
// ...and the AV1 arm must still have run, which the old if/else-if skipped entirely.
// SAFETY: an AV1 session's union arm is `av1Config`.
unsafe {
assert_eq!(
cfg.encodeCodecConfig.av1Config.pixelBitDepthMinus8(),
2,
"AV1 10-bit setup was swallowed by the HEVC 4:4:4 branch"
);
}
}
#[test]
fn split_forces_two_way_at_4k120() {
// The regression this threshold constant exists for: 3840×2160×120 = 995,328,000 sat
// 0.47% under the old `> 1_000_000_000` gate and stayed AUTO — pinned ~107 fps on a
// 4090 because AUTO never engages at 2160 px height.
let four_k_120 = 3840u64 * 2160 * 120;
assert_eq!(
resolve_split_mode(8, four_k_120),
M::NV_ENC_SPLIT_TWO_FORCED_MODE as u32
);
}
#[test]
fn split_leaves_1440p240_auto() {
// 884.7 Mpix/s is comfortably single-engine — the threshold move must not drag it in.
let qhd_240 = 2560u64 * 1440 * 240;
assert_eq!(
resolve_split_mode(8, qhd_240),
M::NV_ENC_SPLIT_AUTO_MODE as u32
);
}
#[test]
fn split_disabled_for_10bit_even_at_high_pixel_rate() {
// The measured Main10 rule: split/merge overhead dominates 10-bit on Ada (7.6 ms forced-2
// vs 2.8 ms single-engine at 5K240) — 10-bit precedes the pixel-rate arm.
let five_k_240 = 5120u64 * 1440 * 240;
assert_eq!(
resolve_split_mode(10, five_k_240),
M::NV_ENC_SPLIT_DISABLE_MODE as u32
);
}
#[test]
fn ceiling_cache_round_trips_and_keys_precisely() {
let key = CeilingKey {
gpu: 0xB0B0,
codec: Codec::H265,
width: 3840,
height: 2160,
fps: 120,
bit_depth: 8,
chroma_444: false,
split_mode: M::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
};
assert_eq!(cached_ceiling(&key), None);
store_ceiling(key, 794_000_000);
assert_eq!(cached_ceiling(&key), Some(794_000_000));
// Any config-identity change is a different ceiling — a miss, never a wrong clamp.
assert_eq!(cached_ceiling(&CeilingKey { fps: 60, ..key }), None);
assert_eq!(
cached_ceiling(&CeilingKey {
split_mode: M::NV_ENC_SPLIT_DISABLE_MODE as u32,
..key
}),
None
);
// A re-search overwrites (the advisory-cache stale-entry path).
store_ceiling(key, 620_000_000);
assert_eq!(cached_ceiling(&key), Some(620_000_000));
}
}
/// Reference-frame DPB depth when RFI is supported (Apollo uses 5). A deeper DPB lets an invalidated
/// reference fall back to an older still-valid frame instead of a full IDR; `numRefL0 = 1` keeps each
/// P-frame single-reference for low latency. Also the window the backends' `invalidate_ref_frames`
/// paths check against (`frame_idx - RFI_DPB` = the oldest frame still in the DPB).
/// P-frame single-reference for low latency. Also the window [`plan_range_recovery`] checks against
/// (`next_ts - RFI_DPB` = the oldest frame still in the DPB).
pub(super) const RFI_DPB: u32 = 5;
/// One loss event's recovery decision for the timestamp-range RFI both direct-NVENC backends run
/// (the range half of WP7.2's policy extraction; the slot half — AMF/QSV/Vulkan — is
/// `crate::rfi`). The mechanism (the per-timestamp `nvEncInvalidateRefFrames` loop, the
/// `last_rfi_range`/`pending_anchor` stores, the null-handle/`rfi_supported` gate) stays in each
/// backend.
pub(super) enum RangePlan {
/// The last successful invalidation already covers this range — no new driver calls, no IDR.
/// The caller must still RE-ARM its recovery anchor: the client re-asking means the previous
/// anchor AU may itself have been lost, and the next frame is just as clean a re-anchor.
Covered,
/// Invalidate `first..=last` (the CLAMPED range — this is also what the caller must record in
/// `last_rfi_range` on success, exactly as the inline code stored the post-clamp values).
Invalidate { first: i64, last: i64 },
/// Recovery without an IDR is impossible (nonsense range, loss older than the DPB, or a range
/// entirely in the future) — the caller returns `false` and its (coalesced) keyframe path
/// recovers. Deliberately NOT paired with any state clearing: neither twin touches
/// `pending_anchor` on decline (matching Vulkan's decline, opposite of AMF/QSV's
/// `pending_force` clear — see `crate::rfi`'s module doc before "harmonizing").
Decline,
}
/// The range-RFI policy, extracted verbatim from the two backends' `invalidate_ref_frames` (they
/// were hand-copied twins). Step order is load-bearing and pinned by tests:
///
/// 1. nonsense range (`first < 0 || first > last`) → [`RangePlan::Decline`];
/// 2. covering-range dedup — checked with the UNCLAMPED `last`, BEFORE the DPB window, so a
/// covered re-ask never touches the driver even when the range has since left the DPB;
/// 3. DPB window: `first < next_ts - RFI_DPB` → Decline (a lost frame older than the DPB cannot
/// be invalidated; the only correct recovery is an IDR);
/// 4. clamp `last` to `next_ts - 1` (never invalidate a timestamp never assigned); an inverted
/// range after the clamp (loss entirely in the future — a prediction desync) → Decline.
///
/// `next_ts` is the backend's `frame_idx`: the NEXT timestamp to assign, which `submit_indexed`
/// pins to the wire frame index — so the client's lost-frame range maps 1:1 onto the timestamps
/// the driver invalidates, across every rebuild/reset. Note `teardown()` clears `last_rfi_range`
/// but NOT `frame_idx`, so a post-reset call legitimately sees a stale-high `next_ts` with a
/// `None` range — the same view the inline code had.
pub(super) fn plan_range_recovery(
first: i64,
last: i64,
next_ts: i64,
last_rfi_range: Option<(i64, i64)>,
) -> RangePlan {
if first < 0 || first > last {
return RangePlan::Decline;
}
if let Some((pf, pl)) = last_rfi_range {
if first >= pf && last <= pl {
return RangePlan::Covered;
}
}
let oldest_in_dpb = next_ts - RFI_DPB as i64;
if first < oldest_in_dpb {
return RangePlan::Decline;
}
let last = last.min(next_ts - 1);
if first > last {
return RangePlan::Decline;
}
RangePlan::Invalidate { first, last }
}
#[cfg(test)]
mod range_policy_tests {
use super::{plan_range_recovery, RangePlan, RFI_DPB};
/// Convenience: the plan with no prior invalidation recorded.
fn plan(first: i64, last: i64, next_ts: i64) -> RangePlan {
plan_range_recovery(first, last, next_ts, None)
}
#[test]
fn nonsense_ranges_decline() {
assert!(matches!(plan(-1, 5, 100), RangePlan::Decline));
assert!(matches!(plan(7, 5, 100), RangePlan::Decline));
}
#[test]
fn covering_range_dedups_partial_overlap_does_not() {
let prior = Some((90i64, 95i64));
// Exact cover and sub-range → Covered. This pins EXISTING behavior, including that a
// covered range survives a forced IDR with zero driver calls (nothing clears
// `last_rfi_range` on a keyframe) — a recorded fact, not an endorsement.
assert!(matches!(
plan_range_recovery(90, 95, 100, prior),
RangePlan::Covered
));
assert!(matches!(
plan_range_recovery(92, 94, 100, prior),
RangePlan::Covered
));
// Partial overlap re-invalidates the FULL new range (next_ts = 98 keeps the window open:
// oldest_in_dpb = 93; at next_ts = 100 the same range would age out and Decline instead).
assert!(matches!(
plan_range_recovery(93, 97, 98, prior),
RangePlan::Invalidate {
first: 93,
last: 97
}
));
}
/// The covering check runs BEFORE the DPB window: a covered re-ask stays Covered (no driver
/// calls needed) even when the range has since aged out of the DPB.
#[test]
fn covered_is_checked_before_the_dpb_window() {
let prior = Some((10i64, 12i64));
assert!(matches!(
plan_range_recovery(10, 12, 100, prior),
RangePlan::Covered
));
// ...whereas the same range with no prior invalidation is outside the window → Decline.
assert!(matches!(plan(10, 12, 100), RangePlan::Decline));
}
#[test]
fn dpb_window_boundary() {
let next_ts = 100i64;
let oldest = next_ts - RFI_DPB as i64; // 95: the oldest timestamp still in the DPB
assert!(matches!(
plan(oldest, oldest, next_ts),
RangePlan::Invalidate { .. }
));
assert!(matches!(
plan(oldest - 1, oldest, next_ts),
RangePlan::Decline
));
}
/// `last` clamps to `next_ts - 1` (the newest encoded frame); the Invalidate carries the
/// CLAMPED value — which is also what the caller records in `last_rfi_range`.
#[test]
fn clamps_to_newest_encoded() {
assert!(matches!(
plan(98, 150, 100),
RangePlan::Invalidate {
first: 98,
last: 99
}
));
// A range entirely in the future inverts under the clamp → Decline (prediction desync).
assert!(matches!(plan(100, 150, 100), RangePlan::Decline));
// Fresh session (`frame_idx == 0`): window passes (oldest = -5) but the clamp gives
// last = -1 < first → Decline. The inline code behaved identically.
assert!(matches!(plan(0, 3, 0), RangePlan::Decline));
}
}
/// The per-session knobs both direct-NVENC backends feed [`apply_low_latency_config`]. `Copy` so the
/// backend fills it from `self` at the call. The two input-format fields bridge the only real
/// divergence between the CUDA and D3D11 paths (which surface formats can carry full chroma / 10-bit
@@ -219,7 +592,27 @@ pub(super) unsafe fn apply_low_latency_config(cfg: &mut nv::NV_ENC_CONFIG, c: Lo
// profile) takes precedence and composes with 10-bit (Main 4:4:4 10); it needs a full-chroma-
// capable input. Otherwise 10-bit selects Main10 (HEVC) or the AV1 output depth — stamping the
// HEVC Main10 GUID onto an AV1 session is an INVALID_PARAM, so bit depth is set PER CODEC.
if c.chroma_444 && c.full_chroma_input {
// `encodeCodecConfig` is a C UNION, so the `hevcConfig` writes below are only meaningful on an
// HEVC session — on an H.264 or AV1 one they reinterpret that codec's own config bytes. The
// codec test is therefore load-bearing, not defensive: without it this branch was gated purely
// on `chroma_444 && full_chroma_input` and stayed non-UB only because `lib.rs` degrades 4:4:4
// for non-HEVC codecs. That was a two-file invariant with nothing asserting it, on the path
// BOTH direct-NVENC backends take.
//
// Being a codec test also fixes a second, quieter bug in the same shape: this is an
// `if`/`else if`, so a non-HEVC session that somehow arrived with `chroma_444` set took this
// branch and skipped the per-codec bit-depth arm entirely — ending up with neither HEVC 4:4:4
// (wrong for it) nor its own 10-bit configuration (simply missing). Non-HEVC now falls through
// to the arm that knows what to do with it.
let want_444 = c.chroma_444 && c.full_chroma_input;
if want_444 && c.codec != Codec::H265 {
tracing::warn!(
codec = ?c.codec,
"4:4:4 requested on a non-HEVC NVENC session — ignoring it (Range Extensions are \
HEVC-only); the negotiator should have degraded this to 4:2:0 before the open"
);
}
if want_444 && c.codec == Codec::H265 {
cfg.profileGUID = nv::NV_ENC_HEVC_PROFILE_FREXT_GUID;
cfg.encodeCodecConfig.hevcConfig.set_chromaFormatIDC(3);
if c.bit_depth == 10 {
+201 -16
View File
@@ -8,27 +8,77 @@
//! means and what the operator should do, and folds that cause into the `anyhow::Error` at
//! construction, so every downstream `{e:#}` log (the encode-recovery loop, session teardown) says
//! the useful thing without extra plumbing.
//!
//! One status needs process state to explain honestly: the driver reports BOTH "your headers are
//! newer than my kernel module" and "I can no longer hand this process a session" as
//! `NV_ENC_ERR_INVALID_VERSION`. [`note_session_opened`] latches the fact that a session already
//! opened here, which tells the two apart — see [`explain`].
use std::sync::atomic::{AtomicBool, Ordering};
use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv;
/// Latched the first time `nvEncOpenEncodeSessionEx` succeeds in this process (the caps probe, a
/// real session open, or the Windows availability probe — every one of them completes the
/// userspace↔kernel-module handshake).
///
/// The load-time gate (`NvEncodeAPIGetMaxSupportedVersion`, both backends' `load_api`) can NOT
/// serve this purpose: it is a pure userspace query, so the genuine "updated the driver, didn't
/// reboot" skew sails through it and fails later, at the open. Only a session that actually opened
/// proves the kernel module agreed.
static SESSION_OPENED: AtomicBool = AtomicBool::new(false);
/// Record that an NVENC session opened. Call right after every successful
/// `open_encode_session_ex`, so [`explain`] can rule a version skew out for the rest of the
/// process.
pub(super) fn note_session_opened() {
SESSION_OPENED.store(true, Ordering::Relaxed);
}
/// The two very different failures the driver reports as `NV_ENC_ERR_INVALID_VERSION`, split on
/// whether a session has already opened here (`session_opened`). Pure, so both halves are testable
/// without touching the process-wide latch.
fn invalid_version(session_opened: bool) -> String {
if session_opened {
// Same status, opposite cause: a session ALREADY opened in this process, so the driver's
// kernel module accepted this exact build's version word minutes ago. A version skew is
// static — it cannot come and go — so "update the driver / reboot" is the wrong advice
// here, and following it costs the operator a reboot per stream (2026-07 field report: one
// stream works, the next fails at the caps probe, forever, until the PROCESS restarts).
// What is left is per-process driver state: a resource the last session did not give back,
// or a wedged/lost device. Say that, and point at the cheap fix.
// Worded for ANY call (`explain` also serves `lock_bitstream`); `call_err` already names
// the entry point ahead of this text, so it must not assume the session open.
return "this process already opened an NVENC session successfully, so this is NOT a driver \
version mismatch that cannot come and go within a process, and a reboot is not \
the fix. The NVIDIA driver state in THIS process is exhausted or wedged: restart \
the Punktfunk host service to clear it, and please report this with the host log \
so it can be fixed properly"
.to_string();
}
// No session has ever opened here, so the version word really is in question. Either the
// driver is genuinely older than our headers, or (the sneaky case) the userspace
// `libnvidia-encode` reports a new-enough version to the pre-flight probe but the running
// kernel module is older and rejects the session — the classic "updated the driver, didn't
// reboot" skew. Both heal the same way.
format!(
"the NVIDIA driver is older than this build's NVENC headers (needs NVENC API {}.{} or \
newer), or the userspace and kernel-module driver versions are mismatched common right \
after a driver update without a reboot. Update the NVIDIA driver, or reboot if you just \
updated it (a host restart is the usual fix).",
nv::NVENCAPI_MAJOR_VERSION,
nv::NVENCAPI_MINOR_VERSION,
)
}
/// A one-line, operator-actionable cause for an NVENC status. Does not repeat the raw code —
/// callers print that alongside (see [`call_err`]). Public for the few sites that build a
/// `String`/`format!` error instead of an `anyhow::Error`.
pub(super) fn explain(status: nv::NVENCSTATUS) -> String {
match status {
// The one this whole module exists for: a version word the driver rejects. Either the
// driver is genuinely older than our headers, or (the sneaky case) the userspace
// `libnvidia-encode` reports a new-enough version to the pre-flight probe but the running
// kernel module is older and rejects the session — the classic "updated the driver, didn't
// reboot" skew. Both heal the same way.
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_VERSION => format!(
"the NVIDIA driver is older than this build's NVENC headers (needs NVENC API {}.{} or \
newer), or the userspace and kernel-module driver versions are mismatched common \
right after a driver update without a reboot. Update the NVIDIA driver, or reboot if \
you just updated it (a host restart is the usual fix).",
nv::NVENCAPI_MAJOR_VERSION,
nv::NVENCAPI_MINOR_VERSION,
),
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_VERSION => {
invalid_version(SESSION_OPENED.load(Ordering::Relaxed))
}
nv::NVENCSTATUS::NV_ENC_ERR_NO_ENCODE_DEVICE => {
"this GPU exposes no usable NVENC engine — it has no hardware video encoder, or NVENC is \
disabled on this card"
@@ -72,9 +122,144 @@ pub(super) fn explain(status: nv::NVENCSTATUS) -> String {
}
}
/// Typed root of a failed NVENC entry-point call: carries the raw status so callers can classify
/// the failure class, not just print it — the bitrate-clamp search must only read a
/// parameter/caps rejection as "above the codec-level ceiling"; a transient failure shrinking the
/// search would discover (and cache) a bogus ceiling. Recover it through an `anyhow` chain with
/// `err.downcast_ref::<NvCallError>()` (see [`is_param_rejection`]).
#[derive(Debug)]
pub(super) struct NvCallError(pub(super) nv::NVENCSTATUS);
impl std::fmt::Display for NvCallError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{:?} — {}", self.0, explain(self.0))
}
}
impl std::error::Error for NvCallError {}
/// Whether `err` is an NVENC parameter/capability rejection: the driver understood the request
/// and says THIS config is not encodable — the clamp search's "bitrate above the ceiling"
/// evidence. Everything else (busy engine, session limit, OOM, device loss, version skew) is
/// environmental and must propagate instead of steering the search.
pub(super) fn is_param_rejection(err: &anyhow::Error) -> bool {
matches!(
err.downcast_ref::<NvCallError>(),
Some(NvCallError(
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_PARAM
| nv::NVENCSTATUS::NV_ENC_ERR_UNSUPPORTED_PARAM
| nv::NVENCSTATUS::NV_ENC_ERR_UNIMPLEMENTED,
))
)
}
/// Build an actionable `anyhow::Error` for a failed NVENC entry-point call. `call` names the API
/// (e.g. `"open_encode_session_ex"`); the message carries both the raw status and its real-world
/// cause, so triage never again reads a version mismatch as "(no NVIDIA GPU?)".
/// (e.g. `"open_encode_session_ex"`); the chain carries both the raw status and its real-world
/// cause, so triage never again reads a version mismatch as "(no NVIDIA GPU?)". The
/// [`NvCallError`] root keeps the status downcastable for failure-class checks.
pub(super) fn call_err(call: &str, status: nv::NVENCSTATUS) -> anyhow::Error {
anyhow::anyhow!("NVENC {call} failed: {status:?} — {}", explain(status))
anyhow::Error::new(NvCallError(status)).context(format!("NVENC {call} failed"))
}
/// Whether a FAILED `nvEncDestroyEncoder` status PROVES the driver holds no session for the
/// handle — i.e. the per-process concurrent-session slot is not consumed, so the session's budget
/// units can be refunded immediately. These are the statuses the driver returns when the session
/// or its device no longer exists on its side (a TDR/device removal reclaims every session with
/// the context). Everything else — `GENERIC`, `ENCODER_BUSY`, OOM, ... — is AMBIGUOUS: the slot
/// may genuinely still be held, so the caller must park the handle fail-closed (units stay
/// charged) and let a later retry-destroy produce the proof. Splitting on proof is what keeps the
/// session budget from drifting low on failures (over-admitting parallel displays) WITHOUT letting
/// one transient wedge episode permanently poison admission until a host restart.
///
/// Used by the Windows D3D11 backend's teardown accounting; the Linux CUDA backend has no session
/// budget (parallel-display admission is a Windows feature), so there this exists for the unit
/// tests only.
#[cfg_attr(not(target_os = "windows"), allow(dead_code))]
pub(super) fn destroy_proves_no_session(status: nv::NVENCSTATUS) -> bool {
matches!(
status,
nv::NVENCSTATUS::NV_ENC_ERR_DEVICE_NOT_EXIST
| nv::NVENCSTATUS::NV_ENC_ERR_INVALID_ENCODERDEVICE
| nv::NVENCSTATUS::NV_ENC_ERR_INVALID_PTR
| nv::NVENCSTATUS::NV_ENC_ERR_ENCODER_NOT_INITIALIZED
)
}
#[cfg(test)]
mod tests {
use super::*;
/// Before any session has opened, the version word IS in question — keep the skew advice.
#[test]
fn invalid_version_before_any_session_blames_the_driver_version() {
let msg = invalid_version(false);
assert!(
msg.contains("older than this build's NVENC headers"),
"{msg}"
);
assert!(msg.contains("reboot if you just updated it"), "{msg}");
}
/// Once a session has opened here, a skew is impossible — the message must stop sending
/// operators to reboot (2026-07 field report: one stream per boot, forever).
#[test]
fn invalid_version_after_a_session_blames_process_state_not_the_driver() {
let msg = invalid_version(true);
assert!(msg.contains("NOT a driver version mismatch"), "{msg}");
assert!(msg.contains("restart the Punktfunk host service"), "{msg}");
assert!(
!msg.contains("older than this build's NVENC headers"),
"must not repeat the version-skew advice: {msg}"
);
assert!(
!msg.contains("Update the NVIDIA driver"),
"must not tell the operator to update a driver that just worked: {msg}"
);
}
/// The latch is one-way and only touches this status.
#[test]
fn note_session_opened_latches() {
note_session_opened();
assert!(SESSION_OPENED.load(Ordering::Relaxed));
note_session_opened();
assert!(SESSION_OPENED.load(Ordering::Relaxed));
assert_eq!(
explain(nv::NVENCSTATUS::NV_ENC_ERR_OUT_OF_MEMORY),
"the GPU is out of memory"
);
}
/// Destroy-failure classification: session-gone statuses refund; everything ambiguous parks.
/// The split is the load-bearing part of the session-budget accounting — a wrong `true`
/// under-counts (over-admits parallel displays), a wrong `false` merely defers the refund to
/// a reap retry.
#[test]
fn destroy_classification_refunds_only_on_proof() {
for gone in [
nv::NVENCSTATUS::NV_ENC_ERR_DEVICE_NOT_EXIST,
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_ENCODERDEVICE,
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_PTR,
nv::NVENCSTATUS::NV_ENC_ERR_ENCODER_NOT_INITIALIZED,
] {
assert!(
destroy_proves_no_session(gone),
"{gone:?} proves no session"
);
}
for ambiguous in [
nv::NVENCSTATUS::NV_ENC_ERR_GENERIC,
nv::NVENCSTATUS::NV_ENC_ERR_ENCODER_BUSY,
nv::NVENCSTATUS::NV_ENC_ERR_OUT_OF_MEMORY,
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_PARAM,
// INVALID_DEVICE sounds like the gone class but is also what a transiently-confused
// driver returns — deliberately fail-closed (park + retry), not refunded.
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_DEVICE,
] {
assert!(
!destroy_proves_no_session(ambiguous),
"{ambiguous:?} must park fail-closed"
);
}
}
}
+11
View File
@@ -246,6 +246,17 @@ mod tests {
assert!(block_count_32x32(3840, 2160, true) <= u16::MAX as u32);
assert!(block_count_32x32(7680, 4320, true) > u16::MAX as u32);
assert!(block_count_32x32(7680, 4320, false) <= u16::MAX as u32);
// …and 4:2:0 wraps it too, just later — the hole the old 4:4:4-only open guard left.
// `Codec::max_dimension()` allows PyroWave 8192px per axis, so these modes were
// reachable from a client-requested `mode=WxHxFPS`, and the negotiator's 4:4:4 → 4:2:0
// downgrade routed oversized modes straight into the unguarded branch.
// `validate_dimensions` now rejects them against this 4:2:0 count.
assert_eq!(block_count_32x32(8192, 6144, false), 73728);
assert_eq!(block_count_32x32(8192, 8192, false), 98304);
assert!(block_count_32x32(8192, 6144, false) > u16::MAX as u32);
assert!(block_count_32x32(8192, 8192, false) > u16::MAX as u32);
// The largest 4:2:0 mode that still fits, for the boundary the validator enforces.
assert!(block_count_32x32(7680, 4320, false) <= u16::MAX as u32);
}
#[test]
+178
View File
@@ -0,0 +1,178 @@
//! The slot-family RFI (reference-frame invalidation) recovery **policy**, shared by the three
//! backends that answer a loss with a re-reference to a known-good older frame instead of an IDR:
//! native AMF (user-LTR bitfield), native QSV (`mfxExtRefListCtrl` LTR) and Vulkan Video (the
//! app-owned DPB slot table). One decision, three mechanisms — the policy lived as three
//! hand-copies and had already diverged once (the fecbec2d taint sweep reached AMF/QSV a commit
//! before the Vulkan backend was carved out, and Vulkan shipped without it until a later fix).
//! The NVENC twins' *range* policy is the other half of WP7.2, in
//! [`super::nvenc_core::plan_range_recovery`].
//!
//! Policy only. Every mechanism — how a force is applied, how distrust is *persisted* (AMF clears
//! its mirror slot to `None`, QSV sets a separate `ltr_tainted` flag because its mirror must keep
//! naming the slot for the RejectedRefList, Vulkan blanks `slot_wire` to `-1` while `slot_poc`
//! keeps the picture resident for the RPS) — stays in its backend. Callers feed their
//! **currently-trusted** references and apply the returned taints through their own marker; that
//! caller-side filter is exactly what makes the three persistence schemes equivalent under one
//! pure function.
//!
//! The decline arm is also the backend's: AMF/QSV clear an un-consumed `pending_force` (the sweep
//! may have emptied the slot it points at), while Vulkan deliberately leaves `pending_loss` armed
//! (a stale arm is re-resolved at frame-build, where a failed re-pick forces the IDR that heals
//! the stream). Do not harmonize them here.
/// One loss event's recovery decision over a slot table: which trusted references become
/// untrustworthy, and which one anchors the recovery.
pub(super) struct SlotPlan {
/// Bitmask of slots whose reference was encoded **at or after** the loss start — inside the
/// client's corrupt window, so the client either never received it or decoded it against a
/// broken chain. Serving one as "known-good" on a LATER loss ships corruption tagged as the
/// recovery anchor; the backend must record the distrust in its own persistent marker, because
/// these wires would otherwise look like valid pre-loss anchors to the next loss event.
pub(super) tainted: u32,
/// The recovery anchor: the newest trusted reference **strictly older** than the loss —
/// `(slot, wire)` — i.e. the most recent picture the client still holds intact, so
/// re-referencing it costs the smallest residual. `None`: every candidate is inside or after
/// the corrupt window — the caller declines and its (coalesced) keyframe path recovers.
pub(super) anchor: Option<(usize, i64)>,
}
/// Plan the recovery for a loss starting at wire frame `loss_first`, over the backend's
/// currently-trusted references (`(slot, wire)`; previously-distrusted entries must already be
/// filtered out by the caller — see the module doc). Sweep and pick are one call so the anchor is
/// chosen from the same snapshot the taints are computed from, by construction: the anchor
/// delegates to [`pick_anchor`], and `wire >= loss_first` (taint) and `wire < loss_first`
/// (anchor) are disjoint, so a slot tainted by this call can never be this call's anchor.
pub(super) fn plan_slot_recovery(refs: &[(usize, i64)], loss_first: i64) -> SlotPlan {
// The callers' validity gate (`first < 0 → decline`) is what makes their sentinel filters
// (-1 / `None`) exact views of "trusted"; this assert keeps the contract visible from inside
// the extracted code. Plain assert: the lint legs run --release, and a compiled-out check
// here would silently drop taints instead of failing loudly.
assert!(
loss_first >= 0,
"loss_first must be validity-gated by the caller"
);
let mut tainted = 0u32;
for &(slot, wire) in refs {
if wire >= loss_first {
assert!(slot < 32, "slot table exceeds the u32 taint mask");
tainted |= 1 << slot;
}
}
SlotPlan {
tainted,
anchor: pick_anchor(refs, loss_first),
}
}
/// The pick half alone: newest trusted reference strictly older than the loss. Ties break to the
/// first entry in `refs` (every caller feeds ascending slot order, so the lowest slot wins —
/// the strict `>` all three backends used). Standalone because Vulkan re-picks at frame-build
/// time: its arm carries the loss start, not the slot, so the slot is resolved against the table
/// as it stands when the recovery frame is actually encoded.
pub(super) fn pick_anchor(refs: &[(usize, i64)], loss_first: i64) -> Option<(usize, i64)> {
let mut best: Option<(usize, i64)> = None;
for &(slot, wire) in refs {
if wire < loss_first && best.is_none_or(|(_, b)| wire > b) {
best = Some((slot, wire));
}
}
best
}
#[cfg(test)]
mod tests {
use super::{pick_anchor, plan_slot_recovery};
/// Adapt a raw slot table (the Vulkan `slot_wire` shape: `-1` = empty) into the trusted view
/// the policy takes — the same filter the backend adapters apply.
fn view(wires: &[i64]) -> Vec<(usize, i64)> {
wires
.iter()
.enumerate()
.filter_map(|(s, &w)| (w >= 0).then_some((s, w)))
.collect()
}
/// Apply a plan's taints the way the Vulkan adapter does (blank the wire) — the persistence
/// half the pure fn hands back to the caller.
fn apply(wires: &mut [i64], tainted: u32) {
for (s, w) in wires.iter_mut().enumerate() {
if tainted & (1 << s) != 0 {
*w = -1;
}
}
}
/// The RFI anchor picker: newest resident wire strictly older than the loss; empty/newer
/// slots never qualify. (Migrated 1:1 from `vulkan_video.rs`'s `pick_recovery_slot` tests —
/// same vectors, now `(slot, wire)`-valued.)
#[test]
fn picks_newest_pre_loss() {
// slots hold wires 5..12 (ring position arbitrary); loss starts at 9 → anchor = wire 8.
let wires = [8i64, 9, 10, 11, 12, 5, 6, 7];
assert_eq!(pick_anchor(&view(&wires), 9), Some((0, 8)));
// loss older than everything resident → no anchor (caller keyframes).
assert_eq!(pick_anchor(&view(&wires), 5), None);
// empty slots (-1) are skipped by the view filter and never anchor.
assert_eq!(pick_anchor(&view(&[-1, 3, -1, 4]), 5), Some((3, 4)));
assert_eq!(pick_anchor(&view(&[-1; 8]), 5), None);
// wire == loss_first is INSIDE the corrupt window: strictly-older only.
assert_eq!(pick_anchor(&view(&[9, 8]), 9), Some((1, 8)));
// tie on the wire → first entry (= lowest slot) wins, the strict `>` all three backends
// used.
assert_eq!(pick_anchor(&[(2, 7), (5, 7)], 9), Some((2, 7)));
// empty view.
assert_eq!(pick_anchor(&[], 9), None);
}
/// The taint sweep (fecbec2d's fix): a slot encoded inside an EARLIER, still unrepaired loss
/// window must not become the "known-good" anchor of a LATER loss. Without persisted
/// distrust the picker accepts it — it is resident and its wire is below the second loss
/// start — and the frame ships tagged `recovery_anchor`, lifting the client's freeze onto a
/// reference it never decoded. (Migrated 1:1 from `vulkan_video.rs`; the hand-replicated
/// sweep is now `plan_slot_recovery` itself.)
#[test]
fn taint_sweep_excludes_slots_from_an_earlier_loss() {
// Slots hold wires 0..7. Loss 1 starts at wire 4, so wires 4..7 are undecodable at the
// client. A second loss report arrives at wire 6 while they are all still resident.
let tainted_wires = [4i64, 5, 6, 7];
// WITHOUT the sweep this is the bug: the newest wire below 6 is wire 5 — squarely inside
// loss 1's unrepaired window — and it would be served as the "known-good" anchor.
let unswept = [0i64, 1, 2, 3, 4, 5, 6, 7];
let (_, picked_wire) = pick_anchor(&view(&unswept), 6).expect("unswept picks something");
assert!(
tainted_wires.contains(&picked_wire),
"precondition: without the sweep the anchor comes from the earlier loss window"
);
// WITH the plan, loss 1 taints 4..7 (and anchors on wire 3 — never a tainted wire, by
// the disjoint predicates), so loss 2 can only reach genuinely clean wires.
let mut wires = unswept;
let plan = plan_slot_recovery(&view(&wires), 4);
assert_eq!(plan.tainted, 0b1111_0000);
assert_eq!(plan.anchor, Some((3, 3)));
apply(&mut wires, plan.tainted);
assert_eq!(wires, [0, 1, 2, 3, -1, -1, -1, -1]);
let (slot, wire) = pick_anchor(&view(&wires), 6).expect("clean wires remain");
assert_eq!((slot, wire), (3, 3), "newest clean survivor is wire 3");
// Encoding resumes after recovery; wires 8..11 refill the swept slots and are clean. A
// later loss at wire 10 legitimately anchors on wire 9 — the sweep must not over-reject.
wires[4] = 8;
wires[5] = 9;
wires[6] = 10;
wires[7] = 11;
let plan = plan_slot_recovery(&view(&wires), 10);
assert_eq!(plan.anchor, Some((5, 9)), "wire 9 is post-recovery, clean");
apply(&mut wires, plan.tainted);
// A loss covering every live wire leaves nothing clean → decline, caller serves an IDR.
let mut all = [5i64, 6, 7, 8, 9, 10, 11, 12];
let plan = plan_slot_recovery(&view(&all), 5);
assert_eq!(plan.tainted, 0b1111_1111);
assert_eq!(plan.anchor, None);
apply(&mut all, plan.tainted);
assert_eq!(pick_anchor(&view(&all), 5), None);
}
}
+64 -1
View File
@@ -51,6 +51,24 @@ pub struct OpenH264Encoder {
// whole value to that thread is therefore sound — there is no concurrent access to the handle.
unsafe impl Send for OpenH264Encoder {}
/// openh264's own ceiling: level 5.2, so 3840x2160 landscape or 2160x3840 portrait.
///
/// The long edge may reach 3840 and the short edge 2160 — the rule is orientation-aware, not a
/// per-axis `w <= 3840 && h <= 2160`, so a portrait 2160x3840 session is legal.
const OPENH264_MAX_LONG_EDGE: u32 = 3840;
const OPENH264_MAX_SHORT_EDGE: u32 = 2160;
/// Whether the bundled openh264 can encode this resolution at all.
///
/// Mirrors the check inside the crate we ship (openh264 0.9.3, `encoder.rs` `reinit`). That check
/// runs on the FIRST ENCODE, not at encoder construction — so without this gate a too-large mode
/// opens perfectly and then fails *every* submit, and the session connects and never delivers a
/// frame. `Codec::max_dimension` does not cover it: it is keyed on the codec, and H.264 legitimately
/// reaches 4096 on every hardware backend — this ceiling belongs to the software backend alone.
fn openh264_supports_dimensions(width: u32, height: u32) -> bool {
width.max(height) <= OPENH264_MAX_LONG_EDGE && width.min(height) <= OPENH264_MAX_SHORT_EDGE
}
impl OpenH264Encoder {
pub fn open(
format: PixelFormat,
@@ -59,7 +77,16 @@ impl OpenH264Encoder {
fps: u32,
bitrate_bps: u64,
) -> Result<Self> {
// validate_dimensions() ran in open_video: even, non-zero, <= 4096.
// validate_dimensions() ran in open_video: even, non-zero, <= 4096. That leaves modes this
// encoder cannot serve (e.g. a legal 4096-wide H.264 mode), so refuse them here — at the
// open, where the caller still gets a real error — rather than at every submit.
ensure!(
openh264_supports_dimensions(width, height),
"openh264 cannot encode {width}x{height}: the software encoder tops out at \
{OPENH264_MAX_LONG_EDGE}x{OPENH264_MAX_SHORT_EDGE} (or \
{OPENH264_MAX_SHORT_EDGE}x{OPENH264_MAX_LONG_EDGE} portrait) lower the client \
resolution, or use a host with a hardware encoder"
);
let bps: u32 = bitrate_bps.try_into().unwrap_or(u32::MAX);
let cfg = EncoderConfig::new()
.usage_type(UsageType::ScreenContentRealTime)
@@ -336,4 +363,40 @@ mod tests {
.any(|w| w[0] == 0 && w[1] == 0 && w[2] == 0 && w[3] == 1 && (w[4] & 0x1f) == 7);
assert!(has_sps, "IDR must carry an SPS NAL (type 7)");
}
/// The modes the software encoder can actually serve — including the portrait orientation,
/// which a naive per-axis `w <= 3840 && h <= 2160` would wrongly reject.
#[test]
fn openh264_accepts_up_to_4k_in_either_orientation() {
assert!(openh264_supports_dimensions(1920, 1080));
assert!(openh264_supports_dimensions(3840, 2160));
assert!(openh264_supports_dimensions(2160, 3840));
assert!(openh264_supports_dimensions(1080, 1920));
}
/// Modes `validate_dimensions` lets through (H.264 legitimately reaches 4096 on hardware) but
/// openh264 rejects on the first encode. Catching them at open is the whole point of the gate:
/// otherwise the session opens and then fails every single submit.
#[test]
fn openh264_rejects_modes_that_would_fail_on_first_submit() {
// 4096-wide is legal H.264 and passes `Codec::max_dimension`, but exceeds the long edge.
assert!(!openh264_supports_dimensions(4096, 2160));
assert!(!openh264_supports_dimensions(2160, 4096));
// Long edge is fine, short edge is not (e.g. an ultrawide-tall composite desktop).
assert!(!openh264_supports_dimensions(3840, 2400));
assert!(!openh264_supports_dimensions(2400, 3840));
}
/// A too-large mode must fail at `open`, not silently at every `submit`.
#[test]
fn open_refuses_a_mode_openh264_cannot_encode() {
// Matched rather than `expect_err`: `OpenH264Encoder` is not `Debug` (it wraps a raw C
// handle), so `expect_err` would not compile.
let err = match OpenH264Encoder::open(PixelFormat::Bgra, 4096, 2160, 60, 20_000_000) {
Ok(_) => panic!("4096x2160 exceeds openh264's long-edge ceiling and must be refused"),
Err(e) => e,
};
let msg = format!("{err:#}");
assert!(msg.contains("openh264 cannot encode 4096x2160"), "{msg}");
}
}
+36 -442
View File
@@ -66,425 +66,11 @@ use windows::Win32::Storage::FileSystem::{
};
use windows::Win32::System::LibraryLoader::GetModuleFileNameW;
// ---------------------------------------------------------------------------------------------
// Mirrored AMF C ABI (written against GPUOpen header release v1.4.36 — amf/public/include; every
// slot below is a base-interface slot whose layout is stable since <= v1.4.34, the loader's
// accepted ABI floor, so the mirror is valid on every runtime the loader admits).
//
// Layout rules this mirror relies on: every AMF interface is a struct whose sole member is a
// pointer to a C vtable; derived interfaces PREPEND their base's slots in order (AMFInterface →
// AMFPropertyStorage → AMFData → AMFBuffer/AMFSurface), so a derived pointer is usable through a
// base vtable mirror. Slots we never call are declared as bare `*const c_void` placeholders —
// same size/alignment as the function pointer they stand in for. `AMF_STD_CALL` is `__stdcall`
// (Rust `extern "system"`); the two DLL entry points are `__cdecl` (`extern "C"`); on x86_64
// both collapse to the one Windows calling convention.
// ---------------------------------------------------------------------------------------------
mod sys {
use std::ffi::c_void;
/// `AMF_RESULT` (core/Result.h) — a plain C enum, sequential from 0. Only the codes this
/// module branches on are named; everything else is reported numerically via [`result_name`].
pub type AmfResult = i32;
pub const AMF_OK: AmfResult = 0;
pub const AMF_EOF: AmfResult = 23;
pub const AMF_REPEAT: AmfResult = 24;
pub const AMF_INPUT_FULL: AmfResult = 25;
pub const AMF_NEED_MORE_INPUT: AmfResult = 44;
/// Human-readable name for an `AMF_RESULT` (diagnostics only — the numeric value rides along
/// so an unnamed code is still identifiable against Result.h).
pub fn result_name(r: AmfResult) -> &'static str {
match r {
0 => "AMF_OK",
1 => "AMF_FAIL",
2 => "AMF_UNEXPECTED",
3 => "AMF_ACCESS_DENIED",
4 => "AMF_INVALID_ARG",
5 => "AMF_OUT_OF_RANGE",
6 => "AMF_OUT_OF_MEMORY",
7 => "AMF_INVALID_POINTER",
8 => "AMF_NO_INTERFACE",
9 => "AMF_NOT_IMPLEMENTED",
10 => "AMF_NOT_SUPPORTED",
11 => "AMF_NOT_FOUND",
12 => "AMF_ALREADY_INITIALIZED",
13 => "AMF_NOT_INITIALIZED",
14 => "AMF_INVALID_FORMAT",
15 => "AMF_WRONG_STATE",
17 => "AMF_NO_DEVICE",
18 => "AMF_DIRECTX_FAILED",
23 => "AMF_EOF",
24 => "AMF_REPEAT",
25 => "AMF_INPUT_FULL",
26 => "AMF_RESOLUTION_CHANGED",
28 => "AMF_INVALID_DATA_TYPE",
29 => "AMF_INVALID_RESOLUTION",
30 => "AMF_CODEC_NOT_SUPPORTED",
31 => "AMF_SURFACE_FORMAT_NOT_SUPPORTED",
32 => "AMF_SURFACE_MUST_BE_SHARED",
36 => "AMF_ENCODER_NOT_PRESENT",
44 => "AMF_NEED_MORE_INPUT",
_ => "AMF_<unnamed>",
}
}
/// The AMF header release this FFI mirror was written against: `AMF_FULL_VERSION` for 1.4.36.0
/// (core/Version.h `AMF_MAKE_FULL_VERSION`). This is the version claimed to `AMFInit` — but
/// capped at the runtime's own reported version (see `load_factory`), so an older-but-accepted
/// runtime is asked only for the ABI it actually provides.
pub const AMF_HEADER_VERSION: u64 = (1u64 << 48) | (4u64 << 32) | (36u64 << 16);
/// The oldest AMF runtime the loader accepts (`AMF_FULL_VERSION` 1.4.34.0 — AMD Adrenalin
/// 24.6.1). This is an **ABI floor, not a feature floor**: every vtable slot mirrored in this
/// module belongs to a base interface (`AMFFactory`/`AMFContext`/`AMFComponent`/`AMFData`/
/// `AMFBuffer`) whose layout has been stable — append-only, no mid-vtable insertions — since
/// well before 1.4.34, so a 1.4.34 runtime is guaranteed to expose every mirrored slot at its
/// mirrored offset. Everything 1.4.35/1.4.36 added that this path can touch (new HQ presets,
/// AV1 B-frame / picture management) is a *string-keyed encoder property*, applied through
/// [`set_prop`] with `required=false` — a runtime that lacks it rejects the property (logged)
/// and the feature degrades, rather than shifting any vtable offset. Below this floor the
/// mirror is not guaranteed to match, so the loader declines cleanly (an old-driver decline,
/// never UB).
pub const AMF_MIN_VERSION: u64 = (1u64 << 48) | (4u64 << 32) | (34u64 << 16);
/// `AMF_SURFACE_FORMAT` (core/Surface.h).
pub const AMF_SURFACE_NV12: i32 = 1;
pub const AMF_SURFACE_P010: i32 = 10;
/// `AMF_DX_VERSION::AMF_DX11_1` (core/Data.h) — the `InitDX11` version argument.
pub const AMF_DX11_1: i32 = 111;
/// `AMF_MEMORY_TYPE::AMF_MEMORY_HOST` (core/Data.h) — the `AllocBuffer` memory type for the
/// CPU-filled HDR-metadata buffer.
pub const AMF_MEMORY_HOST: i32 = 1;
/// `AMFHDRMetadata` (components/ColorSpace.h) — the payload of the `*InHDRMetadata` encoder
/// property (an `AMFBuffer` holding exactly this struct). Same units as the HEVC ST.2086 SEI
/// and [`punktfunk_core::quic::HdrMeta`]: chromaticities in 1/50000, mastering luminance in
/// 0.0001 cd/m², CLL/FALL in nits. 28 bytes, no padding.
#[repr(C)]
pub struct AmfHdrMetadata {
pub red_primary: [u16; 2],
pub green_primary: [u16; 2],
pub blue_primary: [u16; 2],
pub white_point: [u16; 2],
pub max_mastering_luminance: u32,
pub min_mastering_luminance: u32,
pub max_content_light_level: u16,
pub max_frame_average_light_level: u16,
}
/// `AMFGuid` (core/Platform.h) — data41..data48 flattened into an array (identical layout).
#[repr(C)]
pub struct AmfGuid {
pub data1: u32,
pub data2: u16,
pub data3: u16,
pub data4: [u8; 8],
}
/// `IID_AMFBuffer` (core/Buffer.h `AMF_DECLARE_IID`) — for `QueryInterface` on the encoder's
/// output `AMFData` to reach `GetNative`/`GetSize`.
pub const IID_AMF_BUFFER: AmfGuid = AmfGuid {
data1: 0xb04b_7248,
data2: 0xb6f0,
data3: 0x4321,
data4: [0xb6, 0x91, 0xba, 0xa4, 0x74, 0x0f, 0x9f, 0xcb],
};
// `AMF_VARIANT_TYPE` (core/Variant.h) — the tags this module writes/reads.
pub const AMF_VARIANT_BOOL: i32 = 1;
pub const AMF_VARIANT_INT64: i32 = 2;
pub const AMF_VARIANT_RATE: i32 = 7;
pub const AMF_VARIANT_INTERFACE: i32 = 12;
/// `AMFVariantStruct` (core/Variant.h): a 4-byte C-enum tag + a 16-byte union whose largest
/// members are pointer/`amf_int64`/`AMFFloatVector4D` (align 8) → 24 bytes total, tag at 0,
/// payload at 8. Passed BY VALUE to `SetProperty` (Win64 passes >8-byte aggregates by hidden
/// reference on both sides, so declaring it by value matches the C compiler). The payload is
/// stored as two fully-initialised `u64`s — little-endian packing puts a bool in byte 0, an
/// `amf_int64` in word 0, and an `AMFRate{num,den}` as `num | den << 32`, exactly the union's
/// in-memory layout — so no partially-initialised union bytes ever cross the FFI.
#[repr(C)]
pub struct AmfVariant {
pub vtype: i32,
pub payload: [u64; 2],
}
impl AmfVariant {
pub fn zeroed() -> Self {
AmfVariant {
vtype: 0, // AMF_VARIANT_EMPTY
payload: [0, 0],
}
}
pub fn from_i64(v: i64) -> Self {
AmfVariant {
vtype: AMF_VARIANT_INT64,
payload: [v as u64, 0],
}
}
pub fn from_bool(v: bool) -> Self {
AmfVariant {
vtype: AMF_VARIANT_BOOL,
payload: [v as u64, 0],
}
}
/// `AMFRate { num, den }` — two little-endian `amf_uint32`s in the union's first 8 bytes.
pub fn from_rate(num: u32, den: u32) -> Self {
AmfVariant {
vtype: AMF_VARIANT_RATE,
payload: [num as u64 | ((den as u64) << 32), 0],
}
}
/// An `AMFInterface*` payload (`pInterface` in the union's first 8 bytes). The property
/// storage AddRefs the interface when it copies the variant in (the C++ template
/// `SetProperty(name, AMFVariant(value))` passes a temporary whose destructor releases,
/// so `SetProperty` must take its own reference) — the caller keeps sole ownership of the
/// reference it already holds.
pub fn from_interface(p: *mut c_void) -> Self {
AmfVariant {
vtype: AMF_VARIANT_INTERFACE,
payload: [p as usize as u64, 0],
}
}
/// Read back an `amf_int64` payload (only valid when `vtype == AMF_VARIANT_INT64`).
pub fn as_i64(&self) -> Option<i64> {
(self.vtype == AMF_VARIANT_INT64).then_some(self.payload[0] as i64)
}
}
/// Placeholder for a vtable slot this module never calls — same size/align as the function
/// pointer it stands in for, present only to keep the following slots at their C offsets.
pub type Slot = *const c_void;
// -- AMFFactory (core/Factory.h; NOT refcounted — a process singleton) ----------------------
#[repr(C)]
pub struct AmfFactory {
pub vtbl: *const AmfFactoryVtbl,
}
#[repr(C)]
pub struct AmfFactoryVtbl {
pub create_context:
unsafe extern "system" fn(*mut AmfFactory, *mut *mut AmfContext) -> AmfResult,
pub create_component: unsafe extern "system" fn(
*mut AmfFactory,
*mut AmfContext,
*const u16,
*mut *mut AmfComponent,
) -> AmfResult,
pub set_cache_folder: Slot,
pub get_cache_folder: Slot,
pub get_debug: Slot,
pub get_trace: Slot,
pub get_programs: Slot,
}
// -- AMFContext (core/Context.h) ------------------------------------------------------------
#[repr(C)]
pub struct AmfContext {
pub vtbl: *const AmfContextVtbl,
}
#[repr(C)]
pub struct AmfContextVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfContext) -> i32,
pub query_interface: Slot,
// AMFPropertyStorage
pub set_property: Slot,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFContext
pub terminate: unsafe extern "system" fn(*mut AmfContext) -> AmfResult,
pub init_dx9: Slot,
pub get_dx9_device: Slot,
pub lock_dx9: Slot,
pub unlock_dx9: Slot,
pub init_dx11: unsafe extern "system" fn(*mut AmfContext, *mut c_void, i32) -> AmfResult,
pub get_dx11_device: Slot,
pub lock_dx11: Slot,
pub unlock_dx11: Slot,
pub init_opencl: Slot,
pub get_opencl_context: Slot,
pub get_opencl_command_queue: Slot,
pub get_opencl_device_id: Slot,
pub get_opencl_compute_factory: Slot,
pub init_opencl_ex: Slot,
pub lock_opencl: Slot,
pub unlock_opencl: Slot,
pub init_opengl: Slot,
pub get_opengl_context: Slot,
pub get_opengl_drawable: Slot,
pub lock_opengl: Slot,
pub unlock_opengl: Slot,
pub init_xv: Slot,
pub get_xv_device: Slot,
pub lock_xv: Slot,
pub unlock_xv: Slot,
pub init_gralloc: Slot,
pub get_gralloc_device: Slot,
pub lock_gralloc: Slot,
pub unlock_gralloc: Slot,
pub alloc_buffer: unsafe extern "system" fn(
*mut AmfContext,
i32, // AMF_MEMORY_TYPE
usize,
*mut *mut AmfBuffer,
) -> AmfResult,
pub alloc_surface: Slot,
pub alloc_audio_buffer: Slot,
pub create_buffer_from_host_native: Slot,
pub create_surface_from_host_native: Slot,
pub create_surface_from_dx9_native: Slot,
/// Out-param is `AMFSurface**` in the header; declared as the `AmfData` base here because
/// every surface call this module makes (`SetPts`, `SetProperty`, `Release`,
/// `SubmitInput`) lives in the `AMFData` vtable prefix, which `AMFSurfaceVtbl` reproduces
/// slot-for-slot (single inheritance, same object pointer).
pub create_surface_from_dx11_native: unsafe extern "system" fn(
*mut AmfContext,
*mut c_void,
*mut *mut AmfData,
*mut c_void,
) -> AmfResult,
pub create_surface_from_opengl_native: Slot,
pub create_surface_from_gralloc_native: Slot,
pub create_surface_from_opencl_native: Slot,
pub create_buffer_from_opencl_native: Slot,
pub get_compute: Slot,
}
// -- AMFComponent (components/Component.h) --------------------------------------------------
#[repr(C)]
pub struct AmfComponent {
pub vtbl: *const AmfComponentVtbl,
}
#[repr(C)]
pub struct AmfComponentVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfComponent) -> i32,
pub query_interface: Slot,
// AMFPropertyStorage
pub set_property:
unsafe extern "system" fn(*mut AmfComponent, *const u16, AmfVariant) -> AmfResult,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFPropertyStorageEx
pub get_properties_info_count: Slot,
pub get_property_info_at: Slot,
pub get_property_info: Slot,
pub validate_property: Slot,
// AMFComponent
pub init: unsafe extern "system" fn(*mut AmfComponent, i32, i32, i32) -> AmfResult,
pub reinit: Slot,
pub terminate: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub drain: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub flush: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub submit_input: unsafe extern "system" fn(*mut AmfComponent, *mut AmfData) -> AmfResult,
pub query_output:
unsafe extern "system" fn(*mut AmfComponent, *mut *mut AmfData) -> AmfResult,
pub get_context: Slot,
pub set_output_data_allocator_cb: Slot,
pub get_caps: Slot,
pub optimize: Slot,
}
// -- AMFData (core/Data.h) — also the usable prefix of AMFSurface --------------------------
#[repr(C)]
pub struct AmfData {
pub vtbl: *const AmfDataVtbl,
}
#[repr(C)]
pub struct AmfDataVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfData) -> i32,
pub query_interface:
unsafe extern "system" fn(*mut AmfData, *const AmfGuid, *mut *mut c_void) -> AmfResult,
// AMFPropertyStorage
pub set_property:
unsafe extern "system" fn(*mut AmfData, *const u16, AmfVariant) -> AmfResult,
pub get_property:
unsafe extern "system" fn(*mut AmfData, *const u16, *mut AmfVariant) -> AmfResult,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFData
pub get_memory_type: Slot,
pub duplicate: Slot,
pub convert: Slot,
pub interop: Slot,
pub get_data_type: Slot,
pub is_reusable: Slot,
pub set_pts: unsafe extern "system" fn(*mut AmfData, i64),
pub get_pts: Slot,
pub set_duration: Slot,
pub get_duration: Slot,
}
// -- AMFBuffer (core/Buffer.h) — the encoder's output object -------------------------------
#[repr(C)]
pub struct AmfBuffer {
pub vtbl: *const AmfBufferVtbl,
}
#[repr(C)]
pub struct AmfBufferVtbl {
// AMFInterface + AMFPropertyStorage + AMFData prefix (identical order to AmfDataVtbl).
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfBuffer) -> i32,
pub query_interface: Slot,
pub set_property: Slot,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
pub get_memory_type: Slot,
pub duplicate: Slot,
pub convert: Slot,
pub interop: Slot,
pub get_data_type: Slot,
pub is_reusable: Slot,
pub set_pts: Slot,
pub get_pts: Slot,
pub set_duration: Slot,
pub get_duration: Slot,
// AMFBuffer
pub set_size: Slot,
pub get_size: unsafe extern "system" fn(*mut AmfBuffer) -> usize,
pub get_native: unsafe extern "system" fn(*mut AmfBuffer) -> *mut c_void,
pub add_observer_buffer: Slot,
pub remove_observer_buffer: Slot,
}
// -- DLL entry points (core/Factory.h; AMF_CDECL_CALL) --------------------------------------
pub type AmfQueryVersionFn = unsafe extern "C" fn(*mut u64) -> AmfResult;
pub type AmfInitFn = unsafe extern "C" fn(u64, *mut *mut AmfFactory) -> AmfResult;
}
// The hand-mirrored AMF C ABI lives in `amf_sys.rs` (WP7.4): pure FFI surface, no policy —
// isolating the unsafe vtable mirror from the encoder logic is the crate's stated review
// goal, and the `#[path]` keeps the module name (`sys::`) and every call site unchanged.
#[path = "amf_sys.rs"]
mod sys;
use sys::{result_name, AmfVariant};
@@ -1272,7 +858,11 @@ impl AmfEncoder {
if codec == Codec::Av1 && !probe_can_encode(Codec::Av1) {
bail!("this GPU/driver declined AV1 encode (RDNA3+ required) — native AMF probe");
}
let ten_bit = bit_depth >= 10 || matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2);
// Depth follows the delivered pixels, not the negotiated depth ([`crate::ten_bit_input`]).
// With the old `bit_depth >= 10 || …` shape a 10-bit-negotiated session over an 8-bit
// capture derived `expected = P010`, tripped the format check below and ended the session
// at open — on exactly the configuration the capturer had already downgraded on purpose.
let ten_bit = crate::ten_bit_input(format, bit_depth);
// Zero-copy by construction: the input ring is NV12/P010 fed by same-format
// CopySubresourceRegion. Any other capture format (Bgra/Rgb10a2 video-processor fallback,
// CPU frames) has no native input path — and since Phase 3 no ffmpeg readback to degrade
@@ -2369,31 +1959,27 @@ impl Encoder for AmfEncoder {
if !self.ltr_active || first < 0 || first > last {
return false;
}
// Taint sweep BEFORE picking the anchor: an LTR marked at-or-after the loss start was
// encoded inside the client's corrupt window — the client either never received it or
// decoded it against a broken reference chain. Serving it as "known-good" on a LATER
// loss ships corruption as the recovery anchor (and every subsequent mark re-samples
// it). Dropped slots stay dropped; the cadence re-marks a clean frame within ~1/4 s.
for marked in self.ltr_slots.iter_mut() {
if marked.is_some_and(|idx| idx >= first) {
// The taint-sweep + anchor-pick POLICY lives in `rfi::plan_slot_recovery` (one decision
// shared with QSV and Vulkan Video); this backend's mechanism is: distrust = clear the
// mirror slot (dropped slots stay dropped; the cadence re-marks a clean frame within
// ~1/4 s). `ltr_slots` store the WIRE frame index of the marked frame (`submit_indexed`
// pins `frame_idx` to it per submission), so they compare directly against the client's
// `first` — and stay comparable across encoder rebuilds/resets, where an internal counter
// would make the pre-loss check vacuous and risk force-referencing an LTR marked INSIDE
// the lost range.
let view: Vec<(usize, i64)> = self
.ltr_slots
.iter()
.enumerate()
.filter_map(|(s, m)| m.map(|w| (s, w)))
.collect();
let plan = super::rfi::plan_slot_recovery(&view, first);
for (slot, marked) in self.ltr_slots.iter_mut().enumerate() {
if plan.tainted & (1 << slot) != 0 {
*marked = None;
}
}
// Pick the newest LTR strictly OLDER than the loss: the most recent known-good reference the
// client still holds, so re-referencing it costs the least (smallest recovery-frame residual).
// `ltr_slots` store the WIRE frame index of the marked frame (`submit_indexed` pins
// `frame_idx` to it per submission), so they compare directly against the client's `first`
// — and stay comparable across encoder rebuilds/resets, where an internal counter would
// make this check vacuous and risk force-referencing an LTR marked INSIDE the lost range.
let mut best: Option<(usize, i64)> = None;
for (slot, marked) in self.ltr_slots.iter().enumerate() {
if let Some(idx) = *marked {
if idx < first && best.is_none_or(|(_, b)| idx > b) {
best = Some((slot, idx));
}
}
}
match best {
match plan.anchor {
Some((slot, ltr_frame)) => {
// Queue the force for the next submit; that frame ships tagged `recovery_anchor`.
self.pending_force = Some(slot);
@@ -2422,6 +2008,8 @@ impl Encoder for AmfEncoder {
fn caps(&self) -> EncoderCaps {
EncoderCaps {
// As Windows NVENC: the capturer composites; this backend never reads `frame.cursor`.
blends_cursor: false,
// LTR-RFI: AMD's reference invalidation is the user long-term-reference path (mark a
// frame, force a later one to re-reference it). True only when the live driver accepted
// the LTR slots at open — otherwise loss recovery falls back to a full IDR.
@@ -2760,6 +2348,10 @@ mod tests {
}
/// The `p`-quantile of `samples` (µs), sorting in place. `0` when empty.
/// Gated like its only caller, the `amf-qsv`-only §5.2 latency A/B below — otherwise a
/// `--features nvenc,qsv` build compiles this helper with the benchmark cfg'd out and trips
/// `dead_code` (which the crate root no longer blanket-allows).
#[cfg(feature = "amf-qsv")]
fn percentile(samples: &mut [u128], p: f64) -> u128 {
if samples.is_empty() {
return 0;
@@ -2778,6 +2370,8 @@ mod tests {
/// so its submit→AU is the bare ASIC time. The last ~2 unflushed frames on the ffmpeg path
/// are left unmeasured (dropped with the encoder) so every recorded sample is a genuine paced
/// submit→AU.
/// Gated like its only caller (see [`percentile`]).
#[cfg(feature = "amf-qsv")]
#[allow(clippy::too_many_arguments)]
fn drive_and_measure(
enc: &mut dyn Encoder,
+414
View File
@@ -0,0 +1,414 @@
//! Mirrored AMF C ABI (written against GPUOpen header release v1.4.36 — amf/public/include; every
//! slot below is a base-interface slot whose layout is stable since <= v1.4.34, the loader's
//! accepted ABI floor, so the mirror is valid on every runtime the loader admits).
//!
//! Layout rules this mirror relies on: every AMF interface is a struct whose sole member is a
//! pointer to a C vtable; derived interfaces PREPEND their base's slots in order (AMFInterface →
//! AMFPropertyStorage → AMFData → AMFBuffer/AMFSurface), so a derived pointer is usable through a
//! base vtable mirror. Slots we never call are declared as bare `*const c_void` placeholders —
//! same size/alignment as the function pointer they stand in for. `AMF_STD_CALL` is `__stdcall`
//! (Rust `extern "system"`); the two DLL entry points are `__cdecl` (`extern "C"`); on x86_64
//! both collapse to the one Windows calling convention.
use std::ffi::c_void;
/// `AMF_RESULT` (core/Result.h) — a plain C enum, sequential from 0. Only the codes this
/// module branches on are named; everything else is reported numerically via [`result_name`].
pub type AmfResult = i32;
pub const AMF_OK: AmfResult = 0;
pub const AMF_EOF: AmfResult = 23;
pub const AMF_REPEAT: AmfResult = 24;
pub const AMF_INPUT_FULL: AmfResult = 25;
pub const AMF_NEED_MORE_INPUT: AmfResult = 44;
/// Human-readable name for an `AMF_RESULT` (diagnostics only — the numeric value rides along
/// so an unnamed code is still identifiable against Result.h).
pub fn result_name(r: AmfResult) -> &'static str {
match r {
0 => "AMF_OK",
1 => "AMF_FAIL",
2 => "AMF_UNEXPECTED",
3 => "AMF_ACCESS_DENIED",
4 => "AMF_INVALID_ARG",
5 => "AMF_OUT_OF_RANGE",
6 => "AMF_OUT_OF_MEMORY",
7 => "AMF_INVALID_POINTER",
8 => "AMF_NO_INTERFACE",
9 => "AMF_NOT_IMPLEMENTED",
10 => "AMF_NOT_SUPPORTED",
11 => "AMF_NOT_FOUND",
12 => "AMF_ALREADY_INITIALIZED",
13 => "AMF_NOT_INITIALIZED",
14 => "AMF_INVALID_FORMAT",
15 => "AMF_WRONG_STATE",
17 => "AMF_NO_DEVICE",
18 => "AMF_DIRECTX_FAILED",
23 => "AMF_EOF",
24 => "AMF_REPEAT",
25 => "AMF_INPUT_FULL",
26 => "AMF_RESOLUTION_CHANGED",
28 => "AMF_INVALID_DATA_TYPE",
29 => "AMF_INVALID_RESOLUTION",
30 => "AMF_CODEC_NOT_SUPPORTED",
31 => "AMF_SURFACE_FORMAT_NOT_SUPPORTED",
32 => "AMF_SURFACE_MUST_BE_SHARED",
36 => "AMF_ENCODER_NOT_PRESENT",
44 => "AMF_NEED_MORE_INPUT",
_ => "AMF_<unnamed>",
}
}
/// The AMF header release this FFI mirror was written against: `AMF_FULL_VERSION` for 1.4.36.0
/// (core/Version.h `AMF_MAKE_FULL_VERSION`). This is the version claimed to `AMFInit` — but
/// capped at the runtime's own reported version (see `load_factory`), so an older-but-accepted
/// runtime is asked only for the ABI it actually provides.
pub const AMF_HEADER_VERSION: u64 = (1u64 << 48) | (4u64 << 32) | (36u64 << 16);
/// The oldest AMF runtime the loader accepts (`AMF_FULL_VERSION` 1.4.34.0 — AMD Adrenalin
/// 24.6.1). This is an **ABI floor, not a feature floor**: every vtable slot mirrored in this
/// module belongs to a base interface (`AMFFactory`/`AMFContext`/`AMFComponent`/`AMFData`/
/// `AMFBuffer`) whose layout has been stable — append-only, no mid-vtable insertions — since
/// well before 1.4.34, so a 1.4.34 runtime is guaranteed to expose every mirrored slot at its
/// mirrored offset. Everything 1.4.35/1.4.36 added that this path can touch (new HQ presets,
/// AV1 B-frame / picture management) is a *string-keyed encoder property*, applied through
/// [`set_prop`] with `required=false` — a runtime that lacks it rejects the property (logged)
/// and the feature degrades, rather than shifting any vtable offset. Below this floor the
/// mirror is not guaranteed to match, so the loader declines cleanly (an old-driver decline,
/// never UB).
pub const AMF_MIN_VERSION: u64 = (1u64 << 48) | (4u64 << 32) | (34u64 << 16);
/// `AMF_SURFACE_FORMAT` (core/Surface.h).
pub const AMF_SURFACE_NV12: i32 = 1;
pub const AMF_SURFACE_P010: i32 = 10;
/// `AMF_DX_VERSION::AMF_DX11_1` (core/Data.h) — the `InitDX11` version argument.
pub const AMF_DX11_1: i32 = 111;
/// `AMF_MEMORY_TYPE::AMF_MEMORY_HOST` (core/Data.h) — the `AllocBuffer` memory type for the
/// CPU-filled HDR-metadata buffer.
pub const AMF_MEMORY_HOST: i32 = 1;
/// `AMFHDRMetadata` (components/ColorSpace.h) — the payload of the `*InHDRMetadata` encoder
/// property (an `AMFBuffer` holding exactly this struct). Same units as the HEVC ST.2086 SEI
/// and [`punktfunk_core::quic::HdrMeta`]: chromaticities in 1/50000, mastering luminance in
/// 0.0001 cd/m², CLL/FALL in nits. 28 bytes, no padding.
#[repr(C)]
pub struct AmfHdrMetadata {
pub red_primary: [u16; 2],
pub green_primary: [u16; 2],
pub blue_primary: [u16; 2],
pub white_point: [u16; 2],
pub max_mastering_luminance: u32,
pub min_mastering_luminance: u32,
pub max_content_light_level: u16,
pub max_frame_average_light_level: u16,
}
/// `AMFGuid` (core/Platform.h) — data41..data48 flattened into an array (identical layout).
#[repr(C)]
pub struct AmfGuid {
pub data1: u32,
pub data2: u16,
pub data3: u16,
pub data4: [u8; 8],
}
/// `IID_AMFBuffer` (core/Buffer.h `AMF_DECLARE_IID`) — for `QueryInterface` on the encoder's
/// output `AMFData` to reach `GetNative`/`GetSize`.
pub const IID_AMF_BUFFER: AmfGuid = AmfGuid {
data1: 0xb04b_7248,
data2: 0xb6f0,
data3: 0x4321,
data4: [0xb6, 0x91, 0xba, 0xa4, 0x74, 0x0f, 0x9f, 0xcb],
};
// `AMF_VARIANT_TYPE` (core/Variant.h) — the tags this module writes/reads.
pub const AMF_VARIANT_BOOL: i32 = 1;
pub const AMF_VARIANT_INT64: i32 = 2;
pub const AMF_VARIANT_RATE: i32 = 7;
pub const AMF_VARIANT_INTERFACE: i32 = 12;
/// `AMFVariantStruct` (core/Variant.h): a 4-byte C-enum tag + a 16-byte union whose largest
/// members are pointer/`amf_int64`/`AMFFloatVector4D` (align 8) → 24 bytes total, tag at 0,
/// payload at 8. Passed BY VALUE to `SetProperty` (Win64 passes >8-byte aggregates by hidden
/// reference on both sides, so declaring it by value matches the C compiler). The payload is
/// stored as two fully-initialised `u64`s — little-endian packing puts a bool in byte 0, an
/// `amf_int64` in word 0, and an `AMFRate{num,den}` as `num | den << 32`, exactly the union's
/// in-memory layout — so no partially-initialised union bytes ever cross the FFI.
#[repr(C)]
pub struct AmfVariant {
pub vtype: i32,
pub payload: [u64; 2],
}
impl AmfVariant {
pub fn zeroed() -> Self {
AmfVariant {
vtype: 0, // AMF_VARIANT_EMPTY
payload: [0, 0],
}
}
pub fn from_i64(v: i64) -> Self {
AmfVariant {
vtype: AMF_VARIANT_INT64,
payload: [v as u64, 0],
}
}
pub fn from_bool(v: bool) -> Self {
AmfVariant {
vtype: AMF_VARIANT_BOOL,
payload: [v as u64, 0],
}
}
/// `AMFRate { num, den }` — two little-endian `amf_uint32`s in the union's first 8 bytes.
pub fn from_rate(num: u32, den: u32) -> Self {
AmfVariant {
vtype: AMF_VARIANT_RATE,
payload: [num as u64 | ((den as u64) << 32), 0],
}
}
/// An `AMFInterface*` payload (`pInterface` in the union's first 8 bytes). The property
/// storage AddRefs the interface when it copies the variant in (the C++ template
/// `SetProperty(name, AMFVariant(value))` passes a temporary whose destructor releases,
/// so `SetProperty` must take its own reference) — the caller keeps sole ownership of the
/// reference it already holds.
pub fn from_interface(p: *mut c_void) -> Self {
AmfVariant {
vtype: AMF_VARIANT_INTERFACE,
payload: [p as usize as u64, 0],
}
}
/// Read back an `amf_int64` payload (only valid when `vtype == AMF_VARIANT_INT64`).
pub fn as_i64(&self) -> Option<i64> {
(self.vtype == AMF_VARIANT_INT64).then_some(self.payload[0] as i64)
}
}
/// Placeholder for a vtable slot this module never calls — same size/align as the function
/// pointer it stands in for, present only to keep the following slots at their C offsets.
pub type Slot = *const c_void;
// -- AMFFactory (core/Factory.h; NOT refcounted — a process singleton) ----------------------
#[repr(C)]
pub struct AmfFactory {
pub vtbl: *const AmfFactoryVtbl,
}
#[repr(C)]
pub struct AmfFactoryVtbl {
pub create_context:
unsafe extern "system" fn(*mut AmfFactory, *mut *mut AmfContext) -> AmfResult,
pub create_component: unsafe extern "system" fn(
*mut AmfFactory,
*mut AmfContext,
*const u16,
*mut *mut AmfComponent,
) -> AmfResult,
pub set_cache_folder: Slot,
pub get_cache_folder: Slot,
pub get_debug: Slot,
pub get_trace: Slot,
pub get_programs: Slot,
}
// -- AMFContext (core/Context.h) ------------------------------------------------------------
#[repr(C)]
pub struct AmfContext {
pub vtbl: *const AmfContextVtbl,
}
#[repr(C)]
pub struct AmfContextVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfContext) -> i32,
pub query_interface: Slot,
// AMFPropertyStorage
pub set_property: Slot,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFContext
pub terminate: unsafe extern "system" fn(*mut AmfContext) -> AmfResult,
pub init_dx9: Slot,
pub get_dx9_device: Slot,
pub lock_dx9: Slot,
pub unlock_dx9: Slot,
pub init_dx11: unsafe extern "system" fn(*mut AmfContext, *mut c_void, i32) -> AmfResult,
pub get_dx11_device: Slot,
pub lock_dx11: Slot,
pub unlock_dx11: Slot,
pub init_opencl: Slot,
pub get_opencl_context: Slot,
pub get_opencl_command_queue: Slot,
pub get_opencl_device_id: Slot,
pub get_opencl_compute_factory: Slot,
pub init_opencl_ex: Slot,
pub lock_opencl: Slot,
pub unlock_opencl: Slot,
pub init_opengl: Slot,
pub get_opengl_context: Slot,
pub get_opengl_drawable: Slot,
pub lock_opengl: Slot,
pub unlock_opengl: Slot,
pub init_xv: Slot,
pub get_xv_device: Slot,
pub lock_xv: Slot,
pub unlock_xv: Slot,
pub init_gralloc: Slot,
pub get_gralloc_device: Slot,
pub lock_gralloc: Slot,
pub unlock_gralloc: Slot,
pub alloc_buffer: unsafe extern "system" fn(
*mut AmfContext,
i32, // AMF_MEMORY_TYPE
usize,
*mut *mut AmfBuffer,
) -> AmfResult,
pub alloc_surface: Slot,
pub alloc_audio_buffer: Slot,
pub create_buffer_from_host_native: Slot,
pub create_surface_from_host_native: Slot,
pub create_surface_from_dx9_native: Slot,
/// Out-param is `AMFSurface**` in the header; declared as the `AmfData` base here because
/// every surface call this module makes (`SetPts`, `SetProperty`, `Release`,
/// `SubmitInput`) lives in the `AMFData` vtable prefix, which `AMFSurfaceVtbl` reproduces
/// slot-for-slot (single inheritance, same object pointer).
pub create_surface_from_dx11_native: unsafe extern "system" fn(
*mut AmfContext,
*mut c_void,
*mut *mut AmfData,
*mut c_void,
) -> AmfResult,
pub create_surface_from_opengl_native: Slot,
pub create_surface_from_gralloc_native: Slot,
pub create_surface_from_opencl_native: Slot,
pub create_buffer_from_opencl_native: Slot,
pub get_compute: Slot,
}
// -- AMFComponent (components/Component.h) --------------------------------------------------
#[repr(C)]
pub struct AmfComponent {
pub vtbl: *const AmfComponentVtbl,
}
#[repr(C)]
pub struct AmfComponentVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfComponent) -> i32,
pub query_interface: Slot,
// AMFPropertyStorage
pub set_property:
unsafe extern "system" fn(*mut AmfComponent, *const u16, AmfVariant) -> AmfResult,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFPropertyStorageEx
pub get_properties_info_count: Slot,
pub get_property_info_at: Slot,
pub get_property_info: Slot,
pub validate_property: Slot,
// AMFComponent
pub init: unsafe extern "system" fn(*mut AmfComponent, i32, i32, i32) -> AmfResult,
pub reinit: Slot,
pub terminate: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub drain: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub flush: unsafe extern "system" fn(*mut AmfComponent) -> AmfResult,
pub submit_input: unsafe extern "system" fn(*mut AmfComponent, *mut AmfData) -> AmfResult,
pub query_output: unsafe extern "system" fn(*mut AmfComponent, *mut *mut AmfData) -> AmfResult,
pub get_context: Slot,
pub set_output_data_allocator_cb: Slot,
pub get_caps: Slot,
pub optimize: Slot,
}
// -- AMFData (core/Data.h) — also the usable prefix of AMFSurface --------------------------
#[repr(C)]
pub struct AmfData {
pub vtbl: *const AmfDataVtbl,
}
#[repr(C)]
pub struct AmfDataVtbl {
// AMFInterface
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfData) -> i32,
pub query_interface:
unsafe extern "system" fn(*mut AmfData, *const AmfGuid, *mut *mut c_void) -> AmfResult,
// AMFPropertyStorage
pub set_property: unsafe extern "system" fn(*mut AmfData, *const u16, AmfVariant) -> AmfResult,
pub get_property:
unsafe extern "system" fn(*mut AmfData, *const u16, *mut AmfVariant) -> AmfResult,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
// AMFData
pub get_memory_type: Slot,
pub duplicate: Slot,
pub convert: Slot,
pub interop: Slot,
pub get_data_type: Slot,
pub is_reusable: Slot,
pub set_pts: unsafe extern "system" fn(*mut AmfData, i64),
pub get_pts: Slot,
pub set_duration: Slot,
pub get_duration: Slot,
}
// -- AMFBuffer (core/Buffer.h) — the encoder's output object -------------------------------
#[repr(C)]
pub struct AmfBuffer {
pub vtbl: *const AmfBufferVtbl,
}
#[repr(C)]
pub struct AmfBufferVtbl {
// AMFInterface + AMFPropertyStorage + AMFData prefix (identical order to AmfDataVtbl).
pub acquire: Slot,
pub release: unsafe extern "system" fn(*mut AmfBuffer) -> i32,
pub query_interface: Slot,
pub set_property: Slot,
pub get_property: Slot,
pub has_property: Slot,
pub get_property_count: Slot,
pub get_property_at: Slot,
pub clear: Slot,
pub add_to: Slot,
pub copy_to: Slot,
pub add_observer: Slot,
pub remove_observer: Slot,
pub get_memory_type: Slot,
pub duplicate: Slot,
pub convert: Slot,
pub interop: Slot,
pub get_data_type: Slot,
pub is_reusable: Slot,
pub set_pts: Slot,
pub get_pts: Slot,
pub set_duration: Slot,
pub get_duration: Slot,
// AMFBuffer
pub set_size: Slot,
pub get_size: unsafe extern "system" fn(*mut AmfBuffer) -> usize,
pub get_native: unsafe extern "system" fn(*mut AmfBuffer) -> *mut c_void,
pub add_observer_buffer: Slot,
pub remove_observer_buffer: Slot,
}
// -- DLL entry points (core/Factory.h; AMF_CDECL_CALL) --------------------------------------
pub type AmfQueryVersionFn = unsafe extern "C" fn(*mut u64) -> AmfResult;
pub type AmfInitFn = unsafe extern "C" fn(u64, *mut *mut AmfFactory) -> AmfResult;
+65 -10
View File
@@ -95,6 +95,13 @@ struct AVD3D11VAFramesContext {
/// AMD AMF vs Intel QSV — the two libavcodec vendor backends this module covers.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum WinVendor {
/// Benchmark-only, as the module header explains: native AMF replaced the libavcodec AMF path
/// in production, and the only remaining CONSTRUCTOR is the `#[cfg(feature = "amf-qsv")]`
/// latency A/B in `amf.rs` — the measurement that justifies the native backend existing. That
/// is test code, so the *lib* target constructs it nowhere and `dead_code` fires on it (the
/// crate root no longer blanket-allows that). Kept deliberately rather than deleted; the arms
/// below are what the benchmark drives.
#[allow(dead_code)]
Amf,
Qsv,
}
@@ -127,6 +134,44 @@ fn zerocopy_enabled(vendor: WinVendor) -> bool {
.unwrap_or(matches!(vendor, WinVendor::Amf))
}
/// Upper bound on `PUNKTFUNK_FFWIN_POLL_MS`. This knob spins the **encode thread** waiting for an
/// AU, so a value past one frame period is already self-defeating and a full second is far beyond
/// anything an operator would set on purpose. The clamp is also what makes the µs conversion
/// below provably overflow-free.
///
/// The reachable hazard is a slipped digit, not the overflow: pre-clamp, `PUNKTFUNK_FFWIN_POLL_MS=
/// 100000000` was a **27.7-hour** spin with no overflow anywhere near it.
const MAX_POLL_SPIN_MS: u64 = 1_000;
/// Bounded post-submit spin for [`FfmpegWinEncoder::poll`], in microseconds (0 = off, the default
/// and the correct choice on every VCN measured so far).
///
/// Read from the environment **once per process** (WP6.1): `poll` runs once per encode tick, and
/// this was an unconditional `env::var` + parse on it.
///
/// ⚠ The audit proposed `saturating_mul` for the µs conversion. It is still the wrong fix, but for
/// a reason worth stating precisely, because the obvious one is false: `Duration::from_micros(
/// u64::MAX)` is only ~1.8e13 seconds, six orders of magnitude below `Duration`'s `u64::MAX`-second
/// ceiling, so `Instant::now() + Duration::from_micros(u64::MAX)` does **not** overflow and does
/// **not** panic (measured, both with and without debug assertions). What it does instead is set a
/// deadline ~584,000 years out, and the loop below only exits on `Packet`/`Eof` — and this
/// function's own doc explains that a spin here *provably never* produces the owed AU on the
/// measured hardware. So `saturating_mul` converts a bad value into a **permanently wedged encode
/// thread**: a hang, not a panic. Clamping the parsed value first removes the bad value entirely.
///
/// (For the record on the pre-clamp behaviour: the workspace sets no `overflow-checks` in
/// `[profile.release]`, so `ms * 1000` wrapped silently in release and panicked only in debug.)
fn poll_spin_cap_us() -> u64 {
static CAP_US: std::sync::OnceLock<u64> = std::sync::OnceLock::new();
*CAP_US.get_or_init(|| {
std::env::var("PUNKTFUNK_FFWIN_POLL_MS")
.ok()
.and_then(|s| s.trim().parse::<u64>().ok())
.map(|ms| ms.min(MAX_POLL_SPIN_MS) * 1000)
.unwrap_or(0) // default: no spin — the libavcodec AMF buffer can't be spun out
})
}
/// The swscale *source* pixel format for a captured packed-RGB/BGR layout (8-bit BGRA fallback only).
fn sws_src(format: PixelFormat) -> Result<Pixel> {
Ok(match format {
@@ -151,8 +196,14 @@ fn sws_src(format: PixelFormat) -> Result<Pixel> {
}
/// Does this captured format imply a 10-bit encode (P010 / Rgb10a2)?
fn is_10bit_format(format: PixelFormat, bit_depth: u8) -> bool {
bit_depth >= 10 || matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2)
///
/// Depth follows the PIXELS, not the negotiated `bit_depth` — see
/// [`crate::ten_bit_input`] for why, and for the failure this shape used to produce here in
/// particular: a 10-bit-negotiated session over an 8-bit capture built a P010 encoder whose every
/// `submit_d3d11` then failed the depth check below, forever, with `reset()` unable to help
/// because the rebuild re-derived the same wrong answer.
fn is_10bit_format(format: PixelFormat) -> bool {
matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2)
}
/// Build the FFmpeg encoder context shared by both inner paths: name, mode, low-latency RC,
@@ -271,6 +322,11 @@ pub fn probe_can_encode_444(_vendor: WinVendor, _codec: Codec) -> bool {
false
}
/// Gated to the builds that can actually reach it: `lib.rs`'s only caller sits under
/// `cfg(all(not(feature = "qsv"), feature = "amf-qsv"))`, because with the native VPL backend
/// compiled in it is `qsv::probe_can_encode` that answers. So in the SHIPPED Windows combo
/// (`nvenc,amf-qsv,qsv`) this function has no caller at all.
#[cfg(not(feature = "qsv"))]
pub fn probe_can_encode(vendor: WinVendor, codec: Codec) -> bool {
// Deliberately NOT pinned to the selected render adapter (unlike `nvenc::probe_can_encode_444`):
// the system-input probe passes no hwdevice, and the AMF/QSV runtimes only ever bind their own
@@ -350,7 +406,7 @@ impl SystemInner {
bitrate_bps: u64,
bit_depth: u8,
) -> Result<Self> {
let ten_bit = is_10bit_format(format, bit_depth);
let ten_bit = crate::ten_bit_input(format, bit_depth);
let sw_av = if ten_bit {
ffi::AVPixelFormat::AV_PIX_FMT_P010LE
} else {
@@ -472,7 +528,10 @@ impl SystemInner {
pts: i64,
idr: bool,
) -> Result<()> {
let fmt_10 = matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2);
// Same predicate the encoder was built from (`ten_bit_input`), so this can only fire on a
// genuine MID-STREAM depth change — never, as it used to, on every frame of a session that
// merely negotiated 10-bit over an 8-bit capture.
let fmt_10 = is_10bit_format(format);
anyhow::ensure!(
fmt_10 == self.ten_bit,
"captured format {format:?} bit-depth changed under the encoder (built {}-bit)",
@@ -851,7 +910,7 @@ impl ZeroCopyInner {
bit_depth: u8,
device: &ID3D11Device,
) -> Result<Self> {
let ten_bit = is_10bit_format(format, bit_depth);
let ten_bit = crate::ten_bit_input(format, bit_depth);
let sw_av = if ten_bit {
ffi::AVPixelFormat::AV_PIX_FMT_P010LE
} else {
@@ -1311,11 +1370,7 @@ impl Encoder for FfmpegWinEncoder {
Some(Inner::ZeroCopy(z)) => &mut z.enc,
None => return Ok(None),
};
let cap_us = std::env::var("PUNKTFUNK_FFWIN_POLL_MS")
.ok()
.and_then(|s| s.parse::<u64>().ok())
.map(|ms| ms * 1000)
.unwrap_or(0); // default: no spin — the libavcodec AMF buffer can't be spun out
let cap_us = poll_spin_cap_us();
let deadline = (cap_us > 0 && self.in_flight > 0)
.then(|| std::time::Instant::now() + std::time::Duration::from_micros(cap_us));
loop {
+425 -132
View File
@@ -37,8 +37,9 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::nvenc_core::{
apply_low_latency_config, build_init_params, codec_guid, resolve_slices, resolve_subframe,
LowLatencyConfig, NvStatusExt, RFI_DPB,
apply_low_latency_config, build_init_params, cached_ceiling, codec_guid, plan_range_recovery,
resolve_slices, resolve_split_mode, resolve_subframe, store_ceiling, CeilingKey,
LowLatencyConfig, NvStatusExt, RangePlan,
};
use super::nvenc_status;
use super::{ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
@@ -263,6 +264,112 @@ fn split_mode_units(split_mode: u32) -> u32 {
}
}
/// Serializes every `nvEncOpenEncodeSessionEx` in this process against [`reap_parked_sessions`].
/// The reap retry-destroys handles whose driver-side state is unknown; if it overlapped an open,
/// the driver could hand the NEW session the recycled address of the zombie being destroyed and
/// the reap would destroy a live session. Held across `init_session`'s whole open sequence
/// (including its caps probe and bitrate-clamp search) and around the standalone caps probes.
/// Admission (`can_open_another_session`) never takes it — that stays a lock-free atomic load.
static DRIVER_SESSION_GATE: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// A session whose `destroy_encoder` failed with an AMBIGUOUS status (see
/// [`nvenc_status::destroy_proves_no_session`]): the driver may still hold its concurrent-session
/// slot, so its units stay charged in [`LIVE_SESSION_UNITS`] (fail-closed for admission) and the
/// handle is parked here for [`reap_parked_sessions`] to retry. The retry is what keeps a
/// TRANSIENT failure (wedge episode a TDR later cleared) from poisoning the budget until a host
/// restart, while a genuine driver leak keeps failing the retry and stays correctly charged.
struct ParkedSession {
enc: usize,
units: u32,
/// Pins the D3D11 device the session was opened against. Teardown drops the texture
/// registrations (the only other refs), and the reinit-on-device-change path parks exactly
/// when the old device is on its way out — without this, the reap's late `destroy_encoder`
/// could touch a freed device.
_device: Option<ID3D11Device>,
}
// SAFETY: the COM ref is the only non-Send field. The D3D11 device is free-threaded (capture
// creates it without `D3D11_CREATE_DEVICE_SINGLETHREADED` — see pf-frame/src/dxgi.rs), we never
// call methods on it from here, and dropping (Release) a free-threaded COM object from another
// thread is sound. The raw `enc` handle is only ever passed back to `destroy_encoder` under
// [`PARKED`]'s lock with [`DRIVER_SESSION_GATE`] held — single-threaded access in practice.
unsafe impl Send for ParkedSession {}
static PARKED: std::sync::Mutex<Vec<ParkedSession>> = std::sync::Mutex::new(Vec::new());
/// Park a session whose destroy failed ambiguously. Units are NOT refunded — they keep counting
/// against admission until a reap proves the slot free. Bounded: once parked units would exceed
/// the session cap the oldest entry ages out WITH a refund (a repeated wedge-rebuild loop would
/// otherwise grow this without limit, and by then the driver has almost certainly been through
/// the reset that reclaims the early handles anyway).
fn park_session(enc: usize, units: u32, device: Option<ID3D11Device>) {
let mut parked = PARKED.lock().unwrap_or_else(|p| p.into_inner());
while !parked.is_empty() && parked.iter().map(|z| z.units).sum::<u32>() + units > session_cap()
{
let old = parked.remove(0);
LIVE_SESSION_UNITS.fetch_sub(old.units, std::sync::atomic::Ordering::Relaxed);
tracing::warn!(
enc = old.enc,
units = old.units,
"NVENC parked-session graveyard full — aging out the oldest entry (units refunded)"
);
}
parked.push(ParkedSession {
enc,
units,
_device: device,
});
}
/// Retry `destroy_encoder` on every parked session and refund the units of each one that now
/// succeeds (or fails with a session-gone status — same proof). Runs from `init_session` on the
/// encode thread, NEVER from admission (a wedged driver would block the admission registry lock).
///
/// # Safety
/// Caller must hold [`DRIVER_SESSION_GATE`] and there must be NO live NVENC session in the
/// process (checked here: live units == parked units). Under those two conditions no live session
/// can alias a recycled handle address, and no concurrent open can be handed one mid-reap. The
/// residual assumption — documented, unprovable from the SDK — is that a FAILED destroy leaves
/// the handle itself intact for a later retry; NVENC documents no retry semantics either way.
unsafe fn reap_parked_sessions() {
let mut parked = PARKED.lock().unwrap_or_else(|p| p.into_inner());
if parked.is_empty() {
return;
}
let parked_units: u32 = parked.iter().map(|z| z.units).sum();
if LIVE_SESSION_UNITS.load(std::sync::atomic::Ordering::Relaxed) != parked_units {
return; // a live session exists somewhere — its address space is off limits
}
parked.retain(
|z| match (api().destroy_encoder)(z.enc as *mut c_void).nv_ok() {
Ok(()) => {
tracing::info!(
enc = z.enc,
units = z.units,
"NVENC parked session reclaimed — retry-destroy succeeded, budget refunded"
);
LIVE_SESSION_UNITS.fetch_sub(z.units, std::sync::atomic::Ordering::Relaxed);
false
}
Err(e) if nvenc_status::destroy_proves_no_session(e) => {
tracing::info!(
enc = z.enc,
units = z.units,
status = ?e,
"NVENC parked session gone on the driver side — budget refunded"
);
LIVE_SESSION_UNITS.fetch_sub(z.units, std::sync::atomic::Ordering::Relaxed);
false
}
Err(e) => {
tracing::debug!(enc = z.enc, status = ?e,
"NVENC parked session still refuses destroy — units stay charged");
true
}
},
);
}
/// Whether the operator asked for the two-thread async retrieve (`PUNKTFUNK_NVENC_ASYNC` truthy).
/// Combined with the GPU's `NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT` in `init_session`. Opt-in until
/// on-glass validated; note an async-rejecting config surfaces as a failed session open — unset
@@ -279,12 +386,24 @@ fn async_retrieve_requested() -> bool {
/// the ring textures in place, so in-flight depth beyond the ring lets the capturer overwrite a
/// frame mid-encode: visual corruption, not UB). IDD-push rings are sized around
/// `PUNKTFUNK_IDD_DEPTH`; raise both together if deeper pipelining is needed.
///
/// Read from the environment **once per process** (WP6.1): `submit` consults this on EVERY frame —
/// both arms of the `cap` match, in sync mode too, where the result is then discarded because the
/// backpressure loop short-circuits on `async_rt`. Memoizing rather than latching into a session
/// field is deliberate: the composed `cap` also folds in `input_ring_depth`, which
/// `set_input_ring_depth` may change after open, and freezing that half would reintroduce the
/// in-place-overwrite bug the ring term exists to prevent. Nothing in the workspace mutates this
/// variable at runtime, so memoizing a process-constant read is behaviour-preserving by
/// construction.
fn async_inflight_cap() -> usize {
std::env::var("PUNKTFUNK_NVENC_ASYNC_DEPTH")
.ok()
.and_then(|s| s.parse::<usize>().ok())
.unwrap_or(4)
.clamp(2, POOL - 1)
static CAP: std::sync::OnceLock<usize> = std::sync::OnceLock::new();
*CAP.get_or_init(|| {
std::env::var("PUNKTFUNK_NVENC_ASYNC_DEPTH")
.ok()
.and_then(|s| s.trim().parse::<usize>().ok())
.unwrap_or(4)
.clamp(2, POOL - 1)
})
}
/// One in-flight encode handed to the retrieve thread: the output bitstream to lock once its
@@ -323,7 +442,23 @@ fn retrieve_loop(
done_tx: mpsc::Sender<RetrieveDone>,
) {
pf_frame::thread_qos::boost_thread_priority(false);
// Wedged-vs-routine drain (WP5.2b). Teardown drops the job sender and joins this thread, so
// every queued job is WAITED for (never abandoned — abandoning converts the routine drain
// into destroy/unmap/close-while-encoding). But on a wedged session that used to cost the
// full 5 s PER queued job: up to `cap x 5 s` on the encode thread, and the stall-recovery
// ladder rebuilds the session several times per episode, paying it each rebuild. The wedge
// evidence lives right here — a completion wait that burned its full budget — so after one
// full-budget timeout later jobs get a short slice instead. A first timeout is already
// encoder-fatal (its Err reaches `absorb_done` before any later completion is handed out,
// and the host tears the session down), so short-slicing behind an established wedge changes
// only how long teardown blocks, never an outcome. A successful wait resets the latch, which
// keeps a transient GPU stall from cascading short slices into false timeouts. (NVENC does
// not document completion ordering across in-flight frames; nothing here relies on it —
// FIFO only sharpens the latency bound.)
const WEDGED_DRAIN_WAIT_MS: u32 = 250;
let mut wedged = false;
while let Ok(job) = work_rx.recv() {
let wait_ms = if wedged { WEDGED_DRAIN_WAIT_MS } else { 5000 };
// SAFETY: `job.event` is one of the auto-reset events `init_session` created and
// registered for exactly this session, and `job.bs` one of its pool bitstreams; both stay
// valid until `teardown`, which joins this thread first. `WaitForSingleObject` takes the
@@ -334,9 +469,13 @@ fn retrieve_loop(
// Lock/unlock from a secondary thread while the encode thread submits is the NVENC
// guide's documented threading model.
let result = unsafe {
if WaitForSingleObject(HANDLE(job.event as *mut c_void), 5000) != WAIT_OBJECT_0 {
Err("NVENC completion event timeout (5s) — encoder wedged?".to_string())
if WaitForSingleObject(HANDLE(job.event as *mut c_void), wait_ms) != WAIT_OBJECT_0 {
wedged = true;
Err(format!(
"NVENC completion event timeout ({wait_ms} ms) — encoder wedged?"
))
} else {
wedged = false;
let mut lock = nv::NV_ENC_LOCK_BITSTREAM {
version: nv::NV_ENC_LOCK_BITSTREAM_VER,
outputBitstream: job.bs as *mut c_void,
@@ -387,6 +526,12 @@ pub struct NvencD3d11Encoder {
/// `NV_ENC_CAPS_SUPPORT_YUV444_ENCODE` (cleared in `query_caps` on a card that lacks it) and on an
/// RGB input format (NV12/P010 capture can't reconstruct 4:4:4). HEVC-only.
chroma_444: bool,
/// What the SESSION NEGOTIATED, before any per-init downgrade. `chroma_444` above is the
/// EFFECTIVE value for the session currently open and is cleared whenever the capturer hands us
/// subsampled YUV — which used to overwrite the negotiation itself, so a capturer that went
/// back to RGB (HDR toggle, capture-path switch) could never recover 4:4:4 for the rest of the
/// stream. Keeping the request separate lets every re-init recompute the effective value.
chroma_444_requested: bool,
/// `NV_ENC_CAPS_SUPPORT_YUV444_ENCODE` from the caps probe — whether this GPU can 4:4:4 encode at
/// all. `chroma_444` is forced off when this is false (graceful downgrade to 4:2:0).
yuv444_supported: bool,
@@ -394,6 +539,17 @@ pub struct NvencD3d11Encoder {
/// `ABGR10` input format + the BT.2020/PQ colour VUI. Derived per-frame from the capture format
/// (HDR can toggle mid-session); a change re-inits the session.
hdr: bool,
/// The HDR state the CAPTURE FORMAT asks for, independent of whether this GPU can serve it.
///
/// `hdr` above is the effective value and `query_caps` clears it on a card without 10-bit
/// encode. `submit` re-derives the requested value from every frame's pixel format, so
/// comparing that against the post-downgrade `hdr` made them disagree permanently: a P010
/// capturer on such a GPU reported "HDR changed" on EVERY frame and tore down and rebuilt the
/// whole session each time. The re-init trigger compares against THIS instead.
hdr_requested: bool,
/// Latched when `query_caps` finds no 10-bit encode support, so the downgrade is remembered
/// across re-inits instead of being rediscovered (and re-warned) every session.
hdr_unsupported: bool,
/// The source's static HDR mastering metadata (from the capturer's `GetDesc1`), emitted as
/// in-band SEI (`mastering_display_colour_volume` + `content_light_level_info`) on each keyframe
/// when `hdr`. `None` = unknown → no SEI (the VUI still signals BT.2020 PQ). Set per-frame via
@@ -455,6 +611,11 @@ pub struct NvencD3d11Encoder {
/// device on a desktop switch (normal ↔ Winlogon secure); when a frame carries a new device we
/// tear down and re-init NVENC against it.
init_device: *mut c_void,
/// COM ref pinning that device while the session lives. `init_device` alone pins nothing, and
/// `teardown` releases the texture registrations (the only other refs) BEFORE destroying the
/// session — worse, a failed destroy parks the session handle for a later retry, which must
/// never outlive the device it references. Moved into the [`ParkedSession`] on that path.
init_device_com: Option<ID3D11Device>,
/// The hardware-session units THIS encoder holds against [`LIVE_SESSION_UNITS`] (1 plain, 23
/// under forced split-encode — a split session occupies one session per engine). `0` while no
/// session is open; set by `init_session`, returned by `teardown`.
@@ -504,6 +665,9 @@ impl NvencD3d11Encoder {
bit_depth,
// 4:4:4 is HEVC-only; the GPU-support gate is applied in `query_caps`.
chroma_444: chroma.is_444() && codec == Codec::H265,
chroma_444_requested: chroma.is_444() && codec == Codec::H265,
hdr_requested: false,
hdr_unsupported: false,
yuv444_supported: false,
hdr: false,
hdr_meta: None,
@@ -525,6 +689,7 @@ impl NvencD3d11Encoder {
session_async: false,
last_rfi_range: None,
init_device: ptr::null_mut(),
init_device_com: None,
session_units: 0,
})
}
@@ -571,18 +736,41 @@ impl NvencD3d11Encoder {
for &bs in &self.bitstreams {
let _ = (api().destroy_bitstream_buffer)(self.encoder, bs);
}
// A destroy failure means the driver may still hold this session's slot (the concurrent-
// session cap is per process and only a restart clears a leak) — make it visible instead
// of silently discarding the status.
if let Err(e) = (api().destroy_encoder)(self.encoder).nv_ok() {
tracing::warn!(
status = ?e,
"NVENC destroy_encoder failed at teardown — the driver may have leaked this \
session's slot toward the concurrent-session cap"
);
// Session-budget truth (see LIVE_SESSION_UNITS): refund only on PROOF the driver released
// the slot — a successful destroy, or a failure whose status says the session/device no
// longer exists on the driver side (a TDR reclaims every session with the context). The
// old unconditional refund made the counter drift low on real leaks (over-admitting
// parallel displays); the opposite extreme — a permanent charge on ANY failure — let one
// transient wedge episode poison admission until a host restart. Ambiguous failures
// instead PARK the handle with its units still charged (fail-closed), and
// `reap_parked_sessions` retries the destroy once no session is live, so the charge lasts
// exactly as long as the driver keeps refusing — which is the definition of still-leaked.
let dev_pin = self.init_device_com.take();
match (api().destroy_encoder)(self.encoder).nv_ok() {
Ok(()) => {
LIVE_SESSION_UNITS
.fetch_sub(self.session_units, std::sync::atomic::Ordering::Relaxed);
}
Err(e) if nvenc_status::destroy_proves_no_session(e) => {
tracing::warn!(
status = ?e,
"NVENC destroy_encoder failed, but the status proves the driver holds no \
session (device gone/reset) budget refunded"
);
LIVE_SESSION_UNITS
.fetch_sub(self.session_units, std::sync::atomic::Ordering::Relaxed);
}
Err(e) => {
tracing::warn!(
status = ?e,
units = self.session_units,
"NVENC destroy_encoder failed ambiguously — the driver may still hold this \
session's slot; parking the handle (units stay charged until a reap-destroy \
proves the slot free)"
);
park_session(self.encoder as usize, self.session_units, dev_pin);
}
}
// Return this session's units to the budget (see LIVE_SESSION_UNITS).
LIVE_SESSION_UNITS.fetch_sub(self.session_units, std::sync::atomic::Ordering::Relaxed);
self.session_units = 0;
self.regs.clear(); // drops the texture clones, releasing our refs
self.bitstreams.clear();
@@ -641,6 +829,9 @@ impl NvencD3d11Encoder {
e,
));
}
// The handshake with the kernel-mode driver just succeeded — from here on, an
// `NV_ENC_ERR_INVALID_VERSION` in this process cannot be a driver version skew.
nvenc_status::note_session_opened();
let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX);
let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX);
let ten_bit = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_10BIT_ENCODE);
@@ -668,7 +859,12 @@ impl NvencD3d11Encoder {
}
// Degrade gracefully rather than fail: no 10-bit encode on this card → 8-bit SDR.
if self.bit_depth >= 10 && ten_bit == 0 {
tracing::warn!("NVENC: this GPU can't 10-bit encode — falling back to 8-bit SDR");
if !self.hdr_unsupported {
tracing::warn!("NVENC: this GPU can't 10-bit encode — falling back to 8-bit SDR");
}
// Latched: `submit` compares the frame's REQUESTED hdr against `hdr_requested`, not
// against this cleared value, so a 10-bit capturer no longer re-inits every frame.
self.hdr_unsupported = true;
self.bit_depth = 8;
self.hdr = false;
}
@@ -754,6 +950,27 @@ impl NvencD3d11Encoder {
Ok(cfg)
}
/// This session config's identity in the process-lifetime bitrate-ceiling cache
/// (`nvenc_core::{cached_ceiling, store_ceiling}`). GPU identity is the selected render
/// adapter's LUID — the adapter the capturer's device (and so this session) lives on; `0`
/// when unresolved. Best effort by design: the cache is advisory, a colliding identity costs
/// one failed open + re-search, never a wrong session.
fn ceiling_key(&self, split_mode: u32) -> CeilingKey {
let gpu = pf_gpu::resolve_render_adapter_luid()
.map(|l| ((l.HighPart as u32 as u64) << 32) | l.LowPart as u64)
.unwrap_or(0);
CeilingKey {
gpu,
codec: self.codec,
width: self.width,
height: self.height,
fps: self.fps,
bit_depth: self.bit_depth,
chroma_444: self.chroma_444,
split_mode,
}
}
/// Open + configure + initialize ONE NVENC session at `bitrate` (bps) and `split_mode`. Returns
/// the session handle, or destroys it and returns the error. NVENC has no re-init after a failed
/// `initialize_encoder`, so the bitrate-clamp search in `init_session` calls this once per probe.
@@ -780,6 +997,7 @@ impl NvencD3d11Encoder {
}
return Err(nvenc_status::call_err("open_encode_session_ex", e));
}
nvenc_status::note_session_opened();
let mut cfg = match self.build_config(enc, bitrate) {
Ok(cfg) => cfg,
@@ -812,6 +1030,15 @@ impl NvencD3d11Encoder {
/// Lazily create the session on the first frame's D3D11 device (so capture + encode share it).
fn init_session(&mut self, device: &ID3D11Device) -> Result<()> {
// Serialize this whole open sequence (caps probe + bitrate-clamp search + charge) against
// every other open and against the zombie reap — see DRIVER_SESSION_GATE. Cold path:
// sessions open on a stream start or a device-change rebuild, never per frame.
let _gate = DRIVER_SESSION_GATE
.lock()
.unwrap_or_else(|p| p.into_inner());
// SAFETY: gate held (no open can be handed a recycled zombie address mid-reap) and the
// reap itself re-checks that no live session exists before touching any parked handle.
unsafe { reap_parked_sessions() };
// SAFETY: every call below goes through a function pointer resolved once from the
// runtime-loaded [`EncodeApi`] table (`api()`, gated in `open`), or through this type's own
// `unsafe fn`s whose contract is met here. `query_caps`/`try_open_session` receive `device`,
@@ -834,43 +1061,14 @@ impl NvencD3d11Encoder {
// gets the highest the GPU can actually do, not a coarse fraction of it.
const FLOOR_BPS: u64 = 10_000_000;
let requested_bps = self.bitrate_bps;
// 2-way NVENC split-frame encoding (Ada dual-NVENC) — the high-pixel-rate throughput lever
// the Linux host enables via libavcodec `split_encode_mode`. A single Ada NVENC session tops
// out ~0.8 Gpix/s, so at high motion a 5K@240 (1.77 Gpix/s) frame takes ~8 ms to encode and
// the rate caps ~125 fps; splitting across both engines roughly halves that. Force 2-way
// above ~1 Gpix/s (matching encode/linux.rs), AUTO below (the ~2% BD-rate cost isn't worth
// it at low pixel rates). Env override PUNKTFUNK_SPLIT_ENCODE = 0/disable | 1/auto | 2 | 3.
// HEVC/AV1 only; the init-failure fallback below disables it if a codec/config rejects it.
// 2-way NVENC split-frame encoding (Ada dual-NVENC) — the high-pixel-rate throughput lever.
// A single Ada NVENC session tops out ~0.8-1 Gpix/s, so at high motion a 5K@240
// (1.77 Gpix/s) frame takes ~8 ms to encode and the rate caps ~125 fps; splitting across
// both engines roughly halves that. Shared selector — see [`resolve_split_mode`] for the
// precedence (env override / the measured Main10 don't-split rule / pixel rate).
// The init-failure fallback below disables it if a codec/config rejects it.
let pixel_rate = self.width as u64 * self.height as u64 * self.fps.max(1) as u64;
let mut split_mode: u32 = match std::env::var("PUNKTFUNK_SPLIT_ENCODE").ok().as_deref()
{
Some("0") | Some("disable") => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32
}
Some("1") | Some("auto") => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_AUTO_FORCED_MODE as u32
}
Some("3") => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_THREE_FORCED_MODE as u32,
Some("2") => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
// Main10 (10-bit / HDR): 2-way split is measurably SLOWER on Ada — at 5120x1440@240
// Main10, forced-2 took 7.6 ms/frame (~131 fps) vs 2.8 ms (~357 fps) single-engine
// (the split/merge overhead dominates for 10-bit). A single Ada NVENC engine already
// handles 5K@240 Main10 well under the 4.17 ms budget, so DON'T split — splitting was
// the "broken animations in HDR" (the stream capped at ~131 fps). Env still overrides.
_ if self.bit_depth >= 10 => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32
}
_ if pixel_rate > 1_000_000_000 => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_TWO_FORCED_MODE as u32
}
_ => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_AUTO_MODE as u32,
};
tracing::debug!(
split_mode,
bit_depth = self.bit_depth,
pixel_rate,
"NVENC split-encode mode selected"
);
let split_mode: u32 = resolve_split_mode(self.bit_depth, pixel_rate);
// Find the highest bitrate the GPU's codec LEVEL accepts and CLAMP to it. NVENC rejects
// `initialize_encoder` (InvalidParam) when the bitrate exceeds the level ceiling (e.g. a
// 1 Gbps request on HEVC). Strategy: try the requested rate; if the only problem is a forced
@@ -884,39 +1082,69 @@ impl NvencD3d11Encoder {
// built in the right mode from the start.
let use_async = self.async_supported && async_retrieve_requested();
let mut probe = self.try_open_session(device, requested_bps, split_mode, use_async);
// Ceiling cache (process lifetime, `nvenc_core`): a prior clamp search already found
// this config's max accepted rate — open straight AT the ceiling instead of paying
// the ~6-open binary search (and its session churn) on every ABR overshoot.
let mut target_bps = requested_bps;
if let Some(ceiling) = cached_ceiling(&self.ceiling_key(split_mode)) {
if requested_bps > ceiling {
tracing::info!(
requested_mbps = requested_bps / 1_000_000,
ceiling_mbps = ceiling / 1_000_000,
"NVENC: requested bitrate above the cached codec-level ceiling — opening \
at the ceiling"
);
target_bps = ceiling;
}
}
let mut probe = self.try_open_session(device, target_bps, split_mode, use_async);
// The cache is advisory: a stale entry (driver change, identity collision) must not
// wedge the open — retry the requested rate and let the search below rediscover.
if probe.is_err() && target_bps < requested_bps {
target_bps = requested_bps;
probe = self.try_open_session(device, requested_bps, split_mode, use_async);
}
// Disambiguate a forced-split rejection from a bitrate-cap rejection: retry once at the
// requested rate with split disabled — if THAT succeeds, split was the problem, not bitrate.
// ANY non-disabled mode can be the rejection — AUTO included: AV1 rejects the whole
// init with INVALID_PARAM on drivers/configs where auto split isn't valid for it,
// which then masqueraded as a bitrate cap and failed "even at the floor".
// `used_split` tracks the mode sessions ACTUALLY open with from here on — it feeds
// `self.split_mode` (a reconfigure must re-present it) and the ceiling-cache key.
let mut used_split = split_mode;
let split_on =
split_mode != nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
if probe.is_err() && split_on {
let no_split = nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
if let Ok(e) = self.try_open_session(device, requested_bps, no_split, use_async) {
if let Ok(e) = self.try_open_session(device, target_bps, no_split, use_async) {
tracing::warn!("NVENC: split-encode rejected by codec/config — disabled");
split_mode = no_split;
used_split = no_split;
probe = Ok(e);
}
}
let enc = match probe {
Ok(enc) => {
self.bitrate_bps = requested_bps;
self.bitrate_bps = target_bps;
enc
}
// Only a parameter/caps rejection means "the bitrate is above the codec-level
// ceiling". A transient failure (busy engine, session limit, OOM, device loss,
// version skew) must propagate — a search steered by it would discover, and
// cache, a bogus ceiling.
Err(e) if !nvenc_status::is_param_rejection(&e) => return Err(e),
Err(_) => {
// Requested bitrate exceeds the codec-level ceiling — binary-search the max accepted.
// `lo` is the highest known-good rate (FLOOR is assumed to fit), `hi` the lowest
// rejected; `best` holds the live session at `lo` so we end up with the clamped one.
let mut lo = FLOOR_BPS;
let mut hi = requested_bps;
let mut hi = target_bps;
let mut best: *mut c_void = ptr::null_mut();
let mut best_bps = 0u64;
while hi > lo + CLAMP_TOL_BPS {
let mid = lo + (hi - lo) / 2;
match self.try_open_session(device, mid, split_mode, use_async) {
match self.try_open_session(device, mid, used_split, use_async) {
Ok(e) => {
if !best.is_null() {
let _ = (api().destroy_encoder)(best);
@@ -925,7 +1153,15 @@ impl NvencD3d11Encoder {
best_bps = mid;
lo = mid;
}
Err(_) => hi = mid,
Err(e) if nvenc_status::is_param_rejection(&e) => hi = mid,
Err(e) => {
// Environmental mid-search failure: don't let it shrink the
// search — release the partial result and propagate.
if !best.is_null() {
let _ = (api().destroy_encoder)(best);
}
return Err(e);
}
}
}
if best.is_null() {
@@ -933,14 +1169,19 @@ impl NvencD3d11Encoder {
// trying split-disabled in case a forced split (not the bitrate) is the blocker.
let no_split =
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
best = self
.try_open_session(device, FLOOR_BPS, split_mode, use_async)
.or_else(|_| {
self.try_open_session(device, FLOOR_BPS, no_split, use_async)
})
.context(
"NVENC initialize_encoder rejected even at the floor bitrate",
)?;
best = match self.try_open_session(device, FLOOR_BPS, used_split, use_async)
{
Ok(e) => e,
Err(_) => {
let e = self
.try_open_session(device, FLOOR_BPS, no_split, use_async)
.context(
"NVENC initialize_encoder rejected even at the floor bitrate",
)?;
used_split = no_split;
e
}
};
best_bps = FLOOR_BPS;
}
tracing::warn!(
@@ -948,21 +1189,22 @@ impl NvencD3d11Encoder {
clamped_mbps = best_bps / 1_000_000,
"NVENC: requested bitrate above the GPU codec-level ceiling — clamped to the max accepted"
);
store_ceiling(self.ceiling_key(used_split), best_bps);
self.bitrate_bps = best_bps;
best
}
};
self.encoder = enc;
// Session init params a later `reconfigure_bitrate` must re-present verbatim. (Best
// effort: the floor fallback above may have succeeded split-disabled without updating
// `split_mode` — a reconfigure then presents the forced mode, NVENC rejects it, and
// the caller's rebuild fallback covers the mismatch.)
self.split_mode = split_mode;
// Pin the device for the session's lifetime — and for a parked afterlife if the
// eventual destroy fails (see `ParkedSession::_device`).
self.init_device_com = Some(device.clone());
// Session init params a later `reconfigure_bitrate` must re-present verbatim.
self.split_mode = used_split;
self.session_async = use_async;
// Session-budget accounting (Stage W3): record what this open holds so admission can
// decline a parallel display the hardware can't afford. Weighted by the FINAL split
// mode (a split session occupies one hardware session per engine).
self.session_units = split_mode_units(split_mode);
self.session_units = split_mode_units(used_split);
LIVE_SESSION_UNITS.fetch_add(self.session_units, std::sync::atomic::Ordering::Relaxed);
// (The clamp path above already logs the requested→clamped bitrate at warn; no second
// info line for the same event here.)
@@ -986,6 +1228,12 @@ impl NvencD3d11Encoder {
for _ in 0..POOL {
let ev = CreateEventW(None, false, false, PCWSTR::null())
.context("CreateEvent (NVENC completion)")?;
// Push BEFORE registering: a failed registration propagates out of
// `init_session` into submit's teardown call, and only handles that made it
// into `self.events` get closed there — pushing after the fallible call
// leaked the Win32 event on every registration failure. Teardown's
// unregister of the one never-registered event is a harmless error return.
self.events.push(ev.0 as usize);
let mut ep = nv::NV_ENC_EVENT_PARAMS {
version: nv::NV_ENC_EVENT_PARAMS_VER,
completionEvent: ev.0,
@@ -994,7 +1242,6 @@ impl NvencD3d11Encoder {
(api().register_async_event)(enc, &mut ep)
.nv_ok()
.map_err(|e| nvenc_status::call_err("register_async_event", e))?;
self.events.push(ev.0 as usize);
}
let (work_tx, work_rx) = mpsc::sync_channel::<RetrieveJob>(POOL);
let (done_tx, done_rx) = mpsc::channel::<RetrieveDone>();
@@ -1086,7 +1333,11 @@ impl Encoder for NvencD3d11Encoder {
let dev_raw = frame.device.as_raw();
let size_changed =
self.inited && (self.width != captured.width || self.height != captured.height);
let hdr_changed = self.inited && self.hdr != hdr;
// Compare against what was REQUESTED last init, not the effective `self.hdr`: on a GPU
// without 10-bit encode `query_caps` clears `self.hdr`, so comparing it against a P010
// capturer's perpetual `hdr = true` reported a change on every single frame and tore the
// session down and rebuilt it each time.
let hdr_changed = self.inited && self.hdr_requested != hdr;
if self.inited && (self.init_device != dev_raw || size_changed || hdr_changed) {
tracing::info!(
device_changed = self.init_device != dev_raw,
@@ -1107,7 +1358,13 @@ impl Encoder for NvencD3d11Encoder {
// Adopt the current frame size + colour so the encoder always matches the capturer output.
self.width = captured.width;
self.height = captured.height;
self.hdr_requested = hdr;
// Effective until `query_caps` says otherwise (it clears this on a card without 10-bit).
self.hdr = hdr;
// Recompute the EFFECTIVE 4:4:4 from the negotiation on every init. This used to read
// (and overwrite) `chroma_444` itself, so the first subsampled-YUV frame permanently
// demoted the session — 4:4:4 never returned even when the capturer went back to RGB.
self.chroma_444 = self.chroma_444_requested;
// Pick the NVENC input format from the captured pixel format. YUV (NV12/P010) is the
// video-processor path — NVENC encodes it natively (no internal RGB→YUV, which is a hidden
// 3D/compute step that would fight a GPU-saturating game). RGB (ARGB/ABGR10) is the legacy
@@ -1133,9 +1390,11 @@ impl Encoder for NvencD3d11Encoder {
};
// 4:4:4 honesty: the FREXT/chromaFormatIDC=3 config engages only on an RGB input (a
// subsampled NV12/P010 source can't reconstruct full chroma). If the capturer handed
// native YUV despite a 4:4:4 negotiation, this session encodes 4:2:0 — clear the flag
// NOW so `caps().chroma_444` (and native's post-open cross-check) reports what
// the stream really carries instead of silently claiming full chroma.
// native YUV despite a 4:4:4 negotiation, THIS session encodes 4:2:0 — clear the
// effective flag now so `caps().chroma_444` (and native's post-open cross-check)
// reports what the stream really carries instead of silently claiming full chroma.
// Only the effective value is cleared: `chroma_444_requested` keeps the negotiation, so
// a later re-init on an RGB capture recovers 4:4:4 instead of being stuck at 4:2:0.
if self.chroma_444
&& !matches!(
self.buffer_fmt,
@@ -1181,9 +1440,23 @@ impl Encoder for NvencD3d11Encoder {
// despite this comment previously claiming otherwise. Since this backend encodes the
// capturer's textures in place, exceeding the capturer's declared `pipeline_depth` lets it
// rotate a texture out from under a live encode — torn frames, silently.
// FAIL SAFE when nobody told us. `set_input_ring_depth` is a defaulted trait method, so a
// caller that forgets it fails SILENTLY — and one did: the GameStream loop opens an encoder
// (`gamestream/stream.rs`) and never calls it, while the Windows IDD-push capturer declares a
// ring of 2 and `async_inflight_cap()` defaults to 4. That let a Moonlight session pipeline
// four encodes against a two-texture ring, i.e. exactly the in-place overwrite this cap
// exists to prevent, as torn/mixed frames and never an error.
//
// Guarding at the single point of CONSUMPTION rather than at each call site is deliberate:
// it covers every present and future caller, including ones that have not been written yet,
// whereas plumbing the setter into N loops only fixes the N sites someone remembered. An
// unknown ring is treated as the shallowest one any capturer in this tree declares, so the
// unconfigured path degrades to less pipelining — a latency cost, not corruption. Callers
// that DO configure it are unaffected.
const UNCONFIGURED_RING_DEPTH: usize = 2;
let cap = match self.input_ring_depth {
Some(d) => async_inflight_cap().min(d.max(1)),
None => async_inflight_cap(),
None => async_inflight_cap().min(UNCONFIGURED_RING_DEPTH),
};
while self.async_rt.is_some() && self.pending.len() >= cap {
let done = {
@@ -1380,6 +1653,8 @@ impl Encoder for NvencD3d11Encoder {
// RFI is probed once at open (`rfi_supported`); HDR SEI rides keyframes whenever the
// session is in HDR mode. Both are the real capabilities the session glue routes on.
EncoderCaps {
// The Windows capture path composites the pointer; this backend never reads `frame.cursor`.
blends_cursor: false,
supports_rfi: self.rfi_supported,
// In-band mastering/CLL is attached as keyframe SEI on HEVC/H.264 only — AV1 carries
// it in METADATA OBUs (`HDR_MDCV`/`HDR_CLL`), which this backend doesn't emit yet
@@ -1404,61 +1679,56 @@ impl Encoder for NvencD3d11Encoder {
}
fn invalidate_ref_frames(&mut self, first: i64, last: i64) -> bool {
// No live session, the GPU can't invalidate, or a nonsense range → caller forces a full IDR.
// (NVENC handles are single-threaded; this runs on the encode thread, like submit/poll.)
if self.encoder.is_null() || !self.rfi_supported || first < 0 || first > last {
// No live session or the GPU can't invalidate → caller forces a full IDR. (NVENC handles
// are single-threaded; this runs on the encode thread, like submit/poll.) Everything else
// — range validity, covering-range dedup, the DPB window, the clamp — is
// `nvenc_core::plan_range_recovery`, one policy for both direct-NVENC backends.
if self.encoder.is_null() || !self.rfi_supported {
return false;
}
// Already invalidated a covering range for this loss event — no new driver calls needed,
// no IDR. RE-ARM the anchor though: the client re-asking means the previous recovery
// anchor AU may itself have been lost, and the next frame is just as clean a re-anchor
// (it too references only valid frames).
if let Some((pf, pl)) = self.last_rfi_range {
if first >= pf && last <= pl {
match plan_range_recovery(first, last, self.frame_idx, self.last_rfi_range) {
// Already invalidated a covering range for this loss event — no new driver calls
// needed, no IDR. RE-ARM the anchor though: the client re-asking means the previous
// recovery anchor AU may itself have been lost, and the next frame is just as clean a
// re-anchor (it too references only valid frames).
RangePlan::Covered => {
self.pending_anchor = true;
return true;
true
}
}
// `frame_idx` is the NEXT timestamp to assign, so the last encoded frame is `frame_idx - 1`
// and the DPB holds `[frame_idx - RFI_DPB, frame_idx - 1]`. A lost frame older than that
// can't be invalidated, so the only correct recovery is an IDR.
let oldest_in_dpb = self.frame_idx - RFI_DPB as i64;
if first < oldest_in_dpb {
return false;
}
// Clamp to frames we've actually encoded (don't invalidate a timestamp we never assigned).
let last = last.min(self.frame_idx - 1);
if first > last {
return false;
}
// Each input's `inputTimeStamp` is `frame_idx`, which `submit_indexed` pins to the WIRE
// frame index the AU carries — so the client's lost-frame range maps 1:1 onto the
// timestamps NVENC invalidates here, and stays 1:1 across encoder rebuilds/resets (an
// internal counter would desync on the first adaptive-bitrate rebuild and RFI would then
// clamp every range into first > last, silently degrading to IDR-only forever).
// SAFETY: `invalidate_ref_frames` is a function pointer from the runtime-loaded `EncodeApi` table.
// `self.encoder` was checked non-null at the top of this fn and is the live session; this runs
// on the encode thread (like submit/poll), so there is no concurrent NVENC use. Each `ts` was
// clamped to `[oldest_in_dpb, frame_idx - 1]` above, so it names a frame still in the session's
// DPB; the call passes only that `u64` timestamp (no struct), so there is no struct-size or
// lifetime concern.
unsafe {
for ts in first..=last {
if (api().invalidate_ref_frames)(self.encoder, ts as u64)
.nv_ok()
.is_err()
{
return false; // any failure → fall back to IDR
RangePlan::Decline => false,
RangePlan::Invalidate { first, last } => {
// Each input's `inputTimeStamp` is `frame_idx`, which `submit_indexed` pins to the
// WIRE frame index the AU carries — so the client's lost-frame range maps 1:1 onto
// the timestamps NVENC invalidates here, and stays 1:1 across encoder
// rebuilds/resets (an internal counter would desync on the first adaptive-bitrate
// rebuild and RFI would then clamp every range into first > last, silently
// degrading to IDR-only forever).
// SAFETY: `invalidate_ref_frames` is a function pointer from the runtime-loaded
// `EncodeApi` table. `self.encoder` was checked non-null at the top of this fn and
// is the live session; this runs on the encode thread (like submit/poll), so there
// is no concurrent NVENC use. The plan clamped each `ts` to
// `[oldest_in_dpb, frame_idx - 1]`, so it names a frame still in the session's
// DPB; the call passes only that `u64` timestamp (no struct), so there is no
// struct-size or lifetime concern.
unsafe {
for ts in first..=last {
if (api().invalidate_ref_frames)(self.encoder, ts as u64)
.nv_ok()
.is_err()
{
return false; // any failure → fall back to IDR
}
}
}
self.last_rfi_range = Some((first, last));
// The next submitted frame is the first one encoded after the invalidation — the
// clean re-anchor P-frame. Arm the tag so its AU ships with `recovery_anchor` and
// the client lifts its post-loss freeze on it (instead of waiting ~1 s for the
// cooldown-suppressed IDR fallback).
self.pending_anchor = true;
true
}
}
self.last_rfi_range = Some((first, last));
// The next submitted frame is the first one encoded after the invalidation — the clean
// re-anchor P-frame. Arm the tag so its AU ships with `recovery_anchor` and the client
// lifts its post-loss freeze on it (instead of waiting ~1 s for the cooldown-suppressed
// IDR fallback).
self.pending_anchor = true;
true
}
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
@@ -1552,6 +1822,15 @@ impl Encoder for NvencD3d11Encoder {
self.bitrate_bps = bps;
return true;
}
// Cached codec-level ceiling: clamp the target BEFORE the driver call, so a known
// overshoot retargets to the ceiling IN PLACE instead of bouncing off the driver into
// the caller's full-rebuild fallback (an IDR plus ~half a second of session churn per
// ABR overshoot on the pre-cache path). The caller reads the clamp back through
// [`Encoder::applied_bitrate_bps`].
let bps = match cached_ceiling(&self.ceiling_key(self.split_mode)) {
Some(ceiling) => bps.min(ceiling),
None => bps,
};
// SAFETY: `inited` ⟹ `self.encoder` is the live session and this runs on the encode
// thread between submit/poll (`nvEncReconfigureEncoder` is a submit-side call, the
// sanctioned side of the two-thread async split — the retrieve thread only ever locks
@@ -1600,6 +1879,12 @@ impl Encoder for NvencD3d11Encoder {
}
}
fn applied_bitrate_bps(&self) -> Option<u64> {
// `bitrate_bps` is the post-clamp truth: the open path's ceiling search and the
// reconfigure path's cache clamp both write what the session ACTUALLY targets.
Some(self.bitrate_bps)
}
fn flush(&mut self) -> Result<()> {
Ok(()) // P1/ULL + frameIntervalP=1: each submit yields its AU; no internal queue to drain.
}
@@ -1645,6 +1930,11 @@ pub fn probe_can_encode_10bit(codec: Codec) -> bool {
/// on any failure (no loadable NVENC, no device, failed open) — the honest answer for a
/// capability that couldn't be confirmed.
fn probe_encode_cap(codec: Codec, cap: nv::NV_ENC_CAPS) -> bool {
// Same exclusion as `init_session`: this opens a real (throwaway) session, so it must never
// overlap a zombie reap that could be destroying the very address the driver hands us.
let _gate = DRIVER_SESSION_GATE
.lock()
.unwrap_or_else(|p| p.into_inner());
use windows::Win32::Foundation::HMODULE;
use windows::Win32::Graphics::Direct3D::{
D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_UNKNOWN, D3D_FEATURE_LEVEL_11_0,
@@ -1723,6 +2013,9 @@ fn probe_encode_cap(codec: Codec, cap: nv::NV_ENC_CAPS) -> bool {
}
return false;
}
// Availability probe, but a real session open all the same: it proves the driver accepted
// this build's version word, which is what rules a skew out later (see `nvenc_status`).
nvenc_status::note_session_opened();
let mut param = nv::NV_ENC_CAPS_PARAM {
version: nv::NV_ENC_CAPS_PARAM_VER,
capsToQuery: cap,
+22 -9
View File
@@ -189,12 +189,15 @@ impl PyroWaveEncoder {
if !chroma444 && (width % 2 != 0 || height % 2 != 0) {
bail!("pyrowave 4:2:0 needs even dimensions (got {width}x{height})");
}
if chroma444 && !crate::pyrowave_mode_fits_rdo(width, height, true) {
// The negotiator downgrades these modes to 4:2:0 pre-Welcome; refuse if one
// slips through (e.g. the lab override) rather than wrap the RDO block index.
// Checked against the chroma actually being opened, NOT hardcoded 4:4:4 — see the Linux
// twin (`enc/linux/pyrowave.rs`) for the full rationale: the negotiator's 4:4:4 → 4:2:0
// downgrade hands oversized modes to this open AS 4:2:0, so the old `chroma444`-gated
// check was skipped exactly when it was needed.
if !crate::pyrowave_mode_fits_rdo(width, height, chroma444) {
bail!(
"pyrowave 4:4:4 at {width}x{height} exceeds the rate controller's 16-bit \
block index (see pyrowave-sys patches/0002 note) use 4:2:0 at this size"
"pyrowave {} at {width}x{height} exceeds the rate controller's 16-bit block \
index (see pyrowave-sys patches/0002 note) lower the resolution",
if chroma444 { "4:4:4" } else { "4:2:0" }
);
}
let fps = fps.max(1);
@@ -294,8 +297,10 @@ impl PyroWaveEncoder {
/// Import one capturer plane D3D11 texture (`R8_UNORM` Y or `R8G8_UNORM` CbCr) into pyrowave's
/// Vulkan device. Creates a fresh shared NT handle from the texture (the capturer marked the ring
/// `SHARED | SHARED_NTHANDLE`); `pyrowave_image_create` takes ownership of the handle and closes
/// it on import. Single/two-component textures import reliably on NVIDIA at any size — unlike a
/// planar NV12 — so no MUTABLE_FORMAT / planar-layout workaround is involved.
/// it on SUCCESSFUL import only (pyrowave-sys patch 0006 — same contract as the fence import in
/// [`Self::import_fence`]), so this fn closes it on every failure return. Single/two-component
/// textures import reliably on NVIDIA at any size — unlike a planar NV12 — so no
/// MUTABLE_FORMAT / planar-layout workaround is involved.
///
/// # Safety
/// `texture` must be a live `ID3D11Texture2D` of format `vk_format`, sized `w`×`h`, created
@@ -346,7 +351,11 @@ impl PyroWaveEncoder {
};
let mut image: pw::pyrowave_image = std::ptr::null_mut();
if let Err(e) = pw_check(pw::pyrowave_image_create(&info, &mut image), "image_create") {
// pyrowave only closes the handle on a SUCCESSFUL import — close it ourselves on failure.
// pyrowave consumes the handle ONLY on a successful import (pyrowave-sys patch 0006
// pinned this at the API's success boundary) — so on EVERY failure return the handle
// is still ours and this close is the single one. Before the patch, an
// allocate-stage failure inside Granite had already closed it and this was a double
// close of a possibly-recycled handle value.
let _ = CloseHandle(handle);
return Err(e);
}
@@ -416,7 +425,9 @@ impl PyroWaveEncoder {
pw::pyrowave_sync_object_create(&info, &mut sync),
"sync_object_create",
) {
// pyrowave only closes the handle on a SUCCESSFUL import — close the dup on failure.
// pyrowave only closes the handle on a SUCCESSFUL import (Granite's semaphore import
// has always had these semantics; patch 0006 made the image import match) — close
// the dup on failure.
let _ = CloseHandle(dup);
return Err(e);
}
@@ -678,6 +689,8 @@ impl Encoder for PyroWaveEncoder {
// after the caps() default was written — a hardcoded `default()` here mis-reports a 4:4:4
// open as 4:2:0 and fires a spurious "chroma disagrees with the negotiated Welcome" warn).
EncoderCaps {
// The Windows capturer composites the pointer itself; this backend never reads it.
blends_cursor: false,
chroma_444: self.chroma444,
..EncoderCaps::default()
}
+137 -37
View File
@@ -519,9 +519,20 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
}
b.WhitePointX = m.white_point[0];
b.WhitePointY = m.white_point[1];
// Units diverge on the max: VPL wants whole cd/m² (HdrMeta carries 0.0001 cd/m²); the
// min is 0.0001 cd/m² on both sides.
b.MaxDisplayMasteringLuminance = m.max_display_mastering_luminance / 10_000;
// BOTH luminance fields are 0.0001 cd/m² here — do NOT scale the max.
//
// The `/10_000` this used to carry followed the VPL header's *video-processing* unit
// (whole cd/m², which is right for VPP), but on the ENCODE path the runtime passes these
// straight into the ITU-T H.265 Annex D mastering-display SEI, whose unit is 0.0001 cd/m².
// Dividing therefore under-reported peak brightness by 10,000x. Confirmed in a real
// bitstream on Intel UHD 750 (2026-07-25): SEI 137 carried
// max_display_mastering_luminance = 1000, i.e. **0.1 nits** for a 1000-nit display, while
// the undivided min = 500 (0.05 nits) was correct — the asymmetry was the tell. A client
// tone-mapping against 0.1 nits crushes the image.
//
// Unscaled also matches every sibling: `MinDisplayMasteringLuminance` right below,
// `pf_frame::hdr::hevc_mastering_display_sei`, and the AMF backend.
b.MaxDisplayMasteringLuminance = m.max_display_mastering_luminance;
b.MinDisplayMasteringLuminance = m.min_display_mastering_luminance;
b
});
@@ -690,13 +701,17 @@ impl Inner {
}
fn take_bs(&mut self) -> Box<BsBuf> {
match self.bs_pool.pop() {
Some(mut b) => {
// Pooled buffers were sized by whatever `bs_bytes` was when they were allocated. A bitrate
// retarget raises the driver's worst-case AU size, so a recycled buffer can be SMALLER than
// the current requirement — hand those back to the allocator instead of letting the runtime
// write an AU into a short buffer.
while let Some(mut b) = self.bs_pool.pop() {
if b.mfx.MaxLength as usize >= self.bs_bytes {
b.recycle();
b
return b;
}
None => BsBuf::new(self.bs_bytes),
}
BsBuf::new(self.bs_bytes)
}
}
@@ -774,7 +789,12 @@ impl QsvEncoder {
if codec == Codec::Av1 && !probe_can_encode(Codec::Av1) {
bail!("this GPU/driver declined AV1 encode (DG2/Arc or MTL+ required) — QSV probe");
}
let ten_bit = bit_depth >= 10 || matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2);
// Depth follows the delivered pixels, not the negotiated depth ([`crate::ten_bit_input`]).
// With the old `bit_depth >= 10 || …` shape a 10-bit-negotiated session over an 8-bit
// capture derived `expected = P010` and hit the bail below, ending the session at open —
// and taking the ffmpeg fallback with it, since that path had the same defect in a worse
// form (it accepted the open and then failed every frame).
let ten_bit = crate::ten_bit_input(format, bit_depth);
if ten_bit && codec == Codec::H264 {
bail!("native QSV: 10-bit is HEVC/AV1-only (H.264 High10 is not negotiated)");
}
@@ -878,7 +898,8 @@ impl QsvEncoder {
if sts > vpl::MFX_ERR_NONE {
tracing::debug!(status = sts_name(sts), "QSV Init returned a warning");
}
let ir_active = cfg.intra_refresh && set.co2.is_some();
// Whether we ASKED for intra-refresh. Not the same as getting it — confirmed below.
let ir_requested = cfg.intra_refresh && set.co2.is_some();
// The driver's own answer for the worst-case AU size.
// SAFETY: `session` is live; `got` and its (empty) ext chain outlive the call.
let bs_bytes = unsafe {
@@ -892,6 +913,57 @@ impl QsvEncoder {
let kb = enc_of(m).BufferSizeInKB as usize;
(kb * mult * 1000).max(256 * 1024)
};
// Intra-refresh HONESTY: ask the driver what it actually installed instead of trusting that
// it took what we sent. Both `Query` and `Init` can return MFX_WRN_INCOMPATIBLE_VIDEO_PARAM
// — a WARNING, which the code above accepts — while silently dropping the wave. Confirmed
// on Intel UHD 750 (2026-07-25): H.264 reports exactly that, `GetVideoParam` comes back with
// `IntRefType = 0`, and `ir_active` still claimed true. H.265 on the same GPU is genuinely
// active, so this is per-codec and cannot be decided statically.
//
// That lie is not cosmetic: `ir_active` feeds `EncoderCaps::intra_refresh`, so the session
// advertises gradual refresh to the client and then never emits it — the client stops
// requesting the IDRs it would otherwise have asked for on loss, and a lost frame is
// concealed instead of repaired.
//
// Deliberately a SEPARATE, best-effort query rather than a buffer chained onto the
// `BufferSizeInKB` call above (which is what the audit finding suggested): that value is
// load-bearing for every bitstream allocation, and a runtime that dislikes the chained
// buffer would take it down with the readback. A failure here costs only the verdict, and
// is resolved conservatively.
let ir_active = if ir_requested {
// SAFETY: `session` is live on this thread; `got` and `co2_out` (with `ptrs` holding the
// only reference to it) all outlive the synchronous call, and the runtime writes back
// only into the buffer whose header we stamped.
let confirmed = unsafe {
let mut got: vpl::mfxVideoParam = std::mem::zeroed();
let mut co2_out: vpl::mfxExtCodingOption2 = std::mem::zeroed();
co2_out.Header.BufferId = vpl::MFX_EXTBUFF_CODING_OPTION2 as u32;
co2_out.Header.BufferSz = std::mem::size_of::<vpl::mfxExtCodingOption2>() as u32;
let mut ptrs: [*mut vpl::mfxExtBuffer; 1] = [&mut co2_out.Header as *mut _];
got.ExtParam = ptrs.as_mut_ptr();
got.NumExtParam = 1;
let sts = vpl::MFXVideoENCODE_GetVideoParam(session, &mut got);
if sts < vpl::MFX_ERR_NONE {
tracing::debug!(
status = sts_name(sts),
"QSV: could not read back CodingOption2 — trusting the intra-refresh request"
);
true
} else {
co2_out.IntRefType != 0
}
};
if !confirmed {
tracing::warn!(
codec = ?cfg.codec,
"QSV silently dropped intra-refresh (GetVideoParam reports IntRefType=0) — \
advertising it OFF so the client keeps asking for IDRs on loss"
);
}
confirmed
} else {
false
};
Ok((ltr_active, ir_active, bs_bytes))
}
@@ -1344,36 +1416,30 @@ impl Encoder for QsvEncoder {
if !self.ltr_active || first < 0 || first > last {
return false;
}
// Taint sweep BEFORE picking the anchor: an LTR marked at-or-after the loss start was
// encoded inside the client's corrupt window — the client either never received it or
// decoded it against a broken reference chain. Serving it as "known-good" on a LATER
// loss ships corruption as the recovery anchor, and every subsequent mark re-samples
// the soup — the sustained-loss field failure where the picture never healed. Dropped
// slots stay dropped; the cadence re-marks a clean frame within ~1/4 s.
//
// Mark tainted rather than clearing: `ltr_slots` mirrors the HARDWARE DPB, and nulling an
// entry issues no VPL call — the frame stays marked long-term in the encoder. Clearing it
// made the rejection list below (which iterates the post-sweep mirror and only names `Some`
// slots) silently SKIP the one entry the sweep exists to distrust, so the recovery frame
// could still predict from it. With two slots the "exactly one swept" case is the modal
// one, and it was the broken one.
for (slot, marked) in self.ltr_slots.iter().enumerate() {
if marked.is_some_and(|idx| idx >= first) {
self.ltr_tainted[slot] = true;
// The taint-sweep + anchor-pick POLICY lives in `rfi::plan_slot_recovery` (one decision
// shared with AMF and Vulkan Video). This backend's mechanism: distrust is a SEPARATE
// `ltr_tainted` flag, never a cleared mirror slot — `ltr_slots` mirrors the HARDWARE DPB,
// and nulling an entry issues no VPL call, so the frame stays marked long-term in the
// encoder. Clearing it made the rejection list below (which iterates the mirror and only
// names `Some` slots) silently SKIP the one entry the sweep exists to distrust, so the
// recovery frame could still predict from it. With two slots the "exactly one swept" case
// is the modal one, and it was the broken one. The `!ltr_tainted` view filter below is
// what persists that distrust across loss events (this call's taints are excluded from
// this call's anchor by the policy itself).
let view: Vec<(usize, i64)> = self
.ltr_slots
.iter()
.enumerate()
.filter(|&(slot, _)| !self.ltr_tainted[slot])
.filter_map(|(s, m)| m.map(|w| (s, w)))
.collect();
let plan = super::rfi::plan_slot_recovery(&view, first);
for (slot, tainted) in self.ltr_tainted.iter_mut().enumerate() {
if plan.tainted & (1 << slot) != 0 {
*tainted = true;
}
}
let mut best: Option<(usize, i64)> = None;
for (slot, marked) in self.ltr_slots.iter().enumerate() {
if self.ltr_tainted[slot] {
continue; // still in the DPB, but encoded inside the corrupt window
}
if let Some(idx) = *marked {
if idx < first && best.is_none_or(|(_, b)| idx > b) {
best = Some((slot, idx));
}
}
}
match best {
match plan.anchor {
Some((slot, ltr_frame)) => {
self.pending_force = Some(slot);
tracing::info!(
@@ -1402,6 +1468,8 @@ impl Encoder for QsvEncoder {
fn caps(&self) -> EncoderCaps {
EncoderCaps {
// As Windows NVENC: the capturer composites; this backend never reads `frame.cursor`.
blends_cursor: false,
supports_rfi: self.ltr_active,
// In-band mastering/CLL at IDR (HEVC prefix SEI / AV1 metadata OBU); AVC sessions
// are never HDR.
@@ -1576,6 +1644,38 @@ impl Encoder for QsvEncoder {
self.bitrate_bps = old;
return false;
}
// Re-read the driver's worst-case AU size. `Reset` re-derives `BufferSizeInKB` from the new
// rate, and a step-up can raise it well above what `init_encode` computed — leaving
// `bs_bytes` (and every buffer already in the pool) sized for the OLD, lower bitrate. This
// mirrors `init_encode`, which asks the same question post-Init; `take_bs` drops any pooled
// buffer that is now too small.
// SAFETY: `session` is live on this thread and drained above; `got` and its (empty) ext
// chain outlive the synchronous call.
let refreshed = unsafe {
let mut got: vpl::mfxVideoParam = std::mem::zeroed();
let sts = vpl::MFXVideoENCODE_GetVideoParam(session, &mut got);
(sts >= vpl::MFX_ERR_NONE).then(|| {
let m = &mut got.__bindgen_anon_1.mfx;
let mult = m.BRCParamMultiplier.max(1) as usize;
let kb = enc_of(m).BufferSizeInKB as usize;
(kb * mult * 1000).max(256 * 1024)
})
};
match (refreshed, self.inner.as_mut()) {
(Some(bytes), Some(inner)) if bytes > inner.bs_bytes => {
tracing::debug!(
old_bytes = inner.bs_bytes,
new_bytes = bytes,
mbps = bps / 1_000_000,
"QSV retarget raised the worst-case AU size — resizing the bitstream pool"
);
inner.bs_bytes = bytes;
}
(None, _) => tracing::warn!(
"QSV retarget: GetVideoParam failed, keeping the previous AU buffer size"
),
_ => {}
}
true
}
+616 -234
View File
@@ -5,9 +5,11 @@
//! VA surface). One [`Encoder`] trait, selected in [`open_video`]. Extracted into a subsystem crate
//! (plan §W6): depends on the shared frame vocabulary (`pf-frame`) + zero-copy plumbing
//! (`pf-zerocopy`), never on capture — the capture→encode edge is one-way.
// Scaffold: some backend paths + trait defaults are defined ahead of the per-feature build that
// uses them (mirrors the host crate root's allow before the extraction).
#![allow(dead_code)]
// NOTE: no crate-wide `#![allow(dead_code)]`. It was inherited from the pre-extraction host crate
// root as scaffolding for backend paths defined ahead of the build that used them, but a census
// across every feature combination on both platforms found it was hiding exactly two items — so it
// bought nothing and blinded the crate to future rot. Genuinely test-only helpers carry
// `#[cfg(test)]` instead.
// Every unsafe block in this module tree carries a `// SAFETY:` proof; enforce it (unsafe-proof
// program). As a parent module this also covers the child modules (windows/linux backends).
#![deny(clippy::undocumented_unsafe_blocks)]
@@ -39,11 +41,14 @@ impl Codec {
// per-session instead (the host `session_plan::SessionPlan::output_format`) — the EGL→CUDA
// frames the `auto` GPU path would deliver are NVENC-only. Only a software/GPU-less pref
// keeps the bit off (no Vulkan device to open).
// Resolved ONCE for every gate below (pyro bit, software pin, vulkan ceiling, the
// zero-copy plane): this fn backs the codec advertisement on polled paths, and the
// resolver's auto arm samples live GPU-preference state — four samples per call was the
// probe-amplification the WP7.6 diff critic caught.
#[cfg(target_os = "linux")]
let backend = linux_resolved_backend();
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
let pyro = if !matches!(
pf_host_config::config().encoder_pref.as_str(),
"software" | "sw" | "openh264"
) {
let pyro = if backend != LinuxBackend::Software {
punktfunk_core::quic::CODEC_PYROWAVE
} else {
0u8
@@ -70,19 +75,41 @@ impl Codec {
| punktfunk_core::quic::CODEC_AV1;
#[cfg(target_os = "linux")]
{
if matches!(
pf_host_config::config().encoder_pref.as_str(),
"software" | "sw" | "openh264"
) {
if backend == LinuxBackend::Software {
return punktfunk_core::quic::CODEC_H264;
}
if linux_zero_copy_is_vaapi() {
// A pref that FORCES the raw Vulkan Video backend can only ever serve what that
// backend encodes: `open_video_backend`'s `vulkan` arm bails outright for anything
// that is not HEVC/AV1 ("the Vulkan Video encoder supports HEVC + AV1; the session
// negotiated {codec:?}"). Advertising H.264 there let a client negotiate it and die
// at encoder open. Without the `vulkan-encode` feature that arm cannot open at all,
// so the pref advertises nothing.
//
// This is a CEILING intersected with the device probe below, never a replacement
// for it: pinning a static HEVC|AV1 would ADD AV1 on the AMD/Intel hosts whose probe
// currently withholds it (pre-RDNA3, pre-Arc), i.e. it would re-create this very bug
// for a different codec. Intersecting can only narrow.
let pref_ceiling: u8 = match backend {
// Feature-blindness rider (WP7.6 critics): the resolver knows the PREF is
// vulkan; only this cfg! knows the BUILD can open it. Dropping the inner
// branch would advertise HEVC|AV1 on a featureless build — the
// advertise-then-die-at-open bug this ceiling exists to prevent.
LinuxBackend::Vulkan => {
if cfg!(feature = "vulkan-encode") {
punktfunk_core::quic::CODEC_HEVC | punktfunk_core::quic::CODEC_AV1
} else {
0
}
}
_ => GPU_SUPERSET,
};
if linux_zero_copy_is_vaapi_for(backend) {
if let Some(m) = vaapi_codec_support().wire_mask() {
return m;
return m & pref_ceiling;
}
}
// NVENC (static superset, like GameStream) — or an empty VAAPI probe (see above).
GPU_SUPERSET
GPU_SUPERSET & pref_ceiling
}
#[cfg(target_os = "windows")]
{
@@ -175,6 +202,23 @@ pub fn open_video(
},
}
};
// The session asked for a composited pointer; say so loudly if the backend that actually opened
// cannot deliver one. `cursor_blend` was a REQUEST with no answer for most of this crate's life
// (`let _ = cursor_blend;` below), and the result was a stream with no mouse cursor and nothing
// in the logs — confirmed on the VAAPI dmabuf path and the libav-NVENC CUDA path.
//
// A warning is deliberately all this does. `open_video` cannot re-plan capture, so refusing here
// would only trade a missing pointer for a dead session; the host owns `plan.cursor_blend` and is
// the only layer that can fall back to capturer-side compositing. This makes the condition
// visible and queryable (`EncoderCaps::blends_cursor`) so that decision can be made upstream.
if cursor_blend && !inner.caps().blends_cursor {
tracing::warn!(
backend,
"session negotiated a composited cursor but this encode backend does not blend \
CapturedFrame::cursor the pointer will be MISSING from the stream unless the \
capturer composites it"
);
}
Ok(Box::new(TrackedEncoder {
inner,
_session: pf_gpu::session_begin(gpu),
@@ -241,6 +285,12 @@ impl Encoder for TrackedEncoder {
fn reconfigure_bitrate(&mut self, bps: u64) -> bool {
self.inner.reconfigure_bitrate(bps)
}
// Forwarded (the same trap class as `set_wire_chunking`): the unforwarded default `None`
// would tell the session loop "no encoder truth here" and it would keep pacing/acking the
// requested rate even where NVENC clamped to the codec-level ceiling.
fn applied_bitrate_bps(&self) -> Option<u64> {
self.inner.applied_bitrate_bps()
}
fn flush(&mut self) -> Result<()> {
self.inner.flush()
}
@@ -253,6 +303,224 @@ impl Encoder for TrackedEncoder {
#[cfg(any(target_os = "linux", target_os = "windows"))]
const SW_BITRATE_CEIL: u64 = 100_000_000;
/// The Linux half of [`open_video_backend`], with the pref INJECTED — the seam exists so the
/// dispatch is testable without mutating process env (`set_var` races `getenv` in parallel
/// tests) or fighting the once-latched `pf_host_config::config()`. Production takes the
/// config pref via the caller; shared validation (dims/fps/chroma degrade) stays there too.
#[cfg(target_os = "linux")]
#[allow(clippy::too_many_arguments)]
fn open_video_backend_linux(
pref: &str,
codec: Codec,
format: PixelFormat,
width: u32,
height: u32,
fps: u32,
bitrate_bps: u64,
cuda: bool,
bit_depth: u8,
chroma: ChromaFormat,
cursor_blend: bool,
) -> Result<(Box<dyn Encoder>, &'static str)> {
// A NEGOTIATED PyroWave session (client advertised + preferred it, plan §3) routes
// straight to that backend — the PUNKTFUNK_ENCODER pref below stays a lab override.
if codec == Codec::PyroWave {
#[cfg(feature = "pyrowave")]
{
return pyrowave::PyroWaveEncoder::open(width, height, fps, bitrate_bps, chroma)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "pyrowave"));
}
#[cfg(not(feature = "pyrowave"))]
anyhow::bail!(
"session negotiated PyroWave but this host was built without --features \
punktfunk-host/pyrowave (the advertisement bit should not have been set)"
);
}
// Pick the GPU encode backend. NVIDIA → NVENC/CUDA (the original path, unchanged);
// AMD/Intel → VAAPI (one libavcodec backend for both). Auto-detect by default so a single
// Linux binary serves any GPU; `PUNKTFUNK_ENCODER` forces a specific backend (and surfaces
// its errors crisply instead of silently trying the other).
// AMD/Intel opener. Default = libav VAAPI. With `--features vulkan-encode` +
// PUNKTFUNK_VULKAN_ENCODE, an HEVC session instead opens the raw Vulkan Video backend (real
// RFI loss recovery the VAAPI path can't express); a failed open falls back to VAAPI so the
// stream never dies over the new path. `format`/`bit_depth`/`chroma` only matter to VAAPI —
// the Vulkan backend imports the dmabuf and does its own 8-bit 4:2:0 CSC.
let open_amd_intel = || -> Result<(Box<dyn Encoder>, &'static str)> {
// An HDR session (10-bit + a PQ/BT.2020 capture format) must skip the Vulkan Video
// backend — it hardcodes an 8-bit 4:2:0 BT.709 CSC — and take the libav VAAPI path,
// which has the P010/Main10/PQ wiring. SDR sessions keep the Vulkan default.
#[cfg(feature = "vulkan-encode")]
if matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
&& !(bit_depth == 10 && format.is_hdr_rgb10())
{
match vulkan_video::VulkanVideoEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
cursor_blend,
) {
Ok(e) => {
tracing::info!(
codec = ?codec,
"Linux Vulkan Video encode (real RFI via DPB reference slots) — \
set PUNKTFUNK_VULKAN_ENCODE=0 for libav VAAPI"
);
return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan"));
}
// Native NV12 (PUNKTFUNK_PIPEWIRE_NV12 capture) has no VAAPI fallback:
// libav's dmabuf lane would import the two-plane buffer as packed RGB
// (silent garbage) and its CPU lane bails per frame — die crisply instead.
Err(e) if format == PixelFormat::Nv12 => {
return Err(e.context(
"Vulkan Video open failed on a native-NV12 capture \
no VAAPI fallback exists; set PUNKTFUNK_PIPEWIRE_NV12=0 to \
restore the packed-RGB negotiation",
));
}
Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"Vulkan Video encode open failed — falling back to libav VAAPI"
),
}
}
// Same rule when the Vulkan backend was never eligible (H264 session,
// PUNKTFUNK_VULKAN_ENCODE=0, or a build without the feature).
if format == PixelFormat::Nv12 {
anyhow::bail!(
"native NV12 capture requires the Vulkan Video encoder (HEVC/AV1 \
session, --features vulkan-encode, PUNKTFUNK_VULKAN_ENCODE not 0) this \
session resolved to libav VAAPI; set PUNKTFUNK_PIPEWIRE_NV12=0 to restore \
the packed-RGB negotiation"
);
}
vaapi::VaapiEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "vaapi"))
};
let open_nvidia = || -> Result<(Box<dyn Encoder>, &'static str)> {
open_nvenc_probed(
codec,
format,
width,
height,
fps,
bitrate_bps,
cuda,
bit_depth,
chroma,
cursor_blend,
)
.map(|e| (e, "nvenc"))
};
// WP7.6: the dispatch consumes the SAME resolver the capability mirrors consult (the
// Windows shape — `windows_resolved_backend` + `match backend` below) so the alias table
// exists once. Arm bodies and their open-site labels are untouched.
match resolve_linux_backend(pref, linux_auto_is_vaapi, cuda) {
Some(LinuxBackend::Nvenc) => open_nvidia(),
Some(LinuxBackend::AmdIntel) => open_amd_intel(),
// Force the raw Vulkan Video HEVC backend (real RFI). Needs `--features vulkan-encode`.
Some(LinuxBackend::Vulkan) => {
#[cfg(feature = "vulkan-encode")]
{
if !matches!(codec, Codec::H265 | Codec::Av1) {
anyhow::bail!(
"the Vulkan Video encoder supports HEVC + AV1; the session negotiated {codec:?}"
);
}
vulkan_video::VulkanVideoEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
cursor_blend,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "vulkan"))
}
#[cfg(not(feature = "vulkan-encode"))]
{
let _ = (format, bit_depth, chroma);
anyhow::bail!(
"PUNKTFUNK_ENCODER=vulkan requires a build with --features vulkan-encode"
)
}
}
// PyroWave — the opt-in wired-LAN intra-only wavelet codec. Explicit-only, and
// EXPERIMENTAL until CODEC_PYROWAVE negotiation lands (plan Phase 2): no shipping
// client can decode the stream yet, so this arm exists for host-side bring-up and
// latency work only. Vendor-agnostic (any Vulkan 1.3 GPU); ignores the negotiated
// codec — every AU is an independently-decodable wavelet frame.
Some(LinuxBackend::Pyrowave) => {
#[cfg(feature = "pyrowave")]
{
tracing::warn!(
?codec,
"PUNKTFUNK_ENCODER=pyrowave forces the all-intra wavelet stream \
regardless of the negotiated codec only a pyrowave-feature client \
that ALSO preferred CODEC_PYROWAVE can display it (lab override; \
normal sessions negotiate it instead)"
);
// The lab override forces the wavelet stream onto a session negotiated for
// another codec — that session's chroma may be HEVC-4:4:4, which the
// pyrowave encoder doesn't do yet, so pin the override to 4:2:0.
pyrowave::PyroWaveEncoder::open(
width,
height,
fps,
bitrate_bps,
ChromaFormat::Yuv420,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "pyrowave"))
}
#[cfg(not(feature = "pyrowave"))]
{
anyhow::bail!(
"PUNKTFUNK_ENCODER=pyrowave requires a build with --features punktfunk-host/pyrowave"
)
}
}
// GPU-less software H.264 (openh264) — for a headless / GPU-lost box. Explicit-only:
// `auto` never picks it (a box with `/dev/nvidiactl` present but a dead driver would
// otherwise wrongly resolve to NVENC). Needs H.264 (openh264 emits only that) and a CPU
// RGB frame, which the capturer delivers because the software backend resolves `gpu=false`.
Some(LinuxBackend::Software) => {
if codec != Codec::H264 {
anyhow::bail!(
"the software encoder emits H.264 only; the session negotiated {codec:?} \
(a client must advertise CODEC_H264 to reach a software host)"
);
}
let _ = (cuda, bit_depth); // software path is CPU + 8-bit only
sw::OpenH264Encoder::open(
format,
width,
height,
fps,
bitrate_bps.min(SW_BITRATE_CEIL),
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "software"))
}
// The auto arm lives in the resolver now (a CUDA frame can only be consumed by
// NVENC; else the shared manual-preference/NVIDIA-presence decision).
None => anyhow::bail!(
"unknown PUNKTFUNK_ENCODER={pref:?} — use auto (default), nvenc, vaapi, vulkan, pyrowave, or software"
),
}
}
/// Open the platform encoder backend. Returns the encoder together with the display label of the
/// branch that ACTUALLY opened (`nvenc`/`vaapi`/`vulkan`/`amf`/`qsv`/`software`) — the label feeds
/// the mgmt API's live-session record, and only the open site knows which internal fallback won
@@ -295,209 +563,19 @@ fn open_video_backend(
};
#[cfg(target_os = "linux")]
{
// A NEGOTIATED PyroWave session (client advertised + preferred it, plan §3) routes
// straight to that backend — the PUNKTFUNK_ENCODER pref below stays a lab override.
if codec == Codec::PyroWave {
#[cfg(feature = "pyrowave")]
{
return pyrowave::PyroWaveEncoder::open(width, height, fps, bitrate_bps, chroma)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "pyrowave"));
}
#[cfg(not(feature = "pyrowave"))]
anyhow::bail!(
"session negotiated PyroWave but this host was built without --features \
punktfunk-host/pyrowave (the advertisement bit should not have been set)"
);
}
// Pick the GPU encode backend. NVIDIA → NVENC/CUDA (the original path, unchanged);
// AMD/Intel → VAAPI (one libavcodec backend for both). Auto-detect by default so a single
// Linux binary serves any GPU; `PUNKTFUNK_ENCODER` forces a specific backend (and surfaces
// its errors crisply instead of silently trying the other).
let pref = pf_host_config::config().encoder_pref.as_str();
// AMD/Intel opener. Default = libav VAAPI. With `--features vulkan-encode` +
// PUNKTFUNK_VULKAN_ENCODE, an HEVC session instead opens the raw Vulkan Video backend (real
// RFI loss recovery the VAAPI path can't express); a failed open falls back to VAAPI so the
// stream never dies over the new path. `format`/`bit_depth`/`chroma` only matter to VAAPI —
// the Vulkan backend imports the dmabuf and does its own 8-bit 4:2:0 CSC.
let open_amd_intel = || -> Result<(Box<dyn Encoder>, &'static str)> {
// An HDR session (10-bit + a PQ/BT.2020 capture format) must skip the Vulkan Video
// backend — it hardcodes an 8-bit 4:2:0 BT.709 CSC — and take the libav VAAPI path,
// which has the P010/Main10/PQ wiring. SDR sessions keep the Vulkan default.
#[cfg(feature = "vulkan-encode")]
if matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
&& !(bit_depth == 10 && format.is_hdr_rgb10())
{
match vulkan_video::VulkanVideoEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
cursor_blend,
) {
Ok(e) => {
tracing::info!(
codec = ?codec,
"Linux Vulkan Video encode (real RFI via DPB reference slots) — \
set PUNKTFUNK_VULKAN_ENCODE=0 for libav VAAPI"
);
return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan"));
}
// Native NV12 (PUNKTFUNK_PIPEWIRE_NV12 capture) has no VAAPI fallback:
// libav's dmabuf lane would import the two-plane buffer as packed RGB
// (silent garbage) and its CPU lane bails per frame — die crisply instead.
Err(e) if format == PixelFormat::Nv12 => {
return Err(e.context(
"Vulkan Video open failed on a native-NV12 capture \
no VAAPI fallback exists; set PUNKTFUNK_PIPEWIRE_NV12=0 to \
restore the packed-RGB negotiation",
));
}
Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"Vulkan Video encode open failed — falling back to libav VAAPI"
),
}
}
// Same rule when the Vulkan backend was never eligible (H264 session,
// PUNKTFUNK_VULKAN_ENCODE=0, or a build without the feature).
if format == PixelFormat::Nv12 {
anyhow::bail!(
"native NV12 capture requires the Vulkan Video encoder (HEVC/AV1 \
session, --features vulkan-encode, PUNKTFUNK_VULKAN_ENCODE not 0) this \
session resolved to libav VAAPI; set PUNKTFUNK_PIPEWIRE_NV12=0 to restore \
the packed-RGB negotiation"
);
}
vaapi::VaapiEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "vaapi"))
};
let open_nvidia = || -> Result<(Box<dyn Encoder>, &'static str)> {
open_nvenc_probed(
codec,
format,
width,
height,
fps,
bitrate_bps,
cuda,
bit_depth,
chroma,
cursor_blend,
)
.map(|e| (e, "nvenc"))
};
match pref {
"nvenc" | "nvidia" | "cuda" => open_nvidia(),
"vaapi" | "amd" | "intel" => open_amd_intel(),
// Force the raw Vulkan Video HEVC backend (real RFI). Needs `--features vulkan-encode`.
"vulkan" | "vulkan-video" => {
#[cfg(feature = "vulkan-encode")]
{
if !matches!(codec, Codec::H265 | Codec::Av1) {
anyhow::bail!(
"the Vulkan Video encoder supports HEVC + AV1; the session negotiated {codec:?}"
);
}
vulkan_video::VulkanVideoEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
cursor_blend,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "vulkan"))
}
#[cfg(not(feature = "vulkan-encode"))]
{
let _ = (format, bit_depth, chroma);
anyhow::bail!(
"PUNKTFUNK_ENCODER=vulkan requires a build with --features vulkan-encode"
)
}
}
// PyroWave — the opt-in wired-LAN intra-only wavelet codec. Explicit-only, and
// EXPERIMENTAL until CODEC_PYROWAVE negotiation lands (plan Phase 2): no shipping
// client can decode the stream yet, so this arm exists for host-side bring-up and
// latency work only. Vendor-agnostic (any Vulkan 1.3 GPU); ignores the negotiated
// codec — every AU is an independently-decodable wavelet frame.
"pyrowave" => {
#[cfg(feature = "pyrowave")]
{
tracing::warn!(
?codec,
"PUNKTFUNK_ENCODER=pyrowave forces the all-intra wavelet stream \
regardless of the negotiated codec only a pyrowave-feature client \
that ALSO preferred CODEC_PYROWAVE can display it (lab override; \
normal sessions negotiate it instead)"
);
// The lab override forces the wavelet stream onto a session negotiated for
// another codec — that session's chroma may be HEVC-4:4:4, which the
// pyrowave encoder doesn't do yet, so pin the override to 4:2:0.
pyrowave::PyroWaveEncoder::open(
width,
height,
fps,
bitrate_bps,
ChromaFormat::Yuv420,
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "pyrowave"))
}
#[cfg(not(feature = "pyrowave"))]
{
anyhow::bail!(
"PUNKTFUNK_ENCODER=pyrowave requires a build with --features punktfunk-host/pyrowave"
)
}
}
// GPU-less software H.264 (openh264) — for a headless / GPU-lost box. Explicit-only:
// `auto` never picks it (a box with `/dev/nvidiactl` present but a dead driver would
// otherwise wrongly resolve to NVENC). Needs H.264 (openh264 emits only that) and a CPU
// RGB frame, which the capturer delivers because the software backend resolves `gpu=false`.
"software" | "sw" | "openh264" => {
if codec != Codec::H264 {
anyhow::bail!(
"the software encoder emits H.264 only; the session negotiated {codec:?} \
(a client must advertise CODEC_H264 to reach a software host)"
);
}
let _ = (cuda, bit_depth); // software path is CPU + 8-bit only
sw::OpenH264Encoder::open(
format,
width,
height,
fps,
bitrate_bps.min(SW_BITRATE_CEIL),
)
.map(|e| (Box::new(e) as Box<dyn Encoder>, "software"))
}
"auto" | "" => {
// A CUDA frame can ONLY be consumed by NVENC. Otherwise the shared auto decision
// (manual web-console GPU preference, else the NVIDIA-presence probe) picks the
// backend — see `linux_auto_is_vaapi`.
if cuda || !linux_auto_is_vaapi() {
open_nvidia()
} else {
open_amd_intel()
}
}
other => anyhow::bail!(
"unknown PUNKTFUNK_ENCODER={other:?} — use auto (default), nvenc, vaapi, vulkan, pyrowave, or software"
),
}
open_video_backend_linux(
pf_host_config::config().encoder_pref.as_str(),
codec,
format,
width,
height,
fps,
bitrate_bps,
cuda,
bit_depth,
chroma,
cursor_blend,
)
}
#[cfg(target_os = "windows")]
{
@@ -611,8 +689,17 @@ fn open_video_backend(
// Phase 4 after the field-silence gate.
#[cfg(feature = "qsv")]
{
let ffmpeg_forced = std::env::var("PUNKTFUNK_QSV_FFMPEG")
.is_ok_and(|v| v == "1" || v.eq_ignore_ascii_case("true"));
// Trimmed like every sibling knob, so `"1 "` from a shell script or a `.env`
// line takes effect instead of silently doing nothing. Case-insensitivity is
// kept deliberately — this knob already accepted `TRUE` via
// `eq_ignore_ascii_case`, and the bare house `matches!` would have dropped
// that spelling; widening to `yes`/`on` only adds accepted values.
let ffmpeg_forced = std::env::var("PUNKTFUNK_QSV_FFMPEG").is_ok_and(|v| {
matches!(
v.trim().to_ascii_lowercase().as_str(),
"1" | "true" | "yes" | "on"
)
});
if !ffmpeg_forced {
match qsv::QsvEncoder::open(
codec,
@@ -853,18 +940,31 @@ fn vulkan_encode_enabled() -> bool {
/// negotiation (`OutputFormat::nv12_native` → `ZeroCopyPolicy::native_nv12_session`), which
/// then PREFERS gamescope's producer-side NV12 pod (default-on; `PUNKTFUNK_PIPEWIRE_NV12=0`
/// escapes at the capture gate).
///
/// This verdict is load-bearing in a way the other capability helpers are not: once the producer
/// has been asked for two-plane NV12 there is **no fallback**. [`open_video`] deliberately makes a
/// failed Vulkan open FATAL for an NV12 capture rather than degrading to libav VAAPI, because VAAPI
/// would import that buffer as packed RGB and stream silent garbage. So a wrong `true` here does
/// not cost quality — it kills the session at its first frame. Hence the real device probe below,
/// and hence the conjuncts are ordered cheapest-first so it only runs when everything else already
/// said yes.
#[cfg(target_os = "linux")]
pub fn linux_native_nv12_ok(codec: Codec) -> bool {
#[cfg(feature = "vulkan-encode")]
{
matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
// NVENC/PyroWave prefs never open the Vulkan Video backend; every other pref
// (auto/vaapi/amd/intel/vulkan) tries it first on AMD/Intel — see [`open_video`].
&& !matches!(
pf_host_config::config().encoder_pref.as_str(),
"nvenc" | "nvidia" | "cuda" | "pyrowave"
)
// Which backend this host actually resolves to. This used to be a denylist of the
// EXPLICIT prefs that skip Vulkan Video ("nvenc"|"nvidia"|"cuda"|"pyrowave"), which
// silently missed the one that matters: the DEFAULT `encoder_pref` is `""`, and `""`
// resolves to `auto`, which on an NVIDIA box opens NVENC. So a stock NVIDIA host passed
// this gate. `linux_zero_copy_is_vaapi` layers the pref on top of the same auto decision
// `open_video` makes, which is exactly what the note on `linux_auto_is_vaapi` says a
// capability probe must consult — and it is what the downstream consumer of this very
// verdict already uses to pick its zero-copy path.
&& linux_zero_copy_is_vaapi()
// …and only then ask the GPU. Ordered last on purpose: it opens a Vulkan instance.
&& vulkan_encode_available(codec)
}
#[cfg(not(feature = "vulkan-encode"))]
{
@@ -873,6 +973,42 @@ pub fn linux_native_nv12_ok(codec: Codec) -> bool {
}
}
/// Can this GPU + driver actually open a Vulkan Video **encode** session for `codec`? Cached per
/// (selected GPU, codec) — the [`can_encode_10bit`] idiom, with the probe run outside the lock.
///
/// Only [`linux_native_nv12_ok`] consults this, and only for the no-fallback decision described
/// there. It is deliberately NOT wired into [`open_video`]'s dispatch: that path already degrades
/// to VAAPI on a failed open for every non-NV12 capture, so making it pay for a probe would buy
/// nothing. Prediction and truth stay separate — the probe answers "would it open", the open itself
/// reports what actually did.
#[cfg(all(target_os = "linux", feature = "vulkan-encode"))]
fn vulkan_encode_available(codec: Codec) -> bool {
use std::collections::HashMap;
use std::sync::{Mutex, OnceLock};
static CACHE: OnceLock<Mutex<HashMap<(String, &'static str), bool>>> = OnceLock::new();
let key = (pf_gpu::selection_key(), codec.label());
let cache = CACHE.get_or_init(|| Mutex::new(HashMap::new()));
if let Some(v) = cache.lock().unwrap().get(&key) {
return *v;
}
let verdict = vulkan_video::probe_encode_support(codec);
let ok = verdict.is_ok();
match &verdict {
Ok(()) => tracing::info!(
?codec,
"Vulkan Video encode probed OK — producer-native NV12 capture is eligible"
),
Err(why) => tracing::info!(
?codec,
why = *why,
"Vulkan Video encode unavailable — keeping the packed-RGB capture negotiation \
(the native-NV12 path has no VAAPI fallback)"
),
}
cache.lock().unwrap().insert(key, ok);
ok
}
/// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA
/// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT
/// create a CUDA context (that would allocate GPU state on every host that merely *might* be
@@ -904,6 +1040,67 @@ fn linux_auto_is_vaapi() -> bool {
!nvidia_present()
}
/// The resolved Linux encode backend — the Linux twin of [`WindowsBackend`] (WP7.6). ONE alias
/// table, consumed by BOTH [`open_video_backend`]'s dispatch and every capability/advertisement
/// mirror below, so the two can never drift again (the audit counted five partial hand-copies of
/// this decision). Labels still come from the open sites (the mgmt-record convention) — this
/// resolves which ARM runs, never what actually opened.
#[cfg(target_os = "linux")]
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum LinuxBackend {
Nvenc,
AmdIntel,
Vulkan,
Pyrowave,
Software,
}
/// The pure core. `None` = the pref names no backend — [`open_video_backend`] BAILS on that,
/// while the capability mirrors historically resolved it as auto (via
/// [`linux_resolved_backend`]); that divergence is recorded, not accidental. `auto_is_vaapi` is
/// deliberately LAZY: explicit prefs must stay zero-probe — `/serverinfo` polls through the
/// mirrors, and `gamestream/serverinfo.rs` already litigated exactly that probe cost.
#[cfg(target_os = "linux")]
fn resolve_linux_backend(
pref: &str,
auto_is_vaapi: impl FnOnce() -> bool,
cuda: bool,
) -> Option<LinuxBackend> {
Some(match pref {
"nvenc" | "nvidia" | "cuda" => LinuxBackend::Nvenc,
"vaapi" | "amd" | "intel" => LinuxBackend::AmdIntel,
"vulkan" | "vulkan-video" => LinuxBackend::Vulkan,
"pyrowave" => LinuxBackend::Pyrowave,
"software" | "sw" | "openh264" => LinuxBackend::Software,
// A CUDA frame can ONLY be consumed by NVENC. Otherwise the shared auto decision
// (manual web-console GPU preference, else the NVIDIA-presence probe) picks the backend —
// see `linux_auto_is_vaapi`.
"auto" | "" => {
if cuda || !auto_is_vaapi() {
LinuxBackend::Nvenc
} else {
LinuxBackend::AmdIntel
}
}
_ => return None,
})
}
/// Config-reading wrapper for the capability/advertisement mirrors: an unknown pref resolves as
/// auto — exactly what every mirror's old `_` arm did. `cuda = false`: the mirrors answer
/// pre-session questions (a CUDA payload exists only inside a live NVIDIA session).
#[cfg(target_os = "linux")]
fn linux_resolved_backend() -> LinuxBackend {
let pref = pf_host_config::config().encoder_pref.as_str();
resolve_linux_backend(pref, linux_auto_is_vaapi, false).unwrap_or_else(|| {
if linux_auto_is_vaapi() {
LinuxBackend::AmdIntel
} else {
LinuxBackend::Nvenc
}
})
}
/// The dmabuf modifiers the PyroWave encoder's Vulkan device imports for the capture's
/// packed-RGB fourcc — advertised by the capture when the pyrowave passthrough is active
/// (the VAAPI LINEAR-only policy starves it on Mutter+NVIDIA, which allocates tiled only).
@@ -917,13 +1114,25 @@ pub fn pyrowave_capture_modifiers(fourcc: u32) -> Vec<u64> {
/// passthrough for VAAPI vs the EGL→CUDA import for NVENC).
#[cfg(target_os = "linux")]
pub fn linux_zero_copy_is_vaapi() -> bool {
match pf_host_config::config().encoder_pref.as_str() {
"nvenc" | "nvidia" | "cuda" => false,
"vaapi" | "amd" | "intel" => true,
linux_zero_copy_is_vaapi_for(linux_resolved_backend())
}
/// The zero-copy-plane decision for an ALREADY-resolved backend — so a caller that has just
/// resolved (e.g. `host_wire_caps`, which consults several gates per call on a POLLED endpoint)
/// pays for the resolution once instead of once per gate (the `serverinfo.rs` probe-cost class,
/// caught by the WP7.6 diff critic).
#[cfg(target_os = "linux")]
fn linux_zero_copy_is_vaapi_for(backend: LinuxBackend) -> bool {
match backend {
LinuxBackend::Nvenc => false,
LinuxBackend::AmdIntel => true,
// PyroWave ingests the raw capture dmabuf itself (Vulkan import + compute CSC) on ANY
// vendor — it must get the passthrough payload, never the EGL→CUDA import.
"pyrowave" => true,
_ => linux_auto_is_vaapi(),
LinuxBackend::Pyrowave => true,
// EXACTLY the old `_` fallthrough for these two prefs — preserved, not endorsed. The
// vulkan-on-NVIDIA half is FILED (EGL→CUDA capture negotiated for a dmabuf-importing
// backend; reachable only via the explicit lab pref); the software half is latent (a
// software host pins H.264, so the GPU-backend probes this steers never negotiate).
LinuxBackend::Vulkan | LinuxBackend::Software => linux_auto_is_vaapi(),
}
}
@@ -1216,9 +1425,17 @@ pub fn windows_resolved_backend() -> WindowsBackend {
pub fn resolved_backend_is_gpu() -> bool {
!matches!(windows_resolved_backend(), WindowsBackend::Software)
}
/// Linux/other: every backend but the GPU-less software encoder (openh264) is GPU-resident. Config-backed
/// (mirrors `session_plan::resolve_encoder`; the NVENC vs VAAPI split is auto-detected in [`open_video`]).
#[cfg(not(target_os = "windows"))]
/// Linux: every backend but the GPU-less software encoder (openh264) is GPU-resident. Resolver-
/// backed (WP7.6); cross-crate mirrors still exist in `session_plan::resolve_encoder` and
/// `gamestream/serverinfo.rs` — out of this crate's reach, listed in the WP7.6 design doc.
#[cfg(target_os = "linux")]
pub fn resolved_backend_is_gpu() -> bool {
linux_resolved_backend() != LinuxBackend::Software
}
/// Other non-Windows targets (the macOS dev host): keep the string gate — no resolver exists
/// there. `not(any(...))`, NEVER a target list, so no exotic target loses the fn (the
/// `can_encode_444`/`can_encode_10bit` shape).
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
pub fn resolved_backend_is_gpu() -> bool {
!matches!(
pf_host_config::config().encoder_pref.as_str(),
@@ -1393,7 +1610,9 @@ mod nvenc;
#[path = "enc/linux/nvenc_cuda.rs"]
mod nvenc_cuda;
// Actionable `NVENCSTATUS` → cause mapping shared by both direct-NVENC backends, so a failed
// session open logs "update/reboot the driver" instead of the old misleading "(no NVIDIA GPU?)".
// session open names its real cause instead of the old misleading "(no NVIDIA GPU?)" — including
// splitting the two opposite failures the driver reports as the SAME `INVALID_VERSION` (a genuine
// driver skew vs. this process's driver state going bad after a session already opened).
#[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "nvenc"))]
#[path = "enc/nvenc_status.rs"]
mod nvenc_status;
@@ -1402,6 +1621,20 @@ mod nvenc_status;
#[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "nvenc"))]
#[path = "enc/nvenc_core.rs"]
mod nvenc_core;
// The slot-family RFI recovery policy (WP7.2) shared by native AMF, native QSV and Vulkan Video —
// the taint sweep + pre-loss anchor pick extracted from three hand-copies that had already
// diverged once (the fecbec2d sweep predates the Vulkan backend's carve-out and was never ported
// to it until a later fix). Policy only: every mechanism (LTR bitfield, RefListCtrl, DPB slot
// table) stays in its backend. cfg = the union of the callers': amf.rs is featureless on Windows,
// vulkan_video needs `vulkan-encode` on Linux — and each ITEM inside is live under the module's
// whole cfg because `plan_slot_recovery` delegates to `pick_anchor` (dead_code is an item lint;
// module-granular reasoning is how a Windows-dead helper reaches main).
#[cfg(any(
target_os = "windows",
all(target_os = "linux", feature = "vulkan-encode")
))]
#[path = "enc/rfi.rs"]
mod rfi;
// Shared libavcodec glue (`pixel_to_av`, swscale consts) for the three libav backends — Linux
// NVENC + VAAPI and Windows AMF/QSV — so the byte-identical pieces live once (plan §2.2, Tier 2).
#[cfg(any(target_os = "linux", all(target_os = "windows", feature = "amf-qsv")))]
@@ -1504,4 +1737,153 @@ mod tests {
};
assert_eq!(none.wire_mask(), None);
}
/// WP7.7 guard (the cheap half): every `Encoder` trait method must be explicitly forwarded by
/// `TrackedEncoder`. A defaulted trait method that isn't forwarded silently no-ops through the
/// wrapper — the trap has bitten three times (`set_wire_chunking`'s §4.4 chunking probe,
/// `set_pipelined`'s LN3 escalation, `applied_bitrate_bps`'s ABR truth), and every Phase 7
/// consolidation that adds a trait method re-arms it. Source-text parse, not reflection: both
/// blocks are top-level rustfmt items, so each ends at the first column-0 `}` and every method
/// name sits on a line starting `fn ` (a wrapped signature still carries the name there).
#[test]
fn tracked_encoder_forwards_every_trait_method() {
fn item_block<'a>(src: &'a str, marker: &str) -> &'a str {
let start = src
.find(marker)
.unwrap_or_else(|| panic!("marker {marker:?} not found — update this guard"));
let body = &src[start..];
let end = body
.find("\n}")
.unwrap_or_else(|| panic!("no column-0 close brace after {marker:?}"));
&body[..end]
}
fn fn_names(block: &str) -> std::collections::BTreeSet<&str> {
block
.lines()
.map(str::trim_start)
.filter(|l| !l.starts_with("//"))
.filter_map(|l| l.strip_prefix("fn "))
.map(|rest| {
rest.split(|c: char| !(c.is_ascii_alphanumeric() || c == '_'))
.next()
.expect("split yields at least one item")
})
.collect()
}
// `find` takes the FIRST occurrence: the real impl precedes this test's own copy of the
// marker string in the included source.
let trait_fns = fn_names(item_block(
include_str!("enc/codec.rs"),
"pub trait Encoder: Send {",
));
let impl_fns = fn_names(item_block(
include_str!("lib.rs"),
"impl Encoder for TrackedEncoder {",
));
assert!(
trait_fns.len() >= 12,
"only {} trait methods parsed — the extraction markers have rotted, fix the parse \
before trusting this guard",
trait_fns.len()
);
let missing: Vec<_> = trait_fns.difference(&impl_fns).collect();
assert!(
missing.is_empty(),
"Encoder methods NOT forwarded by TrackedEncoder: {missing:?} — the host loop only \
ever holds the wrapped box, so an unforwarded default silently disables the feature \
for every session. Forward each one in `impl Encoder for TrackedEncoder`."
);
// The reverse direction (impl fn absent from the trait) is a compile error, so equality
// here is pure belt-and-braces against a parse regression.
assert_eq!(trait_fns, impl_fns);
}
/// WP7.6: the resolver's full alias table, pinned. The panicking closure is the laziness
/// contract — an explicit pref must resolve WITHOUT running the auto probe (`/serverinfo`
/// polls through the mirrors; `gamestream/serverinfo.rs` litigated exactly that cost).
#[cfg(target_os = "linux")]
#[test]
fn linux_backend_resolver_table() {
use LinuxBackend::*;
let no_probe = || -> bool { panic!("explicit prefs must not run the auto probe") };
for pref in ["nvenc", "nvidia", "cuda"] {
assert_eq!(resolve_linux_backend(pref, no_probe, false), Some(Nvenc));
}
for pref in ["vaapi", "amd", "intel"] {
assert_eq!(resolve_linux_backend(pref, no_probe, false), Some(AmdIntel));
}
for pref in ["vulkan", "vulkan-video"] {
assert_eq!(resolve_linux_backend(pref, no_probe, false), Some(Vulkan));
}
assert_eq!(
resolve_linux_backend("pyrowave", no_probe, false),
Some(Pyrowave)
);
for pref in ["software", "sw", "openh264"] {
assert_eq!(resolve_linux_backend(pref, no_probe, false), Some(Software));
}
// auto (and the default empty pref): the probe decides…
assert_eq!(resolve_linux_backend("", || true, false), Some(AmdIntel));
assert_eq!(resolve_linux_backend("auto", || false, false), Some(Nvenc));
// …except a CUDA frame, which only NVENC can consume — and that short-circuits the
// probe too (`||` order).
assert_eq!(resolve_linux_backend("auto", no_probe, true), Some(Nvenc));
// Unknown prefs name no backend: the dispatch bails; the mirrors' wrapper maps this to
// auto (the historical `_`-arm behavior, recorded in the design doc).
assert_eq!(resolve_linux_backend("banana", no_probe, false), None);
// An explicit pref is never overridden by `cuda` (a CUDA payload cannot appear in a
// session whose pref forced a non-NVENC backend; the table must not mask such a bug).
assert_eq!(
resolve_linux_backend("vaapi", no_probe, true),
Some(AmdIntel)
);
}
/// WP7.6: the REAL dispatch through the resolver, GPU-free via the software arm (openh264 is
/// vendored + CPU-only; its own tests already run on every leg). This is the only in-CI
/// execution of the Linux dispatch — before this test it had zero test call sites. The pref
/// is INJECTED through `open_video_backend_linux` — the first draft mutated `PUNKTFUNK_ENCODER`
/// via `set_var`, the binary's first non-ignored env mutation, racing `getenv` in the sw/codec/
/// nvenc_core tests (diff-critic catch); the seam removes the race and the config-latch
/// coupling outright.
#[cfg(target_os = "linux")]
#[test]
fn open_video_backend_dispatches_software() {
let (enc, label) = open_video_backend_linux(
"software",
Codec::H264,
PixelFormat::Bgrx,
64,
64,
30,
1_000_000,
false,
8,
ChromaFormat::Yuv420,
false,
)
.expect("software arm must open GPU-free");
assert_eq!(label, "software");
drop(enc);
// The software arm is codec-gated — a non-H.264 session must take the bail, not fall
// through to another backend.
let err = match open_video_backend_linux(
"software",
Codec::H265,
PixelFormat::Bgrx,
64,
64,
30,
1_000_000,
false,
8,
ChromaFormat::Yuv420,
false,
) {
// `expect_err` needs `Ok: Debug` and `Box<dyn Encoder>` isn't — match instead.
Ok(_) => panic!("software emits H.264 only; an H.265 session must be refused"),
Err(e) => e,
};
assert!(err.to_string().contains("H.264"), "{err:#}");
}
}
+6 -4
View File
@@ -32,10 +32,12 @@ pub struct WinCaptureTarget {
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32,
/// The ADD reply flagged this target as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer from its
/// frames forever, so a session WITHOUT the cursor channel must self-composite (the IDD-push
/// capturer's forced-composite gate) or the streamed desktop has no cursor at all.
/// The ADD reply flagged the ADAPTER as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23, a GameStream session's fresh target streamed cursor-less): DWM
/// excludes the pointer from its frames until adapter reset, so a session WITHOUT the cursor
/// channel must self-composite (the IDD-push capturer's forced-composite gate) or the
/// streamed desktop has no cursor at all.
pub cursor_excluded: bool,
}
+8
View File
@@ -48,6 +48,9 @@ xkbcommon = "0.8"
usbip-sim = { path = "../punktfunk-host/vendor/usbip-sim" }
[target.'cfg(target_os = "windows")'.dependencies]
# The streamed-output desktop rect (CCD source rect) absolute input maps into — the same
# resolver the cursor-readback poller uses, so inject and readback always agree.
pf-win-display = { path = "../pf-win-display" }
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_Security",
@@ -61,5 +64,10 @@ windows = { version = "0.62", features = [
"Win32_System_StationsAndDesktops",
"Win32_System_Threading",
"Win32_UI_Input_KeyboardAndMouse",
# Synthetic pointer devices (PT_PEN / PT_TOUCH) — the pen/touch injectors. The device
# create/destroy + POINTER_TYPE_INFO live under Controls in this metadata layout;
# InjectSyntheticPointerInput + POINTER_*_INFO under Input_Pointer.
"Win32_UI_Controls",
"Win32_UI_Input_Pointer",
"Win32_UI_WindowsAndMessaging",
] }
+396
View File
@@ -0,0 +1,396 @@
//! Virtual tablet ("Punktfunk Pen"): a uinput stylus device carrying the pen plane's full
//! fidelity — pressure, tilt, barrel roll, hover distance, eraser, barrel buttons
//! (design/pen-tablet-input.md §5).
//!
//! Deliberately a **uinput device, not a compositor-protocol citizen**: no virtual-tablet
//! protocol exists in EI, the RemoteDesktop portal, KWin `fake_input`, or wlroots — while
//! every compositor consumes evdev tablets via libinput and forwards them to apps over
//! `zwp_tablet_v2` with nothing to configure. udev's `input_id` builtin classifies the device
//! from its capabilities (`BTN_TOOL_PEN` + `ABS_X/Y` ⇒ `ID_INPUT_TABLET`), so Krita/GIMP/
//! Xournal++ see a real pen. Output mapping is the compositor's own tablet-mapping default
//! (single output ⇒ correct; multi-monitor pinning is the documented follow-up, which is why
//! the device carries a stable, distinctive identity to key rules on).
//!
//! The consumer feeds it [`PenTransition`]s straight from the core's
//! [`PenTracker`](punktfunk_core::quic::PenTracker); this file only translates transitions to
//! evdev events and groups them into SYN frames so proximity-enter carries its position in the
//! same frame (libinput would otherwise report an entry at a stale point).
//!
//! ioctl numbers/struct layouts mirror `gamepad.rs` (verified against the same kernel
//! generation); each backend file stays self-contained by convention.
use anyhow::{bail, Result};
use punktfunk_core::quic::{PenSample, PenTool, PenTransition, PEN_BARREL1, PEN_BARREL2};
use std::os::fd::{AsRawFd, OwnedFd};
// ioctls (x86_64).
const UI_DEV_CREATE: libc::c_ulong = 0x5501;
const UI_DEV_DESTROY: libc::c_ulong = 0x5502;
const UI_DEV_SETUP: libc::c_ulong = 0x405c_5503;
const UI_ABS_SETUP: libc::c_ulong = 0x401c_5504;
const UI_SET_EVBIT: libc::c_ulong = 0x4004_5564;
const UI_SET_KEYBIT: libc::c_ulong = 0x4004_5565;
const UI_SET_PROPBIT: libc::c_ulong = 0x4004_556e;
// input-event-codes.h subset.
const EV_SYN: u16 = 0x00;
const EV_KEY: u16 = 0x01;
const EV_ABS: u16 = 0x03;
const SYN_REPORT: u16 = 0;
const ABS_X: u16 = 0x00;
const ABS_Y: u16 = 0x01;
/// Barrel roll rides ABS_Z (the Wacom Art-Pen rotation convention); libinput normalizes the
/// declared min..max onto its 0..360° rotation axis.
const ABS_Z: u16 = 0x02;
const ABS_PRESSURE: u16 = 0x18;
const ABS_DISTANCE: u16 = 0x19;
const ABS_TILT_X: u16 = 0x1a;
const ABS_TILT_Y: u16 = 0x1b;
const BTN_TOOL_PEN: u16 = 0x140;
const BTN_TOOL_RUBBER: u16 = 0x141;
const BTN_TOUCH: u16 = 0x14a;
const BTN_STYLUS: u16 = 0x14b;
const BTN_STYLUS2: u16 = 0x14c;
/// The pen writes on the display it is mapped to (a "screen tablet"), not a desk pad —
/// libinput then maps the full ABS range onto the output rect, exactly the wire's normalized
/// coordinate contract.
const INPUT_PROP_DIRECT: libc::c_int = 0x01;
/// Full-scale wire pressure (u16) → the declared 0..4095 axis.
const PRESSURE_SHIFT: u32 = 4;
/// Wire hover distance (u16, 0xFFFF = unknown) → the declared 0..1023 axis.
const DISTANCE_SHIFT: u32 = 6;
const ABS_RANGE: f32 = 65535.0;
#[repr(C)]
struct InputId {
bustype: u16,
vendor: u16,
product: u16,
version: u16,
}
#[repr(C)]
struct UinputSetup {
id: InputId,
name: [u8; 80],
ff_effects_max: u32,
}
#[repr(C)]
#[derive(Default, Clone, Copy)]
struct AbsInfo {
value: i32,
minimum: i32,
maximum: i32,
fuzz: i32,
flat: i32,
resolution: i32,
}
#[repr(C)]
struct UinputAbsSetup {
code: u16,
_pad: u16,
absinfo: AbsInfo,
}
#[repr(C)]
#[derive(Clone, Copy)]
struct InputEventRaw {
time: libc::timeval,
type_: u16,
code: u16,
value: i32,
}
fn ioctl_int(fd: i32, req: libc::c_ulong, arg: libc::c_int, what: &str) -> Result<()> {
// SAFETY: every caller passes a UI_SET_*/UI_DEV_* request whose argument the kernel reads
// as a plain int; `fd` is a live uinput fd owned by the caller. No memory is handed over.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
Ok(())
}
fn ioctl_ptr<T>(fd: i32, req: libc::c_ulong, arg: *mut T, what: &str) -> Result<()> {
// SAFETY: every caller passes a pointer to a live, initialized `#[repr(C)]` struct matching
// the request's expected layout (UI_DEV_SETUP/UI_ABS_SETUP); the kernel reads it during the
// call and retains nothing.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
Ok(())
}
/// The active tool's evdev key, one in proximity at a time (Wacom semantics — the core's
/// [`PenTracker`](punktfunk_core::quic::PenTracker) already re-enters proximity on a tool
/// switch, so this only tracks which key to release on `ProximityOut`).
fn tool_key(tool: PenTool) -> u16 {
match tool {
PenTool::Eraser => BTN_TOOL_RUBBER,
// Unknown = a newer client's future tool — nearest ink-capable behavior is the pen.
PenTool::Pen | PenTool::Unknown => BTN_TOOL_PEN,
}
}
/// One per-session virtual tablet. Created lazily on the first pen batch (a session that never
/// draws never creates a device), destroyed with the session (Drop → `UI_DEV_DESTROY`).
pub struct VirtualPen {
fd: OwnedFd,
/// The tool key currently held in proximity (release target for `ProximityOut`).
tool: u16,
/// Whether the current SYN frame already carries a Motion — the frame-split trigger for
/// consecutive samples in one batch.
frame_has_motion: bool,
/// Whether the current SYN frame has any unflushed events.
frame_dirty: bool,
}
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
use std::os::fd::FromRawFd;
// SAFETY: `c"/dev/uinput"` is a 'static NUL-terminated C string literal; `open` reads it
// as a path, returns a fresh fd (or -1) and retains nothing.
let raw = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if raw < 0 {
bail!(
"open /dev/uinput: {} (install the udev rule granting the 'input' group access \
see scripts/60-punktfunk.rules and add the user to the 'input' group)",
std::io::Error::last_os_error()
);
}
// SAFETY: `raw >= 0` here, a freshly-opened fd owned nowhere else; `OwnedFd` becomes the
// unique owner and closes it exactly once on drop.
let fd = unsafe { OwnedFd::from_raw_fd(raw) };
ioctl_int(raw, UI_SET_EVBIT, EV_KEY as i32, "UI_SET_EVBIT(EV_KEY)")?;
ioctl_int(raw, UI_SET_EVBIT, EV_ABS as i32, "UI_SET_EVBIT(EV_ABS)")?;
for key in [
BTN_TOOL_PEN,
BTN_TOOL_RUBBER,
BTN_TOUCH,
BTN_STYLUS,
BTN_STYLUS2,
] {
ioctl_int(raw, UI_SET_KEYBIT, key as i32, "UI_SET_KEYBIT")?;
}
ioctl_int(
raw,
UI_SET_PROPBIT,
INPUT_PROP_DIRECT,
"UI_SET_PROPBIT(DIRECT)",
)?;
// Position spans the full u16 range; `resolution` (units/mm) only feeds libinput's mm
// math (nothing pen-relevant), but tablets without one trip its missing-resolution
// fixup — 100 declares a plausible ~655 mm drawing surface.
let pos = AbsInfo {
minimum: 0,
maximum: 65535,
resolution: 100,
..Default::default()
};
// Tilt in degrees from vertical, per evdev convention; resolution = units/radian (57
// ⇔ 1 unit = 1°, what the Wacom driver declares).
let tilt = AbsInfo {
minimum: -90,
maximum: 90,
resolution: 57,
..Default::default()
};
for (code, info) in [
(ABS_X, pos),
(ABS_Y, pos),
(
ABS_PRESSURE,
AbsInfo {
minimum: 0,
maximum: 4095,
..Default::default()
},
),
(
ABS_DISTANCE,
AbsInfo {
minimum: 0,
maximum: 1023,
..Default::default()
},
),
(ABS_TILT_X, tilt),
(ABS_TILT_Y, tilt),
(
// Barrel roll: libinput maps the declared range linearly onto 0..360°.
ABS_Z,
AbsInfo {
minimum: 0,
maximum: 359,
..Default::default()
},
),
] {
let mut a = UinputAbsSetup {
code,
_pad: 0,
absinfo: info,
};
ioctl_ptr(raw, UI_ABS_SETUP, &mut a, "UI_ABS_SETUP")?;
}
// A stable, distinctive identity (pid.codes open-source VID) so compositor
// tablet-mapping rules — GNOME's per-`vendor:product` gsettings path, sway
// `map_to_output` — can target exactly this device.
let mut setup = UinputSetup {
id: InputId {
bustype: 0x0006, // BUS_VIRTUAL
vendor: 0x1209,
product: 0x5046, // "PF"
version: 1,
},
name: [0; 80],
ff_effects_max: 0,
};
let name = b"Punktfunk Pen";
setup.name[..name.len()].copy_from_slice(name);
ioctl_ptr(raw, UI_DEV_SETUP, &mut setup, "UI_DEV_SETUP")?;
ioctl_int(raw, UI_DEV_CREATE, 0, "UI_DEV_CREATE")?;
tracing::info!("virtual tablet created (Punktfunk Pen, uinput)");
Ok(VirtualPen {
fd,
tool: BTN_TOOL_PEN,
frame_has_motion: false,
frame_dirty: false,
})
}
fn emit(&self, type_: u16, code: u16, value: i32) {
let ev = InputEventRaw {
time: libc::timeval {
tv_sec: 0,
tv_usec: 0,
},
type_,
code,
value,
};
// SAFETY: `ev` is a live local `#[repr(C)]` all-integer struct (no padding: timeval=16 +
// u16 + u16 + i32 = 24), so every byte is initialized; the slice spans exactly `ev`'s
// bytes and is used immediately below with no concurrent mutation.
let bytes = unsafe {
std::slice::from_raw_parts(
&ev as *const _ as *const u8,
std::mem::size_of::<InputEventRaw>(),
)
};
// Best-effort like the gamepad path: a full kernel queue drops the event; pen samples
// are state-full, so the next frame re-syncs axes (and the tracker re-syncs state).
// SAFETY: `self.fd` stays open for the synchronous call; `write` only reads
// `bytes.len()` bytes from the still-live local and retains nothing.
let _ = unsafe {
libc::write(
self.fd.as_raw_fd(),
bytes.as_ptr() as *const libc::c_void,
bytes.len(),
)
};
}
fn flush(&mut self) {
if self.frame_dirty {
self.emit(EV_SYN, SYN_REPORT, 0);
self.frame_dirty = false;
self.frame_has_motion = false;
}
}
fn motion(&mut self, s: &PenSample) {
self.emit(EV_ABS, ABS_X, (s.x * ABS_RANGE) as i32);
self.emit(EV_ABS, ABS_Y, (s.y * ABS_RANGE) as i32);
self.emit(EV_ABS, ABS_PRESSURE, (s.pressure >> PRESSURE_SHIFT) as i32);
if s.distance != punktfunk_core::quic::PEN_DISTANCE_UNKNOWN {
self.emit(EV_ABS, ABS_DISTANCE, (s.distance >> DISTANCE_SHIFT) as i32);
}
// Polar → tiltX/tiltY needs both angles; azimuth clockwise from north, so east (90°)
// tilts +X and south (180°, toward the user) tilts +Y — the evdev/W3C signs.
if s.tilt_deg != punktfunk_core::quic::PEN_TILT_UNKNOWN
&& s.azimuth_deg != punktfunk_core::quic::PEN_ANGLE_UNKNOWN
{
let az = (s.azimuth_deg as f32).to_radians();
let tilt = s.tilt_deg as f32;
self.emit(EV_ABS, ABS_TILT_X, (tilt * az.sin()).round() as i32);
self.emit(EV_ABS, ABS_TILT_Y, (-tilt * az.cos()).round() as i32);
}
if s.roll_deg != punktfunk_core::quic::PEN_ANGLE_UNKNOWN {
self.emit(EV_ABS, ABS_Z, (s.roll_deg % 360) as i32);
}
self.frame_dirty = true;
self.frame_has_motion = true;
}
/// Apply one decoded batch's transitions (the core tracker's output, in its documented
/// order), grouping them into SYN frames: a frame closes before a `ProximityIn` (an entry
/// is a new instant — and must carry its own position, not inherit the stale frame) and
/// before a second `Motion` (consecutive samples are consecutive instants), plus a final
/// close. So `[ProxIn, Motion, TipDown]` lands as ONE frame — libinput reports the entry
/// already at the right point with contact — while a drag batch's `[Motion, Motion]`
/// stays two.
pub fn apply_batch(&mut self, transitions: &[PenTransition]) {
for t in transitions {
match t {
PenTransition::ProximityIn { tool } => {
self.flush();
self.tool = tool_key(*tool);
self.emit(EV_KEY, self.tool, 1);
self.frame_dirty = true;
}
PenTransition::Motion { sample } => {
if self.frame_has_motion {
self.flush();
}
self.motion(sample);
}
PenTransition::TipDown => {
self.emit(EV_KEY, BTN_TOUCH, 1);
self.frame_dirty = true;
}
PenTransition::ButtonsChanged { pressed, released } => {
for (bit, key) in [(PEN_BARREL1, BTN_STYLUS), (PEN_BARREL2, BTN_STYLUS2)] {
if pressed & bit != 0 {
self.emit(EV_KEY, key, 1);
self.frame_dirty = true;
}
if released & bit != 0 {
self.emit(EV_KEY, key, 0);
self.frame_dirty = true;
}
}
}
PenTransition::TipUp => {
self.emit(EV_KEY, BTN_TOUCH, 0);
self.emit(EV_ABS, ABS_PRESSURE, 0);
self.frame_dirty = true;
}
PenTransition::ProximityOut => {
self.emit(EV_KEY, self.tool, 0);
self.frame_dirty = true;
}
}
}
self.flush();
}
}
impl Drop for VirtualPen {
fn drop(&mut self) {
// SAFETY: `self.fd` is still open (OwnedFd closes only after this body returns);
// UI_DEV_DESTROY takes no pointer argument. Errors are moot on teardown.
let _ = unsafe { libc::ioctl(self.fd.as_raw_fd(), UI_DEV_DESTROY, 0) };
}
}
@@ -0,0 +1,659 @@
//! Windows synthetic-pointer injection (design/pen-tablet-input.md §6): a per-session `PT_PEN`
//! device carrying the pen plane's full fidelity — pressure (rescaled to Windows' 0..1024),
//! polar tilt → tiltX/tiltY, barrel roll on `rotation` (0..359 — Windows Ink renders Pencil
//! Pro roll natively), barrel button, eraser (`PEN_FLAG_INVERTED`/`ERASER`), hover — plus a
//! `PT_TOUCH` device that closes the long-standing SendInput touch no-op for wire touches.
//!
//! Both follow Apollo's proven recipe (`design/apollo-comparison.md`): synthetic pointer state
//! goes STALE if not re-injected (~100 ms), so each device runs a small refresh thread that
//! re-asserts the last frame every [`REFRESH_MS`] while the pen is in range / contacts are
//! held — a stationary stylus must not hover-out (and a held finger must not auto-lift) just
//! because no new samples arrived. `CreateSyntheticPointerDevice` needs Win10 1809+;
//! [`crate::pen_supported`] probes it, so older hosts simply never advertise pen.
//!
//! Frame grouping mirrors the Linux uinput backend: a proximity-enter is injected together
//! with its position (never at a stale point), tip edges get their own DOWN/UP frames, and a
//! range-leave is a final frame without `INRANGE`.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{Context, Result};
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{
PenSample, PenTool, PenTransition, PEN_ANGLE_UNKNOWN, PEN_BARREL1, PEN_TILT_UNKNOWN,
};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use windows::Win32::Foundation::POINT;
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetThreadDesktop, OpenInputDesktop, SetThreadDesktop, DESKTOP_ACCESS_FLAGS,
DESKTOP_CONTROL_FLAGS, HDESK,
};
use windows::Win32::System::Threading::GetCurrentThreadId;
use windows::Win32::UI::Controls::{
CreateSyntheticPointerDevice, DestroySyntheticPointerDevice, HSYNTHETICPOINTERDEVICE,
POINTER_FEEDBACK_DEFAULT, POINTER_TYPE_INFO, POINTER_TYPE_INFO_0,
};
use windows::Win32::UI::Input::Pointer::{
InjectSyntheticPointerInput, POINTER_FLAGS, POINTER_FLAG_DOWN, POINTER_FLAG_FIRSTBUTTON,
POINTER_FLAG_INCONTACT, POINTER_FLAG_INRANGE, POINTER_FLAG_NEW, POINTER_FLAG_UP,
POINTER_FLAG_UPDATE, POINTER_INFO, POINTER_PEN_INFO, POINTER_TOUCH_INFO,
};
use windows::Win32::UI::WindowsAndMessaging::{
PEN_FLAG_BARREL, PEN_FLAG_ERASER, PEN_FLAG_INVERTED, PEN_FLAG_NONE, PEN_MASK_PRESSURE,
PEN_MASK_ROTATION, PEN_MASK_TILT_X, PEN_MASK_TILT_Y, PT_PEN, PT_TOUCH, TOUCH_FLAG_NONE,
TOUCH_MASK_NONE,
};
/// Re-inject cadence while state is held (in-range pen / touching contacts). Apollo uses
/// 50 ms against the ~100 ms staleness window; 40 ms keeps two refreshes inside it.
const REFRESH_MS: u64 = 40;
/// `GENERIC_ALL` for the desktop open — the `windows` crate models desktop rights as their own
/// flag type and doesn't re-export the generic ones (same constant `sendinput.rs` uses).
const DESKTOP_GENERIC_ALL: u32 = 0x1000_0000;
/// This thread's binding to the input desktop, restored on drop.
///
/// `SendInput` (`sendinput.rs`) solves the same problem by binding its DEDICATED injector thread
/// once and keeping it. That can't be borrowed here: `inject_pen`/`inject_touch_frame` are driven
/// from TWO threads — the caller's `apply_batch` and the `pf-pen-refresh`/`pf-touch-refresh`
/// staleness threads — and the batch caller is a shared task thread, which must not be left parked
/// on a `Winlogon` desktop that disappears when the prompt is dismissed. So the binding is scoped
/// to the retry: `GetThreadDesktop` hands back a BORROWED handle (never closed), and only the
/// `OpenInputDesktop` handle is closed, after the thread has moved back off it.
struct InputDesktopBinding {
previous: HDESK,
input: HDESK,
}
impl InputDesktopBinding {
fn bind() -> Option<Self> {
// SAFETY: FFI calls taking by-value args only. `OpenInputDesktop` yields an owned `HDESK`
// solely on `Ok`; it is either installed by `SetThreadDesktop` (then owned by this guard,
// closed exactly once in `Drop`) or closed here on failure — closed once on every path,
// never used after. `SetThreadDesktop` rebinds only the calling thread, which owns no
// windows or hooks, so it cannot fail on that account.
unsafe {
let previous = GetThreadDesktop(GetCurrentThreadId()).ok()?;
let input = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(DESKTOP_GENERIC_ALL),
)
.ok()?;
if SetThreadDesktop(input).is_err() {
let _ = CloseDesktop(input);
return None;
}
Some(Self { previous, input })
}
}
}
impl Drop for InputDesktopBinding {
fn drop(&mut self) {
// SAFETY: `previous` is the borrowed desktop this thread started on, `input` the handle
// this guard uniquely owns. The thread moves back FIRST, so the handle is not the thread's
// desktop when closed — closed exactly once, never used after.
unsafe {
let _ = SetThreadDesktop(self.previous);
let _ = CloseDesktop(self.input);
}
}
}
/// Inject one synthetic-pointer frame, following the input desktop if Windows refuses it.
///
/// While a UAC consent prompt (or the lock / logon screen) owns input, the secure desktop does, and
/// injection from the host's own `WinSta0\Default` thread comes back `ERROR_ACCESS_DENIED` — which
/// is why pen and touch went dead on a prompt a user could SEE in the stream (capture already
/// renders it, and `SendInput` mouse/keyboard already followed the switch, so only these two were
/// left behind). Field-reported 2026-07-23; the exact rc reproduced in a probe the same night.
///
/// Measured then, with a real consent prompt up and a SYSTEM host in the console session:
///
/// ```text
/// device created on Default, thread on Default -> 0x80070005 (the field failure)
/// device created on Default, thread on INPUT desktop -> OK
/// device created on INPUT, thread on INPUT desktop -> OK
/// ```
///
/// The middle row is the load-bearing one: the synthetic pointer device is NOT desktop-affine, so
/// rebinding the THREAD is sufficient and the device never has to be destroyed and recreated
/// across a desktop switch (which would drop in-flight contacts and the pen's in-range state).
///
/// # Safety
/// `dev` must be a live synthetic-pointer device and `frame` a live slice for the call.
unsafe fn inject_following_desktop(
dev: HSYNTHETICPOINTERDEVICE,
frame: &[POINTER_TYPE_INFO],
) -> windows::core::Result<()> {
// SAFETY: per this fn's contract — `dev` is live and `frame` outlives the call, which only
// reads it. Best-effort, exactly as the direct call was.
match InjectSyntheticPointerInput(dev, frame) {
Ok(()) => Ok(()),
Err(first) => {
// Only a desktop switch is worth a rebind; anything else would just fail identically.
let Some(_binding) = InputDesktopBinding::bind() else {
return Err(first);
};
// SAFETY: same live `dev`/`frame`, re-issued with this thread on the input desktop.
InjectSyntheticPointerInput(dev, frame)
}
}
}
/// Windows pen pressure is 0..1024 (vs the wire's full-scale u16).
const WIN_PEN_PRESSURE_MAX: u32 = 1024;
/// Map a normalized [0,1] coordinate pair onto desktop pixels over the STREAMED output's rect
/// ([`crate::stream_target`]) — the surface the SendInput absolute mouse targets too, so pen,
/// touch, and pointer all land identically. The wire normalizes over the streamed display's
/// frame, not the desktop: mapping over the whole virtual desktop is only right when they
/// coincide (Exclusive topology — the fallback when no stream target is live).
fn to_screen(x: f32, y: f32) -> POINT {
let (px, py) = crate::stream_target::map_normalized(x as f64, y as f64);
POINT { x: px, y: py }
}
/// An owned `HSYNTHETICPOINTERDEVICE` (destroyed exactly once on drop).
struct Device(HSYNTHETICPOINTERDEVICE);
// SAFETY: the handle is a plain kernel object identifier with no thread affinity —
// `InjectSyntheticPointerInput`/`DestroySyntheticPointerDevice` are documented callable from
// any thread; ownership transfer/sharing does not alias memory.
unsafe impl Send for Device {}
impl Drop for Device {
fn drop(&mut self) {
// SAFETY: `self.0` is the device this wrapper uniquely owns; destroyed once here.
unsafe { DestroySyntheticPointerDevice(self.0) };
}
}
/// Probe: can this Windows build create a synthetic pen device (Win10 1809+)?
pub fn synthetic_pen_available() -> bool {
// SAFETY: FFI create with by-value args; on success the returned handle is destroyed
// immediately by the `Device` wrapper, exactly once.
match unsafe { CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) } {
Ok(h) => {
drop(Device(h));
true
}
Err(_) => false,
}
}
/// The tracked pen state a refresh tick re-asserts.
#[derive(Default)]
struct PenState {
in_range: bool,
touching: bool,
barrel: bool,
eraser: bool,
x: f32,
y: f32,
pressure: u16,
tilt_deg: u8,
azimuth_deg: u16,
roll_deg: u16,
}
struct PenShared {
dev: Device,
state: PenState,
/// First injection failure logs at WARN (see `TouchShared::fail_warned`).
fail_warned: bool,
}
/// One per-session virtual pen (the Windows sibling of the Linux uinput tablet — same
/// [`PenTransition`] consumer API). Wire barrel button 2 has no Windows pen equivalent
/// (one barrel + eraser is the platform model) and is ignored here.
pub struct VirtualPen {
shared: Arc<Mutex<PenShared>>,
stop: Arc<AtomicBool>,
refresher: Option<std::thread::JoinHandle<()>>,
// Batch-local frame grouping (single-threaded within apply_batch).
edge_down: bool,
edge_up: bool,
is_new: bool,
frame_dirty: bool,
frame_has_motion: bool,
}
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
// SAFETY: FFI create with by-value args; the handle's sole owner becomes `Device`.
let dev = unsafe { CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) }
.context("CreateSyntheticPointerDevice(PT_PEN) — needs Windows 10 1809+")?;
let shared = Arc::new(Mutex::new(PenShared {
dev: Device(dev),
state: PenState::default(),
fail_warned: false,
}));
let stop = Arc::new(AtomicBool::new(false));
// The staleness guard: re-assert the last frame while in range so a stationary pen
// (native plane: between 100 ms heartbeats; GameStream: indefinitely) never hovers out.
let refresher = {
let shared = Arc::clone(&shared);
let stop = Arc::clone(&stop);
std::thread::Builder::new()
.name("pf-pen-refresh".into())
.spawn(move || {
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(std::time::Duration::from_millis(REFRESH_MS));
let s = &mut *shared.lock().unwrap();
if s.state.in_range {
inject_pen(s, POINTER_FLAG_UPDATE, false);
}
}
})
.context("spawn pen refresh thread")?
};
tracing::info!("virtual pen created (Windows synthetic pointer, PT_PEN)");
Ok(VirtualPen {
shared,
stop,
refresher: Some(refresher),
edge_down: false,
edge_up: false,
is_new: false,
frame_dirty: false,
frame_has_motion: false,
})
}
/// Apply one batch of tracker transitions — same grouping contract as the Linux backend:
/// `[ProxIn, Motion, TipDown]` is ONE frame (entry lands at its position, in contact),
/// consecutive `Motion`s split, tip edges own their frames, range-leave is a final
/// no-`INRANGE` frame.
pub fn apply_batch(&mut self, transitions: &[PenTransition]) {
for t in transitions {
match t {
PenTransition::ProximityIn { tool } => {
self.flush();
let mut s = self.shared.lock().unwrap();
s.state.in_range = true;
s.state.eraser = *tool == PenTool::Eraser;
drop(s);
self.is_new = true;
self.frame_dirty = true;
}
PenTransition::Motion { sample } => {
if self.frame_has_motion {
self.flush();
}
self.set_axes(sample);
self.frame_dirty = true;
self.frame_has_motion = true;
}
PenTransition::TipDown => {
self.shared.lock().unwrap().state.touching = true;
self.edge_down = true;
self.frame_dirty = true;
}
PenTransition::ButtonsChanged { pressed, released } => {
let mut s = self.shared.lock().unwrap();
if pressed & PEN_BARREL1 != 0 {
s.state.barrel = true;
}
if released & PEN_BARREL1 != 0 {
s.state.barrel = false;
}
drop(s);
self.frame_dirty = true;
}
PenTransition::TipUp => {
self.shared.lock().unwrap().state.touching = false;
self.edge_up = true;
self.frame_dirty = true;
self.flush(); // UP owns its frame (still INRANGE)
}
PenTransition::ProximityOut => {
self.shared.lock().unwrap().state.in_range = false;
self.frame_dirty = true;
self.flush(); // final frame without INRANGE
}
}
}
self.flush();
}
fn set_axes(&mut self, s: &PenSample) {
let mut sh = self.shared.lock().unwrap();
sh.state.x = s.x;
sh.state.y = s.y;
sh.state.pressure = s.pressure;
sh.state.tilt_deg = s.tilt_deg;
sh.state.azimuth_deg = s.azimuth_deg;
sh.state.roll_deg = s.roll_deg;
}
fn flush(&mut self) {
if !self.frame_dirty {
return;
}
let edge = if self.edge_down {
POINTER_FLAG_DOWN
} else if self.edge_up {
POINTER_FLAG_UP
} else {
POINTER_FLAG_UPDATE
};
inject_pen(&mut self.shared.lock().unwrap(), edge, self.is_new);
self.edge_down = false;
self.edge_up = false;
self.is_new = false;
self.frame_dirty = false;
self.frame_has_motion = false;
}
}
impl Drop for VirtualPen {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(h) = self.refresher.take() {
let _ = h.join();
}
// The device itself dies with `shared` (Device::drop) — Windows releases any held
// in-range/contact state when the synthetic device is destroyed.
}
}
/// Build + inject one pen frame from tracked state. `edge` is DOWN/UP/UPDATE;
/// `INRANGE`/`INCONTACT` derive from the state itself.
fn inject_pen(sh: &mut PenShared, edge: POINTER_FLAGS, is_new: bool) {
let st = &sh.state;
let mut flags = edge;
if st.in_range {
flags |= POINTER_FLAG_INRANGE;
}
if st.touching {
flags |= POINTER_FLAG_INCONTACT | POINTER_FLAG_FIRSTBUTTON;
}
if is_new {
flags |= POINTER_FLAG_NEW;
}
let mut pen_flags = PEN_FLAG_NONE;
if st.barrel {
pen_flags |= PEN_FLAG_BARREL;
}
if st.eraser {
pen_flags |= PEN_FLAG_INVERTED;
if st.touching {
pen_flags |= PEN_FLAG_ERASER;
}
}
let mut pen_mask = PEN_MASK_PRESSURE;
// Contact needs a nonzero pressure to ink; hover reports 0 (Windows convention).
let pressure = if st.touching {
((st.pressure as u32 * WIN_PEN_PRESSURE_MAX) / u16::MAX as u32).max(1)
} else {
0
};
let (mut tilt_x, mut tilt_y) = (0i32, 0i32);
if st.tilt_deg != PEN_TILT_UNKNOWN && st.azimuth_deg != PEN_ANGLE_UNKNOWN {
let az = (st.azimuth_deg as f32).to_radians();
let tilt = st.tilt_deg as f32;
tilt_x = (tilt * az.sin()).round() as i32;
tilt_y = (-tilt * az.cos()).round() as i32;
pen_mask |= PEN_MASK_TILT_X | PEN_MASK_TILT_Y;
}
let mut rotation = 0u32;
if st.roll_deg != PEN_ANGLE_UNKNOWN {
rotation = (st.roll_deg % 360) as u32;
pen_mask |= PEN_MASK_ROTATION;
}
let pt = to_screen(st.x, st.y);
let info = POINTER_TYPE_INFO {
r#type: PT_PEN,
Anonymous: POINTER_TYPE_INFO_0 {
penInfo: POINTER_PEN_INFO {
pointerInfo: POINTER_INFO {
pointerType: PT_PEN,
pointerId: 0,
pointerFlags: flags,
ptPixelLocation: pt,
ptPixelLocationRaw: pt,
..Default::default()
},
penFlags: pen_flags,
penMask: pen_mask,
pressure,
rotation,
tiltX: tilt_x,
tiltY: tilt_y,
},
},
};
// SAFETY: `sh.dev.0` is the live device this wrapper owns; the one-element array is a live
// stack value the call only reads. Best-effort like every injector write — a transient
// failure (desktop switch) is healed by the next refresh tick re-asserting state.
if let Err(e) = unsafe { inject_following_desktop(sh.dev.0, &[info]) } {
if !sh.fail_warned {
sh.fail_warned = true;
tracing::warn!(
error = %e,
flags = format!("{:#x}", flags.0),
pen_flags = format!("{pen_flags:#x}"),
pen_mask = format!("{pen_mask:#x}"),
pressure,
rotation,
tilt_x,
tilt_y,
x = pt.x,
y = pt.y,
"pen inject failed"
);
} else {
tracing::trace!(error = %e, "pen inject failed (transient)");
}
} else {
sh.fail_warned = false;
}
}
/// One live wire-touch contact. `slot` is the SMALL, DENSE pointer id handed to Windows —
/// synthetic-pointer injection rejects arbitrary large ids, and clients (Moonlight's
/// `pointerId` especially) send exactly those, so wire ids compact into the lowest free slot
/// for the contact's lifetime (Apollo's slot-contiguity rule; the on-glass symptom of passing
/// wire ids through was pen working while every touch silently failed to inject).
#[derive(Clone, Copy)]
struct Contact {
id: u32,
slot: u32,
x: f32,
y: f32,
}
/// Lowest slot not held by a live contact.
fn free_slot(contacts: &[Contact]) -> u32 {
let mut slot = 0u32;
while contacts.iter().any(|c| c.slot == slot) {
slot += 1;
}
slot
}
/// Windows can inject at most this many simultaneous synthetic touch contacts.
const MAX_CONTACTS: usize = 10;
struct TouchShared {
dev: Device,
contacts: Vec<Contact>,
/// First injection failure logs at WARN (an on-glass "touch does nothing" must be
/// visible in the host log); repeats stay at trace.
fail_warned: bool,
}
/// The `PT_TOUCH` device servicing wire `TouchDown/Move/Up` events — closes the SendInput
/// touch no-op. Contacts are keyed by the wire's finger id; every frame re-injects the FULL
/// active set (the synthetic-pointer contract), and a refresh thread re-asserts held contacts
/// against the ~100 ms staleness auto-lift.
pub struct SyntheticTouch {
shared: Arc<Mutex<TouchShared>>,
stop: Arc<AtomicBool>,
refresher: Option<std::thread::JoinHandle<()>>,
}
impl SyntheticTouch {
pub fn create() -> Result<SyntheticTouch> {
// SAFETY: FFI create with by-value args; the handle's sole owner becomes `Device`.
let dev = unsafe {
CreateSyntheticPointerDevice(PT_TOUCH, MAX_CONTACTS as u32, POINTER_FEEDBACK_DEFAULT)
}
.context("CreateSyntheticPointerDevice(PT_TOUCH) — needs Windows 10 1809+")?;
let shared = Arc::new(Mutex::new(TouchShared {
dev: Device(dev),
contacts: Vec::new(),
fail_warned: false,
}));
let stop = Arc::new(AtomicBool::new(false));
let refresher = {
let shared = Arc::clone(&shared);
let stop = Arc::clone(&stop);
std::thread::Builder::new()
.name("pf-touch-refresh".into())
.spawn(move || {
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(std::time::Duration::from_millis(REFRESH_MS));
let s = &mut *shared.lock().unwrap();
if !s.contacts.is_empty() {
inject_touch_frame(s, None);
}
}
})
.context("spawn touch refresh thread")?
};
tracing::info!("virtual touchscreen created (Windows synthetic pointer, PT_TOUCH)");
Ok(SyntheticTouch {
shared,
stop,
refresher: Some(refresher),
})
}
/// Apply one wire touch event (`code` = finger id, pixel x/y against the
/// `flags = (w << 16) | h` reference extent, exactly like `MouseMoveAbs`).
pub fn apply(&mut self, ev: &InputEvent) {
let (w, h) = ((ev.flags >> 16) as f32, (ev.flags & 0xFFFF) as f32);
if (w < 1.0 || h < 1.0) && ev.kind != InputKind::TouchUp {
return; // the documented zero-extent drop, as for MouseMoveAbs
}
let (x, y) = (ev.x as f32 / w.max(1.0), ev.y as f32 / h.max(1.0));
let s = &mut *self.shared.lock().unwrap();
match ev.kind {
InputKind::TouchDown => {
match s.contacts.iter().position(|c| c.id == ev.code) {
Some(i) => (s.contacts[i].x, s.contacts[i].y) = (x, y),
None if s.contacts.len() < MAX_CONTACTS => {
let slot = free_slot(&s.contacts);
s.contacts.push(Contact {
id: ev.code,
slot,
x,
y,
})
}
None => return, // beyond the platform max — drop, never evict a live finger
}
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_DOWN)));
}
InputKind::TouchMove => {
match s.contacts.iter().position(|c| c.id == ev.code) {
Some(i) => {
(s.contacts[i].x, s.contacts[i].y) = (x, y);
inject_touch_frame(s, None);
}
// A move for an unknown id (its DOWN was dropped/lost): synthesize the
// contact so the stroke self-heals, like the pen plane does.
None if s.contacts.len() < MAX_CONTACTS => {
let slot = free_slot(&s.contacts);
s.contacts.push(Contact {
id: ev.code,
slot,
x,
y,
});
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_DOWN)));
}
None => {}
}
}
InputKind::TouchUp => {
let Some(idx) = s.contacts.iter().position(|c| c.id == ev.code) else {
return;
};
// The UP frame still carries the lifting contact (with UP flags), then it
// leaves the active set.
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_UP)));
s.contacts.remove(idx);
}
_ => {}
}
}
}
impl Drop for SyntheticTouch {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(h) = self.refresher.take() {
let _ = h.join();
}
}
}
/// Inject the full active-contact frame; `edge` marks one contact's DOWN/UP transition (by
/// WIRE id — everyone else is a held UPDATE). Windows sees the compacted `slot` ids only.
fn inject_touch_frame(sh: &mut TouchShared, edge: Option<(u32, POINTER_FLAGS)>) {
let contacts = &sh.contacts;
if contacts.is_empty() {
return;
}
let mut frame: Vec<POINTER_TYPE_INFO> = Vec::with_capacity(contacts.len());
for c in contacts {
let mut flags = POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT | POINTER_FLAG_UPDATE;
if let Some((id, e)) = edge {
if id == c.id {
if e == POINTER_FLAG_DOWN {
flags = POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT | POINTER_FLAG_DOWN;
} else if e == POINTER_FLAG_UP {
flags = POINTER_FLAG_UP; // contact + range end together
}
}
}
let pt = to_screen(c.x, c.y);
frame.push(POINTER_TYPE_INFO {
r#type: PT_TOUCH,
Anonymous: POINTER_TYPE_INFO_0 {
touchInfo: POINTER_TOUCH_INFO {
pointerInfo: POINTER_INFO {
pointerType: PT_TOUCH,
pointerId: c.slot,
pointerFlags: flags,
ptPixelLocation: pt,
ptPixelLocationRaw: pt,
..Default::default()
},
touchFlags: TOUCH_FLAG_NONE,
touchMask: TOUCH_MASK_NONE,
..Default::default()
},
},
});
}
// SAFETY: `sh.dev.0` is the live owned device; `frame` is a live Vec the call only reads.
// Best-effort — a transient failure heals on the next event/refresh re-assertion.
if let Err(e) = unsafe { inject_following_desktop(sh.dev.0, &frame) } {
if !sh.fail_warned {
sh.fail_warned = true;
tracing::warn!(error = %e, contacts = frame.len(), "touch inject failed");
} else {
tracing::trace!(error = %e, "touch inject failed (transient)");
}
} else {
sh.fail_warned = false;
}
}
@@ -1,6 +1,6 @@
//! Windows input injection via `SendInput` (Win32 KeyboardAndMouse) — the Windows analogue of
//! [`super::wlr`]: absolute mouse normalized to the virtual desktop, relative mouse for games,
//! scancode keyboard, scroll, buttons. Survives UAC/lock desktop switches with Sunshine's
//! [`super::wlr`]: absolute mouse mapped over the streamed output's rect
//! ([`crate::stream_target`]), relative mouse for games, scancode keyboard, scroll, buttons. Survives UAC/lock desktop switches with Sunshine's
//! retry-on-failure model: the thread stays bound to its desktop and only reattaches
//! (`OpenInputDesktop`/`SetThreadDesktop`) when `SendInput` reports a short write (the input
//! desktop switched) — no per-event reattach overhead.
@@ -32,20 +32,21 @@ use windows::Win32::UI::Input::KeyboardAndMouse::{
MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK,
MOUSEEVENTF_WHEEL, MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY,
};
use windows::Win32::UI::WindowsAndMessaging::{
GetForegroundWindow, GetSystemMetrics, GetWindowThreadProcessId, SM_CXVIRTUALSCREEN,
SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN, SM_YVIRTUALSCREEN,
};
use windows::Win32::UI::WindowsAndMessaging::{GetForegroundWindow, GetWindowThreadProcessId};
use super::InputInjector;
const ABS_MAX: f64 = 65535.0; // SendInput absolute coords are 0..65535 over the chosen surface.
const GENERIC_ALL: u32 = 0x1000_0000;
const XBUTTON1: u32 = 0x0001;
const XBUTTON2: u32 = 0x0002;
pub struct SendInputInjector {
desktop: Option<HDESK>,
/// PT_TOUCH synthetic device, created on the first wire-touch event (a session that never
/// touches never creates one). `None` after a failed create (pre-1809) — touch then stays
/// the historical no-op.
touch: Option<crate::pen::SyntheticTouch>,
touch_failed: bool,
}
// SAFETY: `SendInputInjector` holds only an `Option<HDESK>` (a desktop handle). The host creates
@@ -58,7 +59,11 @@ unsafe impl Send for SendInputInjector {}
impl SendInputInjector {
pub fn open() -> Result<Self> {
let mut me = Self { desktop: None };
let mut me = Self {
desktop: None,
touch: None,
touch_failed: false,
};
me.reattach_input_desktop(); // best-effort
tracing::info!("SendInput injector ready (Win32 KeyboardAndMouse)");
Ok(me)
@@ -156,13 +161,16 @@ impl InputInjector for SendInputInjector {
if w == 0 || h == 0 {
return Ok(()); // contract: drop zero extent
}
let (_vx, _vy, vw, vh) = virtual_desktop_rect();
// One virtual output spanning the virtual desktop: map client (0..w,0..h) -> 0..65535.
// Client (0..w,0..h) → the STREAMED output's desktop rect
// ([`crate::stream_target`]; the whole virtual desktop only as fallback) →
// 0..65535 over the virtual desktop for MOUSEEVENTF_VIRTUALDESK. Mapping over
// the desktop alone is the Extend-topology offset bug the pen plane exposed
// (design/pen-tablet-input.md): correct only when the streamed display IS the
// whole desktop.
let cx = (event.x.clamp(0, w as i32)) as f64 / w as f64;
let cy = (event.y.clamp(0, h as i32)) as f64 / h as f64;
let ax = (cx * ABS_MAX).round() as i32;
let ay = (cy * ABS_MAX).round() as i32;
let _ = (vw, vh); // virtual-desktop rect reserved for multi-output mapping
let px = crate::stream_target::map_normalized(cx, cy);
let (ax, ay) = crate::stream_target::desktop_px_to_virtualdesk(px);
let mi = MOUSEINPUT {
dx: ax,
dy: ay,
@@ -324,15 +332,33 @@ impl InputInjector for SendInputInjector {
}
self.send(&inputs)
}
// Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op.
// Gamepad goes through the XUSB backend.
InputKind::GamepadButton
| InputKind::GamepadAxis
| InputKind::GamepadState
| InputKind::GamepadRemove
| InputKind::GamepadArrival
| InputKind::TouchDown
| InputKind::TouchMove
| InputKind::TouchUp => Ok(()),
| InputKind::GamepadArrival => Ok(()),
// Wire touch → the PT_TOUCH synthetic pointer device (design/pen-tablet-input.md
// §6; closes the historical SendInput no-op). Lazily created — a session that never
// touches never creates one; a pre-1809 create failure latches back to the no-op.
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp => {
if self.touch.is_none() && !self.touch_failed {
match crate::pen::SyntheticTouch::create() {
Ok(t) => self.touch = Some(t),
Err(e) => {
self.touch_failed = true;
tracing::warn!(
error = %format!("{e:#}"),
"touch: synthetic pointer unavailable — wire touch stays a no-op"
);
}
}
}
if let Some(t) = self.touch.as_mut() {
t.apply(event);
}
Ok(())
}
}
}
}
@@ -351,19 +377,6 @@ fn key(ki: KEYBDINPUT) -> INPUT {
}
}
fn virtual_desktop_rect() -> (i32, i32, i32, i32) {
// SAFETY: each `GetSystemMetrics` takes a single by-value `SYSTEM_METRICS_INDEX` constant and
// returns an `i32`; it dereferences no pointer and has no side effects — FFI-`unsafe` only.
unsafe {
(
GetSystemMetrics(SM_XVIRTUALSCREEN),
GetSystemMetrics(SM_YVIRTUALSCREEN),
GetSystemMetrics(SM_CXVIRTUALSCREEN),
GetSystemMetrics(SM_CYVIRTUALSCREEN),
)
}
}
// VKs Windows wants flagged extended even when the scancode high bits aren't set: the editing
// cluster (Ins/Del/Home/End/PgUp/PgDn = 0x21..0x28, 0x2D, 0x2E), the Win keys (0x5B/0x5C/0x5D),
// RCtrl (0xA3), RAlt (0xA5), Pause (0x90). MAPVK_VK_TO_VSC_EX already encodes E0 for most; this is a
@@ -0,0 +1,181 @@
//! The streamed display every absolute coordinate maps into (design/pen-tablet-input.md field
//! fix). Pen, touch, and absolute-mouse positions arrive normalized to the STREAMED output's
//! frame, but the injectors historically mapped them over the whole virtual desktop — correct
//! only when the virtual display is the sole active display (Exclusive topology, normalized to
//! origin). In Extend — a physical monitor kept on beside the virtual output, or an Exclusive
//! isolate degraded to the keep-physicals fallback — the streamed output sits at a non-zero
//! origin, so every sample landed shifted and mis-scaled (the pen exposed it first: a stylus is
//! strictly absolute, with no closed-loop correction onto the target like a cursor).
//!
//! The host publishes the streamed output's CCD target id at capture bring-up
//! ([`set_stream_target`]); the mapping sites resolve its CURRENT desktop rect through
//! [`pf_win_display::win_display::source_desktop_rect`] — the same resolver the cursor-readback
//! poller maps frames with, so the two directions always agree — TTL-cached because a
//! group-layout re-arrange moves a live output's origin mid-session. With no target set, or none
//! resolved yet, mapping falls back to the whole virtual desktop: the historical behavior, still
//! correct for Exclusive topology and the client-less devtest paths.
use std::sync::Mutex;
use std::time::{Duration, Instant};
use windows::Win32::UI::WindowsAndMessaging::{
GetSystemMetrics, SM_CXVIRTUALSCREEN, SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN, SM_YVIRTUALSCREEN,
};
/// `(x, y, w, h)` in desktop coordinates, physical pixels (`source_desktop_rect` order).
type Rect = (i32, i32, i32, i32);
/// How long a resolved rect stays fresh: long enough that the CCD query cost vanishes at input
/// rates (pen samples + the 40 ms refresh threads), short enough that a mid-session layout move
/// (a parallel session joining the auto-row) is picked up within a blink.
const RECT_TTL: Duration = Duration::from_millis(250);
struct State {
target_id: Option<u32>,
rect: Option<Rect>,
queried: Option<Instant>,
}
static STATE: Mutex<State> = Mutex::new(State {
target_id: None,
rect: None,
queried: None,
});
/// Publish the streamed output (its CCD target id) that absolute input maps into. The host calls
/// this at capture bring-up; it is never cleared at teardown — a deactivated target simply stops
/// resolving (the last-known rect is kept, and nothing injects between sessions), and the next
/// session's bring-up re-targets. One slot per process: with parallel sessions the LAST bring-up
/// wins for every session's absolute input — per-session routing needs source-tagged input
/// events (parallel-displays plan) and the single slot is never worse than the historical
/// whole-desktop mapping.
pub fn set_stream_target(target_id: Option<u32>) {
let mut st = STATE.lock().unwrap();
if st.target_id != target_id {
tracing::info!(?target_id, "absolute-input stream target set");
st.target_id = target_id;
st.rect = None;
st.queried = None;
}
}
/// The streamed output's current desktop rect, TTL-cached. `None` = no target set / never
/// resolved (callers fall back to the whole virtual desktop).
fn stream_rect() -> Option<Rect> {
let mut st = STATE.lock().unwrap();
let target_id = st.target_id?;
let fresh = st.queried.is_some_and(|at| at.elapsed() < RECT_TTL);
if !fresh {
st.queried = Some(Instant::now());
// SAFETY: read-only QueryDisplayConfig over owned locals (`source_desktop_rect`'s
// contract — the same call the cursor-readback poller makes at spawn).
match unsafe { pf_win_display::win_display::source_desktop_rect(target_id) } {
Some(r) => {
if st.rect != Some(r) {
tracing::info!(target_id, rect = ?r, "stream-target desktop rect resolved");
}
st.rect = Some(r);
}
// Not an active path right now (teardown, or a topology commit in flight): keep the
// last-known rect — snapping mid-stroke to the whole-desktop mapping would visibly
// jump, and after teardown nothing injects until the next session re-targets.
None => {
if st.rect.is_some() {
tracing::debug!(
target_id,
"stream target not an active path — keeping last rect"
);
}
}
}
}
st.rect
}
/// Desktop-space pixel for a normalized `[0,1]²` coordinate over the streamed output's rect,
/// falling back to the whole virtual desktop when no stream target is live.
pub(crate) fn map_normalized(nx: f64, ny: f64) -> (i32, i32) {
map_into(stream_rect().unwrap_or_else(virtual_desktop_rect), nx, ny)
}
/// Pure mapping: `[0,1]²` over `(x, y, w, h)`, inclusive edges (1.0 lands on the last pixel).
fn map_into((x, y, w, h): Rect, nx: f64, ny: f64) -> (i32, i32) {
(
x + (nx.clamp(0.0, 1.0) * (w - 1).max(0) as f64).round() as i32,
y + (ny.clamp(0.0, 1.0) * (h - 1).max(0) as f64).round() as i32,
)
}
/// The virtual-desktop bounds `(x, y, w, h)` — the mapping fallback, and the surface
/// `MOUSEEVENTF_VIRTUALDESK` absolute coordinates normalize over.
pub(crate) fn virtual_desktop_rect() -> Rect {
// SAFETY: each `GetSystemMetrics` takes a single by-value `SYSTEM_METRICS_INDEX` constant and
// returns an `i32`; it dereferences no pointer and has no side effects — FFI-`unsafe` only.
unsafe {
(
GetSystemMetrics(SM_XVIRTUALSCREEN),
GetSystemMetrics(SM_YVIRTUALSCREEN),
GetSystemMetrics(SM_CXVIRTUALSCREEN).max(1),
GetSystemMetrics(SM_CYVIRTUALSCREEN).max(1),
)
}
}
/// A desktop-space pixel as the `MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_VIRTUALDESK` 0..65535
/// coordinate pair `SendInput` wants.
pub(crate) fn desktop_px_to_virtualdesk(px: (i32, i32)) -> (i32, i32) {
px_to_abs(virtual_desktop_rect(), px)
}
/// SendInput absolute coordinates span 0..65535 over the chosen surface.
const ABS_MAX: f64 = 65535.0;
/// Pure normalization: a desktop pixel inside `(x, y, w, h)` → 0..65535 over that surface.
fn px_to_abs((vx, vy, vw, vh): Rect, (px, py): (i32, i32)) -> (i32, i32) {
(
((px - vx) as f64 * ABS_MAX / (vw - 1).max(1) as f64).round() as i32,
((py - vy) as f64 * ABS_MAX / (vh - 1).max(1) as f64).round() as i32,
)
}
#[cfg(test)]
mod tests {
use super::*;
/// The Extend-topology field bug: physical 1920x1080 at (0,0), streamed virtual 2560x1440
/// beside it at (1920,0) — samples must land inside the VIRTUAL output, not at the desktop
/// origin.
#[test]
fn maps_over_the_streamed_rect_not_the_desktop() {
let r = (1920, 0, 2560, 1440);
assert_eq!(map_into(r, 0.0, 0.0), (1920, 0));
assert_eq!(map_into(r, 1.0, 1.0), (1920 + 2559, 1439));
assert_eq!(map_into(r, 0.5, 0.5), (1920 + 1280, 720));
}
#[test]
fn clamps_out_of_range_and_handles_negative_origins() {
// An output placed LEFT of / ABOVE the primary has a negative desktop origin.
let r = (-2560, -100, 2560, 1440);
assert_eq!(map_into(r, 0.0, 0.0), (-2560, -100));
assert_eq!(map_into(r, 2.0, -1.0), (-2560 + 2559, -100));
}
#[test]
fn degenerate_rect_pins_to_its_origin() {
assert_eq!(map_into((10, 20, 0, 0), 0.7, 0.7), (10, 20));
}
/// The VIRTUALDESK round trip: win32k maps an absolute coordinate back to a pixel roughly as
/// `px = ax * vw / 65536` (floor) — edge pixels and the streamed output's origin must survive.
#[test]
fn virtualdesk_normalization_round_trips() {
let v = (0, 0, 4480, 1080);
assert_eq!(px_to_abs(v, (0, 0)), (0, 0));
assert_eq!(px_to_abs(v, (4479, 1079)), (65535, 65535));
let (ax, _) = px_to_abs(v, (1920, 0));
assert_eq!((ax as i64 * 4480 / 65536) as i32, 1920);
// Negative-origin desktops (a monitor left of the primary) still normalize from 0.
let v = (-2560, 0, 4480, 1440);
assert_eq!(px_to_abs(v, (-2560, 0)), (0, 0));
}
}
+78
View File
@@ -197,6 +197,50 @@ pub fn text_input_supported() -> bool {
false
}
/// Whether this host can inject full-fidelity stylus input (`HOST_CAP_PEN` —
/// design/pen-tablet-input.md): Linux only, via the [`pen::VirtualPen`] uinput tablet, so the
/// probe is "can we open /dev/uinput" (the same permission the virtual gamepads need) plus the
/// `PUNKTFUNK_PEN=0` operator kill-switch. Consulted at Welcome time; clients without the bit
/// keep folding pen into touch/pointer. Windows PT_PEN synthetic pointers are the design's P3.
#[cfg(target_os = "linux")]
pub fn pen_supported() -> bool {
if std::env::var("PUNKTFUNK_PEN").as_deref() == Ok("0") {
return false;
}
// SAFETY: 'static NUL-terminated path literal; `open` returns a fresh fd (or -1) and
// retains nothing.
let fd = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if fd < 0 {
return false;
}
// SAFETY: `fd >= 0` is the fd opened above, owned by no one else; closed exactly once here.
unsafe { libc::close(fd) };
true
}
/// Windows: pen (and touch) inject via synthetic pointer devices — available on Win10 1809+,
/// probed by actually creating (and immediately destroying) a PT_PEN device. Same
/// `PUNKTFUNK_PEN=0` kill-switch as Linux. The probe result also stands in for PT_TOUCH
/// (both APIs arrived together in 1809).
#[cfg(target_os = "windows")]
pub fn pen_supported() -> bool {
if std::env::var("PUNKTFUNK_PEN").as_deref() == Ok("0") {
return false;
}
pen::synthetic_pen_available()
}
/// See the Linux/Windows variants — no pen injection elsewhere.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn pen_supported() -> bool {
false
}
#[path = "inject/service.rs"]
mod service;
pub use service::InjectorService;
@@ -362,6 +406,40 @@ pub mod gamepad {
pub fn pump_rumble(&mut self, _send: impl FnMut(u16, u16, u16)) {}
}
}
/// Linux: the "Punktfunk Pen" uinput virtual tablet (design/pen-tablet-input.md §5) — the
/// per-session stylus device the native pen plane injects through.
#[cfg(target_os = "linux")]
#[path = "inject/linux/pen.rs"]
pub mod pen;
/// Windows: PT_PEN/PT_TOUCH synthetic pointer devices (design/pen-tablet-input.md §6).
/// `pen::VirtualPen` here is the PT_PEN device; `pen::SyntheticTouch` backs the SendInput
/// injector's wire-touch path.
#[cfg(target_os = "windows")]
#[path = "inject/windows/pointer_windows.rs"]
pub mod pen;
/// Windows: the streamed output's desktop rect that every absolute coordinate (pen, touch,
/// absolute mouse) maps into — published by the host at capture bring-up, resolved through the
/// CCD source rect (the cursor-readback poller's resolver, so both directions agree). Mapping
/// over the whole virtual desktop instead is the Extend-topology offset bug the pen exposed
/// (design/pen-tablet-input.md).
#[cfg(target_os = "windows")]
#[path = "inject/windows/stream_target.rs"]
pub mod stream_target;
#[cfg(target_os = "windows")]
pub use stream_target::set_stream_target;
/// Stub — pen injection needs the Linux uinput tablet or Windows synthetic pointers;
/// `pen_supported()` is false here, so no host advertises the cap and no batches arrive.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub mod pen {
use anyhow::{bail, Result};
pub struct VirtualPen;
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
bail!("no pen injection backend on this platform")
}
pub fn apply_batch(&mut self, _transitions: &[punktfunk_core::quic::PenTransition]) {}
}
}
#[cfg(target_os = "linux")]
#[path = "inject/linux/kwin_fake_input.rs"]
mod kwin_fake_input;
+144 -37
View File
@@ -5,10 +5,19 @@
//! negotiated it (`HOST_CAP_CURSOR` in the Welcome — the host stopped compositing then) and
//! only applied while the DESKTOP mouse model is engaged: under capture the pointer is
//! relative-locked (SDL hides it) and games draw their own cursor in-frame.
//!
//! The host sends the bitmap in host-FRAMEBUFFER pixels, whose size tracks the host virtual
//! display's DPI scaling (32 px at 100%, 96 px at 300%). Drawn 1:1 it balloons on a high-DPI
//! host; instead we scale it by the SAME aspect-fit factor the video is drawn at
//! (`min(window_px/mode)`), so the pointer stays sized to the streamed desktop at any host
//! scaling. SDL cursors are fixed-size from their surface (no draw-time scaling), so we cache
//! shapes RAW and resample per install — rebuilding when the serial OR the fit changes.
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR};
use sdl3::mouse::{Cursor, MouseUtil, SystemCursor};
use sdl3::pixels::PixelFormat;
use sdl3::surface::Surface;
use std::collections::HashMap;
use std::time::Duration;
@@ -17,14 +26,27 @@ use std::time::Duration;
/// on the reliable stream via the serial-miss path).
const SHAPE_CACHE_MAX: usize = 64;
/// A forwarded cursor shape held RAW (host-framebuffer-pixel bytes + hotspot), so it can be
/// rebuilt into a scaled OS cursor whenever the video-fit changes (a window resize). Caching a
/// fixed-size `Cursor` instead would freeze the pointer at its build-time size.
struct RawShape {
rgba: Vec<u8>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
}
pub struct CursorChannel {
/// The Welcome carried `HOST_CAP_CURSOR` — the host forwards instead of compositing.
negotiated: bool,
/// Serial → built OS cursor. An SDL `Cursor` must outlive its `set()`, so the cache owns
/// every shape ever applied this session (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, Cursor>,
/// The serial currently installed via `Cursor::set` (`None` = default/system cursor).
installed: Option<u32>,
/// Serial → raw forwarded shape (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, RawShape>,
/// The serial + fit scale the currently-installed OS cursor was built at (`None` =
/// default/system cursor). A change in EITHER forces a rebuild.
installed: Option<(u32, f32)>,
/// Keeps the installed `Cursor` alive — SDL requires it to outlive its `set()`.
installed_cursor: Option<Cursor>,
/// Latest `0xD0` state (latest-wins across a drained batch).
state: Option<CursorState>,
}
@@ -39,6 +61,7 @@ impl CursorChannel {
negotiated,
shapes: HashMap::new(),
installed: None,
installed_cursor: None,
state: None,
}
}
@@ -57,8 +80,16 @@ impl CursorChannel {
/// Drain the two planes and apply the newest state — once per run-loop iteration.
/// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then
/// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns
/// it, and released the system cursor must look normal.
pub fn pump(&mut self, connector: &NativeClient, mouse: &MouseUtil, desktop_active: bool) {
/// it, and released the system cursor must look normal. `fit_scale` is host-framebuffer
/// pixels → window pixels (the aspect-fit factor the video is drawn at); the shape is
/// resampled by it so the pointer matches the streamed desktop at any host DPI.
pub fn pump(
&mut self,
connector: &NativeClient,
mouse: &MouseUtil,
desktop_active: bool,
fit_scale: f32,
) {
if !self.negotiated {
return;
}
@@ -68,30 +99,25 @@ impl CursorChannel {
self.shapes.clear();
self.installed = None;
}
let mut data = shape.rgba;
let built = sdl3::surface::Surface::from_data(
&mut data,
shape.w as u32,
shape.h as u32,
shape.w as u32 * 4,
sdl3::pixels::PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())
.and_then(|surf| {
Cursor::from_surface(&surf, shape.hot_x as i32, shape.hot_y as i32)
.map_err(|e| e.to_string())
});
match built {
Ok(cursor) => {
// A re-sent serial replaces its entry; force re-install if it's current.
if self.installed == Some(shape.serial) {
self.installed = None;
}
self.shapes.insert(shape.serial, cursor);
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
let (w, h) = (shape.w as u32, shape.h as u32);
if w == 0 || h == 0 || shape.rgba.len() < (w * h * 4) as usize {
tracing::warn!(w, h, "cursor shape malformed — ignored");
continue;
}
// A re-sent serial replaces its entry; force re-install if it's current.
if matches!(self.installed, Some((s, _)) if s == shape.serial) {
self.installed = None;
}
self.shapes.insert(
shape.serial,
RawShape {
rgba: shape.rgba,
w,
h,
hot_x: shape.hot_x as u32,
hot_y: shape.hot_y as u32,
},
);
}
while let Ok(st) = connector.next_cursor_state(Duration::ZERO) {
self.state = Some(st); // latest wins
@@ -101,17 +127,25 @@ impl CursorChannel {
// Capture mode / released: hand the cursor back to the system default so a
// released pointer over the window doesn't wear the host's shape.
if self.installed.take().is_some() {
Cursor::from_system(SystemCursor::Arrow)
.map(|c| c.set())
.ok();
if let Ok(c) = Cursor::from_system(SystemCursor::Arrow) {
c.set();
self.installed_cursor = Some(c); // keep it alive past set()
}
}
return;
}
let Some(st) = self.state else { return };
if st.visible() && self.installed != Some(st.serial) {
if let Some(cursor) = self.shapes.get(&st.serial) {
cursor.set();
self.installed = Some(st.serial);
if st.visible() && self.installed != Some((st.serial, fit_scale)) {
if let Some(shape) = self.shapes.get(&st.serial) {
match build_scaled_cursor(shape, fit_scale) {
Ok(cursor) => {
cursor.set();
self.installed = Some((st.serial, fit_scale));
self.installed_cursor = Some(cursor); // outlive set()
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
}
// Serial miss: the (reliable) shape hasn't landed yet — keep the previous
// cursor for the RTT rather than flashing default.
@@ -123,3 +157,76 @@ impl CursorChannel {
}
}
}
/// Resample a raw shape by `fit_scale` and build an SDL color cursor from it. The hotspot scales
/// with the bitmap so the click point stays true. `fit_scale <= 0` (or a degenerate result) is
/// clamped so we always hand SDL a ≥1×1 surface.
fn build_scaled_cursor(shape: &RawShape, fit_scale: f32) -> Result<Cursor, String> {
let scale = if fit_scale.is_finite() && fit_scale > 0.0 {
fit_scale
} else {
1.0
};
let dw = ((shape.w as f32 * scale).round() as u32).max(1);
let dh = ((shape.h as f32 * scale).round() as u32).max(1);
let hot_x = ((shape.hot_x as f32 * scale).round() as u32).min(dw - 1) as i32;
let hot_y = ((shape.hot_y as f32 * scale).round() as u32).min(dh - 1) as i32;
if dw == shape.w && dh == shape.h {
// 1:1 fit (mode == window) — no resample, feed the bytes straight through.
let mut data = shape.rgba.clone();
let surf = Surface::from_data(
&mut data,
shape.w,
shape.h,
shape.w * 4,
PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())?;
return Cursor::from_surface(&surf, hot_x, hot_y).map_err(|e| e.to_string());
}
let mut scaled = resample_rgba(&shape.rgba, shape.w, shape.h, dw, dh);
let surf = Surface::from_data(&mut scaled, dw, dh, dw * 4, PixelFormat::RGBA32)
.map_err(|e| e.to_string())?;
Cursor::from_surface(&surf, hot_x, hot_y).map_err(|e| e.to_string())
}
/// Area-average resample of a straight-alpha RGBA bitmap `(sw×sh) → (dw×dh)`. Averaging is done on
/// PREMULTIPLIED colour (weighting each source texel by its alpha) so transparent-pixel colour
/// can't bleed into the fringe, then un-premultiplied back to straight alpha. Handles both down-
/// and up-scale; for a cursor the common case is a downscale (high-DPI host → smaller pointer).
fn resample_rgba(src: &[u8], sw: u32, sh: u32, dw: u32, dh: u32) -> Vec<u8> {
let mut out = vec![0u8; (dw * dh * 4) as usize];
let fx = sw as f32 / dw as f32;
let fy = sh as f32 / dh as f32;
for dy in 0..dh {
let sy0 = (dy as f32 * fy).floor() as u32;
let sy1 = (((dy + 1) as f32 * fy).ceil() as u32).clamp(sy0 + 1, sh);
for dx in 0..dw {
let sx0 = (dx as f32 * fx).floor() as u32;
let sx1 = (((dx + 1) as f32 * fx).ceil() as u32).clamp(sx0 + 1, sw);
let (mut r, mut g, mut b, mut a_sum, mut n) = (0f32, 0f32, 0f32, 0f32, 0f32);
for sy in sy0..sy1 {
for sx in sx0..sx1 {
let i = ((sy * sw + sx) * 4) as usize;
let a = src[i + 3] as f32 / 255.0;
r += src[i] as f32 * a;
g += src[i + 1] as f32 * a;
b += src[i + 2] as f32 * a;
a_sum += a;
n += 1.0;
}
}
let di = ((dy * dw + dx) * 4) as usize;
if a_sum > 0.0 {
out[di] = (r / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 1] = (g / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 2] = (b / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 3] = (a_sum / n * 255.0).round().clamp(0.0, 255.0) as u8;
}
// else fully transparent — already zero-filled.
}
}
out
}
+31 -1
View File
@@ -360,6 +360,28 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// identity and the session window gets the default-Wayland icon (the Linux analog of
// the AppUserModelID adoption above).
sdl3::hint::set("SDL_APP_ID", "io.unom.Punktfunk");
// `PUNKTFUNK_DRM_CARD=<n>` → SDL's KMSDRM device index, for a compositor-less (kiosk/embedded)
// run. SDL enumerates /dev/dri/card* and takes the first one it can open, which on a
// multi-GPU box is regularly the WRONG one: measured on a two-card machine it chose the card
// a live compositor already held DRM master on and failed at swapchain creation, while the
// idle card with the connected display sat unused. There is no reliable way to pick from
// inside the process (detecting "already mastered" needs the ioctl that taking master IS), so
// this stays an explicit operator choice rather than fragile auto-detection.
// Ignored unless the kmsdrm backend is actually in use.
if let Ok(card) = std::env::var("PUNKTFUNK_DRM_CARD") {
if card.chars().all(|c| c.is_ascii_digit()) && !card.is_empty() {
tracing::info!(
card,
"PUNKTFUNK_DRM_CARD: pinning SDL's KMSDRM device index"
);
sdl3::hint::set("SDL_KMSDRM_DEVICE_INDEX", &card);
} else {
tracing::warn!(
card,
"PUNKTFUNK_DRM_CARD must be a card NUMBER (e.g. 0) — ignoring"
);
}
}
let sdl = sdl3::init().context("SDL init")?;
let video = sdl.video().context("SDL video")?;
let events = sdl.event().context("SDL events")?;
@@ -773,12 +795,20 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// Cursor channel (M2): drain forwarded shape/state and drive the local OS cursor —
// only meaningful in the desktop mouse model (capture's relative lock hides it).
if let Some(st) = stream.as_mut() {
// Host-framebuffer px → window px: the aspect-fit factor the video is drawn at
// (same `min(surface/content)` as `finger_to_content`). The forwarded pointer is
// resampled by it so a high-DPI host's oversized bitmap lands sized to the streamed
// desktop rather than ballooning. 1:1 until the first frame gives `last_video`.
let fit_scale = st.last_video.map_or(1.0, |(vw, vh)| {
let (pw, ph) = window.size_in_pixels();
(pw as f32 / vw.max(1) as f32).min(ph as f32 / vh.max(1) as f32)
});
if let (Some(chan), Some(c)) = (st.cursor_chan.as_mut(), st.connector.as_ref()) {
let desktop_active = st
.capture
.as_ref()
.is_some_and(|cap| cap.captured() && cap.desktop());
chan.pump(c, &mouse, desktop_active);
chan.pump(c, &mouse, desktop_active, fit_scale);
// §8 mid-stream render flip: tell the host who renders the pointer whenever
// the local model changes. Desktop-active = we draw it (host excludes +
// forwards); anything else — the capture model OR a released pointer — the
+28 -3
View File
@@ -3,9 +3,34 @@
use super::setup::pick_formats;
use super::{OverlayPipe, Presenter};
use crate::csc::CscPass;
use anyhow::{Context as _, Result};
use anyhow::{anyhow, Context as _, Result};
use ash::vk;
/// Extra guidance appended to a swapchain-creation failure when SDL is on the KMSDRM backend —
/// i.e. a compositor-less kiosk/embedded run, where the bare Vulkan error is close to useless.
///
/// Measured on an NVIDIA proprietary + KMSDRM box: SDL opens the card, Vulkan enumerates the GPU
/// *and* the display (`VK_KHR_display` reports the connected HDMI connector), then
/// `vkCreateSwapchainKHR` returns ERROR_INITIALIZATION_FAILED — as root, with `nvidia_drm.modeset`
/// on, on a card no compositor was using. So it is neither permissions nor DRM master: NVIDIA's
/// direct-to-display path wants the display leased (`vkAcquireDrmDisplayEXT`) and SDL's KMSDRM
/// surface path does not do that for it. Empty on every other backend, where the message would be
/// noise.
fn kmsdrm_swapchain_hint() -> String {
let kmsdrm = std::env::var("SDL_VIDEODRIVER").is_ok_and(|v| v.eq_ignore_ascii_case("kmsdrm"));
if !kmsdrm {
return String::new();
}
let card = std::env::var("PUNKTFUNK_DRM_CARD").unwrap_or_else(|_| "unset".into());
format!(
" — under SDL_VIDEODRIVER=kmsdrm (PUNKTFUNK_DRM_CARD={card}). Check, in order: the card \
has a CONNECTED connector (`cat /sys/class/drm/card*-*/status`); nothing else holds DRM \
master on it (a running compositor does pin another card with PUNKTFUNK_DRM_CARD=<n>); \
and the driver is Mesa. NVIDIA's proprietary direct-display path is known to fail here \
even as root with a display Vulkan can enumerate."
)
}
impl Presenter {
/// (Re)build the swapchain for the window's current pixel size. Also the resize path.
pub fn recreate_swapchain(&mut self, window: &sdl3::video::Window) -> Result<()> {
@@ -66,8 +91,8 @@ impl Presenter {
.present_mode(self.present_mode)
.clipped(true)
.old_swapchain(old);
let swapchain =
unsafe { self.swap_d.create_swapchain(&info, None) }.context("vkCreateSwapchainKHR")?;
let swapchain = unsafe { self.swap_d.create_swapchain(&info, None) }
.map_err(|e| anyhow!("vkCreateSwapchainKHR: {e}{}", kmsdrm_swapchain_hint()))?;
// The old swapchain and everything tied to its images dies NOW: the fence
// quiesce covered our own command buffers, the queue drain above covered the
// presentation engine's semaphore waits — nothing can still reference them.
+5 -2
View File
@@ -11,7 +11,7 @@ use crate::dmabuf;
use anyhow::{anyhow, bail, Context as _, Result};
use ash::vk;
use ash::vk::Handle as _;
use std::ffi::CString;
use std::ffi::{c_char, CString};
impl Presenter {
/// Bring up instance → surface → device → swapchain over an SDL window.
@@ -40,7 +40,10 @@ impl Presenter {
.iter()
.map(|e| CString::new(e.as_str()).unwrap())
.collect();
let ext_ptrs: Vec<*const i8> = ext_cstrings.iter().map(|e| e.as_ptr()).collect();
// `c_char`, not `i8`: plain `char` is SIGNED on x86_64 but UNSIGNED on aarch64, so a
// hardcoded `*const i8` compiles on the desktop targets and fails to match ash's
// `&[*const c_char]` on ARM.
let ext_ptrs: Vec<*const c_char> = ext_cstrings.iter().map(|e| e.as_ptr()).collect();
let instance = unsafe {
entry.create_instance(
&vk::InstanceCreateInfo::default()
+3
View File
@@ -45,6 +45,9 @@ futures-util = "0.3"
wayland-client = "0.31"
wayland-scanner = "0.31"
wayland-backend = "0.3"
# wayland-scanner emits `bitflags::bitflags!` for the KDE output-device protocol's bitfield enums
# (kde-output-device-v2 `capability`/`flags`); needs the crate in scope (kwin_output_mgmt.rs).
bitflags = "2"
[target.'cfg(target_os = "windows")'.dependencies]
# The host<->driver wire contract for the pf-vdisplay IddCx backend (control IOCTLs + Pod structs).
@@ -0,0 +1,653 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="kde_output_device_v2">
<copyright><![CDATA[
SPDX-FileCopyrightText: 2008-2011 Kristian Høgsberg
SPDX-FileCopyrightText: 2010-2011 Intel Corporation
SPDX-FileCopyrightText: 2012-2013 Collabora, Ltd.
SPDX-FileCopyrightText: 2015 Sebastian Kügler <sebas@kde.org>
SPDX-FileCopyrightText: 2021 Méven Car <meven.car@enioka.com>
SPDX-License-Identifier: MIT-CMU
]]></copyright>
<interface name="kde_output_device_registry_v2" version="24">
<description summary="output devices">
This interface can be used to list output devices.
If this global is bound with a version less than 21, the unsupported_version
protocol error will be posted.
</description>
<enum name="error">
<description summary="kde_output_device_registry_v2 error values">
These errors can be emitted in response to some requests.
</description>
<entry name="unsupported_version" value="0"
summary="the registry was bound with an unsupported version"/>
</enum>
<event name="finished" type="destructor" since="21">
<description summary="no new output announcements">
This event is sent in response to the stop request. The compositor will
immediately destroy the object after sending this event.
</description>
</event>
<request name="stop" since="21">
<description summary="stop receiving updates">
This request indicates that the client no longer wants to receive new
output announcements. The compositor will send the
kde_output_device_registry_v2.finished event in response to this request.
The compositor may still send new output announcements after calling this
request until the kde_output_device_registry_v2.finished event is sent.
</description>
</request>
<event name="output" since="21">
<description summary="new available output">
This event is sent when a new output is connected or after binding this
global to list all available outputs.
</description>
<arg name="output" type="new_id" interface="kde_output_device_v2"/>
</event>
</interface>
<interface name="kde_output_device_v2" version="24">
<description summary="output configuration representation">
An output device describes a display device available to the compositor.
output_device is similar to wl_output, but focuses on output
configuration management.
A client can query all global output_device objects to enlist all
available display devices, even those that may currently not be
represented by the compositor as a wl_output.
The client sends configuration changes to the server through the
outputconfiguration interface, and the server applies the configuration
changes to the hardware and signals changes to the output devices
accordingly.
This object is published as global during start up for every available
display devices, or when one later becomes available, for example by
being hotplugged via a physical connector.
Warning! The protocol described in this file is a desktop environment
implementation detail. Regular clients must not use this protocol.
Backward incompatible changes may be added without bumping the major
version of the extension.
</description>
<enum name="subpixel">
<description summary="subpixel geometry information">
This enumeration describes how the physical pixels on an output are
laid out.
</description>
<entry name="unknown" value="0"/>
<entry name="none" value="1"/>
<entry name="horizontal_rgb" value="2"/>
<entry name="horizontal_bgr" value="3"/>
<entry name="vertical_rgb" value="4"/>
<entry name="vertical_bgr" value="5"/>
</enum>
<enum name="transform">
<description summary="transform from framebuffer to output">
This describes the transform, that a compositor will apply to a
surface to compensate for the rotation or mirroring of an
output device.
The flipped values correspond to an initial flip around a
vertical axis followed by rotation.
The purpose is mainly to allow clients to render accordingly and
tell the compositor, so that for fullscreen surfaces, the
compositor is still able to scan out directly client surfaces.
</description>
<entry name="normal" value="0"/>
<entry name="90" value="1"/>
<entry name="180" value="2"/>
<entry name="270" value="3"/>
<entry name="flipped" value="4"/>
<entry name="flipped_90" value="5"/>
<entry name="flipped_180" value="6"/>
<entry name="flipped_270" value="7"/>
</enum>
<event name="geometry">
<description summary="geometric properties of the output">
The geometry event describes geometric properties of the output.
The event is sent when binding to the output object and whenever
any of the properties change.
</description>
<arg name="x" type="int"
summary="x position within the global compositor space"/>
<arg name="y" type="int"
summary="y position within the global compositor space"/>
<arg name="physical_width" type="int"
summary="width in millimeters of the output"/>
<arg name="physical_height" type="int"
summary="height in millimeters of the output"/>
<arg name="subpixel" type="int"
summary="subpixel orientation of the output"/>
<arg name="make" type="string"
summary="textual description of the manufacturer"/>
<arg name="model" type="string"
summary="textual description of the model"/>
<arg name="transform" type="int"
summary="transform that maps framebuffer to output"/>
</event>
<event name="current_mode">
<description summary="current mode">
This event describes the mode currently in use for this head. It is only
sent if the output is enabled.
</description>
<arg name="mode" type="object" interface="kde_output_device_mode_v2"/>
</event>
<event name="mode">
<description summary="advertise available output modes and current one">
The mode event describes an available mode for the output.
When the client binds to the output_device object, the server sends this
event once for every available mode the output_device can be operated by.
There will always be at least one event sent out on initial binding,
which represents the current mode.
Later if an output changes, its mode event is sent again for the
eventual added modes and lastly the current mode. In other words, the
current mode is always represented by the latest event sent with the current
flag set.
The size of a mode is given in physical hardware units of the output device.
This is not necessarily the same as the output size in the global compositor
space. For instance, the output may be scaled, as described in
kde_output_device_v2.scale, or transformed, as described in
kde_output_device_v2.transform.
</description>
<arg name="mode" type="new_id" interface="kde_output_device_mode_v2"/>
</event>
<event name="done">
<description summary="sent all information about output">
This event is sent after all other properties have been
sent on binding to the output object as well as after any
other output property change have been applied later on.
This allows to see changes to the output properties as atomic,
even if multiple events successively announce them.
</description>
</event>
<event name="scale">
<description summary="output scaling properties">
This event contains scaling geometry information
that is not in the geometry event. It may be sent after
binding the output object or if the output scale changes
later. If it is not sent, the client should assume a
scale of 1.
A scale larger than 1 means that the compositor will
automatically scale surface buffers by this amount
when rendering. This is used for high resolution
displays where applications rendering at the native
resolution would be too small to be legible.
It is intended that scaling aware clients track the
current output of a surface, and if it is on a scaled
output it should use wl_surface.set_buffer_scale with
the scale of the output. That way the compositor can
avoid scaling the surface, and the client can supply
a higher detail image.
</description>
<arg name="factor" type="fixed" summary="scaling factor of output"/>
</event>
<event name="edid">
<description summary="advertise EDID data for the output">
The edid event encapsulates the EDID data for the outputdevice.
The event is sent when binding to the output object. The EDID
data may be empty, in which case this event is sent anyway.
If the EDID information is empty, you can fall back to the name
et al. properties of the outputdevice.
</description>
<arg name="raw" type="string" summary="base64-encoded EDID string"/>
</event>
<event name="enabled">
<description summary="output is enabled or disabled">
The enabled event notifies whether this output is currently
enabled and used for displaying content by the server.
The event is sent when binding to the output object and
whenever later on an output changes its state by becoming
enabled or disabled.
</description>
<arg name="enabled" type="int" summary="output enabled state"/>
</event>
<event name="uuid">
<description summary="A unique id for this outputdevice">
The uuid can be used to identify the output. It's controlled by
the server entirely. The server should make sure the uuid is
persistent across restarts. An empty uuid is considered invalid.
</description>
<arg name="uuid" type="string" summary="output devices ID"/>
</event>
<event name="serial_number">
<description summary="Serial Number">
Serial ID of the monitor, sent on startup before the first done event.
</description>
<arg name="serialNumber" type="string"
summary="textual representation of serial number"/>
</event>
<event name="eisa_id">
<description summary="EISA ID">
EISA ID of the monitor, sent on startup before the first done event.
</description>
<arg name="eisaId" type="string"
summary="textual representation of EISA identifier"/>
</event>
<enum name="capability" bitfield="true">
<description summary="describes capabilities of the outputdevice">
Describes what capabilities this device has.
</description>
<entry name="overscan" value="0x1"
summary="if this output_device can use overscan"/>
<entry name="vrr" value="0x2"
summary="if this outputdevice supports variable refresh rate"/>
<entry name="rgb_range" value="0x4"
summary="if setting the rgb range is possible"/>
<entry name="high_dynamic_range" value="0x8" since="3"
summary="if this outputdevice supports high dynamic range"/>
<entry name="wide_color_gamut" value="0x10" since="3"
summary="if this outputdevice supports a wide color gamut"/>
<entry name="auto_rotate" value="0x20" since="4"
summary="if this outputdevice supports autorotation"/>
<entry name="icc_profile" value="0x40" since="5"
summary="if this outputdevice supports icc profiles"/>
<entry name="brightness" value="0x80" since="9"
summary="if this outputdevice supports the brightness setting"/>
<entry name="built_in_color" value="0x100" since="12"
summary="if this outputdevice supports the built-in color profile"/>
<entry name="ddc_ci" value="0x200" since="14"
summary="if this outputdevice supports DDC/CI"/>
<entry name="max_bits_per_color" value="0x400" since="15"
summary="if this outputdevice supports setting max bpc"/>
<entry name="edr" value="0x800" since="16"
summary="if this outputdevice supports EDR"/>
<entry name="sharpness" value="0x1000" since="17"
summary="if this outputdevice supports the sharpness setting"/>
<entry name="custom_modes" value="0x2000" since="18"
summary="if this outputdevice supports custom modes"/>
<entry name="auto_brightness" value = "0x4000" since="19"/>
<entry name="hdr_icc_profile" value="0x8000" since="22"
summary="if this outputdevice supports HDR ICC profiles"/>
<entry name="abm_level" value="0x10000" since="23"
summary="if this outputdevice supports the abm level setting"/>
</enum>
<event name="capabilities">
<description summary="capability flags">
What capabilities this device has, sent on startup before the first
done event.
</description>
<arg name="flags" type="uint" enum="capability"/>
</event>
<event name="overscan">
<description summary="overscan">
Overscan value of the monitor in percent, sent on startup before the
first done event.
</description>
<arg name="overscan" type="uint"
summary="amount of overscan of the monitor"/>
</event>
<enum name="vrr_policy">
<description summary="describes vrr policy">
Describes when the compositor may employ variable refresh rate
</description>
<entry name="never" value="0"/>
<entry name="always" value="1"/>
<entry name="automatic" value="2"/>
</enum>
<event name="vrr_policy">
<description summary="Variable Refresh Rate Policy">
What policy the compositor will employ regarding its use of variable
refresh rate.
</description>
<arg name="vrr_policy" type="uint" enum="vrr_policy"/>
</event>
<enum name="rgb_range">
<description summary="describes RGB range policy">
Whether full or limited color range should be used
</description>
<entry name="automatic" value="0"/>
<entry name="full" value="1"/>
<entry name="limited" value="2"/>
</enum>
<event name="rgb_range">
<description summary="RGB range">
What rgb range the compositor is using for this output
</description>
<arg name="rgb_range" type="uint" enum="rgb_range"/>
</event>
<event name="name" since="2">
<description summary="Output's name">
Name of the output, it's useful to cross-reference to an zxdg_output_v1 and ultimately QScreen
</description>
<arg name="name" type="string"/>
</event>
<event name="high_dynamic_range" since="3">
<description summary="if HDR is enabled">
Whether or not high dynamic range is enabled for this output
</description>
<arg name="hdr_enabled" type="uint" summary="1 if enabled, 0 if disabled"/>
</event>
<event name="sdr_brightness" since="3">
<description summary="the brightness of sdr if hdr is enabled">
If high dynamic range is used, this value defines the brightness in nits for content
that's in standard dynamic range format. Note that while the value is in nits, that
doesn't necessarily translate to the same brightness on the screen.
</description>
<arg name="sdr_brightness" type="uint"/>
</event>
<event name="wide_color_gamut" since="3">
<description summary="if WCG is enabled">
Whether or not the use of a wide color gamut is enabled for this output
</description>
<arg name="wcg_enabled" type="uint" summary="1 if enabled, 0 if disabled"/>
</event>
<enum name="auto_rotate_policy">
<description summary="describes when auto rotate should be used"/>
<entry name="never" value="0"/>
<entry name="in_tablet_mode" value="1"/>
<entry name="always" value="2"/>
</enum>
<event name="auto_rotate_policy" since="4">
<description summary="describes when auto rotate is used"/>
<arg name="policy" type="uint" enum="auto_rotate_policy"/>
</event>
<event name="icc_profile_path" since="5">
<description summary="describes the path to the ICC profile used in SDR mode"/>
<arg name="profile_path" type="string"/>
</event>
<event name="brightness_metadata" since="6">
<description summary="metadata about the screen's brightness limits"/>
<arg name="max_peak_brightness" type="uint" summary="in nits"/>
<arg name="max_frame_average_brightness" type="uint" summary="in nits"/>
<arg name="min_brightness" type="uint" summary="in 0.0001 nits"/>
</event>
<event name="brightness_overrides" since="6">
<description summary="overrides for the screen's brightness limits"/>
<arg name="max_peak_brightness" type="int" summary="-1 for no override, positive values are the brightness in nits"/>
<arg name="max_average_brightness" type="int" summary="-1 for no override, positive values are the brightness in nits"/>
<arg name="min_brightness" type="int" summary="-1 for no override, positive values are the brightness in 0.0001 nits"/>
</event>
<event name="sdr_gamut_wideness" since="6">
<description summary="describes which gamut is assumed for sRGB applications">
This can be used to provide the colors users assume sRGB applications should have based on the
default experience on many modern sRGB screens.
</description>
<arg name="gamut_wideness" type="uint" summary="0 means rec.709 primaries, 10000 means native primaries"/>
</event>
<enum name="color_profile_source" since="7">
<description summary="which source the compositor should use for the color profile on an output"/>
<entry name="sRGB" value="0"/>
<entry name="ICC" value="1"/>
<entry name="EDID" value="2"/>
</enum>
<event name="color_profile_source" since="7">
<description summary="describes which source the compositor uses for the color profile on an output in SDR mode"/>
<arg name="source" type="uint" enum="color_profile_source"/>
</event>
<event name="brightness" since="8">
<description summary="brightness multiplier">
This is the brightness modifier of the output. It doesn't specify
any absolute values, but is merely a multiplier on top of other
brightness values, like sdr_brightness and brightness_metadata.
0 is the minimum brightness (not completely dark) and 10000 is
the maximum brightness.
This is currently only supported / meaningful while HDR is active.
</description>
<arg name="brightness" type="uint" summary="brightness in 0-10000"/>
</event>
<enum name="color_power_tradeoff">
<description summary="tradeoff between power and accuracy">
The compositor can do a lot of things that trade between
performance, power and color accuracy. This setting describes
a high level preference from the user about in which direction
that tradeoff should be made.
</description>
<entry name="efficiency" value="0" summary="prefer efficiency and performance"/>
<entry name="accuracy" value="1" summary="prefer accuracy"/>
</enum>
<event name="color_power_tradeoff" since="10">
<description summary="the preferred color/power tradeoff"/>
<arg name="preference" type="uint" enum="color_power_tradeoff"/>
</event>
<event name="dimming" since="11">
<description summary="dimming multiplier">
This is the dimming multiplier of the output. This is similar to
the brightness setting, except it's meant to be a temporary setting
only, not persistent and may be implemented differently depending
on the display.
0 is the minimum dimming factor (not completely dark) and 10000
means the output is not dimmed.
</description>
<arg name="multiplier" type="uint" summary="multiplier in 0-10000"/>
</event>
<event name="replication_source" since="13">
<description summary="source output for mirroring"/>
<arg name="source" type="string" summary="uuid of the source output"/>
</event>
<event name="ddc_ci_allowed" since="14">
<description summary="if DDC/CI should be used to control brightness etc.">
If the ddc_ci capability is present, this determines if settings
such as brightness, contrast or others should be set using DDC/CI.
</description>
<arg name="allowed" type="uint" summary="1 if allowed, 0 if disabled"/>
</event>
<event name="max_bits_per_color" since="15">
<description summary="override max bpc">
This limits the amount of bits per color that are sent to the display.
</description>
<arg name="max_bpc" type="uint" summary="0 for the default / automatic"/>
</event>
<event name="max_bits_per_color_range" since="15">
<description summary="range of max bits per color value"/>
<arg name="min_value" type="uint" summary="the minimum supported by the driver"/>
<arg name="max_value" type="uint" summary="the maximum supported by the driver"/>
</event>
<event name="automatic_max_bits_per_color_limit" since="15">
<description summary="if and to what value automatic max bpc is limited"/>
<arg name="max_bpc_limit" type="uint"
summary="which value automatic bpc gets limited to. 0 if not limited"/>
</event>
<enum name="edr_policy" since="16">
<description summary="when the compositor may make use of EDR"/>
<entry name="never" value="0"/>
<entry name="always" value="1"/>
</enum>
<event name="edr_policy" since="16">
<description summary="when the compositor may apply EDR">
When EDR is enabled, the compositor may increase the backlight beyond
the user-specified setting, in order to present HDR content on displays
without native HDR support.
This will usually result in better visuals, but also increases battery
usage.
</description>
<arg name="policy" type="uint" enum="edr_policy"/>
</event>
<event name="sharpness" since="17">
<description summary="sharpness strength">
This is the sharpness modifier of the output.
0 is sharpness disabled and 10000 is the maximum sharpness
</description>
<arg name="sharpness" type="uint" summary="sharpness in 0-10000"/>
</event>
<event name="priority" since="18">
<description summary="output priority">
Describes the position of the output in the output order list,
with lower values being earlier in the list. There's no specific
value the list has to start at, this value is only used in sorting
outputs.
Note that the output order protocol is not sufficient for this,
as an output may not be in the output order if it's disabled or
mirroring another screen.
</description>
<arg name="priority" type="uint" summary="priority"/>
</event>
<event name="auto_brightness" since="20">
<description summary="whether or not automatic brightness is enabled"/>
<arg name="enabled" type="uint" summary="1 for enabled, 0 for disabled"/>
</event>
<request name="release" type="destructor" since="21">
<description summary="destroy the output device">
This notifies the compositor that the client no longer wishes to use
the kde_output_device_v2 object.
</description>
</request>
<event name="removed" since="21">
<description summary="the output has been removed">
This event is sent when the output device is disconnected and no new
updates will be sent. The client should call the kde_output_device_v2.release
request after receiving this event.
</description>
</event>
<event name="hdr_icc_profile_path" since="22">
<description summary="describes the path to the ICC profile used in HDR mode"/>
<arg name="profile_path" type="string"/>
</event>
<event name="hdr_color_profile_source" since="22">
<description summary="describes which source the compositor uses for the color profile on an output in HDR mode"/>
<arg name="source" type="uint" enum="color_profile_source"/>
</event>
<event name="abm_level" since="23">
<description summary="allowed level of adaptive backlight modulation">
Adaptive backlight modulation is a feature that reduces the backlight
and increases contrast of colors on the screen to improve power usage.
</description>
<arg name="level" type="uint" summary="0 is off, 4 is the maximum level"/>
</event>
</interface>
<interface name="kde_output_device_mode_v2" version="24">
<description summary="output mode">
This object describes an output mode.
Some heads don't support output modes, in which case modes won't be
advertised.
Properties sent via this interface are applied atomically via the
kde_output_device.done event. No guarantees are made regarding the order
in which properties are sent.
</description>
<event name="size">
<description summary="mode size">
This event describes the mode size. The size is given in physical
hardware units of the output device. This is not necessarily the same as
the output size in the global compositor space. For instance, the output
may be scaled or transformed.
</description>
<arg name="width" type="int" summary="width of the mode in hardware units"/>
<arg name="height" type="int" summary="height of the mode in hardware units"/>
</event>
<event name="refresh">
<description summary="mode refresh rate">
This event describes the mode's fixed vertical refresh rate. It is only
sent if the mode has a fixed refresh rate.
</description>
<arg name="refresh" type="int" summary="vertical refresh rate in mHz"/>
</event>
<event name="preferred">
<description summary="mode is preferred">
This event advertises this mode as preferred.
</description>
</event>
<event name="removed">
<description summary="the mode has been destroyed">
The compositor will destroy the object immediately after sending this
event, so it will become invalid and the client should release any
resources associated with it.
</description>
</event>
<enum name="flags">
<description summary="mode flags"/>
<entry name="custom" value="0x1"/>
<entry name="reduced_blanking" value="0x2" deprecated-since="24"/>
</enum>
<event name="flags" since="19">
<description summary="mode flags">
This event describes the mode's flags.
</description>
<arg name="flags" type="uint" enum="flags"/>
</event>
<event name="cvt" since="24">
<description summary="cvt timings">
This event describes the CVT timings associated with the mode.
If an output mode has no CVT timings, this event will not be sent.
</description>
<arg name="dot_clock" type="uint" summary="pixel clock in kHz"/>
<arg name="hdisplay" type="uint" summary="horizontal display size"/>
<arg name="hsync_start" type="uint" summary="horizontal sync start"/>
<arg name="hsync_end" type="uint" summary="horizontal sync end"/>
<arg name="htotal" type="uint" summary="horizontal total size"/>
<arg name="hskew" type="uint" summary="horizontal skew"/>
<arg name="vdisplay" type="uint" summary="vertical display size"/>
<arg name="vsync_start" type="uint" summary="vertical sync start"/>
<arg name="vsync_end" type="uint" summary="vertical sync end"/>
<arg name="vtotal" type="uint" summary="vertical total size"/>
<arg name="vscan" type="uint" summary="vertical scan"/>
<arg name="flags" type="uint" summary="flags, see DRM_MODE_FLAG_*"/>
</event>
</interface>
</protocol>
@@ -0,0 +1,539 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="kde_output_management_v2">
<copyright><![CDATA[
SPDX-FileCopyrightText: 2008-2011 Kristian Høgsberg
SPDX-FileCopyrightText: 2010-2011 Intel Corporation
SPDX-FileCopyrightText: 2012-2013 Collabora, Ltd.
SPDX-FileCopyrightText: 2015 Sebastian Kügler <sebas@kde.org>
SPDX-FileCopyrightText: 2021 Méven Car <meven.car@enioka.com>
SPDX-FileCopyrightText: 2023 Xaver Hugl <xaver.hugl@kde.org>
SPDX-License-Identifier: MIT-CMU
]]></copyright>
<interface name="kde_output_management_v2" version="22">
<description summary="configuration of server outputs through clients">
This interface enables clients to set properties of output devices for screen
configuration purposes via the server. To this end output devices are referenced
by global kde_output_device_v2 objects.
outputmanagement (wl_global)
--------------------------
request:
* create_configuration -> outputconfiguration (wl_resource)
outputconfiguration (wl_resource)
--------------------------
requests:
* enable(outputdevice, bool)
* mode(outputdevice, mode)
* transformation(outputdevice, flag)
* position(outputdevice, x, y)
* apply
events:
* applied
* failed
The server registers one outputmanagement object as a global object. In order
to configure outputs a client requests create_configuration, which provides a
resource referencing an outputconfiguration for one-time configuration. That
way the server knows which requests belong together and can group them by that.
On the outputconfiguration object the client calls for each output whether the
output should be enabled, which mode should be set (by referencing the mode from
the list of announced modes) and the output's global position. Once all outputs
are configured that way, the client calls apply.
At that point and not earlier the server should try to apply the configuration.
If this succeeds the server emits the applied signal, otherwise the failed
signal, such that the configuring client is noticed about the success of its
configuration request.
Through this design the interface enables atomic output configuration changes if
internally supported by the server.
Warning! The protocol described in this file is a desktop environment implementation
detail. Regular clients must not use this protocol. Backward incompatible
changes may be added without bumping the major version of the extension.
</description>
<request name="create_configuration">
<description summary="provide outputconfiguration object for configuring outputs">
Request an outputconfiguration object through which the client can configure
output devices.
</description>
<arg name="id" type="new_id" interface="kde_output_configuration_v2"/>
</request>
<request name="create_mode_list">
<description summary="create a list of custom modes">
For details, see the description of kde_mode_list_v2 and
kde_output_configuration_v2.set_custom_modes.
</description>
<arg name="id" type="new_id" interface="kde_mode_list_v2"/>
</request>
</interface>
<interface name="kde_output_configuration_v2" version="22">
<description summary="configure single output devices">
outputconfiguration is a client-specific resource that can be used to ask
the server to apply changes to available output devices.
The client receives a list of output devices from the registry. When it wants
to apply new settings, it creates a configuration object from the
outputmanagement global, writes changes through this object's enable, scale,
transform and mode calls. It then asks the server to apply these settings in
an atomic fashion, for example through Linux' DRM interface.
The server signals back whether the new settings have applied successfully
or failed to apply. outputdevice objects are updated after the changes have been
applied to the hardware and before the server side sends the applied event.
</description>
<enum name="error">
<description summary="kde_output_configuration_v2 error values">
These error can be emitted in response to kde_output_configuration_v2 requests.
</description>
<entry name="already_applied" value="0" summary="the config is already applied"/>
</enum>
<request name="enable">
<description summary="enable or disable an output">
Mark the output as enabled or disabled.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice to be en- or disabled"/>
<arg name="enable" type="int" summary="1 to enable or 0 to disable this output"/>
</request>
<request name="mode">
<description summary="switch output-device to mode">
Sets the mode for a given output.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this mode change applies to"/>
<arg name="mode" type="object" interface="kde_output_device_mode_v2" summary="the mode to apply"/>
</request>
<request name="transform">
<description summary="transform output-device">
Sets the transformation for a given output.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this transformation change applies to"/>
<arg name="transform" type="int" summary="transform enum"/>
</request>
<request name="position">
<description summary="position output in global space">
Sets the position for this output device. (x,y) describe the top-left corner
of the output in global space, whereby the origin (0,0) of the global space
has to be aligned with the top-left corner of the most left and in case this
does not define a single one the top output.
There may be no gaps or overlaps between outputs, i.e. the outputs are
stacked horizontally, vertically, or both on each other.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this position applies to"/>
<arg name="x" type="int" summary="position on the x-axis"/>
<arg name="y" type="int" summary="position on the y-axis"/>
</request>
<request name="scale">
<description summary="set scaling factor of this output">
Sets the scaling factor for this output device.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this scale change applies to"/>
<arg name="scale" type="fixed" summary="scaling factor"/>
</request>
<request name="apply">
<description summary="apply configuration changes to all output devices">
Asks the server to apply property changes requested through this outputconfiguration
object to all outputs on the server side.
The output configuration can be applied only once. The already_applied protocol error
will be posted if the apply request is called the second time.
</description>
</request>
<event name="applied">
<description summary="configuration changes have been applied">
Sent after the server has successfully applied the changes.
.
</description>
</event>
<event name="failed">
<description summary="configuration changes failed to apply">
Sent if the server rejects the changes or failed to apply them.
</description>
</event>
<request name="destroy" type="destructor">
<description summary="release the outputconfiguration object"/>
</request>
<request name="overscan">
<description summary="set overscan value">
Set the overscan value of this output device with a value in percent.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice overscan applies to"/>
<arg name="overscan" type="uint" summary="overscan value"/>
</request>
<enum name="vrr_policy">
<description summary="describes vrr policy">
Describes when the compositor may employ variable refresh rate
</description>
<entry name="never" value="0"/>
<entry name="always" value="1"/>
<entry name="automatic" value="2"/>
</enum>
<request name="set_vrr_policy">
<description summary="set the VRR policy">
Set what policy the compositor should employ regarding its use of
variable refresh rate.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this VRR policy applies to"/>
<arg name="policy" type="uint" enum="vrr_policy" summary="the vrr policy to apply"/>
</request>
<enum name="rgb_range">
<description summary="describes RGB range policy">
Whether this output should use full or limited rgb.
</description>
<entry name="automatic" value="0"/>
<entry name="full" value="1"/>
<entry name="limited" value="2"/>
</enum>
<request name="set_rgb_range">
<description summary="RGB range">
Whether full or limited color range should be used
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice the rgb range applies to"/>
<arg name="rgb_range" type="uint" enum="rgb_range"/>
</request>
<request name="set_primary_output" since="2">
<description summary="Select which primary output to use" />
<arg name="output" type="object" interface="kde_output_device_v2" allow-null="false"/>
</request>
<request name="set_priority" since="3">
<description summary="Set the order of outputs">
Set the position of the output in the output order list, with lower values
being earlier in the list. There's no specific value the list has to start
at, this value is only used in sorting outputs.
The order of outputs can be used to assign desktop environment components
to a specific screen, see kde_output_order_v1 and kde-output-device-v2 for
details. Note that for consistent behavior, the priority value needs to be
unique among all enabled outputs.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice the index applies to" />
<arg name="priority" type="uint" summary="the priority of the output" />
</request>
<request name="set_high_dynamic_range" since="4">
<description summary="change if HDR should be enabled">
Sets whether or not the output should be set to HDR mode.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="enable_hdr" type="uint" summary="1 to enable, 0 to disable hdr"/>
</request>
<request name="set_sdr_brightness" since="4">
<description summary="set the brightness for sdr content">
Sets the brightness of standard dynamic range content in nits. Only has an effect while the output is in HDR mode.
Note that while the value is in nits, that doesn't necessarily translate to the same brightness on the screen.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="sdr_brightness" type="uint"/>
</request>
<request name="set_wide_color_gamut" since="4">
<description summary="change if a wide color gamut should be used">
Whether or not the output should use a wide color gamut
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="enable_wcg" type="uint" summary="1 to enable, 0 to disable wcg"/>
</request>
<enum name="auto_rotate_policy">
<description summary="describes when auto rotate should be used"/>
<entry name="never" value="0"/>
<entry name="in_tablet_mode" value="1"/>
<entry name="always" value="2"/>
</enum>
<request name="set_auto_rotate_policy" since="5">
<description summary="change when auto rotate should be used"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="policy" type="uint" enum="auto_rotate_policy"/>
</request>
<request name="set_icc_profile_path" since="6">
<description summary="change the used icc profile for SDR mode"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="profile_path" type="string"/>
</request>
<request name="set_brightness_overrides" since="7">
<description summary="override metadata about the screen's brightness limits"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="max_peak_brightness" type="int" summary="-1 for not overriding, or positive values in nits"/>
<arg name="max_frame_average_brightness" type="int" summary="-1 for not overriding, or positive values in nits"/>
<arg name="min_brightness" type="int" summary="-1 for not overriding, or positive values in 0.0001 nits"/>
</request>
<request name="set_sdr_gamut_wideness" since="7">
<description summary="describes which gamut is assumed for sRGB applications">
This can be used to provide the colors users assume sRGB applications should have based on the
default experience on many modern sRGB screens.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="gamut_wideness" type="uint" summary="0 means rec.709 primaries, 10000 means native primaries"/>
</request>
<enum name="color_profile_source" since="7">
<description summary="which source the compositor should use for the color profile on an output"/>
<entry name="sRGB" value="0"/>
<entry name="ICC" value="1"/>
<entry name="EDID" value="2"/>
</enum>
<request name="set_color_profile_source" since="8">
<description summary="which source the compositor should use for the color profile on an output in SDR mode"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="color_profile_source" type="uint" enum="color_profile_source" summary="the color profile source"/>
</request>
<request name="set_brightness" since="9">
<description summary="brightness multiplier">
Set the brightness modifier of the output. It doesn't specify
any absolute values, but is merely a multiplier on top of other
brightness values, like sdr_brightness and brightness_metadata.
0 is the minimum brightness (not completely dark) and 10000 is
the maximum brightness.
This is supported while HDR is active in versions 8 and below,
or when the device supports the "brightness" capability in
versions 9 and above.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="brightness" type="uint" summary="brightness in 0-10000"/>
</request>
<enum name="color_power_tradeoff">
<description summary="tradeoff between power and accuracy">
The compositor can do a lot of things that trade between
performance, power and color accuracy. This setting describes
a high level preference from the user about in which direction
that tradeoff should be made.
</description>
<entry name="efficiency" value="0" summary="prefer efficiency and performance"/>
<entry name="accuracy" value="1" summary="prefer accuracy"/>
</enum>
<request name="set_color_power_tradeoff" since="10">
<description summary="set the preferred color/power tradeoff"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="preference" type="uint" enum="color_power_tradeoff"/>
</request>
<request name="set_dimming" since="11">
<description summary="dimming multiplier">
Set the dimming multiplier of the output. This is similar to the
brightness setting, except it's meant to be a temporary setting
only, not persistent and may be implemented differently depending
on the display.
0 is the minimum dimming factor (not completely dark) and 10000
means the output is not dimmed.
This is supported only when the "brightness" capability is
also supported.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="multiplier" type="uint" summary="multiplier in 0-10000"/>
</request>
<event name="failure_reason" since="12">
<description summary="reason for failure">
Describes why applying the output configuration failed. Is only
sent before the failure event.
</description>
<arg name="reason" type="string" summary="reason for failure"/>
</event>
<request name="set_replication_source" since="13">
<description summary="source output for mirroring">
Set the source output that the outputdevice should mirror its
viewport from.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="source" type="string" summary="uuid of the source output"/>
</request>
<request name="set_ddc_ci_allowed" since="14">
<description summary="if DDC/CI should be used to control brightness etc."/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="allowed" type="uint" summary="1 if allowed, 0 if disabled"/>
</request>
<request name="set_max_bits_per_color" since="15">
<description summary="override the max bpc">
This limits the amount of bits per color that are sent to the display.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="max_bpc" type="uint" summary="0 for the default / automatic"/>
</request>
<enum name="edr_policy" since="16">
<description summary="when the compositor may make use of EDR"/>
<entry name="never" value="0"/>
<entry name="always" value="1"/>
</enum>
<request name="set_edr_policy" since="16">
<description summary="set when the compositor may apply EDR">
When EDR is enabled, the compositor may increase the backlight beyond
the user-specified setting, in order to present HDR content on displays
without native HDR support.
This will usually result in better visuals, but also increases battery
usage.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="policy" type="uint" enum="edr_policy"/>
</request>
<request name="set_sharpness" since="17">
<description summary="sharpness strength">
This is the sharpness modifier of the output.
0 is sharpness disabled and 10000 is the maximum sharpness
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="sharpness" type="uint" summary="sharpness in 0-10000"/>
</request>
<request name="set_custom_modes" since="18">
<description summary="set the custom mode list">
Set the list of custom modes for this output. The compositor
will in response generate the requested modes and add them to
the output (or delete ones no longer in the list).
This can be useful for overclocking displays, or for working
around broken EDIDs.
Note that there is no guarantee for any custom mode to
actually work, or even to leave the display undamaged (in the
case of CRTs). It's entirely the responsibility of the user
to ensure each added mode is the right one for their display.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="modes" type="object" interface="kde_mode_list_v2"/>
</request>
<request name="set_auto_brightness" since="19">
<description summary="whether or not automatic brightness is enabled"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="enabled" type="uint" summary="1 for enabled, 0 for disabled"/>
</request>
<request name="set_hdr_icc_profile_path" since="20">
<description summary="change the used icc profile for HDR mode"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="profile_path" type="string"/>
</request>
<request name="set_hdr_color_profile_source" since="20">
<description summary="which source the compositor should use for the color profile on an output in HDR mode"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="color_profile_source" type="uint" enum="color_profile_source" summary="the color profile source"/>
</request>
<request name="set_abm_level" since="21">
<description summary="set the allowed level of adaptive backlight modulation">
Adaptive backlight modulation is a feature that reduces the backlight
and increases contrast of colors on the screen to improve power usage.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="level" type="uint" summary="0 is off, 4 is the maximum level"/>
</request>
</interface>
<interface name="kde_mode_list_v2" version="22">
<description summary="a list of custom modes">
This list is populated by first setting each relevant property,
and then calling add_mode to add a mode with these properties.
One would for example call
- set_resolution
- set_refresh_rate
- set_reduced_blanking
- add_mode
add_mode does not reset the properties that were previously set,
they are valid until the object is destroyed.
The compositor may additionally have sensible defaults for some
properties like reduced_blanking, but for consistent results,
it's best to always set each known property every time.
One can also specify custom modes with CVT timings, for example
- add_cvt
- add_cvt
The parameters resolution and refresh rate are required, if they
are not set, the missing_parameters error will be emitted.
</description>
<enum name="error">
<description summary="kde_mode_list_v2 error values">
These errors can be emitted in response to add_mode requests.
</description>
<entry name="missing_parameters" value="0" summary="a required parameter wasn't set"/>
</enum>
<request name="destroy" type="destructor">
<description summary="destroy the mode list object"/>
</request>
<request name="add_mode">
<description summary="Add the current mode configuration to the list"/>
</request>
<request name="set_resolution">
<arg name="width" type="uint"/>
<arg name="height" type="uint"/>
</request>
<request name="set_refresh_rate">
<arg name="rate" type="uint" summary="in milliHz"/>
</request>
<request name="set_reduced_blanking">
<description summary="whether or not the mode should have reduced blanking">
Reduced blanking is an optimization that can reduce bandwidth / timing
requirements for a display mode by reducing the time vblank takes.
As not all displays support it, it may be desired to still turn it off
though (like with CRTs, where full blanking is required).
</description>
<arg name="reduced" type="uint"
summary="1 for reduced blanking, 0 for normal vblank duration"/>
</request>
<request name="add_cvt" since="22">
<description summary="new mode with CVT timings">
Adds a new mode with the specified CVT timings.
</description>
<arg name="dot_clock" type="uint" summary="pixel clock in kHz"/>
<arg name="hdisplay" type="uint" summary="horizontal display size"/>
<arg name="hsync_start" type="uint" summary="horizontal sync start"/>
<arg name="hsync_end" type="uint" summary="horizontal sync end"/>
<arg name="htotal" type="uint" summary="horizontal total size"/>
<arg name="hskew" type="uint" summary="horizontal skew"/>
<arg name="vdisplay" type="uint" summary="vertical display size"/>
<arg name="vsync_start" type="uint" summary="vertical sync start"/>
<arg name="vsync_end" type="uint" summary="vertical sync end"/>
<arg name="vtotal" type="uint" summary="vertical total size"/>
<arg name="vscan" type="uint" summary="vertical scan"/>
<arg name="flags" type="uint" summary="flags, see DRM_MODE_FLAG_*"/>
</request>
</interface>
</protocol>
+12
View File
@@ -58,6 +58,11 @@ pub(crate) fn emit_display_event(ev: DisplayEvent) {
pub(crate) mod backend;
pub use backend::{DisplayOwnership, VirtualDisplay, VirtualOutput};
/// Time-bounded child-process helpers — every compositor query shells out, and an unbounded one
/// can wedge the calling (session) thread forever.
#[path = "vdisplay/proc.rs"]
pub(crate) mod proc;
/// Live-session detection + session-epoch + env retargeting (plan §W3).
#[path = "vdisplay/session.rs"]
pub(crate) mod session;
@@ -440,6 +445,13 @@ mod hyprland;
#[path = "vdisplay/linux/kwin.rs"]
mod kwin;
// In-process KDE output management (kde_output_management_v2) — the topology path that used to shell
// out to `kscreen-doctor`, driven over the compositor's own Wayland instead so it can't be wedged by
// a stuck libkscreen/kscreen-KDED backend. Consumed by `kwin` (best-effort, with kscreen fallback).
#[cfg(target_os = "linux")]
#[path = "vdisplay/linux/kwin_output_mgmt.rs"]
mod kwin_output_mgmt;
#[cfg(target_os = "windows")]
#[path = "vdisplay/windows/manager.rs"]
pub mod manager;
@@ -61,6 +61,32 @@ static MANAGED_SESSION: std::sync::Mutex<Option<SessionState>> = std::sync::Mute
/// (single-instance), so [`schedule_restore_tv_session`] can restart them when the client disconnects.
static STOPPED_AUTOLOGIN: std::sync::Mutex<Vec<String>> = std::sync::Mutex::new(Vec::new());
/// The display-manager unit we stopped for the takeover on a mask-fragile DM flavor (Nobara's
/// `plasmalogin` — see [`dm_survives_masked_unit`]), so the restore brings the box back via
/// `reset-failed` + `restart` of the DM instead of a `--user start` of the gamescope unit (which
/// cannot work there: without a DM login session there is no seat, so gamescope never gets DRM
/// master — live-proven on the Nobara repro VM 2026-07-24).
static STOPPED_DM: std::sync::Mutex<Option<String>> = std::sync::Mutex::new(None);
/// mtime of the `steamos-session-select` sentinel at managed-session launch — the baseline the
/// in-stream "Switch to Desktop" detector compares against. Steam's session-select script writes
/// `~/.config/steamos-session-select` unconditionally in its USER pass, before any of its
/// display-manager checks — so it advances even under a DM-stop takeover, where the script's
/// config-rewrite tail is a silent no-op (every write branch is gated on the DM *running*;
/// diagnosed live on the Nobara repro VM 2026-07-24). An advanced mtime after a capture loss is
/// therefore the one durable trace of the user's switch request.
static SESSION_SELECT_BASELINE: std::sync::Mutex<Option<std::time::SystemTime>> =
std::sync::Mutex::new(None);
/// When [`honor_session_select_switch`] last ran. While recent, a managed (re)launch is refused —
/// the rebuild loop would otherwise race the booting desktop back into game mode (gamescope+Steam
/// come up faster than KWin, and a delivering managed pipeline ends the rebuild's re-detection).
static SWITCH_HONORED_AT: std::sync::Mutex<Option<Instant>> = std::sync::Mutex::new(None);
/// How long after honoring an in-stream desktop switch the managed path refuses to relaunch,
/// giving the DM's desktop session time to come up so re-detection follows it instead.
const SWITCH_HONOR_GRACE: Duration = Duration::from_secs(120);
/// A pending debounced TV-session restore: the instant [`do_restore_tv_session`] should fire after
/// the last client disconnect. A reconnect inside the window clears it (and reuses the still-warm
/// managed session), so we never stop+relaunch gamescope per connect — that per-connect teardown is
@@ -115,6 +141,10 @@ struct TakeoverState {
stopped_autologin: Vec<String>,
/// Whether we took over SteamOS's `gamescope-session.target` (restore = remove drop-in + restart).
steamos: bool,
/// The display-manager unit we stopped on a mask-fragile DM flavor (restore = `reset-failed` +
/// `restart` of the DM). `default` so takeover files from older hosts still parse.
#[serde(default)]
stopped_dm: Option<String>,
}
/// Path of the persisted [`TakeoverState`], under `$XDG_RUNTIME_DIR` (per-user, 0700, tmpfs — cleared
@@ -133,8 +163,9 @@ fn persist_takeover() {
.unwrap_or_else(|e| e.into_inner())
.clone(),
steamos: *STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()),
stopped_dm: STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()).clone(),
};
if state.stopped_autologin.is_empty() && !state.steamos {
if state.stopped_autologin.is_empty() && !state.steamos && state.stopped_dm.is_none() {
clear_takeover();
return;
}
@@ -162,17 +193,22 @@ pub fn restore_takeover_on_startup() {
clear_takeover();
return;
};
if state.stopped_autologin.is_empty() && !state.steamos {
if state.stopped_autologin.is_empty() && !state.steamos && state.stopped_dm.is_none() {
clear_takeover();
return;
}
tracing::warn!(
units = ?state.stopped_autologin,
steamos = state.steamos,
stopped_dm = ?state.stopped_dm,
"gamescope: found a stranded takeover from a previous host instance — scheduling TV restore"
);
*STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = state.stopped_autologin;
*STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()) = state.steamos;
*STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()) = state.stopped_dm;
// Re-baseline the session-select sentinel: after a crash-restore the launch-time baseline is
// gone, and a long-existing sentinel file must not read as a fresh in-stream switch request.
record_session_select_baseline();
// A generous grace so a client reconnecting right after the restart cancels it (create_managed_session
// clears PENDING_RESTORE) and keeps the streamed session rather than bouncing to gaming mode.
*PENDING_RESTORE.lock().unwrap_or_else(|e| e.into_inner()) =
@@ -263,7 +299,10 @@ impl VirtualDisplay for GamescopeDisplay {
// A3 takeover machinery (recorded in STOPPED_AUTOLOGIN + persisted; restarted on session end via
// schedule_restore_tv_session). Non-Steam launches don't conflict, so they skip this.
if self.cmd.as_deref().is_some_and(is_steam_launch) {
stop_autologin_sessions();
// A dedicated launch NEEDS Steam's single instance — no attach degrade exists here, so
// a mask-fragile-DM box without takeover privilege fails with the actionable error.
stop_autologin_sessions()
.context("dedicated Steam launch needs the box's gaming session freed")?;
// B1b: a Steam running in a plain DESKTOP session (GNOME/KDE) holds the instance just
// the same, and the autologin stop above can't see it — free it too, or fail loudly.
free_desktop_steam()?;
@@ -326,6 +365,40 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
if steamos_session_present() {
return create_managed_session_steamos(mode);
}
// In-stream "Switch to Desktop" under a DM-stop takeover: the user's session-select inside
// the streamed game mode advanced the sentinel, but its config rewrite was a silent no-op
// (every write branch needs the DM running, and the takeover stopped it) — so without this,
// the capture loss it caused would just relaunch game mode ("thrown back in", field-tested
// 2026-07-24). Honor the request instead: restore the DM and replay the switch.
let dm_takeover = STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()).clone();
if let Some(dm) = dm_takeover {
if session_select_requested() {
*STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()) = None;
honor_session_select_switch(dm);
return Err(anyhow!(
"the user switched the box to the desktop session — display manager restored; \
re-detection follows the desktop compositor as it comes up"
));
}
}
// Post-honor grace: while the selected desktop boots, a managed relaunch would win the race
// (gamescope+Steam start faster than KWin) and a delivering pipeline ends the rebuild's
// re-detection — right back in game mode. A live box-owned game-mode unit supersedes the
// grace: the user already switched back, so managed may proceed.
let honor_pending = SWITCH_HONORED_AT
.lock()
.unwrap_or_else(|e| e.into_inner())
.is_some_and(|t| t.elapsed() < SWITCH_HONOR_GRACE);
if honor_pending {
if running_autologin_gamescope_unit().is_some() {
*SWITCH_HONORED_AT.lock().unwrap_or_else(|e| e.into_inner()) = None;
} else {
return Err(anyhow!(
"waiting for the desktop session the user selected — refusing to relaunch game \
mode (re-detection follows the desktop once it's up)"
));
}
}
// Attach-only rebuild probe: reuse a live same-mode session, but NEVER stop/relaunch box
// sessions — right after a capture loss the caller's session detection can be stale, and a
// destructive rebuild here would fight the session the user just switched to.
@@ -353,7 +426,28 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
// Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and
// renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by
// stopping it; [`schedule_restore_tv_session`] (on disconnect) brings it back after a debounce.
stop_autologin_sessions();
// On a mask-fragile-DM box without the privilege to stop the DM, the takeover would destabilize
// the seat — degrade to ATTACH instead: mirror the box's own live game-mode session (capture +
// inject, no lifecycle ownership), which needs no takeover at all.
if let Err(e) = stop_autologin_sessions() {
tracing::warn!(
error = %format!("{e:#}"),
"gamescope: managed takeover unavailable — degrading to ATTACH (mirroring the box's \
own game-mode session)"
);
let node_id = ensure_box_gamescope_mode(mode)?;
point_injector_at_eis();
return Ok(VirtualOutput {
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(()),
ownership: DisplayOwnership::External,
reused_gen: None,
pool_gen: None,
expect_exact_dims: false,
});
}
// B1b: a desktop-session Steam (outside any gamescope unit) also holds the single instance and
// would make the managed session's own Steam exit at birth. The managed session's Steam itself
// is exempt (it lives in the SESSION_UNIT cgroup), so the same-mode reuse below is unaffected.
@@ -379,7 +473,21 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
}
// (Re)launch at the new mode. `launch_session` stops the old unit by name first, so there is
// exactly one gamescope `Video/Source` node for discovery.
let node_id = launch_session(client, SESSION_UNIT, mode)?;
let node_id = match launch_session(client, SESSION_UNIT, mode) {
Ok(id) => id,
Err(e) => {
// The takeover already happened (autologin units stopped, possibly the DM down) — arm
// the restore now, or a failed launch strands the box sessionless until a host
// restart. Policy-timed; a quick client retry cancels it and relaunches warm.
// MANAGED_SESSION must be released first: the scheduler reads it (orphan detection).
drop(guard);
schedule_restore_tv_session();
return Err(e);
}
};
// Baseline the session-select sentinel NOW: only a write from INSIDE this session (the user's
// "Switch to Desktop") should read as a switch request, not the one that led here.
record_session_select_baseline();
point_injector_at_eis();
*guard = Some(SessionState {
width: mode.width,
@@ -807,6 +915,38 @@ fn ensure_box_gamescope_mode(mode: Mode) -> Result<u32> {
return Ok(node);
}
}
// Attach-only rebuild probe (parity with both managed paths — this gap was the attach-path
// stale-detection hazard): right after a capture loss the caller's session detection can be
// stale, and a set-environment + unit restart here would fight the session the user just
// switched to. Mirror whatever live node exists at its own mode; refuse otherwise.
if crate::rebuild_probe_active() {
if let Some(node) = find_gamescope_node() {
tracing::info!(
node,
"gamescope: attach-only rebuild probe — mirroring the live node at its own mode"
);
return Ok(node);
}
return Err(anyhow!(
"no live gamescope node — attach-only rebuild probe refuses to restart the box's \
session (re-detection follows the live session)"
));
}
// A box driving a PHYSICAL display is mirrored at its own mode, never re-moded: the re-mode
// restart is the headless-box model (no panel ⇒ the game-mode resolution is ours to set);
// on-glass it would flip the user's own screen to the client's resolution — and on a
// DM-session-driven box (Nobara) the unit restart bounces the login session with it.
if physical_display_connected() {
if let Some(node) = find_gamescope_node() {
tracing::info!(
node,
client_w = mode.width,
client_h = mode.height,
"gamescope: box drives a physical display — attaching at its own mode (no re-mode)"
);
return Ok(node);
}
}
let Some(unit) = running_autologin_gamescope_unit() else {
// No box-owned autologin session to reconfigure (a bare/foreign gamescope): attach to
// whatever node exists, accepting its resolution.
@@ -972,17 +1112,245 @@ fn unmask_unit(unit: &str) {
.status();
}
/// The unit name of the display manager driving this box's graphical logins, from the
/// `display-manager.service` alias symlink (the Fedora/Arch/openSUSE convention every
/// gamescope-session distro follows). `None` when no DM is installed (a box that boots straight
/// into a user session — getty autologin / an enabled user unit).
fn display_manager_unit() -> Option<String> {
display_manager_unit_under(std::path::Path::new("/etc/systemd/system"))
}
/// [`display_manager_unit`] against an arbitrary root (the unit-testable core).
fn display_manager_unit_under(base: &std::path::Path) -> Option<String> {
let target = std::fs::read_link(base.join("display-manager.service")).ok()?;
target.file_name().map(|n| n.to_string_lossy().into_owned())
}
/// Does this display manager's autologin loop SURVIVE the gamescope unit being masked? Only SDDM is
/// proven to (Bazzite/SteamOS `Relogin=true` — the mask is what stops its relogin from restarting
/// the unit mid-stream, diagnosed live on .181 2026-07-07; a failing autologin leaves sddm itself
/// running). Nobara's `plasmalogin` (KDE's SDDM successor) is proven FATAL: against a masked unit
/// its session Exec fails instantly, `Relogin=true` retries, and `plasmalogin.service` trips
/// systemd's start limit within ~1 s — the DM dies and the box is a permanent black screen that
/// only a root `reset-failed` + `restart` recovers (live-proven on the Nobara repro VM
/// 2026-07-24). Unknown DMs are treated as fragile: the fragile path degrades gracefully, a wrong
/// "safe" kills the seat.
fn dm_survives_masked_unit(dm: &str) -> bool {
dm == "sddm.service"
}
/// The packaged privileged fallback for the display-manager takeover verbs: a root helper behind
/// its own polkit action (`io.unom.punktfunk.dm-helper`, `allow_any` — the mechanism these
/// distros use for their own session switcher, e.g. Nobara's `os-session-select`), so the managed
/// takeover works out of the box on mask-fragile DM flavors with no hand-installed polkit rule.
/// The helper derives the DM unit from the `display-manager.service` symlink itself, so this
/// process never gets to name an arbitrary unit across the privilege boundary. Two layouts: the
/// rpm/deb `libexec` path (what the shipped policy annotates) and Arch's `/usr/lib/<pkg>` (its
/// PKGBUILD rewrites the annotation to match).
const DM_HELPER_PATHS: &[&str] = &[
"/usr/libexec/punktfunk/pf-dm-helper",
"/usr/lib/punktfunk/pf-dm-helper",
];
/// Run the packaged DM helper (`stop` | `restore`) via pkexec. `false` when the helper isn't
/// installed (tarball/old package), pkexec is missing, or polkit denies the action.
fn dm_helper(verb: &str) -> bool {
let Some(helper) = DM_HELPER_PATHS
.iter()
.find(|p| std::path::Path::new(p).exists())
else {
return false;
};
Command::new("pkexec")
.arg(helper)
.arg(verb)
.status()
.map(|s| s.success())
.unwrap_or(false)
}
/// Stop the display manager for a takeover on a mask-fragile DM flavor. Plain `systemctl stop` on
/// the SYSTEM bus first — succeeds as root or under an operator polkit rule scoped to the DM unit
/// (see docs); fails cleanly otherwise ("interactive authentication required") — then the
/// packaged pkexec helper. `false` means no privilege path exists and the caller degrades to
/// attach.
fn try_stop_display_manager(dm: &str) -> bool {
let direct = Command::new("systemctl")
.args(["stop", dm])
.status()
.map(|s| s.success())
.unwrap_or(false);
direct || dm_helper("stop")
}
/// Restore the display manager: `reset-failed` (a relogin loop may have tripped the unit's start
/// limit, and a plain restart is refused until the accounting clears) + `restart` — its autologin
/// session Exec brings the box's own session back up. Plain system-bus verbs first (root / an
/// operator polkit rule), then the packaged pkexec helper, whose `restore` verb performs the same
/// two steps as root.
fn restore_display_manager(dm: &str) -> bool {
let _ = Command::new("systemctl")
.args(["reset-failed", dm])
.status();
let direct = Command::new("systemctl")
.args(["restart", dm])
.status()
.map(|s| s.success())
.unwrap_or(false);
direct || dm_helper("restore")
}
/// The distro's session-switch helper (ChimeraOS/Nobara layout). Its USER pass records the
/// sentinel + self-pkexecs (authorized `allow_any` by the distro's own polkit action policy, so
/// it works from our sessionless context); its ROOT pass rewrites the DM autologin config — but
/// only while the DM is RUNNING, which is why the takeover must restart the DM before calling it.
const OS_SESSION_SELECT: &str = "/usr/libexec/os-session-select";
/// The sentinel Steam's `steamos-session-select` writes in its user pass
/// (`~/.config/steamos-session-select`) — see [`SESSION_SELECT_BASELINE`].
fn session_select_sentinel() -> Option<std::path::PathBuf> {
let home = std::env::var("HOME").ok()?;
Some(
std::path::Path::new(&home)
.join(".config")
.join("steamos-session-select"),
)
}
/// Current mtime of the session-select sentinel (`None` when it doesn't exist yet).
fn session_select_mtime() -> Option<std::time::SystemTime> {
let path = session_select_sentinel()?;
std::fs::metadata(path).ok()?.modified().ok()
}
/// Record the sentinel baseline at managed-session launch, so a LATER write (the user's in-stream
/// "Switch to Desktop") is distinguishable from the switch that led into this session.
fn record_session_select_baseline() {
*SESSION_SELECT_BASELINE
.lock()
.unwrap_or_else(|e| e.into_inner()) = session_select_mtime();
}
/// Did a session-select run inside the managed session since its launch (sentinel newer than the
/// recorded baseline, or newly created)? Inside a managed game session the only switch Steam
/// offers is TO the desktop, so an advanced sentinel reads as that request.
fn session_select_requested() -> bool {
let baseline = *SESSION_SELECT_BASELINE
.lock()
.unwrap_or_else(|e| e.into_inner());
match (baseline, session_select_mtime()) {
(Some(base), Some(now)) => now > base,
(None, Some(_)) => true, // created during the session
_ => false,
}
}
/// Honor the user's in-stream "Switch to Desktop" under a DM-stop takeover. The OS flow was a
/// silent no-op (the switch script's config rewrite requires a running DM, which the takeover
/// stopped), so replay it with the DM up — every verb live-validated on the Nobara repro VM:
/// 1. consume the takeover and start the DM (its autologin heads back into game mode briefly —
/// the config still names it);
/// 2. run the distro's own `os-session-select desktop` as the user (its internal pkexec is
/// `allow_any`-authorized), which rewrites the DM autologin config to the desktop session;
/// 3. stop the autologin gamescope unit — the login session exits, and `Relogin=true` relogs
/// into the now-selected desktop.
///
/// The caller then refuses managed relaunches for [`SWITCH_HONOR_GRACE`] so the capture-loss
/// re-detection follows the desktop compositor once it's up instead of racing it.
fn honor_session_select_switch(dm: String) {
tracing::info!(
%dm,
"gamescope: in-stream session-select detected — restoring the display manager and \
switching the box to the desktop session"
);
// Consume the takeover state up front: from here on the box is the DM's again.
std::mem::take(&mut *STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()));
clear_takeover();
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
stop_session(SESSION_UNIT); // dead already (the switch shut its Steam down) — clear the unit
if !restore_display_manager(&dm) {
tracing::warn!(
%dm,
"gamescope: display-manager start was denied — the desktop switch may need a manual \
`systemctl restart` of the DM"
);
}
let deadline = Instant::now() + Duration::from_secs(10);
while Instant::now() < deadline {
let active = Command::new("systemctl")
.args(["is-active", &dm])
.output()
.map(|o| String::from_utf8_lossy(&o.stdout).trim() == "active")
.unwrap_or(false);
if active {
break;
}
std::thread::sleep(Duration::from_millis(500));
}
// Rewrite the autologin session via the distro's own switch helper (needs the DM running).
// Absent/failing helper degrades to a plain DM restore — the box lands back in game mode on
// glass and the stream follows that instead (no black screen either way).
if std::path::Path::new(OS_SESSION_SELECT).exists() {
match Command::new(OS_SESSION_SELECT).arg("desktop").status() {
Ok(s) if s.success() => {
// The relogin only fires when the CURRENT (game-mode) login session exits: wait
// for its autologin unit to come up, then stop it. Never mask here — the mask is
// what start-limit-kills this DM flavor.
let deadline = Instant::now() + Duration::from_secs(15);
loop {
if let Some(unit) = running_autologin_gamescope_unit() {
systemctl_user(&["stop", &unit]);
tracing::info!(
%unit,
"gamescope: desktop selected — stopped the game-mode session so the \
DM relogs into the desktop"
);
break;
}
if Instant::now() >= deadline {
tracing::warn!(
"gamescope: game-mode session never appeared after the DM restart — \
the desktop switch may need a manual session exit"
);
break;
}
std::thread::sleep(Duration::from_millis(500));
}
}
other => tracing::warn!(
status = ?other,
"gamescope: os-session-select failed — leaving the box in its configured session"
),
}
} else {
tracing::warn!(
"gamescope: no {OS_SESSION_SELECT} on this box — restored the DM into its configured \
session instead of switching to the desktop"
);
}
record_session_select_baseline();
*SWITCH_HONORED_AT.lock().unwrap_or_else(|e| e.into_inner()) = Some(Instant::now());
}
/// Stop every autologin gaming-mode session (`gamescope-session-plus@*.service`) so its
/// single-instance Steam is free for our own host-managed session. Records the units so
/// [`schedule_restore_tv_session`] can restart them on disconnect. Our own session is the transient
/// `punktfunk-gamescope` unit (not a `@`-instance), so it's never matched here. No-op when nothing
/// is autologged in (e.g. a box that boots headless). Each unit is **masked first** ([`mask_unit`] —
/// SDDM's `Relogin=true` would otherwise restart it instantly), then torn down with **SIGKILL**
/// ([`kill_unit`]) to avoid the F44 GPU-context leak that the autologin's SIGTERM stop triggers.
/// Matches every loaded instance, not just `running` ones — under the SDDM relogin churn the unit
/// flaps through `activating`/`failed` between cycles, and an unmasked flapping unit re-enters the
/// fight the moment the supervisor restarts it.
fn stop_autologin_sessions() {
/// is autologged in (e.g. a box that boots headless).
///
/// The teardown is DM-flavor-aware ([`dm_survives_masked_unit`]):
/// * **SDDM / no DM**: each unit is **masked first** ([`mask_unit`] — SDDM's `Relogin=true` would
/// otherwise restart it instantly), then torn down with **SIGKILL** ([`kill_unit`]) to avoid the
/// F44 GPU-context leak that the autologin's SIGTERM stop triggers. Matches every loaded
/// instance, not just `running` ones — under the SDDM relogin churn the unit flaps through
/// `activating`/`failed` between cycles, and an unmasked flapping unit re-enters the fight the
/// moment the supervisor restarts it.
/// * **Mask-fragile DM** (Nobara's `plasmalogin`, unknown DMs): masking start-limit-kills the DM
/// itself (permanent black screen), so instead **stop the DM** — no supervisor left to relogin —
/// then SIGKILL the units unmasked. Needs privilege (root / an operator polkit rule); without it
/// nothing is touched and the error tells the caller to degrade to ATTACH (mirror the box's own
/// session) rather than destabilize the seat.
fn stop_autologin_sessions() -> Result<()> {
let Ok(out) = Command::new("systemctl")
.args([
"--user",
@@ -995,26 +1363,67 @@ fn stop_autologin_sessions() {
])
.output()
else {
return;
return Ok(());
};
let mut stopped = Vec::new();
for line in String::from_utf8_lossy(&out.stdout).lines() {
if let Some(unit) = line.split_whitespace().next() {
if unit.starts_with("gamescope-session-plus@") && unit.ends_with(".service") {
mask_unit(unit); // block the SDDM relogin loop from restarting it mid-stream
kill_unit(unit); // SIGKILL teardown — avoid the F44 GPU-context leak
tracing::info!(
unit,
"freed Steam: masked + SIGKILL-stopped the autologin gaming session for this stream"
);
stopped.push(unit.to_string());
}
// `(unit, ACTIVE state)` — the `--plain` columns are UNIT LOAD ACTIVE SUB DESCRIPTION.
let listed: Vec<(String, String)> = String::from_utf8_lossy(&out.stdout)
.lines()
.filter_map(|l| {
let mut cols = l.split_whitespace();
let unit = cols.next()?;
let active = cols.nth(1).unwrap_or("");
(unit.starts_with("gamescope-session-plus@") && unit.ends_with(".service"))
.then(|| (unit.to_string(), active.to_string()))
})
.collect();
if listed.is_empty() {
return Ok(()); // nothing autologged in — Steam is already free
}
let dm = display_manager_unit();
let mask_safe = dm.as_deref().is_none_or(dm_survives_masked_unit);
if !mask_safe {
// Only a LIVE instance holds Steam / justifies touching the DM. A loaded-but-inactive
// leftover (the box switched back to the desktop earlier) must not stop the DM — that
// would kill the user's live desktop to free nothing.
if !listed
.iter()
.any(|(_, active)| matches!(active.as_str(), "active" | "activating"))
{
return Ok(());
}
let dm = dm.expect("!is_none_or ⇒ Some");
if !try_stop_display_manager(&dm) {
bail!(
"the box's gaming session is driven by {dm}, which does not survive a masked \
session unit, and stopping it needs privilege the packaged pf-dm-helper \
polkit action is missing or was denied (reinstall the punktfunk package, or \
install the display-manager polkit rule from the docs) so the managed takeover \
is unavailable"
);
}
tracing::info!(
%dm,
"freed Steam: stopped the display manager for this stream (mask-fragile DM flavor)"
);
*STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()) = Some(dm);
}
if !stopped.is_empty() {
*STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = stopped;
persist_takeover(); // A3: survive a host crash mid-stream
let units: Vec<String> = listed.into_iter().map(|(u, _)| u).collect();
let mut stopped = Vec::new();
for unit in units {
if mask_safe {
mask_unit(&unit); // block the SDDM relogin loop from restarting it mid-stream
}
kill_unit(&unit); // SIGKILL teardown — avoid the F44 GPU-context leak
tracing::info!(
%unit,
masked = mask_safe,
"freed Steam: stopped the autologin gaming session for this stream"
);
stopped.push(unit);
}
*STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = stopped;
persist_takeover(); // A3: survive a host crash mid-stream
Ok(())
}
/// How long a desktop Steam gets to honor `steam -shutdown` before the spawn fails. Steam tears
@@ -1159,7 +1568,16 @@ pub fn schedule_restore_tv_session() {
.lock()
.unwrap_or_else(|e| e.into_inner())
.is_empty()
&& !*STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner());
&& !*STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner())
&& STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()).is_none()
// A managed session that took nothing over (started beside a live desktop — e.g. a client
// gamescope pin on a KDE box) still owns the transient SESSION_UNIT: without this arm it
// was ORPHANED forever after disconnect ("closing the app does not end the session",
// field report 2026-07-24) — the restore stops it even with no autologin to bring back.
&& MANAGED_SESSION
.lock()
.unwrap_or_else(|e| e.into_inner())
.is_none();
if nothing_to_restore {
return; // nothing was taken over → nothing to restore (also the non-managed path)
}
@@ -1254,8 +1672,24 @@ fn do_restore_tv_session() {
}
}
let units = std::mem::take(&mut *STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()));
if units.is_empty() {
return; // nothing was stolen → nothing to restore (also the non-Bazzite path)
let dm = std::mem::take(&mut *STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()));
if units.is_empty() && dm.is_none() {
// Nothing was stolen — but a managed session that started BESIDE a live desktop (client
// gamescope pin on a KDE box) still owns the transient unit; stop it so it doesn't run
// orphaned forever after the disconnect. No-op when the unit isn't running.
if MANAGED_SESSION
.lock()
.unwrap_or_else(|e| e.into_inner())
.take()
.is_some()
{
stop_session(SESSION_UNIT);
tracing::info!(
"gamescope: stopped the idle managed session (nothing was taken over — no box \
session to restore)"
);
}
return;
}
clear_takeover(); // A3: takeover consumed — drop the persisted crash-restore marker
stop_session(SESSION_UNIT); // our gamescope/Steam session, so Steam is free for the autologin
@@ -1266,18 +1700,47 @@ fn do_restore_tv_session() {
}
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
// Only bring the gaming autologin BACK if the box is still meant to be in gaming mode. If the
// user switched to a desktop session (KDE/GNOME/wlroots) in the meantime, don't yank them back
// to gaming — leave the desktop alone. (We still stopped our idle managed session above.)
// user switched to a desktop session (KDE/GNOME/wlroots/Hyprland) in the meantime, don't yank
// them back to gaming — leave the desktop alone. (We still stopped our idle managed session
// above.)
use super::ActiveKind;
if matches!(
super::detect_active_session().kind,
ActiveKind::DesktopKde | ActiveKind::DesktopGnome | ActiveKind::DesktopWlroots
ActiveKind::DesktopKde
| ActiveKind::DesktopGnome
| ActiveKind::DesktopWlroots
| ActiveKind::DesktopHyprland
) {
tracing::info!(
"gamescope: a desktop session is active — not restoring the TV gaming session"
);
return;
}
// Mask-fragile-DM takeover: the gamescope unit CANNOT be `--user start`ed back — without a DM
// login session there is no seat, so gamescope never gets DRM master (unit goes `failed`,
// screen stays black — live-proven on the Nobara repro VM). Restore the DM instead
// (`reset-failed` clears the relogin start-limit accounting, then `restart`); its autologin
// session Exec starts the gamescope unit itself.
if let Some(dm) = dm {
let restart = restore_display_manager(&dm);
if restart {
tracing::info!(%dm, "restored the display manager (its autologin brings gaming mode back)");
} else if crate::try_recover_session() {
tracing::warn!(
%dm,
"display-manager restart lost its privilege — fired PUNKTFUNK_RECOVER_SESSION_CMD \
to bring the session back"
);
} else {
tracing::error!(
%dm,
"could not restart the display manager and no PUNKTFUNK_RECOVER_SESSION_CMD is \
configured the box has no graphical session until someone runs \
`systemctl reset-failed {dm} && systemctl restart {dm}` as root"
);
}
return;
}
for unit in units {
let _ = Command::new("systemctl")
.args(["--user", "start", &unit])
@@ -1641,10 +2104,35 @@ impl Drop for GamescopeProc {
#[cfg(test)]
mod tests {
use super::{
cgroup_is_punktfunk_owned, connected_connector_under, is_steam_launch,
shape_dedicated_command,
cgroup_is_punktfunk_owned, connected_connector_under, display_manager_unit_under,
dm_survives_masked_unit, is_steam_launch, shape_dedicated_command,
};
#[test]
fn display_manager_flavor_detection() {
let base = std::env::temp_dir().join(format!("pf-dm-scan-{}", std::process::id()));
std::fs::create_dir_all(&base).unwrap();
// No alias symlink (no DM installed — getty autologin boxes) → None.
assert_eq!(display_manager_unit_under(&base), None);
// The Fedora-style alias symlink resolves to its target's basename (read_link, not
// canonicalize — the target needn't exist on the build box).
std::os::unix::fs::symlink(
"/usr/lib/systemd/system/plasmalogin.service",
base.join("display-manager.service"),
)
.unwrap();
assert_eq!(
display_manager_unit_under(&base).as_deref(),
Some("plasmalogin.service")
);
// Only SDDM is proven to survive a masked session unit; plasmalogin start-limit-kills
// itself (live-proven), and unknown DMs default to fragile.
assert!(dm_survives_masked_unit("sddm.service"));
assert!(!dm_survives_masked_unit("plasmalogin.service"));
assert!(!dm_survives_masked_unit("gdm.service"));
std::fs::remove_dir_all(&base).unwrap();
}
#[test]
fn connector_status_scan() {
let base = std::env::temp_dir().join(format!("pf-drm-scan-{}", std::process::id()));
@@ -340,9 +340,19 @@ pub(crate) fn is_available() -> bool {
/// against PipeWire ≥ 1.6 (a loop-lock bug) and a stuck link head-blocks the whole daemon.
const MIN_GAMESCOPE: (u32, u32, u32) = (3, 16, 22);
/// Best-effort: warn loudly if the installed gamescope is older than [`MIN_GAMESCOPE`]. Parsing
/// failures are silent (don't block a possibly-fine custom build) — this is a diagnostic, not a
/// gate. Returns the parsed version when it could read one.
/// First gamescope that paints the Steam overlay (Shift+Tab / Quick Access Menu) into its built-in
/// PipeWire node. `paint_pipewire()` is a *separate, reduced* composite from the display scanout;
/// the overlay-window paint (gated on the consumer negotiating `gamescope_focus_appid == 0`, which
/// we do by never advertising that property — see the capturer's EnumFormat builders) first ships
/// in 3.16.23 (gamescope commits `ccd62074` + `f8b33d38`). Below this the overlay is *never* in the
/// node, so it cannot appear in the stream no matter what the host does. The cursor and
/// external-overlay / notification layers are excluded on *every* version (handled host-side).
const MIN_GAMESCOPE_OVERLAY: (u32, u32, u32) = (3, 16, 23);
/// Best-effort: warn if the installed gamescope is older than [`MIN_GAMESCOPE`] (capture is
/// unreliable) or than [`MIN_GAMESCOPE_OVERLAY`] (capture works but the Steam overlay can't reach
/// the stream). Parsing failures are silent (don't block a possibly-fine custom build) — this is a
/// diagnostic, not a gate. Returns the parsed version when it could read one.
pub(super) fn check_gamescope_version() -> Option<(u32, u32, u32)> {
let out = Command::new("gamescope").arg("--version").output().ok()?;
// gamescope prints the version banner to stderr on some builds, stdout on others.
@@ -360,6 +370,18 @@ pub(super) fn check_gamescope_version() -> Option<(u32, u32, u32)> {
capture deadlock against PipeWire 1.6 (a wedged link head-blocks the daemon); \
upgrade gamescope or use PUNKTFUNK_COMPOSITOR=kwin|mutter"
);
} else if ver < MIN_GAMESCOPE_OVERLAY {
// Capture is fine; the Steam overlay just won't be in the frame gamescope hands us.
tracing::warn!(
found = %format!("{}.{}.{}", ver.0, ver.1, ver.2),
min = %format!(
"{}.{}.{}",
MIN_GAMESCOPE_OVERLAY.0, MIN_GAMESCOPE_OVERLAY.1, MIN_GAMESCOPE_OVERLAY.2
),
"gamescope is older than the first version that paints the Steam overlay (Shift+Tab / \
Quick Access Menu) into its PipeWire node the overlay will be absent from the \
stream until you upgrade gamescope (the cursor is composited host-side regardless)"
);
}
Some(ver)
}
@@ -379,7 +401,7 @@ fn parse_version(text: &str) -> Option<(u32, u32, u32)> {
#[cfg(test)]
mod tests {
use super::{parse_version, steam_appid_from_launch, MIN_GAMESCOPE};
use super::{parse_version, steam_appid_from_launch, MIN_GAMESCOPE, MIN_GAMESCOPE_OVERLAY};
#[test]
fn parses_steam_appid_from_launch() {
@@ -425,4 +447,15 @@ mod tests {
assert!(parse_version("gamescope version 3.16.22").unwrap() >= MIN_GAMESCOPE);
assert!(parse_version("gamescope version 3.17.0").unwrap() >= MIN_GAMESCOPE);
}
#[test]
fn overlay_threshold_brackets_the_fix() {
// 3.16.22 captures fine but predates the overlay-in-pipewire paint — it sits in the
// "capture works, overlay absent" window `[MIN_GAMESCOPE, MIN_GAMESCOPE_OVERLAY)`, which is
// exactly the `else if` warn arm; 3.16.23 is the first to include the overlay.
assert!(parse_version("gamescope version 3.16.22").unwrap() >= MIN_GAMESCOPE);
assert!(parse_version("gamescope version 3.16.22").unwrap() < MIN_GAMESCOPE_OVERLAY);
assert!(parse_version("gamescope version 3.16.23").unwrap() >= MIN_GAMESCOPE_OVERLAY);
assert!(parse_version("gamescope version 3.16.25").unwrap() >= MIN_GAMESCOPE_OVERLAY);
}
}
+448 -132
View File
@@ -89,6 +89,11 @@ pub struct KwinDisplay {
/// [`apply_position`](VirtualDisplay::apply_position) addresses OUR output even while a
/// superseded same-name sibling is still alive.
last_name: Option<String>,
/// The RESOLVED `kde_output_device_v2` UUID of the last `create`'s output, when the in-process
/// output-management path handled the topology. A stable per-output id (unlike the shared name),
/// so [`apply_position`](VirtualDisplay::apply_position) and restore address exactly OUR output
/// across a supersede — preferred over `last_name`, which is only the kscreen-doctor fallback.
our_uuid: Option<String>,
/// The topology-restore action the last `create` prepared (re-enable the outputs an `exclusive`
/// topology disabled), pending pickup by the registry via [`take_topology_restore`] — so the
/// physical is re-enabled only when the display GROUP's last member drops (§6.1), not this session's.
@@ -114,6 +119,48 @@ impl KwinDisplay {
pub fn new() -> Result<Self> {
Ok(KwinDisplay::default())
}
/// Apply the effective display topology for the just-created output `our_prefix` (current size
/// `dims`), preferring the in-process `kde_output_management_v2` path and falling back to
/// `kscreen-doctor` if the compositor doesn't answer in budget or the management global is
/// absent. Records the output's UUID (in-process) or kscreen address (fallback) for
/// [`apply_position`](VirtualDisplay::apply_position), and returns the disabled outputs (each
/// `(name, "WxH@Hz")`) for the group teardown restore. `Extend`/`Auto` disable nothing.
fn apply_topology(
&mut self,
name: &str,
our_prefix: &str,
dims: (u32, u32),
) -> Vec<(String, String)> {
use crate::kwin_output_mgmt::TopologyKind;
use crate::policy::Topology;
let topology = crate::effective_topology();
let kind = match topology {
Topology::Exclusive => TopologyKind::Exclusive,
Topology::Primary => TopologyKind::Primary,
Topology::Extend | Topology::Auto => return Vec::new(),
};
// In-process over Wayland — immune to whatever wedges the standalone kscreen-doctor.
let outcome = crate::kwin_output_mgmt::apply_topology(our_prefix, dims.0, dims.1, kind);
if outcome.handled {
self.our_uuid = outcome.our_uuid;
return outcome.disabled;
}
// Fallback: kscreen-doctor — resolve our address the old way, then shell out the topology.
tracing::info!(
"KWin topology: kde_output_management unavailable — kscreen-doctor fallback"
);
let addr = resolve_kscreen_addr(name, dims.0, dims.1);
self.last_name = Some(addr.clone());
match topology {
Topology::Exclusive => apply_virtual_primary(&addr),
Topology::Primary => {
apply_virtual_primary_only(&addr);
Vec::new()
}
Topology::Extend | Topology::Auto => Vec::new(),
}
}
}
impl VirtualDisplay for KwinDisplay {
@@ -142,18 +189,21 @@ impl VirtualDisplay for KwinDisplay {
}
fn apply_position(&mut self, x: i32, y: i32) {
// `last_name` holds the RESOLVED kscreen address (numeric output id, or the
// `Virtual-<name>` fallback) — never re-derive from the name: during a supersede two
// Prefer the in-process path: address OUR output by its stable UUID (supersede-robust) over
// kde_output_management_v2 — immune to a wedged kscreen-doctor backend (see kwin_output_mgmt).
if let Some(uuid) = self.our_uuid.clone() {
if crate::kwin_output_mgmt::set_position(&uuid, x, y) {
return;
}
}
// Fallback: kscreen-doctor. `last_name` holds the RESOLVED kscreen address (numeric output id,
// or the `Virtual-<name>` fallback) — never re-derive from the name: during a supersede two
// outputs share it and the command would hit the old one (see `create`).
let Some(output) = self.last_name.clone() else {
return;
};
// kscreen-doctor position syntax: `output.<name-or-id>.position.<x>,<y>`.
let ok = std::process::Command::new("kscreen-doctor")
.arg(format!("output.{output}.position.{x},{y}"))
.status()
.map(|s| s.success())
.unwrap_or(false);
let ok = kscreen_ok(&[format!("output.{output}.position.{x},{y}")]);
if ok {
tracing::info!(output, x, y, "KWin: placed output in the desktop layout");
} else {
@@ -210,14 +260,16 @@ impl VirtualDisplay for KwinDisplay {
// resize handling: when the source's texture size changes while recording, KWin re-runs
// `buildFormats` — picking up the output's CURRENT refresh — and renegotiates the live
// stream via `pw_stream_update_params`. So above 60 Hz the output is born at a
// SACRIFICIAL height: installing + selecting the real `WxH@hz` custom mode (supported on
// virtual outputs since KWin 6.6) then changes the SIZE, and the first buffers recorded
// after the consumer connects trigger KWin's resize → a renegotiation to `WxH@hz`. The
// SACRIFICIAL height: installing + selecting the real high-refresh custom mode (supported
// on virtual outputs since KWin 6.6) then changes the SIZE, and the first buffers recorded
// after the consumer connects trigger KWin's resize → a renegotiation to that mode. The
// capturer holds frames until that lands (`expect_exact_dims`), so the pipeline never
// builds against the birth mode. First cut shells out to kscreen-doctor; the in-process
// kde_output_management_v2 client is a follow-up. `set_custom_refresh` reads back what
// KWin *actually* gave so the encoder paces to the real source rate. At ≤60 Hz there's
// nothing to install — the output is born at the real size and 60 Hz is the offer anyway.
// builds against the birth mode. The install/select runs in-process over
// kde_output_management_v2 (`kwin_output_mgmt::set_custom_mode`), with kscreen-doctor
// (`set_custom_refresh`) as the fallback; either reads back what KWin *actually* gave — both
// the rate (so the encoder paces to the real source) and the size, which KWin's CVT
// generator may have aligned down (see `CVT_H_GRANULARITY`). At ≤60 Hz there's nothing to
// install — the output is born at the real size and 60 Hz is the offer anyway.
let want_high = mode.refresh_hz > 60;
let birth_h = if want_high { height + 16 } else { height };
let (mut node_id, mut stop) = spawn_vout(width, birth_h)?;
@@ -229,48 +281,75 @@ impl VirtualDisplay for KwinDisplay {
embedded_pointer = !self.hw_cursor,
"KWin virtual output ready"
);
// ⚠️ ADDRESS BY NUMERIC KSCREEN ID, NEVER BY NAME: a supersede (mode switch) creates the
// replacement output — SAME per-slot name, deliberately, for KWin's per-name config
// persistence — while the superseded sibling is still alive (create-before-drop). Every
// name-addressed kscreen command then hits the FIRST match = the OLD output: on-glass this
// resized the LIVE session's display out from under it (wrong-res/black), read back the
// OLD output as "custom mode applied", made the OLD output primary, and positioned it —
// while the new output never left its birth mode and the capturer's dims gate starved.
// Resolve OUR output's kscreen id once (match: managed-prefix name AND current mode ==
// the just-created birth size; newest id wins), and use it for every kscreen operation.
let mut addr = resolve_kscreen_addr(&name, width, birth_h);
self.last_name = Some(addr.clone()); // for apply_position (registry-driven §6.2 layout)
// Topology + positioning address OUR output by its kde_output_management UUID (resolved
// in-process in `apply_topology`, supersede-robust) — no early kscreen-doctor resolve, so
// the path never shells out. `Virtual-<name>` is the name KWin exposes our output as.
let our_prefix = format!("Virtual-{name}");
let mut expect_exact_dims = false;
// The size the output actually ENDS UP at — the request, unless KWin's CVT generator had to
// shrink the width to the cell grain (see `CVT_H_GRANULARITY`). Reported as the output's
// `preferred_mode`, which is what the capturer's renegotiation gate waits for and what the
// encoder opens against, so a CVT-aligned mode flows end-to-end instead of starving.
let mut final_dims = (width, height);
let achieved_hz = if want_high {
let (achieved, size_applied) =
set_custom_refresh(width, height, mode.refresh_hz, &addr);
if size_applied {
// Real mode selected: the recording stream will renegotiate to it (see above).
expect_exact_dims = true;
achieved
} else {
// Custom-mode install/select rejected (pre-6.6 KWin / stale kscreen-doctor): the
// output is STUCK at the sacrificial birth size — unusable. Recreate plain at the
// real size (the pre-sacrifice behavior: correct size, KWin's native 60 Hz).
tracing::warn!(
"KWin rejected the custom mode — recreating the virtual output at the real \
size (60 Hz ceiling on this KWin)"
);
stop.store(true, Ordering::Relaxed);
// Let KWin retire the doomed output before re-using its name.
std::thread::sleep(Duration::from_millis(300));
let (nid, st) = spawn_vout(width, height)?;
node_id = nid;
stop = st;
addr = resolve_kscreen_addr(&name, width, height);
// >60 Hz needs the real high-refresh custom mode installed + selected (sacrificial-birth,
// see above). In-process over kde_output_management_v2 first (no kscreen-doctor); fall
// back to the kscreen-doctor shell-out on pre-6.6 KWin (no `set_custom_modes`) or if the
// compositor doesn't answer in budget.
let active = crate::kwin_output_mgmt::set_custom_mode(
&our_prefix,
width,
birth_h,
width,
height,
mode.refresh_hz,
)
.or_else(|| {
// ⚠️ ADDRESS BY NUMERIC KSCREEN ID, NEVER BY NAME: a supersede reuses the per-slot
// name while the superseded sibling is still alive, so a name-addressed kscreen
// command hits the OLD output. Resolve our kscreen id for the shell-out install +
// keep it as apply_position's kscreen fallback.
let addr = resolve_kscreen_addr(&name, width, birth_h);
self.last_name = Some(addr.clone());
tracing::info!(
node_id,
width,
height,
"KWin virtual output ready (fallback)"
);
60
set_custom_refresh(width, height, mode.refresh_hz, &addr)
});
// Accept only an active mode that IS our custom one: the exact requested height, and a
// width at or just below the request (a CVT alignment). That also proves the output
// left the sacrificial birth size, so the recording stream will renegotiate to it.
match active {
Some((aw, ah, ahz))
if ah == height && aw <= width && width - aw < CVT_H_GRANULARITY =>
{
expect_exact_dims = true;
final_dims = (aw, ah);
ahz
}
other => {
// Custom-mode install/select rejected (pre-6.6 KWin / stale kscreen-doctor): the
// output is STUCK at the sacrificial birth size — unusable. Recreate plain at the
// real size (the pre-sacrifice behavior: correct size, KWin's native 60 Hz).
tracing::warn!(
active = ?other,
requested_w = width,
requested_h = height,
requested_hz = mode.refresh_hz,
"KWin rejected the custom mode — recreating the virtual output at the real \
size (60 Hz ceiling on this KWin)"
);
stop.store(true, Ordering::Relaxed);
// Let KWin retire the doomed output before re-using its name.
std::thread::sleep(Duration::from_millis(300));
let (nid, st) = spawn_vout(width, height)?;
node_id = nid;
stop = st;
tracing::info!(
node_id,
width,
height,
"KWin virtual output ready (fallback)"
);
60
}
}
} else {
mode.refresh_hz
@@ -279,16 +358,16 @@ impl VirtualDisplay for KwinDisplay {
// `Primary` makes it the primary output but keeps the bootstrap/physical outputs enabled;
// `Exclusive` makes it the SOLE desktop (others disabled, restored on teardown) — so
// plasmashell + windows land on the streamed surface, not the headless `kwin --virtual`
// bootstrap output. Read from the policy (replacing the PUNKTFUNK_KWIN_VIRTUAL_PRIMARY boolean).
use crate::policy::Topology;
let disabled = match crate::effective_topology() {
Topology::Exclusive => apply_virtual_primary(&addr),
Topology::Primary => {
apply_virtual_primary_only(&addr);
Vec::new() // nothing disabled → nothing to restore
}
Topology::Extend | Topology::Auto => Vec::new(),
};
// bootstrap output. Applied over kde_output_management_v2 in-process (immune to a wedged
// kscreen-doctor backend; see `apply_topology`), with a kscreen-doctor fallback. `disabled`
// is the physical/bootstrap outputs, each `(name, "WxH@Hz")`, to restore on teardown.
let disabled = self.apply_topology(&name, &our_prefix, final_dims);
// A plain managed name is enough for apply_position's kscreen-doctor fallback when the
// in-process UUID path isn't set (single-output sessions are unambiguous; a supersede uses
// the UUID path instead). `want_high` already set `last_name` to the resolved kscreen id.
if self.last_name.is_none() {
self.last_name = Some(our_prefix);
}
// Per-group restore (§6.1): DON'T bind the re-enable to this session's keepalive (a per-session
// `StopGuard` restore would re-enable the physical the moment the FIRST of several exclusive
// sessions drops — under a still-live sibling). Instead stash it as a closure the registry lifts
@@ -296,13 +375,18 @@ impl VirtualDisplay for KwinDisplay {
// that display's output is reclaimed, so KWin never sees zero outputs). Empty ⇒ nothing to restore.
self.pending_restore = (!disabled.is_empty()).then(|| {
let disabled = disabled.clone();
Box::new(move || reenable_outputs(&disabled)) as Box<dyn FnOnce() + Send>
// In-process first; fall back to kscreen-doctor if the compositor doesn't answer in budget.
Box::new(move || {
if !crate::kwin_output_mgmt::reenable_outputs(&disabled) {
reenable_outputs_kscreen(&disabled);
}
}) as Box<dyn FnOnce() + Send>
});
// Layout position (§6.2) is applied by the registry via `apply_position` right after create
// (it owns the display group, so it computes auto-row / manual placement over the whole group).
let mut out = VirtualOutput::owned(
node_id,
Some((mode.width, mode.height, achieved_hz)),
Some((final_dims.0, final_dims.1, achieved_hz)),
Box::new(StopGuard { stop }),
);
out.expect_exact_dims = expect_exact_dims;
@@ -310,10 +394,12 @@ impl VirtualDisplay for KwinDisplay {
}
}
/// Re-enable the outputs an `exclusive` topology disabled (bootstrap / physical), so KWin re-homes onto
/// them. Called by the registry when the display group's last member is torn down (design §6.1), BEFORE
/// that member's output is reclaimed — so KWin is never momentarily left with zero enabled outputs.
fn reenable_outputs(outputs: &[(String, String)]) {
/// Re-enable the outputs an `exclusive` topology disabled (bootstrap / physical) via `kscreen-doctor`
/// the fallback for the in-process [`crate::kwin_output_mgmt::reenable_outputs`], run by the restore
/// closure only when the in-process path reports the compositor didn't answer. Called by the registry
/// when the display group's last member is torn down (design §6.1), BEFORE that member's output is
/// reclaimed — so KWin is never momentarily left with zero enabled outputs.
fn reenable_outputs_kscreen(outputs: &[(String, String)]) {
if outputs.is_empty() {
return;
}
@@ -324,9 +410,7 @@ fn reenable_outputs(outputs: &[(String, String)]) {
.iter()
.map(|(name, _)| format!("output.{name}.enable"))
.collect();
let _ = std::process::Command::new("kscreen-doctor")
.args(&enable_args)
.status();
let _ = kscreen_ok(&enable_args);
// THEN re-assert each captured mode, best-effort — a bare re-enable lets KWin fall back to the
// EDID-preferred mode (a 120 Hz panel returns at ~60 Hz); this restores the exact refresh. The
// output is enabled now, so the mode set is valid; a rejected mode just leaves KWin's default.
@@ -336,9 +420,7 @@ fn reenable_outputs(outputs: &[(String, String)]) {
.map(|(name, mode)| format!("output.{name}.mode.{mode}"))
.collect();
if !mode_args.is_empty() {
let _ = std::process::Command::new("kscreen-doctor")
.args(&mode_args)
.status();
let _ = kscreen_ok(&mode_args);
}
std::thread::sleep(Duration::from_millis(200));
tracing::info!(reenabled = ?outputs, "KWin: restored the physical/bootstrap outputs at their captured modes (group empty)");
@@ -386,13 +468,37 @@ fn resolve_kscreen_addr(name: &str, w: u32, h: u32) -> String {
fallback
}
/// Budget for one `kscreen-doctor` call.
///
/// It is a Wayland client of the very compositor it configures, so against a wedged KWin it blocks
/// in its own connect and never returns — and these calls run on the session's stream thread, whose
/// only way to end a session is to return. Generous next to a healthy call (tens of ms).
const KSCREEN_BUDGET: Duration = Duration::from_secs(5);
/// `kscreen-doctor <args>` run for its exit status, bounded by [`KSCREEN_BUDGET`]. A timeout reads
/// as a failed apply — the same best-effort path a rejected argument already takes.
fn kscreen_ok(args: &[String]) -> bool {
crate::proc::status_within(
std::process::Command::new("kscreen-doctor").args(args),
KSCREEN_BUDGET,
)
.map(|s| s.success())
.unwrap_or(false)
}
/// `kscreen-doctor -j` stdout, bounded by [`KSCREEN_BUDGET`]; `None` on any failure.
fn kscreen_json_bytes() -> Option<Vec<u8>> {
crate::proc::output_within(
std::process::Command::new("kscreen-doctor").arg("-j"),
KSCREEN_BUDGET,
)
.ok()
.map(|o| o.stdout)
}
/// `kscreen-doctor -j` parsed, `None` on any failure.
fn kscreen_json() -> Option<serde_json::Value> {
let out = std::process::Command::new("kscreen-doctor")
.arg("-j")
.output()
.ok()?;
serde_json::from_slice(&out.stdout).ok()
serde_json::from_slice(&kscreen_json_bytes()?).ok()
}
/// The `(width, height)` of an output's CURRENT mode from its `kscreen-doctor -j` entry.
@@ -415,40 +521,184 @@ fn output_active_size(o: &serde_json::Value) -> Option<(u32, u32)> {
))
}
/// Best-effort: install + select the `width`x`height`@`hz` custom mode on the just-created
/// virtual output via `kscreen-doctor` (`output` is the RESOLVED kscreen address — numeric id or
/// name, see [`resolve_kscreen_addr`] — refresh given in mHz), then **read back the active mode**
/// and return `(achieved_hz, size_applied)`. The apply command can report success yet leave the
/// output on its old mode (rejected), and a silent rate mismatch surfaces downstream as judder /
/// duplicated frames — so the caller paces the encoder to the *achieved* rate, not the requested
/// one. `size_applied` tells the sacrificial-birth caller (see `create`) whether the SIZE half of
/// the mode actually landed — that, not the refresh, is what triggers KWin's stream
/// renegotiation.
fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> (u32, bool) {
/// CVT's horizontal cell granularity. KWin generates every custom mode's timing with **libxcvt**,
/// whose first step is `hdisplay_rnd = hdisplay - (hdisplay % 8)` — so a width that isn't a multiple
/// of 8 comes back NARROWER than asked, and the clock-step rounding that follows lands a fractional
/// refresh. A 2868x1320@120 request (an iPhone 16 Pro Max panel) becomes **2864x1320@119.92**.
///
/// That is why a custom mode must never be selected by the `WxH@Hz` string we *requested*:
/// kscreen-doctor's `findMode` matches a mode's id or its own `WxH@qRound(Hz)` name, so
/// `2868x1320@120` matches nothing, the select silently no-ops, the output stays on its sacrificial
/// birth mode, and the caller falls back to 60 Hz — while KDE's display list shows the perfectly
/// good 2864x1320@119.92 mode sitting there unselected. Widths like 1920/2560/3840 are all
/// multiples of 8, which is why only phone-shaped clients ever hit it.
const CVT_H_GRANULARITY: u32 = 8;
/// One row of an output's mode list, as parsed from `kscreen-doctor -j`.
#[derive(Clone, Debug, PartialEq)]
struct KModeRow {
/// kscreen's mode id — what we address the mode by (never the requested `WxH@Hz` string).
id: String,
w: u32,
h: u32,
hz: f64,
}
/// A kscreen JSON id, which is a string on some KWin versions and a number on others.
fn json_id(v: &serde_json::Value) -> Option<String> {
v.as_str()
.map(|s| s.to_string())
.or_else(|| v.as_u64().map(|n| n.to_string()))
}
/// The full mode list of `output` (a RESOLVED kscreen address — numeric id or name) from a parsed
/// `kscreen-doctor -j` document. Split from the process call so the picker can be tested on
/// captured JSON.
fn modes_from_json(doc: &serde_json::Value, output: &str) -> Vec<KModeRow> {
let Some(o) = doc
.get("outputs")
.and_then(|v| v.as_array())
.and_then(|outs| {
outs.iter().find(|o| {
o.get("name").and_then(|n| n.as_str()) == Some(output)
|| o.get("id").and_then(json_id).as_deref() == Some(output)
})
})
else {
return Vec::new();
};
o.get("modes")
.and_then(|m| m.as_array())
.map(|ms| {
ms.iter()
.filter_map(|m| {
let size = m.get("size")?;
Some(KModeRow {
id: m.get("id").and_then(json_id)?,
w: size.get("width").and_then(|v| v.as_u64())? as u32,
h: size.get("height").and_then(|v| v.as_u64())? as u32,
hz: m.get("refreshRate").and_then(|r| r.as_f64())?,
})
})
.collect()
})
.unwrap_or_default()
}
/// [`modes_from_json`] against a live `kscreen-doctor -j`.
fn output_modes(output: &str) -> Vec<KModeRow> {
kscreen_json()
.map(|doc| modes_from_json(&doc, output))
.unwrap_or_default()
}
/// The mode in `modes` that actually fulfils a `width`x`height`@`hz` request, tolerating the CVT
/// alignment KWin applies when it generates the timing (see [`CVT_H_GRANULARITY`]): the height must
/// match exactly (CVT never touches the vertical active), the width may be up to one cell narrower
/// than asked (never wider — that would be a different mode), and the refresh must land within 1 Hz
/// of the request (which excludes the output's native 60 Hz entry for every rate we install a custom
/// mode for). Widest wins, then fastest — so an exact-width mode always beats an aligned one, and a
/// list carrying duplicate custom modes from earlier sessions still resolves.
fn pick_custom_mode<'a>(
modes: &'a [KModeRow],
width: u32,
height: u32,
hz: u32,
) -> Option<&'a KModeRow> {
modes
.iter()
.filter(|m| {
m.h == height
&& m.w <= width
&& width - m.w < CVT_H_GRANULARITY
&& (m.hz - f64::from(hz)).abs() < 1.0
})
.max_by(|a, b| {
a.w.cmp(&b.w)
.then(a.hz.partial_cmp(&b.hz).unwrap_or(std::cmp::Ordering::Equal))
})
}
/// Best-effort: install + select the `width`x`height`@`hz` custom mode on the just-created virtual
/// output via `kscreen-doctor` (`output` is the RESOLVED kscreen address — numeric id or name, see
/// [`resolve_kscreen_addr`] — refresh given in mHz), then **read back the active mode** and return
/// it as `(width, height, refresh_hz)`. `None` if the read-back failed entirely.
///
/// The apply command can report success yet leave the output on its old mode (rejected), and a
/// silent size/rate mismatch surfaces downstream as a starved capture gate or judder — so the
/// caller drives the pipeline off the *achieved* mode, not the requested one. The mode is selected
/// by kscreen **mode id** resolved from the output's own list, never by the requested `WxH@Hz`
/// string, because KWin's CVT generator may hand back a slightly different one
/// ([`CVT_H_GRANULARITY`]).
fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> Option<(u32, u32, u32)> {
let output = output.to_string();
let mhz = hz.saturating_mul(1000);
let run = |arg: String| {
std::process::Command::new("kscreen-doctor")
.arg(arg)
.status()
.map(|s| s.success())
.unwrap_or(false)
let run = |arg: String| kscreen_ok(&[arg]);
// Install the mode only if the output doesn't already carry a usable one: kscreen-doctor
// APPENDS to the output's custom-mode list and KWin PERSISTS that list per output name
// (`kwinoutputconfig.json`, which is why the same per-slot name is reused across sessions) — so
// re-adding on every connect would grow the user's display list without bound.
let mut modes = output_modes(&output);
if pick_custom_mode(&modes, width, height, hz).is_none() {
let _ = run(format!(
"output.{output}.addCustomMode.{width}.{height}.{mhz}.full"
));
modes = output_modes(&output);
}
let applied = match pick_custom_mode(&modes, width, height, hz) {
Some(target) => {
if (target.w, target.h) != (width, height) {
tracing::info!(
output,
requested_w = width,
requested_h = height,
mode_w = target.w,
mode_h = target.h,
mode_hz = target.hz,
"KWin aligned the custom mode to the CVT cell grain — streaming at its size"
);
}
// By id first; the human `WxH@Hz` form (built from the mode's OWN size/refresh, not the
// request) is the fallback for builds whose ids don't round-trip through the CLI.
run(format!("output.{output}.mode.{}", target.id))
|| run(format!(
"output.{output}.mode.{}x{}@{}",
target.w,
target.h,
target.hz.round() as u32
))
}
None => {
tracing::warn!(
output,
requested_w = width,
requested_h = height,
requested_hz = hz,
offered = ?modes,
"KWin offers no mode matching the request after addCustomMode — is kscreen-doctor \
up to date, and KWin 6.6 (custom modes on virtual outputs)?"
);
false
}
};
// Add the custom mode (a fresh output has none), then select it.
let _ = run(format!(
"output.{output}.addCustomMode.{width}.{height}.{mhz}.full"
));
let applied = run(format!("output.{output}.mode.{width}x{height}@{hz}"));
match read_active_mode(&output) {
Some((w, h, achieved)) => {
let size_applied = (w, h) == (width, height);
if achieved >= hz && size_applied {
if achieved >= hz && (w, h) == (width, height) {
tracing::info!(
output,
requested = hz,
achieved,
"KWin virtual output: custom refresh applied"
);
} else if achieved >= hz {
tracing::info!(
output,
requested = hz,
achieved,
active_w = w,
active_h = h,
"KWin virtual output: custom refresh applied at a CVT-aligned size"
);
} else {
tracing::warn!(
output,
@@ -461,7 +711,7 @@ fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> (u32, b
achieved rate (custom-mode install rejected? is kscreen-doctor up to date?)"
);
}
(achieved.max(1), size_applied)
Some((w, h, achieved.max(1)))
}
None => {
tracing::warn!(
@@ -471,7 +721,7 @@ fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> (u32, b
"could not read back KWin virtual output refresh — assuming 60 Hz (is \
kscreen-doctor installed?)"
);
(60, false)
None
}
}
}
@@ -542,14 +792,11 @@ fn output_current_mode_spec(o: &serde_json::Value) -> Option<String> {
/// session's own output — so a 2nd `exclusive` session (with a distinct per-slot name) never disables
/// the 1st session's live output. Parsed from `kscreen-doctor -j` (same source as [`read_active_mode`]).
fn other_enabled_outputs() -> Vec<(String, String)> {
let out = match std::process::Command::new("kscreen-doctor")
.arg("-j")
.output()
{
Ok(o) => o,
Err(_) => return Vec::new(),
let out = match kscreen_json_bytes() {
Some(o) => o,
None => return Vec::new(),
};
let doc: serde_json::Value = match serde_json::from_slice(&out.stdout) {
let doc: serde_json::Value = match serde_json::from_slice(&out) {
Ok(d) => d,
Err(_) => return Vec::new(),
};
@@ -578,13 +825,10 @@ fn other_enabled_outputs() -> Vec<(String, String)> {
/// then sets itself primary — the pre-group behavior). Recent kscreen marks the primary with
/// `"priority": 1`; older builds used a `"primary": true` bool — accept either.
fn a_managed_output_is_primary() -> bool {
let Ok(out) = std::process::Command::new("kscreen-doctor")
.arg("-j")
.output()
else {
let Some(out) = kscreen_json_bytes() else {
return false;
};
let Ok(doc) = serde_json::from_slice::<serde_json::Value>(&out.stdout) else {
let Ok(doc) = serde_json::from_slice::<serde_json::Value>(&out) else {
return false;
};
doc.get("outputs")
@@ -609,13 +853,7 @@ fn a_managed_output_is_primary() -> bool {
/// the keepalive to re-enable on teardown. Best-effort: on failure, streaming continues (just possibly
/// showing only the wallpaper) rather than failing the session.
fn apply_virtual_primary(ours: &str) -> Vec<(String, String)> {
let kscreen = |args: &[String]| {
std::process::Command::new("kscreen-doctor")
.args(args)
.status()
.map(|s| s.success())
.unwrap_or(false)
};
let kscreen = |args: &[String]| kscreen_ok(args);
// First-slot-wins (§6.1): only grab primary if no managed group member is primary yet — so a 2nd
// exclusive session joins as a secondary monitor of the shared desktop instead of stealing the
// shell off the 1st session's output. KWin usually then re-homes the desktop + disables the
@@ -647,11 +885,7 @@ fn apply_virtual_primary(ours: &str) -> Vec<(String, String)> {
/// (don't disable the bootstrap/physical) — so the shell re-homes onto the streamed surface while a
/// physical screen stays usable. Nothing to restore on teardown (we disabled nothing).
fn apply_virtual_primary_only(ours: &str) {
let ok = std::process::Command::new("kscreen-doctor")
.arg(format!("output.{ours}.primary"))
.status()
.map(|s| s.success())
.unwrap_or(false);
let ok = kscreen_ok(&[format!("output.{ours}.primary")]);
if ok {
tracing::info!("KWin: streamed output set primary (physical outputs kept)");
} else {
@@ -661,8 +895,9 @@ fn apply_virtual_primary_only(ours: &str) {
/// Dropping this releases the KWin virtual output: it flips the keepalive thread's `stop`, which
/// drops the Wayland connection and makes KWin reclaim the output. The topology **restore** is no
/// longer bound here — it moved to the registry's display group (§6.1, [`reenable_outputs`]), which
/// runs it once when the group's last member drops, BEFORE this keepalive is dropped.
/// longer bound here — it moved to the registry's display group (§6.1, restored in-process via
/// [`crate::kwin_output_mgmt::reenable_outputs`], `kscreen-doctor` fallback), which runs it once when
/// the group's last member drops, BEFORE this keepalive is dropped.
struct StopGuard {
stop: Arc<AtomicBool>,
}
@@ -884,7 +1119,88 @@ fn run(
#[cfg(test)]
mod tests {
use super::MANAGED_PREFIX;
use super::{modes_from_json, pick_custom_mode, KModeRow, MANAGED_PREFIX};
fn row(id: &str, w: u32, h: u32, hz: f64) -> KModeRow {
KModeRow {
id: id.to_string(),
w,
h,
hz,
}
}
/// The reported regression: an iPhone 16 Pro Max asks for 2868x1320@120; libxcvt rounds the
/// width down to the 8-pixel cell grain and the clock step lands 119.92, so KWin's list holds
/// 2864x1320@119.92. Selecting by the REQUESTED `2868x1320@120` string matched nothing — the
/// output stayed on its birth mode and the session fell back to 60 Hz. The picker must find it.
#[test]
fn picks_the_cvt_aligned_mode() {
let modes = [
row("1", 2868, 1320, 60.0), // the virtual output's native/birth mode
row("2", 2864, 1320, 119.92), // the custom mode KWin actually generated
];
let got = pick_custom_mode(&modes, 2868, 1320, 120).expect("CVT-aligned mode");
assert_eq!((got.id.as_str(), got.w, got.h), ("2", 2864, 1320));
}
/// A width already on the cell grain (every PC resolution: 1920/2560/3840) round-trips exactly,
/// and an exact-width mode outranks an aligned one when both are offered.
#[test]
fn exact_width_outranks_an_aligned_one() {
let modes = [
row("1", 2560, 1440, 60.0),
row("2", 2552, 1440, 119.93), // a stale narrower custom mode from an earlier session
row("3", 2560, 1440, 119.98),
];
let got = pick_custom_mode(&modes, 2560, 1440, 120).expect("exact mode");
assert_eq!(got.id, "3");
}
/// The picker must never wander onto an unrelated mode: not the 60 Hz native entry (the old
/// fallback the reporter got stuck on), not a different height, not a wider width, and not a
/// mode more than one cell narrower than asked.
#[test]
fn rejects_modes_that_are_not_the_request() {
let modes = [
row("1", 2868, 1320, 60.0), // native — refresh too far off
row("2", 2868, 1080, 119.92), // wrong height
row("3", 2880, 1320, 119.92), // wider than requested
row("4", 2856, 1320, 119.92), // two cells narrower — not a CVT alignment of 2868
row("5", 1920, 1080, 120.0), // unrelated
];
assert!(pick_custom_mode(&modes, 2868, 1320, 120).is_none());
}
/// Mode + output ids come through as JSON strings on some KWin versions and numbers on others;
/// both must parse, and a mode row missing its size/refresh is skipped rather than poisoning
/// the list.
#[test]
fn parses_both_id_encodings() {
let doc: serde_json::Value = serde_json::from_str(
r#"{"outputs":[
{"id":7,"name":"Virtual-punktfunk","modes":[
{"id":"m1","size":{"width":2868,"height":1320},"refreshRate":60.0},
{"id":42,"size":{"width":2864,"height":1320},"refreshRate":119.92},
{"id":"broken","size":{"width":800}}
]},
{"id":1,"name":"eDP-1","modes":[
{"id":"x","size":{"width":2864,"height":1320},"refreshRate":119.92}
]}
]}"#,
)
.expect("fixture parses");
// Addressable by numeric id (how `resolve_kscreen_addr` returns it) and by name.
for addr in ["7", "Virtual-punktfunk"] {
let modes = modes_from_json(&doc, addr);
assert_eq!(modes.len(), 2, "the malformed row is dropped ({addr})");
assert_eq!(modes[1].id, "42", "numeric mode ids stringify ({addr})");
let got = pick_custom_mode(&modes, 2868, 1320, 120).expect("aligned mode");
assert_eq!(got.id, "42");
}
// Never reads another output's list (the eDP-1 entry carries a matching mode).
assert!(modes_from_json(&doc, "Virtual-nope").is_empty());
}
/// Group-aware exclusive (§6.1): with two managed group members + a physical panel enabled,
/// exclusive disables ONLY the non-managed panel — never a sibling session's per-slot output
@@ -0,0 +1,820 @@
//! In-process KDE output management (`kde_output_management_v2` + `kde_output_device_v2`).
//!
//! Topology — make the streamed output primary, disable the physical/bootstrap outputs, capture
//! their modes for restore, re-enable them on teardown, position the output — used to shell out to
//! `kscreen-doctor` (see [`super::kwin`]). But `kscreen-doctor` drives a *separate* stack:
//! libkscreen picks a backend and, depending on the setup, waits on the kscreen KDED module over
//! D-Bus. On a machine where THAT layer is wedged it blocks in its own connect and never returns —
//! a field report (Nobara, KWin 6.6.4) showed every topology `kscreen-doctor` timing out at its 5 s
//! budget, so the streamed output never became the desktop (`also_disabled=[]`) and bring-up took
//! ~26 s.
//!
//! The compositor's OWN Wayland is provably responsive on that same session — the host just created
//! a virtual output over it via `zkde_screencast` — so we drive `kde_output_management_v2` directly
//! over Wayland here, sidestepping whatever wedges the standalone tool. Every wait is time-bounded,
//! so a genuinely wedged compositor degrades to `handled = false` and the caller falls back to the
//! `kscreen-doctor` path rather than hanging.
//!
//! KWin advertises one `kde_output_device_v2` global per output (the classic model; verified live:
//! `kde_output_management_v2` v19, `kde_output_device_v2` v20 on KWin 6.6.4). We bind them all, read
//! each output's name / enabled / priority / current-mode size, then build a
//! `kde_output_configuration_v2` and `apply()` it, waiting for `applied` / `failed`.
#![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
#![deny(clippy::undocumented_unsafe_blocks)]
use std::collections::HashMap;
use std::os::fd::{AsFd, AsRawFd};
use std::time::{Duration, Instant};
use wayland_client::backend::ObjectId;
use wayland_client::protocol::wl_callback::{self, WlCallback};
use wayland_client::protocol::wl_registry::{self, WlRegistry};
use wayland_client::{event_created_child, Connection, Dispatch, Proxy, QueueHandle};
// Generate the client bindings for the two vendored KDE protocols inline (no build.rs). The
// management protocol references the device interfaces, so they can't share one `__interfaces`
// module (each `generate_interfaces!` emits its own helper items, which collide). Instead — the
// interdependent-protocol pattern from the `wayland-protocols` crate — `device` is a self-contained
// module and `management` pulls in `device`'s interface statics + generated proxy types before its
// own codegen, so its cross-protocol object args (`kde_output_device_v2`, `…_mode_v2`) resolve.
#[allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
pub mod device {
use wayland_client;
use wayland_client::protocol::*;
pub mod __interfaces {
use wayland_client::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("protocols/kde-output-device-v2.xml");
}
use self::__interfaces::*;
wayland_scanner::generate_client_code!("protocols/kde-output-device-v2.xml");
}
#[allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
pub mod management {
use wayland_client;
use wayland_client::protocol::*;
pub mod __interfaces {
use super::super::device::__interfaces::*;
use wayland_client::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("protocols/kde-output-management-v2.xml");
}
use self::__interfaces::*;
// The device protocol's generated modules/types, so the foreign object args resolve.
use super::device::*;
wayland_scanner::generate_client_code!("protocols/kde-output-management-v2.xml");
}
use device::kde_output_device_mode_v2::{Event as ModeEvent, KdeOutputDeviceModeV2 as DeviceMode};
use device::kde_output_device_v2::{Event as DeviceEvent, KdeOutputDeviceV2 as OutputDevice};
use management::kde_mode_list_v2::KdeModeListV2 as ModeList;
use management::kde_output_configuration_v2::{
Event as ConfigEvent, KdeOutputConfigurationV2 as OutputConfig,
};
use management::kde_output_management_v2::KdeOutputManagementV2 as OutputManagement;
/// Highest interface versions we drive; we bind `min(advertised, MAX)`. Every request we issue is
/// `since ≤ 2` (`create_configuration`/`enable`/`mode`/`position`/`apply` are v1, `set_primary_output`
/// is v2) and every event we read is `since ≤ 18` (`priority`), so binding high and calling low is
/// always in range on any KWin that advertises the globals.
const MGMT_MAX: u32 = 22;
const DEVICE_MAX: u32 = 24;
/// The opcode of `kde_output_device_v2.mode` (0-based event index) — the event that creates a child
/// `kde_output_device_mode_v2`. Kept in sync with the vendored `kde-output-device-v2.xml`.
const DEVICE_MODE_EVENT_OPCODE: u16 = 2;
/// Overall budget for one enumerate-then-apply operation. Generous next to a healthy roundtrip (a
/// few ms); it exists only so a wedged compositor can't pin the session's stream thread.
const OP_BUDGET: Duration = Duration::from_secs(3);
/// Poll slice while waiting on the Wayland fd (matches the keepalive loop's cadence in `kwin.rs`).
const POLL_MS: i32 = 100;
/// KWin's CVT generator aligns a custom mode's width DOWN to a multiple of this (libxcvt's cell
/// grain), so the mode it builds for a `set_custom_modes` request may be a few px narrower than
/// asked — matches `kwin::CVT_H_GRANULARITY`. Used when matching the generated mode back.
const CVT_H_GRANULARITY: u32 = 8;
/// Which topology to apply once our output is resolved.
#[derive(Clone, Copy, PartialEq, Eq)]
pub(crate) enum TopologyKind {
/// Make ours the sole desktop: primary + disable every other enabled output.
Exclusive,
/// Make ours primary but leave the other outputs enabled.
Primary,
}
/// Outcome of [`apply_topology`].
pub(crate) struct TopologyOutcome {
/// UUID of our resolved virtual output — a stable per-output id that survives a mode-switch
/// supersede (unlike the shared name) — for later [`set_position`] / restore addressing.
pub our_uuid: Option<String>,
/// The outputs we disabled, each `(name, "WxH@Hz")`, so teardown can restore them at their exact
/// mode. Empty for `Primary`, or when nothing else was enabled.
pub disabled: Vec<(String, String)>,
/// `true` if the in-process path bound management, resolved our output, and applied (or tried to)
/// a configuration. `false` ⇒ the compositor didn't answer in budget or our output never
/// appeared, so the caller should fall back to `kscreen-doctor`.
pub handled: bool,
}
/// One output as read from `kde_output_device_v2`.
#[derive(Default, Clone)]
struct DeviceState {
/// The global `name` number (higher = more recently advertised) — used to pick the newest of two
/// same-named outputs during a supersede.
global: u32,
name: Option<String>,
uuid: Option<String>,
enabled: bool,
/// KWin's output priority; 1 is the primary. `None` until the `priority` event (device ≥ v18).
priority: Option<u32>,
/// The `current_mode` object id; its size is looked up in [`State::mode_dims`].
current_mode: Option<ObjectId>,
/// Every mode this output advertised, in announce order — `(mode object id, proxy)` — so restore
/// can pick the one matching a captured `WxH@Hz`.
modes: Vec<(ObjectId, DeviceMode)>,
/// Set once this output's `done` burst has been seen (its state is coherent to read).
seen_done: bool,
proxy: Option<OutputDevice>,
}
/// Everything the enumerate/apply queue accumulates on one connection.
#[derive(Default)]
struct State {
management: Option<OutputManagement>,
mgmt_name_version: Option<(u32, u32)>,
devices: HashMap<ObjectId, DeviceState>,
/// mode object id → `(width, height, refresh_mHz)`.
mode_dims: HashMap<ObjectId, (u32, u32, u32)>,
/// Highest `wl_callback` serial whose `done` has arrived — the barrier the pump waits on.
sync_done: u32,
/// Configuration apply verdict: `Some(true)` = applied, `Some(false)` = failed.
applied: Option<bool>,
failure_reason: Option<String>,
}
impl Dispatch<WlRegistry, ()> for State {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wl_registry::Event,
_: &(),
_: &Connection,
qh: &QueueHandle<Self>,
) {
match event {
wl_registry::Event::Global {
name,
interface,
version,
} => {
if interface == OutputManagement::interface().name {
let v = version.min(MGMT_MAX);
state.management =
Some(registry.bind::<OutputManagement, _, _>(name, v, qh, ()));
state.mgmt_name_version = Some((name, v));
} else if interface == OutputDevice::interface().name {
let v = version.min(DEVICE_MAX);
// The device's `name` global carries into the device's UserData so the event
// handler can record it (newest-wins tie-break during a supersede).
let dev = registry.bind::<OutputDevice, _, _>(name, v, qh, name);
let id = dev.id();
state.devices.entry(id).or_default().proxy = Some(dev);
}
}
wl_registry::Event::GlobalRemove { .. } => {}
_ => {}
}
}
}
// Management has no events.
impl Dispatch<OutputManagement, ()> for State {
fn event(
_: &mut Self,
_: &OutputManagement,
_: management::kde_output_management_v2::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
// A client-built custom-mode list has no events; it just needs a Dispatch impl to be created.
impl Dispatch<ModeList, ()> for State {
fn event(
_: &mut Self,
_: &ModeList,
_: management::kde_mode_list_v2::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
/// The device's UserData is its global `name` number.
impl Dispatch<OutputDevice, u32> for State {
fn event(
state: &mut Self,
device: &OutputDevice,
event: DeviceEvent,
global: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
let entry = state.devices.entry(device.id()).or_default();
entry.global = *global;
if entry.proxy.is_none() {
entry.proxy = Some(device.clone());
}
match event {
DeviceEvent::Name { name } => entry.name = Some(name),
DeviceEvent::Uuid { uuid } => entry.uuid = Some(uuid),
DeviceEvent::Enabled { enabled } => entry.enabled = enabled != 0,
DeviceEvent::Priority { priority } => entry.priority = Some(priority),
DeviceEvent::CurrentMode { mode } => entry.current_mode = Some(mode.id()),
DeviceEvent::Mode { mode } => entry.modes.push((mode.id(), mode)),
DeviceEvent::Done => entry.seen_done = true,
_ => {}
}
}
// The `mode` event hands us a server-created `kde_output_device_mode_v2`. The opcode is a bare
// literal (the macro's fragment matcher rejects a `const` in some wayland-client versions); it is
// pinned to `DEVICE_MODE_EVENT_OPCODE` by `mode_event_opcode_is_two` below.
event_created_child!(State, OutputDevice, [
2 => (DeviceMode, ()),
]);
}
impl Dispatch<DeviceMode, ()> for State {
fn event(
state: &mut Self,
mode: &DeviceMode,
event: ModeEvent,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
let entry = state.mode_dims.entry(mode.id()).or_insert((0, 0, 0));
match event {
ModeEvent::Size { width, height } => {
entry.0 = width.max(0) as u32;
entry.1 = height.max(0) as u32;
}
ModeEvent::Refresh { refresh } => entry.2 = refresh.max(0) as u32,
_ => {}
}
}
}
impl Dispatch<OutputConfig, ()> for State {
fn event(
state: &mut Self,
_: &OutputConfig,
event: ConfigEvent,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
match event {
ConfigEvent::Applied => state.applied = Some(true),
ConfigEvent::Failed => state.applied = Some(false),
ConfigEvent::FailureReason { reason } => state.failure_reason = Some(reason),
_ => {}
}
}
}
impl Dispatch<WlCallback, u32> for State {
fn event(
state: &mut Self,
_: &WlCallback,
event: wl_callback::Event,
serial: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
if let wl_callback::Event::Done { .. } = event {
state.sync_done = state.sync_done.max(*serial);
}
}
}
/// A connected, bound output-management session on its own Wayland connection.
struct Session {
conn: Connection,
queue: wayland_client::EventQueue<State>,
state: State,
next_sync: u32,
}
impl Session {
/// Connect to the KWin Wayland socket, bind `kde_output_management_v2` + every
/// `kde_output_device_v2`, and read each output's state — all bounded by `OP_BUDGET`. `None` if
/// we can't connect, the management global isn't advertised, or the compositor doesn't answer in
/// budget (the wedge case — the caller then falls back to `kscreen-doctor`).
fn open() -> Option<Session> {
let conn = Connection::connect_to_env().ok()?;
let queue = conn.new_event_queue();
let qh = queue.handle();
let _registry = conn.display().get_registry(&qh, ());
let mut s = Session {
conn,
queue,
state: State::default(),
next_sync: 0,
};
let deadline = Instant::now() + OP_BUDGET;
// Phase 1: process the registry globals (binds management + every device in the handler).
if !s.sync_barrier(deadline) {
return None;
}
if s.state.management.is_none() {
tracing::debug!(
"KWin does not advertise kde_output_management_v2 to this client — kscreen-doctor \
fallback"
);
return None;
}
// Phase 2: flush the device binds issued in phase 1 and drain each output's state burst
// (name / enabled / priority / current_mode / mode sizes / done).
if !s.sync_barrier(deadline) {
return None;
}
Some(s)
}
/// Send a `wl_display.sync` and pump the queue until its `done` arrives or `deadline` passes.
/// Returns `true` on the barrier, `false` on timeout.
fn sync_barrier(&mut self, deadline: Instant) -> bool {
self.next_sync += 1;
let serial = self.next_sync;
let qh = self.queue.handle();
let _cb = self.conn.display().sync(&qh, serial);
self.pump_until(deadline, |st| st.sync_done >= serial)
}
/// Bounded manual event loop: flush, dispatch what's queued, then poll the connection fd for up
/// to `POLL_MS` and read. Mirrors the keepalive loop in `kwin.rs::run` (blocking_dispatch can't
/// be interrupted, so we poll the fd instead). Returns `true` once `done(&state)` holds.
fn pump_until(&mut self, deadline: Instant, done: impl Fn(&State) -> bool) -> bool {
loop {
if done(&self.state) {
return true;
}
if self.queue.dispatch_pending(&mut self.state).is_err() {
return false;
}
if done(&self.state) {
return true;
}
if Instant::now() >= deadline {
return false;
}
if self.conn.flush().is_err() {
return false;
}
let Some(guard) = self.conn.prepare_read() else {
continue; // events already queued — loop dispatches them
};
let mut pfd = libc::pollfd {
fd: self.conn.as_fd().as_raw_fd(),
events: libc::POLLIN,
revents: 0,
};
let remaining = deadline.saturating_duration_since(Instant::now());
let timeout = (remaining.as_millis() as i32).clamp(0, POLL_MS);
// SAFETY: `&mut pfd` points at one live, fully-initialized `libc::pollfd` on the stack and
// the count `1` matches that single element, so `poll` reads `fd`/`events` and writes
// `revents` strictly within `pfd`. `pfd.fd` is the Wayland connection's fd, valid because
// `self.conn` (and the `prepare_read` guard) outlive the call. `poll` blocks up to
// `timeout` ms and writes only `revents`; `pfd` is a fresh local that aliases nothing.
let r = unsafe { libc::poll(&mut pfd, 1, timeout) };
if r > 0 && (pfd.revents & libc::POLLIN) != 0 {
let _ = guard.read();
} // else: timeout/signal — drop the guard, re-check the deadline
}
}
/// A fresh `kde_output_configuration_v2` on this connection.
fn new_config(&self) -> OutputConfig {
let qh = self.queue.handle();
self.state
.management
.as_ref()
.unwrap()
.create_configuration(&qh, ())
}
/// A fresh (empty) `kde_mode_list_v2` on this connection, for a `set_custom_modes` request.
fn new_mode_list(&self) -> ModeList {
let qh = self.queue.handle();
self.state
.management
.as_ref()
.unwrap()
.create_mode_list(&qh, ())
}
/// `apply()` the config and pump until `applied`/`failed` or the deadline. Returns the verdict
/// (`true` applied, `false` failed/timeout).
fn apply(&mut self, config: &OutputConfig, deadline: Instant) -> bool {
self.state.applied = None;
config.apply();
let ok = self.pump_until(deadline, |st| st.applied.is_some());
if !ok {
return false;
}
matches!(self.state.applied, Some(true))
}
/// The current-mode size of a device as `(w, h, refresh_mHz)`, if known.
fn current_dims(&self, dev: &DeviceState) -> Option<(u32, u32, u32)> {
let id = dev.current_mode.as_ref()?;
self.state.mode_dims.get(id).copied()
}
}
/// `(width, height, "WxH@Hz")` capture of a device's current mode, Hz rounded — the same shape the
/// `kscreen-doctor` restore path used, so teardown can put a panel back at its real refresh.
fn mode_spec(dims: (u32, u32, u32)) -> String {
let hz = ((dims.2 as f64) / 1000.0).round() as u32;
format!("{}x{}@{}", dims.0, dims.1, hz)
}
/// Prefix EVERY managed KWin output shares (mirrors `kwin::MANAGED_PREFIX`) — the streamed outputs
/// are `Virtual-punktfunk` / `Virtual-punktfunk-<id>`, so a same-family sibling session is never
/// treated as a physical to disable, and its primary is never stolen (first-slot-wins).
const MANAGED_PREFIX: &str = "Virtual-punktfunk";
/// Make the streamed output (name starts with `our_prefix`, current size `our_w`×`our_h`) the
/// primary — and, for `Exclusive`, disable every other enabled output — over `kde_output_management_v2`.
/// See the module docs for why this is done in-process instead of via `kscreen-doctor`.
pub(crate) fn apply_topology(
our_prefix: &str,
our_w: u32,
our_h: u32,
kind: TopologyKind,
) -> TopologyOutcome {
let miss = || TopologyOutcome {
our_uuid: None,
disabled: Vec::new(),
handled: false,
};
let Some(mut sess) = Session::open() else {
return miss();
};
let deadline = Instant::now() + OP_BUDGET;
// Resolve OUR output: managed-prefix name AND current size == the birth size (only the
// just-created output sits there during a supersede); newest global wins the tie.
let ours = sess
.state
.devices
.values()
.filter(|d| {
d.name.as_deref().is_some_and(|n| n.starts_with(our_prefix))
&& sess.current_dims(d).map(|(w, h, _)| (w, h)) == Some((our_w, our_h))
})
.max_by_key(|d| d.global)
.cloned();
let Some(ours) = ours else {
tracing::warn!(
our_prefix,
our_w,
our_h,
"KWin output management: our virtual output hasn't appeared yet — kscreen-doctor fallback"
);
return miss();
};
let our_uuid = ours.uuid.clone();
let our_id = ours.proxy.as_ref().map(|p| p.id());
// First-slot-wins (§6.1): don't steal primary if another managed sibling already holds it
// (priority 1) — a 2nd exclusive session joins as a secondary of the shared desktop.
let sibling_is_primary = sess.state.devices.values().any(|d| {
d.enabled
&& d.priority == Some(1)
&& d.proxy.as_ref().map(|p| p.id()) != our_id
&& d.name
.as_deref()
.is_some_and(|n| n.starts_with(MANAGED_PREFIX))
});
// The physical/bootstrap outputs to disable for `Exclusive`: enabled, not any managed sibling,
// not ours. Captured WITH their current mode so teardown restores the exact refresh.
let mut to_disable: Vec<(OutputDevice, String, String)> = Vec::new();
if kind == TopologyKind::Exclusive {
for d in sess.state.devices.values() {
let is_ours = d.proxy.as_ref().map(|p| p.id()) == our_id;
let managed = d
.name
.as_deref()
.is_some_and(|n| n.starts_with(MANAGED_PREFIX));
if d.enabled && !is_ours && !managed {
if let (Some(name), Some(proxy)) = (d.name.clone(), d.proxy.clone()) {
let spec = sess.current_dims(d).map(mode_spec).unwrap_or_default();
to_disable.push((proxy, name, spec));
}
}
}
}
// Build one configuration: ensure ours is enabled, take primary (unless a sibling holds it),
// disable the others. `apply()` is atomic — KWin re-homes the shell onto the remaining desktop.
let config = sess.new_config();
if let Some(proxy) = ours.proxy.as_ref() {
config.enable(proxy, 1);
if !sibling_is_primary {
config.set_primary_output(proxy);
}
}
for (proxy, _, _) in &to_disable {
config.enable(proxy, 0);
}
let applied = sess.apply(&config, deadline);
config.destroy();
// Always report the outputs we ASKED to disable so teardown restores them: re-enabling an output
// that never actually got disabled is a harmless no-op, whereas dropping them here would strand a
// physical dark if KWin processed the disable but the `applied` ack didn't land in budget.
let disabled: Vec<(String, String)> = to_disable
.into_iter()
.map(|(_, name, spec)| (name, spec))
.collect();
if applied {
tracing::info!(
also_disabled = ?disabled,
primary_taken = !sibling_is_primary,
"KWin output management: streamed output set as the desktop (in-process)"
);
} else {
tracing::warn!(
reason = ?sess.state.failure_reason,
also_disabled = ?disabled,
"KWin output management: apply() not confirmed in budget — proceeding (restore will re-enable)"
);
}
// We resolved our output and drove the config over Wayland; don't ALSO run kscreen-doctor — that
// would double-apply (and on a wedged box it would just add 26 s of timeouts). `handled` is true
// even on an unconfirmed apply; a genuinely absent management global / unresolved output took the
// `handled = false` early returns above and falls back to kscreen-doctor.
TopologyOutcome {
our_uuid,
disabled,
handled: true,
}
}
/// Install + select a `want_w`×`want_h`@`want_hz` custom mode on the just-created virtual output
/// (name starts with `our_prefix`, currently at its sacrificial birth size `birth_w`×`birth_h`) —
/// entirely over `kde_output_management_v2`, the in-process replacement for the `kscreen-doctor`
/// `addCustomMode` + `mode` shell-out (`set_custom_refresh`).
///
/// `set_custom_modes` hands KWin a one-entry mode list; KWin generates the CVT timing (so the width
/// may align DOWN — see [`CVT_H_GRANULARITY`]) and adds the mode. We then SELECT it, which changes
/// the output's size and triggers the screencast stream's renegotiation to the real refresh (the
/// sacrificial-birth mechanism in `kwin::create`). Returns the ACTIVE mode read back after selection
/// (Hz rounded), or `None` if management is absent, the output/generated mode never appeared, or an
/// apply didn't confirm — the caller then falls back to `kscreen-doctor`. `set_custom_modes` REPLACES
/// the custom list (idempotent across reconnects — no per-connect list growth), and it is `since 18`,
/// so pre-6.6 KWin without it simply takes the `None` → kscreen-doctor path.
pub(crate) fn set_custom_mode(
our_prefix: &str,
birth_w: u32,
birth_h: u32,
want_w: u32,
want_h: u32,
want_hz: u32,
) -> Option<(u32, u32, u32)> {
let mut sess = Session::open()?;
let deadline = Instant::now() + OP_BUDGET;
// `set_custom_modes` is `since 18`; calling it on an older bound management object is a protocol
// error, so bail to the kscreen-doctor fallback there (pre-6.6 KWin). Our bound version is
// `min(advertised, MGMT_MAX)`.
if sess.state.mgmt_name_version.map(|(_, v)| v).unwrap_or(0) < 18 {
return None;
}
// Resolve our output at its birth size (newest global wins a supersede).
let our_proxy = sess
.state
.devices
.values()
.filter(|d| {
d.name.as_deref().is_some_and(|n| n.starts_with(our_prefix))
&& sess.current_dims(d).map(|(w, h, _)| (w, h)) == Some((birth_w, birth_h))
})
.max_by_key(|d| d.global)
.and_then(|d| d.proxy.clone())?;
let our_key = our_proxy.id();
let want_mhz = want_hz.saturating_mul(1000);
// A generated mode IS our custom one iff: exact height, width at/just-below the request (a CVT
// alignment), and refresh within 1 Hz — which excludes the sacrificial 60 Hz birth mode.
let mode_matches = move |st: &State, mid: &ObjectId| -> bool {
st.mode_dims.get(mid).is_some_and(|&(w, h, mhz)| {
h == want_h
&& w <= want_w
&& want_w - w < CVT_H_GRANULARITY
&& (mhz as i64 - want_mhz as i64).abs() <= 1000
})
};
// Build a one-entry custom-mode list (full blanking, like kscreen-doctor's `.full`) and install it.
let mode_list = sess.new_mode_list();
mode_list.set_resolution(want_w, want_h);
mode_list.set_refresh_rate(want_mhz);
mode_list.set_reduced_blanking(0);
mode_list.add_mode();
let config = sess.new_config();
config.set_custom_modes(&our_proxy, &mode_list);
let installed = sess.apply(&config, deadline);
config.destroy();
mode_list.destroy();
if !installed {
return None;
}
// Wait for KWin to generate the mode and advertise it on the output.
let found = |st: &State| -> bool {
st.devices
.get(&our_key)
.is_some_and(|d| d.modes.iter().any(|(mid, _)| mode_matches(st, mid)))
};
if !sess.pump_until(deadline, found) {
tracing::warn!(
want_w,
want_h,
want_hz,
"KWin output management: generated custom mode never appeared — kscreen-doctor fallback"
);
return None;
}
// Grab the generated mode's proxy, then select it (this is what changes the size).
let mode_proxy = {
let dev = sess.state.devices.get(&our_key)?;
dev.modes
.iter()
.find(|(mid, _)| mode_matches(&sess.state, mid))
.map(|(_, p)| p.clone())?
};
let config = sess.new_config();
config.mode(&our_proxy, &mode_proxy);
let selected = sess.apply(&config, deadline);
config.destroy();
if !selected {
return None;
}
// Read back the active mode after selection — the size that really landed paces the encoder.
let want_dims = sess
.state
.mode_dims
.get(&mode_proxy.id())
.map(|&(w, h, _)| (w, h));
let landed = |st: &State| -> bool {
st.devices
.get(&our_key)
.and_then(|d| d.current_mode.as_ref())
.and_then(|mid| st.mode_dims.get(mid))
.map(|&(w, h, _)| (w, h))
== want_dims
};
sess.pump_until(deadline, landed);
let dev = sess.state.devices.get(&our_key)?;
let (cw, ch, cmhz) = sess.current_dims(dev)?;
let hz = ((cmhz as f64) / 1000.0).round() as u32;
tracing::info!(
want_w,
want_h,
want_hz,
active_w = cw,
active_h = ch,
active_hz = hz,
"KWin output management: custom mode installed + selected (in-process)"
);
Some((cw, ch, hz.max(1)))
}
/// Re-enable outputs by name at their captured `WxH@Hz` modes (teardown), in-process. Returns
/// `true` if the config applied; `false` (compositor unresponsive / management absent) tells the
/// caller to fall back to `kscreen-doctor`.
pub(crate) fn reenable_outputs(outputs: &[(String, String)]) -> bool {
if outputs.is_empty() {
return true;
}
let Some(mut sess) = Session::open() else {
return false;
};
let deadline = Instant::now() + OP_BUDGET;
let config = sess.new_config();
for (name, spec) in outputs {
// Find the device by name (physical names are stable across a session).
let Some(dev) = sess
.state
.devices
.values()
.find(|d| d.name.as_deref() == Some(name.as_str()))
.cloned()
else {
continue;
};
let Some(proxy) = dev.proxy.as_ref() else {
continue;
};
// Enable first — a bare enable always succeeds, so a physical is never left dark.
config.enable(proxy, 1);
// Then re-assert the captured mode so a 120 Hz panel doesn't return at KWin's ~60 Hz default.
if let Some(mode) = find_mode(&sess, &dev, spec) {
config.mode(proxy, &mode);
}
}
let ok = sess.apply(&config, deadline);
config.destroy();
if ok {
tracing::info!(reenabled = ?outputs, "KWin output management: restored outputs (in-process)");
}
ok
}
/// Position the output identified by `uuid` at `(x, y)` in the desktop layout, in-process. Returns
/// `true` if applied; `false` tells the caller to fall back to `kscreen-doctor`.
pub(crate) fn set_position(uuid: &str, x: i32, y: i32) -> bool {
let Some(mut sess) = Session::open() else {
return false;
};
let deadline = Instant::now() + OP_BUDGET;
let Some(dev) = sess
.state
.devices
.values()
.find(|d| d.uuid.as_deref() == Some(uuid))
.cloned()
else {
return false;
};
let Some(proxy) = dev.proxy.as_ref() else {
return false;
};
let config = sess.new_config();
config.position(proxy, x, y);
let ok = sess.apply(&config, deadline);
config.destroy();
if ok {
tracing::info!(
uuid,
x,
y,
"KWin output management: placed output (in-process)"
);
}
ok
}
/// Find a device's advertised mode proxy matching a captured `"WxH@Hz"` spec (Hz rounded), for
/// restore. `None` if the spec is empty or no mode matches (the caller then enables without a mode
/// and lets KWin pick its preferred one).
fn find_mode(sess: &Session, dev: &DeviceState, spec: &str) -> Option<DeviceMode> {
if spec.is_empty() {
return None;
}
let (wh, hz) = spec.split_once('@')?;
let (w, h) = wh.split_once('x')?;
let (w, h, hz): (u32, u32, u32) = (w.parse().ok()?, h.parse().ok()?, hz.parse().ok()?);
dev.modes.iter().find_map(|(id, proxy)| {
let (mw, mh, mmhz) = sess.state.mode_dims.get(id).copied()?;
let mhz = ((mmhz as f64) / 1000.0).round() as u32;
((mw, mh, mhz) == (w, h, hz)).then(|| proxy.clone())
})
}
#[cfg(test)]
mod tests {
use super::*;
/// The `WxH@Hz` capture rounds mHz to whole Hz — the shape teardown parses back.
#[test]
fn mode_spec_rounds_millihertz() {
assert_eq!(mode_spec((2560, 1440, 59940)), "2560x1440@60");
assert_eq!(mode_spec((1920, 1080, 60000)), "1920x1080@60");
assert_eq!(mode_spec((3840, 2160, 119880)), "3840x2160@120");
}
/// The vendored device XML must keep `mode` at the opcode the `event_created_child!` macro
/// hardcodes — a reorder there would bind the child to the wrong event and desync mode sizes.
#[test]
fn mode_event_opcode_is_two() {
assert_eq!(DEVICE_MODE_EVENT_OPCODE, 2);
}
}
+114
View File
@@ -0,0 +1,114 @@
//! Time-bounded child-process helpers.
//!
//! Every compositor query this crate makes shells out to a helper (`kscreen-doctor`, `systemctl`,
//! `pw-dump`, …), and most of them are *clients of the very thing being diagnosed*: `kscreen-doctor`
//! is a Wayland client, so against a wedged KWin it blocks in its own connect and **never returns**.
//! `Command::status()` / `Command::output()` have no timeout, so one hung helper pinned the calling
//! thread forever — and on the host that thread is the session's stream thread, whose only way to
//! end a session is to return. A stuck query therefore became a permanently stuck session.
//!
//! These wrappers bound the wait: poll for exit until the budget runs out, then kill the child and
//! report [`std::io::ErrorKind::TimedOut`], so callers see a plain "the helper failed" error and
//! take their existing failure path instead of hanging.
use std::io::{Error, ErrorKind, Result};
use std::process::{Command, ExitStatus, Output};
use std::time::{Duration, Instant};
/// Poll interval while waiting for a child to exit. Short enough that a fast helper (the normal
/// case — `kscreen-doctor` answers in tens of ms) isn't measurably delayed.
const POLL: Duration = Duration::from_millis(20);
/// Run `cmd` to completion, killing it if it outlives `budget`.
///
/// Stdout/stderr are left as the caller configured them (inherited by default), so this is for
/// commands run for their exit status alone — see [`output_within`] when the output is read.
pub(crate) fn status_within(cmd: &mut Command, budget: Duration) -> Result<ExitStatus> {
let mut child = cmd.spawn()?;
let deadline = Instant::now() + budget;
loop {
match child.try_wait()? {
Some(status) => return Ok(status),
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait(); // reap it — never leave a zombie behind
return Err(timed_out(cmd, budget));
}
None => std::thread::sleep(POLL),
}
}
}
/// Run `cmd` to completion and capture its stdout/stderr, killing it if it outlives `budget`.
///
/// The output is read only after the child has exited, so a helper that fills the pipe buffer and
/// stalls is caught by the budget rather than deadlocking the reader (these helpers emit at most a
/// few hundred KiB, well under any real pipe pressure).
pub(crate) fn output_within(cmd: &mut Command, budget: Duration) -> Result<Output> {
let mut child = cmd
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()?;
let deadline = Instant::now() + budget;
loop {
match child.try_wait()? {
// Exited: `wait_with_output` now only drains already-buffered pipes.
Some(_) => return child.wait_with_output(),
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait();
return Err(timed_out(cmd, budget));
}
None => std::thread::sleep(POLL),
}
}
}
fn timed_out(cmd: &Command, budget: Duration) -> Error {
let program = cmd.get_program().to_string_lossy().to_string();
tracing::warn!(
program,
budget_ms = budget.as_millis() as u64,
"helper did not exit within its budget — killed it (a wedged compositor/session bus is the \
usual cause); treating it as a failed query"
);
Error::new(
ErrorKind::TimedOut,
format!("`{program}` did not exit within {budget:?}"),
)
}
#[cfg(test)]
mod tests {
use super::*;
/// A helper that never exits must be killed at the budget and reported as `TimedOut` — the
/// whole point of the module (an unbounded `status()` here is what wedged a whole session).
#[test]
fn a_hung_child_is_killed_at_the_budget() {
let started = Instant::now();
let err = status_within(Command::new("sleep").arg("30"), Duration::from_millis(150))
.expect_err("must time out");
assert_eq!(err.kind(), ErrorKind::TimedOut);
assert!(
started.elapsed() < Duration::from_secs(5),
"must return at its budget, not the child's lifetime (took {:?})",
started.elapsed()
);
}
/// The normal path is unaffected: a quick command still yields its status and its output.
#[test]
fn a_quick_child_returns_normally() {
let st = status_within(&mut Command::new("true"), Duration::from_secs(5)).expect("ran");
assert!(st.success());
let out = output_within(
Command::new("echo").arg("punktfunk"),
Duration::from_secs(5),
)
.expect("ran");
assert!(out.status.success());
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "punktfunk");
}
}
+40 -19
View File
@@ -6,6 +6,15 @@
use super::*;
/// Budget for one `systemctl --user` / `dbus-update-activation-environment` call.
///
/// These talk to the session bus, and a bus that is itself restarting or wedged answers nothing —
/// unbounded, that pinned the caller (on the host, the session's stream thread) forever. A restart
/// of the portal units is the slowest legitimate case, hence the generous window; missing it just
/// means the portal env settles late, which the callers already treat as best-effort.
#[cfg(target_os = "linux")]
const SYSTEMD_BUDGET: std::time::Duration = std::time::Duration::from_secs(10);
/// The **session epoch** — bumped whenever session detection observes a different compositor
/// *instance*: an [`ActiveKind`] change, **or** a new compositor PID for the same kind (the
/// Desktop→Game→Desktop bounce that brings up a fresh KWin/gamescope with an unrelated node-id space).
@@ -86,9 +95,15 @@ pub fn observe_session_instance(active: &ActiveSession) {
/// via the next [`settle_desktop_portal`], so scrubbing on a bounce is harmless.)
#[cfg(target_os = "linux")]
fn scrub_desktop_manager_env() {
let _ = std::process::Command::new("systemctl")
.args(["--user", "unset-environment", "WAYLAND_DISPLAY", "DISPLAY"])
.status();
let _ = crate::proc::status_within(
std::process::Command::new("systemctl").args([
"--user",
"unset-environment",
"WAYLAND_DISPLAY",
"DISPLAY",
]),
SYSTEMD_BUDGET,
);
}
#[cfg(not(target_os = "linux"))]
@@ -499,40 +514,46 @@ pub fn settle_desktop_portal(chosen: Compositor) {
];
// Push our (correct) env into the systemd --user manager + the D-Bus activation environment so a
// re-activated portal/backend inherits the live session.
let _ = std::process::Command::new("systemctl")
.args(["--user", "import-environment"])
.args(VARS)
.status();
let _ = std::process::Command::new("dbus-update-activation-environment")
.arg("--systemd")
.args(VARS)
.status();
let _ = crate::proc::status_within(
std::process::Command::new("systemctl")
.args(["--user", "import-environment"])
.args(VARS),
SYSTEMD_BUDGET,
);
let _ = crate::proc::status_within(
std::process::Command::new("dbus-update-activation-environment")
.arg("--systemd")
.args(VARS),
SYSTEMD_BUDGET,
);
// KWin input goes through the xdg RemoteDesktop portal; the frontend routes RemoteDesktop to a
// backend by its OWN startup XDG_CURRENT_DESKTOP, so restart it (+ the KDE backend) to re-read
// the now-live session, then let it settle before the injector reopens against it.
if chosen == Compositor::Kwin {
let _ = std::process::Command::new("systemctl")
.args([
let _ = crate::proc::status_within(
std::process::Command::new("systemctl").args([
"--user",
"try-restart",
"xdg-desktop-portal-kde.service",
"xdg-desktop-portal.service",
])
.status();
]),
SYSTEMD_BUDGET,
);
std::thread::sleep(std::time::Duration::from_millis(600));
}
// Hyprland capture rides the xdg ScreenCast portal serviced by xdph (G5): on a mid-stream switch
// xdph may still hold the old session's Wayland/instance env, so restart it (+ the frontend) to
// re-read the now-live session, mirroring the KWin settling above.
if chosen == Compositor::Hyprland {
let _ = std::process::Command::new("systemctl")
.args([
let _ = crate::proc::status_within(
std::process::Command::new("systemctl").args([
"--user",
"try-restart",
"xdg-desktop-portal-hyprland.service",
"xdg-desktop-portal.service",
])
.status();
]),
SYSTEMD_BUDGET,
);
std::thread::sleep(std::time::Duration::from_millis(600));
}
tracing::info!(
@@ -300,6 +300,21 @@ pub fn control_device_handle() -> Option<HANDLE> {
VDM.get().and_then(VirtualDisplayManager::device_handle)
}
/// Re-commit the CURRENT display config under the manager `state` lock (the sole-topology-mutator
/// contract of [`force_mode_reenumeration`]). The secure-desktop guard's actuator: the OS only
/// reverts a path to its software-cursor default ON a mode commit, so standing the hardware-cursor
/// declare down (`IOCTL_SET_CURSOR_FORWARD` off) needs this nudge for UAC/Winlogon to actually
/// render. `false` (no-op) before the first backend open — no monitors exist to re-commit for.
pub fn force_recommit() -> bool {
let Some(m) = VDM.get() else {
return false;
};
let _guard = m.state.lock().unwrap();
// SAFETY: `force_mode_reenumeration`'s contract is "call under the manager `state` lock";
// held above. The call reads + re-applies the current CCD config over owned locals.
unsafe { pf_win_display::win_display::force_mode_reenumeration() }
}
/// Best-effort "is this WUDFHost pid still alive?" — the monitor-liveness probe for the JOIN path.
/// `OpenProcess` failing (pid reaped) or the process being signaled ⇒ dead. Pid reuse could
/// theoretically alias a fresh process and read "alive"; the joining session then just retries into
+3
View File
@@ -31,5 +31,8 @@ windows = { version = "0.62", features = [
"Win32_System_LibraryLoader",
# console_session_mismatch: WTSGetActiveConsoleSessionId + ProcessIdToSessionId + GetCurrentProcessId.
"Win32_System_RemoteDesktop",
# input_desktop: OpenInputDesktop/SetThreadDesktop/GetUserObjectInformationW — display writes
# follow the input desktop so a UAC/lock screen can't refuse them.
"Win32_System_StationsAndDesktops",
"Win32_System_Threading",
] }
+200
View File
@@ -0,0 +1,200 @@
//! Make display-config writes follow the INPUT desktop.
//!
//! Windows refuses `ChangeDisplaySettingsEx` / `SetDisplayConfig` issued from a thread that is not
//! on the desktop currently receiving input. While a UAC consent prompt (or the lock / logon
//! screen) is up, the input desktop is `Winlogon`; a host thread sitting on `WinSta0\Default` — the
//! desktop the service explicitly launches it onto — then gets `DISP_CHANGE_FAILED` /
//! `ERROR_ACCESS_DENIED` for EVERY write. A session starting in that window can never set its
//! virtual display's mode, so the capturer sizes its ring to a stale mode, every composed frame is
//! dropped for a size mismatch, and the client sits on a black screen until bring-up runs out of
//! retries. Field-reported 2026-07-23: a consent prompt left up on an unattended host made the box
//! unreachable until someone walked over to it.
//!
//! Measured on-glass that same day (RTX 4090 box, SYSTEM host in console session 2, a real
//! interactive consent prompt up, virtual display `\\.\DISPLAY42` active):
//!
//! ```text
//! INPUT desktop = Winlogon
//! UNBOUND CDS_TEST -> -1 (DISP_CHANGE_FAILED) <- the field failure, reproduced
//! UNBOUND SDC_VALIDATE -> 0x5 (ERROR_ACCESS_DENIED) <- the field failure, reproduced
//! bound thread desktop -> Winlogon
//! BOUND CDS_TEST -> 0 (DISP_CHANGE_SUCCESSFUL)
//! BOUND SDC_VALIDATE -> 0x0 (ERROR_SUCCESS)
//! ```
//!
//! The retry model mirrors `pf-inject`'s `sendinput.rs`: do NOT pay
//! `OpenInputDesktop`/`SetThreadDesktop` on every write — issue the write, and only when it fails
//! the way a wrong-desktop write fails do we rebind and retry once. That keeps the normal path
//! byte-for-byte as it was (a working write is never touched) and makes this strictly additive.
//!
//! Unlike sendinput's injector — a dedicated thread that KEEPS its binding — these helpers run on
//! shared/task threads, so the binding here is SCOPED: [`InputDesktopBinding`] restores the
//! thread's original desktop on drop. A thread left bound to a `Winlogon` desktop that is later
//! destroyed (prompt dismissed) would fail every subsequent display write for the life of the
//! process — the exact wedge this module exists to remove, so it must not be introduced here.
use windows::Win32::Foundation::HANDLE;
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetThreadDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, HDESK, UOI_NAME,
};
use windows::Win32::System::Threading::GetCurrentThreadId;
/// `GENERIC_ALL` for the desktop open. The `windows` crate models desktop access as its own flag
/// type and doesn't export the generic rights, so spell it out (same constant `sendinput.rs` and
/// the cursor poller use).
const GENERIC_ALL: u32 = 0x1000_0000;
/// This thread's binding to the input desktop, restored on drop.
///
/// `GetThreadDesktop` returns a BORROWED handle (documented: it creates no new handle and must not
/// be closed), so only the handle `OpenInputDesktop` produced is closed here — and only after the
/// thread has been moved back off it.
pub(crate) struct InputDesktopBinding {
previous: HDESK,
input: HDESK,
/// `UOI_NAME` of the desktop bound to, for the "this write only landed because we rebound"
/// log. Read once at bind time (the failure path only), never in steady state.
name: String,
}
impl InputDesktopBinding {
/// Bind the calling thread to the desktop currently receiving input. `None` when there is no
/// reachable input desktop (not privileged for `Winlogon` — a host that is not SYSTEM) or the
/// rebind is refused, in which case the caller keeps whatever result it already had rather than
/// changing behaviour.
pub(crate) fn bind() -> Option<Self> {
// SAFETY: all four are FFI calls taking by-value args only. `GetThreadDesktop` yields a
// borrowed handle for THIS thread (never closed here). `OpenInputDesktop` yields an owned
// `HDESK` only on `Ok`; it is either installed by `SetThreadDesktop` (and then owned by the
// returned guard, which closes it exactly once in `Drop`) or closed right here on failure —
// so it is closed exactly once on every path and never used after close. `SetThreadDesktop`
// rebinds only the calling thread, which owns no windows or hooks (these are display-config
// worker threads), so it cannot fail on that account.
unsafe {
let previous = GetThreadDesktop(GetCurrentThreadId()).ok()?;
let input = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
)
.ok()?;
let name = desktop_name(input).unwrap_or_else(|| "<unnamed>".into());
if SetThreadDesktop(input).is_err() {
let _ = CloseDesktop(input);
return None;
}
Some(Self {
previous,
input,
name,
})
}
}
}
impl Drop for InputDesktopBinding {
fn drop(&mut self) {
// SAFETY: `self.previous` is the borrowed desktop this thread started on and `self.input`
// is the handle this guard uniquely owns. The thread is moved back FIRST, so the handle is
// no longer the thread's desktop when it is closed — closed exactly once, never used after.
unsafe {
let _ = SetThreadDesktop(self.previous);
let _ = CloseDesktop(self.input);
}
}
}
/// `UOI_NAME` of the current input desktop — `Some("Winlogon")` while a UAC consent prompt, the
/// lock screen or the logon screen owns input, `Some("Default")` in normal operation, `None` when
/// it cannot be opened at all.
pub(crate) fn input_desktop_name() -> Option<String> {
// SAFETY: `OpenInputDesktop` yields an owned handle only on `Ok`, which is closed exactly once
// below and not used after.
unsafe {
let h = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
)
.ok()?;
let name = desktop_name(h);
let _ = CloseDesktop(h);
name
}
}
/// `UOI_NAME` of an already-open desktop handle. Borrows `h` — closing it stays the caller's job.
///
/// # Safety
/// `h` must be a live desktop handle for the duration of the call.
unsafe fn desktop_name(h: HDESK) -> Option<String> {
let mut name = [0u16; 64]; // "Default"/"Winlogon"/"Screen-saver" all fit with room to spare
let mut needed = 0u32;
// SAFETY: `h` is live per this fn's contract; `name`/`needed` are live out-params and the call
// writes at most `nlength` bytes, exactly the size passed.
GetUserObjectInformationW(
HANDLE(h.0),
UOI_NAME,
Some(name.as_mut_ptr().cast()),
(name.len() * 2) as u32,
Some(&mut needed),
)
.ok()?;
let len = name.iter().position(|&c| c == 0).unwrap_or(name.len());
Some(String::from_utf16_lossy(&name[..len]))
}
/// `true` when the input desktop is a SECURE one (`UOI_NAME` != `Default`: `Winlogon` for UAC
/// consent / lock / logon, or a screen-saver desktop) — i.e. when display writes from the host's
/// own desktop are being refused for that reason. `false` when it is normal OR unreadable: this
/// only ever phrases a diagnostic, so an unknown desktop must not claim the secure one is up.
pub(crate) fn input_desktop_is_secure() -> bool {
input_desktop_name().is_some_and(|n| !n.eq_ignore_ascii_case("Default"))
}
/// `ERROR_ACCESS_DENIED` — exactly how Windows refuses a `SetDisplayConfig` issued from a thread
/// that is not on the input desktop (measured, see the module header). Narrower than "any error" on
/// purpose: the CCD path also returns `ERROR_INVALID_PARAMETER` (0x57) for the unrelated
/// exclusive-mode topology bug, and re-issuing THAT on another desktop would only confuse its
/// diagnosis.
const SDC_ACCESS_DENIED: i32 = 5;
/// [`retry_on_input_desktop`] specialised for the CCD writes, which all return a Win32 error code.
pub(crate) fn retry_set_display_config(write: impl FnMut() -> i32) -> i32 {
retry_on_input_desktop(|rc| *rc == SDC_ACCESS_DENIED, write)
}
/// Run a display-config write; if it comes back the way a write from the wrong desktop comes back,
/// bind this thread to the CURRENT input desktop and run it exactly once more.
///
/// `denied` decides which results are worth a retry — keep it NARROW (the specific
/// wrong-desktop verdict), so a genuinely bad mode or a driver-level refusal is not re-issued and
/// mis-attributed. The binding is dropped before returning, so the calling thread always leaves on
/// the desktop it arrived on.
pub(crate) fn retry_on_input_desktop<T>(
denied: impl Fn(&T) -> bool,
mut write: impl FnMut() -> T,
) -> T {
let first = write();
if !denied(&first) {
return first;
}
// Only worth the rebind when the input desktop is genuinely elsewhere; on a normal desktop the
// refusal means something else entirely and a retry would just repeat it.
let Some(binding) = InputDesktopBinding::bind() else {
return first;
};
let second = write();
if !denied(&second) {
// Say so. A silent save is indistinguishable in a log from a write that never needed
// saving, which makes "did the fix fire?" unanswerable in the field — and made the first
// on-glass verification of this very change inconclusive.
tracing::info!(
desktop = %binding.name,
"display write was refused off the input desktop — retried bound to it and it applied \
(a UAC prompt / lock screen owns input; the session continues normally)"
);
}
second
}
+3
View File
@@ -11,6 +11,9 @@
#[cfg(target_os = "windows")]
pub mod display_events;
/// Bind display-config writes to the input desktop so a UAC / lock screen can't refuse them.
#[cfg(target_os = "windows")]
mod input_desktop;
#[cfg(target_os = "windows")]
pub mod monitor_devnode;
#[cfg(target_os = "windows")]
+287 -69
View File
@@ -44,8 +44,8 @@ use windows::Win32::Devices::Display::{
use windows::Win32::Foundation::POINTL;
use windows::Win32::Graphics::Gdi::{
ChangeDisplaySettingsExW, EnumDisplaySettingsW, CDS_TEST, CDS_UPDATEREGISTRY, DEVMODEW,
DISP_CHANGE_SUCCESSFUL, DM_BITSPERPEL, DM_DISPLAYFREQUENCY, DM_PELSHEIGHT, DM_PELSWIDTH,
ENUM_CURRENT_SETTINGS, ENUM_DISPLAY_SETTINGS_MODE,
DISP_CHANGE_FAILED, DISP_CHANGE_SUCCESSFUL, DM_BITSPERPEL, DM_DISPLAYFREQUENCY, DM_PELSHEIGHT,
DM_PELSWIDTH, ENUM_CURRENT_SETTINGS, ENUM_DISPLAY_SETTINGS_MODE,
};
use punktfunk_core::Mode;
@@ -62,7 +62,9 @@ use punktfunk_core::Mode;
pub unsafe fn force_extend_topology() {
// A topology flag with no supplied path/mode arrays tells the OS to recompute + apply that preset
// for the currently-connected displays (the same code path DisplaySwitch.exe drives).
let rc = SetDisplayConfig(None, None, SDC_APPLY | SDC_TOPOLOGY_EXTEND);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(None, None, SDC_APPLY | SDC_TOPOLOGY_EXTEND)
});
if rc == 0 {
tracing::info!(
"display topology forced to EXTEND (a new IddCx monitor would otherwise be CLONED onto the \
@@ -152,11 +154,13 @@ pub unsafe fn activate_target_path(target_id: u32) -> bool {
// SAVE_TO_DATABASE so Windows remembers the arrangement — the next same-identity ADD (the driver
// reuses the slot's EDID serial/ConnectorIndex) then auto-activates from the persistence DB and
// skips this whole fallback ladder.
let rc = SetDisplayConfig(
Some(supplied.as_slice()),
Some(modes.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES | SDC_SAVE_TO_DATABASE,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(supplied.as_slice()),
Some(modes.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES | SDC_SAVE_TO_DATABASE,
)
});
if rc == 0 {
tracing::info!(
target_id,
@@ -273,14 +277,16 @@ pub unsafe fn force_mode_reenumeration() -> bool {
let Some((paths, modes)) = query_active_config() else {
return false;
};
let rc = SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
)
});
if rc != 0 {
tracing::debug!("force mode re-enumeration: SetDisplayConfig rc={rc:#x}");
}
@@ -625,9 +631,15 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
// SAFETY: `wname` is a live NUL-terminated UTF-16 device name and `&dm` is a live DEVMODEW describing
// the requested mode; both outlive the call. CDS_TEST only validates the mode (no apply), the two
// trailing args are null, and the API only reads its inputs.
let test = unsafe {
ChangeDisplaySettingsExW(PCWSTR(wname.as_ptr()), Some(&dm), None, CDS_TEST, None)
};
// A CDS write from a thread that is not on the input desktop is refused with DISP_CHANGE_FAILED
// (UAC consent / lock screen up) — retry it bound to that desktop rather than declaring the mode
// unsupported, which is what stranded sessions on a black screen for a whole bring-up.
let test = crate::input_desktop::retry_on_input_desktop(
|rc| *rc == DISP_CHANGE_FAILED,
|| unsafe {
ChangeDisplaySettingsExW(PCWSTR(wname.as_ptr()), Some(&dm), None, CDS_TEST, None)
},
);
if test != DISP_CHANGE_SUCCESSFUL {
tracing::warn!(
result = test.0,
@@ -642,15 +654,20 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
// SAFETY: same inputs as the CDS_TEST call above — `wname` (live NUL-terminated device name) and
// `&dm` (live DEVMODEW) both outlive the call; CDS_UPDATEREGISTRY applies the already-validated mode,
// and the API only reads its inputs.
let apply = unsafe {
ChangeDisplaySettingsExW(
PCWSTR(wname.as_ptr()),
Some(&dm),
None,
CDS_UPDATEREGISTRY,
None,
)
};
// Same wrong-desktop retry as the validate above: the two calls bind independently, so an apply
// still lands even when the secure desktop came up between them.
let apply = crate::input_desktop::retry_on_input_desktop(
|rc| *rc == DISP_CHANGE_FAILED,
|| unsafe {
ChangeDisplaySettingsExW(
PCWSTR(wname.as_ptr()),
Some(&dm),
None,
CDS_UPDATEREGISTRY,
None,
)
},
);
if apply == DISP_CHANGE_SUCCESSFUL {
tracing::info!(
"{gdi_name}: active mode set to {}x{}@{}",
@@ -672,12 +689,20 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
/// Human decode for a failed `ChangeDisplaySettingsExW` result. The two codes worth telling apart
/// in a field log: `BADMODE` (the display's mode list genuinely lacks the mode) vs `FAILED` (the
/// write itself was rejected — on a healthy driver that is the signature of a host process without
/// console-session access, e.g. one trapped in a disconnected RDP session). An earlier revision
/// printed "mode not advertised?" for BOTH, which sent a lid-closed field report chasing the wrong
/// cause.
/// write itself was rejected). An earlier revision printed "mode not advertised?" for BOTH, which
/// sent a lid-closed field report chasing the wrong cause.
///
/// `FAILED` itself has two causes needing opposite fixes — no console-session access (disconnected
/// RDP) versus the right session but the wrong desktop (UAC / lock / logon screen owns input) — so
/// it asks which before naming one. See [`sdc_access_denied_hint`] for the same split on the CCD
/// side.
fn disp_change_reason(rc: i32) -> &'static str {
match rc {
-1 if crate::input_desktop::input_desktop_is_secure() => {
"DISP_CHANGE_FAILED: the SECURE desktop owns input — a UAC consent prompt, the lock \
screen or the logon screen is up, and display writes are refused off it. Dismiss the \
prompt on the host"
}
-1 => {
"DISP_CHANGE_FAILED: the display write was rejected — a host without console-session \
access (disconnected RDP session / non-console session) fails ALL display writes \
@@ -691,15 +716,28 @@ fn disp_change_reason(rc: i32) -> &'static str {
}
}
/// Appended to `SetDisplayConfig` failure logs when rc is `ERROR_ACCESS_DENIED` (0x5) — per MS
/// docs "the caller does not have access to the console session", the field signature of a host
/// running in a disconnected RDP / non-console session. Every other rc gets no hint.
/// Appended to `SetDisplayConfig` failure logs when rc is `ERROR_ACCESS_DENIED` (0x5). Every other
/// rc gets no hint.
///
/// `ERROR_ACCESS_DENIED` has TWO field causes and they need opposite fixes, so ask which one before
/// naming it. MS docs say only "the caller does not have access to the console session", which is
/// the disconnected-RDP / non-console case — but the SAME rc comes back when the host IS in the
/// console session and merely off the input desktop (UAC consent, lock or logon screen up). Naming
/// only the first sent a 2026-07-23 investigation chasing a phantom RDP session while a consent
/// prompt was the actual cause, on a host the message told to "run via the installed service" —
/// which it already was. [`crate::input_desktop`] now retries these bound to the input desktop, so
/// seeing this at all means even that did not help.
fn sdc_access_denied_hint(rc: i32) -> &'static str {
if rc == 5 {
if rc != 5 {
return "";
}
if crate::input_desktop::input_desktop_is_secure() {
" (ERROR_ACCESS_DENIED: the SECURE desktop owns input — a UAC consent prompt, the lock \
screen or the logon screen is up, and display writes are refused off it. Dismiss the \
prompt on the host)"
} else {
" (ERROR_ACCESS_DENIED: the host has no console-session access — disconnected RDP \
session? run via the installed service so it tracks the console session)"
} else {
""
}
}
@@ -901,7 +939,7 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
// live topology each attempt and re-apply until ONLY the keep set is active. Secure-desktop
// correctness depends on this — the lock screen must not land on a stray panel while we stream.
for attempt in 1..=4u32 {
let (mut paths, modes) = query_active_config()?;
let (mut paths, mut modes) = query_active_config()?;
let mut others = 0u32;
for p in paths.iter_mut() {
if keep_target_ids.contains(&p.targetInfo.id) {
@@ -923,19 +961,53 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
others += 1;
}
}
// The doomed display may have HELD the desktop origin (the physical is primary in the
// single-slot exclusive case): re-anchor the kept sources so the supplied config still
// contains a primary — an origin-less desktop is rejected 0x57 no matter which array
// shape carries it (see `anchor_kept_sources_at_origin`).
if others > 0 {
anchor_kept_sources_at_origin(&paths, &mut modes);
}
// Commit the config. Even when nothing needed deactivating we re-commit: a legacy mode-set does
// NOT drive the IddCx adapter's EVT_IDD_CX_ADAPTER_COMMIT_MODES, and without COMMIT_MODES the OS
// never calls ASSIGN_SWAPCHAIN, so the driver receives no frames. SDC_FORCE_MODE_ENUMERATION
// forces the re-commit; SAVE_TO_DATABASE only in the sole-path case (matches prior behavior —
// don't permanently rewrite the user's multi-display layout; the teardown restore handles it).
let mut flags = SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION;
if others == 0 {
flags |= SDC_SAVE_TO_DATABASE;
}
let rc = SetDisplayConfig(Some(paths.as_slice()), Some(modes.as_slice()), flags);
// The supplied shape is decided (and its escalation announced) ONCE, outside the write, so a
// wrong-desktop retry re-issues the identical config instead of logging the escalation twice.
let keep_only = (others > 0 && attempt >= 2).then(|| {
// ESCALATION (attempt 2+): supply ONLY the keep paths. Kept as belt-and-braces —
// the field 0x57 this was built for turned out to be the missing desktop origin
// (see `anchor_kept_sources_at_origin`), which rejected BOTH shapes identically;
// but should some other validator still choke on the full array, the minimal
// shape is the best last word. The final attempt also drops
// SDC_FORCE_MODE_ENUMERATION in case the driver rejects it combined with a real
// topology change — an actual path removal drives COMMIT_MODES on its own, so the
// re-commit rationale doesn't need the flag here.
let (kp, km) = keep_only_supplied(&paths, &modes);
let mut esc = SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES;
if attempt < 4 {
esc |= SDC_FORCE_MODE_ENUMERATION;
}
tracing::info!(
"display isolate (CCD): escalating to a keep-only supplied config (attempt {attempt}/4, paths {}→{}, modes {}→{})",
paths.len(), kp.len(), modes.len(), km.len()
);
(kp, km, esc)
});
let rc = crate::input_desktop::retry_set_display_config(|| match &keep_only {
Some((kp, km, esc)) => SetDisplayConfig(Some(kp.as_slice()), Some(km.as_slice()), *esc),
None => {
let mut flags = SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION;
if others == 0 {
flags |= SDC_SAVE_TO_DATABASE;
}
SetDisplayConfig(Some(paths.as_slice()), Some(modes.as_slice()), flags)
}
});
// A failed apply must be VISIBLE even when the verification below passes vacuously (nothing
// else was active to deactivate — the lid-closed laptop case, where the success INFO used to
// swallow rc=0x5): the re-commit above is load-bearing (COMMIT_MODES → ASSIGN_SWAPCHAIN),
@@ -958,10 +1030,132 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
tracing::warn!("display isolate (CCD): {survivors} display(s) STILL active after attempt {attempt}/4 (deactivated {others}, rc={rc:#x}) — re-querying + retrying");
std::thread::sleep(std::time::Duration::from_millis(250));
}
tracing::error!("display isolate (CCD): failed to isolate target set {keep_target_ids:?} after 4 attempts — a non-virtual display stayed active (field-reported exclusive-mode bug)");
// Name the survivors instead of assuming their kind — the field logs showed this path fire
// with a sibling VIRTUAL display as the survivor (linger-expiry shrink), where the old
// "a non-virtual display stayed active" wording sent the triage the wrong way.
let survivors: Vec<String> = target_inventory()
.iter()
.filter(|t| t.active && !keep_target_ids.contains(&t.target_id))
.map(|t| format!("{} {} \"{}\"", t.target_id, t.tech, t.friendly))
.collect();
tracing::error!(
"display isolate (CCD): failed to isolate target set {keep_target_ids:?} after 4 attempts — still active: [{}] (field-reported exclusive-mode bug)",
survivors.join(", ")
);
Some(saved)
}
/// Build the ESCALATED supplied config for [`isolate_displays_ccd`]: ONLY the paths still flagged
/// ACTIVE (the keep set — the caller already cleared ACTIVE on every doomed path), with the mode
/// table rebuilt to just the entries those paths reference (indexes remapped). Docs-wise the
/// dropped inactive entries were declared ignored anyway ("Only the paths within this array that
/// have the DISPLAYCONFIG_PATH_ACTIVE flag set are set"), so this shape asks for the identical
/// topology — minus the array contents some driver/OS validation combos reject with 0x57.
unsafe fn keep_only_supplied(
paths: &[DISPLAYCONFIG_PATH_INFO],
modes: &[DISPLAYCONFIG_MODE_INFO],
) -> (Vec<DISPLAYCONFIG_PATH_INFO>, Vec<DISPLAYCONFIG_MODE_INFO>) {
let mut out_paths = Vec::new();
let mut out_modes = Vec::new();
// old mode index → new. Shared entries dedup through here: a clone-style pair references ONE
// source mode, and the docs require each source/target mode to appear in the table only once.
let mut remap = std::collections::HashMap::new();
for p in paths {
if p.flags & DISPLAYCONFIG_PATH_ACTIVE == 0 {
continue;
}
let mut q = *p;
q.sourceInfo.Anonymous.modeInfoIdx = remap_mode_idx(
q.sourceInfo.Anonymous.modeInfoIdx,
modes,
&mut out_modes,
&mut remap,
);
q.targetInfo.Anonymous.modeInfoIdx = remap_mode_idx(
q.targetInfo.Anonymous.modeInfoIdx,
modes,
&mut out_modes,
&mut remap,
);
out_paths.push(q);
}
(out_paths, out_modes)
}
/// Move `modes[old]` into `out` (once — `remap` dedups) and return its new index. INVALID and
/// out-of-range indexes stay INVALID — `SDC_ALLOW_CHANGES` lets best-mode logic fill the gap.
fn remap_mode_idx(
old: u32,
modes: &[DISPLAYCONFIG_MODE_INFO],
out: &mut Vec<DISPLAYCONFIG_MODE_INFO>,
remap: &mut std::collections::HashMap<u32, u32>,
) -> u32 {
if old == DISPLAYCONFIG_PATH_MODE_IDX_INVALID {
return old;
}
let Some(m) = modes.get(old as usize) else {
return DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
};
*remap.entry(old).or_insert_with(|| {
out.push(*m);
(out.len() - 1) as u32
})
}
/// A committable desktop must still contain a PRIMARY — a source pinned exactly at the origin
/// `(0,0)`. Deactivating the display that held the origin (the physical, in the exclusive
/// topology) while the kept virtual stays pinned at its EXTEND offset supplies an origin-less
/// desktop, and Windows rejects that wholesale with 0x57 ERROR_INVALID_PARAMETER no matter the
/// array shape — the field box failed identically with the doomed path carried inactive AND with
/// the keep-only escalation, yet the very same call converged rc=0 whenever a kept member already
/// sat at `(0,0)`; the origin was the real variable all along. Translate the kept sources RIGIDLY
/// (relative arrangement preserved) so the top-left-most lands exactly on the origin. A set that
/// already covers `(0,0)` is left untouched, so a plain re-commit stays byte-identical and a
/// user's negative-coordinate multi-monitor layout is never rearranged.
unsafe fn anchor_kept_sources_at_origin(
paths: &[DISPLAYCONFIG_PATH_INFO],
modes: &mut [DISPLAYCONFIG_MODE_INFO],
) {
// Unique source-mode entries of the still-ACTIVE (kept) paths — clone-style pairs share one,
// and a shared entry must be translated once.
let mut idxs: Vec<usize> = Vec::new();
for p in paths {
if p.flags & DISPLAYCONFIG_PATH_ACTIVE == 0 {
continue;
}
let idx = p.sourceInfo.Anonymous.modeInfoIdx as usize;
let Some(m) = modes.get(idx) else { continue };
if m.infoType == DISPLAYCONFIG_MODE_INFO_TYPE_SOURCE && !idxs.contains(&idx) {
idxs.push(idx);
}
}
let positions: Vec<(i32, i32)> = idxs
.iter()
.map(|&i| {
let pos = modes[i].Anonymous.sourceMode.position;
(pos.x, pos.y)
})
.collect();
if positions.contains(&(0, 0)) {
return; // the kept set already holds the primary — don't touch a valid layout
}
// Lexicographic min over the actual positions — the anchor IS one of the kept sources, so
// after translation one source sits exactly at (0,0), which is what the validator wants.
let Some((ax, ay)) = positions.iter().copied().min() else {
return; // no pinned kept sources — placement is the OS's (SDC_ALLOW_CHANGES), nothing to anchor
};
for &i in &idxs {
let sm = &mut modes[i].Anonymous.sourceMode;
sm.position.x -= ax;
sm.position.y -= ay;
}
tracing::info!(
"display isolate (CCD): kept source(s) re-anchored onto the desktop origin (primary) — the doomed display held (0,0) delta=({},{})",
-ax,
-ay
);
}
/// The desktop-space rectangle `(x, y, w, h)` of `target_id`'s SOURCE — where this display's
/// region lives in the desktop coordinate space. `None` while the target isn't an active path.
/// Used by the IDD-push compose kick to dirty THE TARGET display: with parallel displays the
@@ -1056,14 +1250,16 @@ pub unsafe fn apply_source_positions(positions: &[(u32, i32, i32)]) {
if moved == 0 {
return;
}
let rc = SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
)
});
if rc == 0 {
tracing::info!(
?positions,
@@ -1148,14 +1344,16 @@ pub unsafe fn set_virtual_primary_ccd(keep_target_id: u32) -> Option<SavedConfig
}
}
let rc = SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
)
});
if rc == 0 {
tracing::info!("display primary (CCD): virtual target {keep_target_id} set PRIMARY at (0,0); {others} other display(s) kept ACTIVE + packed to its right");
} else {
@@ -1175,11 +1373,13 @@ pub unsafe fn restore_displays_ccd(saved: &SavedConfig) {
if paths.is_empty() {
return;
}
let rc = SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES,
)
});
if rc == 0 {
tracing::info!("display isolate (CCD): restored original topology");
} else {
@@ -1188,4 +1388,22 @@ pub unsafe fn restore_displays_ccd(saved: &SavedConfig) {
sdc_access_denied_hint(rc)
);
}
// GUARANTEE the desk is never left all-dark. The saved config can be unappliable (field
// rc=0x64a ERROR_BAD_CONFIGURATION: it pinned a virtual target incarnation that was since
// removed) or even apply rc=0 yet re-light nothing (snapshotted while an earlier failed
// teardown had the physicals off — the poisoned-snapshot chain from the field logs). Either
// way, if no external physical panel is active after the apply while at least one is
// connected, fall back to the OS database EXTEND preset, which re-activates every connected
// display. Internal panels deliberately don't count as lit-able here — a closed clamshell
// lid must not be forced back on.
let (connected, lit) = target_inventory()
.iter()
.filter(|t| t.external_physical)
.fold((0u32, 0u32), |(c, a), t| (c + 1, a + u32::from(t.active)));
if connected > 0 && lit == 0 {
tracing::warn!(
"display isolate (CCD): no external physical display active after the restore (rc={rc:#x}, connected={connected}) — forcing the EXTEND preset so the desk is not left dark"
);
force_extend_topology();
}
}
+4
View File
@@ -30,3 +30,7 @@ ash = "0.38"
# (SCM_RIGHTS carries the fds; pixels never cross the socket).
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# Linux-only like the code under test: the render-node scan is exercised against a fixture tree.
[target.'cfg(target_os = "linux")'.dev-dependencies]
tempfile = "3"
+169 -18
View File
@@ -34,6 +34,62 @@ const EGL_DMA_BUF_PLANE0_MODIFIER_HI_EXT: egl::Attrib = 0x3444;
mod gl;
use gl::*;
/// PCI vendor id of the only GPU this importer can drive — everything below it ends in a CUDA
/// context. Same constant the Vulkan bridge selects its physical device by.
const PCI_VENDOR_NVIDIA: u32 = 0x10de;
/// The NVIDIA DRM render node: `PUNKTFUNK_ZEROCOPY_RENDER_NODE` > the first `/dev/dri/renderD*`
/// whose sysfs PCI vendor is NVIDIA > `/dev/dri/renderD128`.
///
/// The fallback used to be the *whole* selection, which quietly assumed NVIDIA owns the first
/// render node. On a hybrid laptop it does not — the iGPU is bound first, so `renderD128` is
/// Intel, and this importer opened a Mesa GBM display, asked it for a pbuffer-capable OpenGL
/// config (Mesa's GBM platform only ever advertises window-capable ones), got none, and reported
/// "no EGL config for OpenGL" on a machine whose NVIDIA EGL stack was working perfectly. Pick the
/// device by what it *is*.
///
/// Deliberately a local scan rather than a `pf_gpu` dependency: this crate is a leaf (the import
/// worker is its own process, spawned for fault isolation), and `pf_gpu::linux_render_node` answers
/// a different question anyway — it follows the operator's VAAPI/GPU *preference*, which may well
/// name the iGPU. The answer needed here is not "which GPU should we use" but "where is CUDA".
fn nvidia_render_node() -> std::path::PathBuf {
use std::path::{Path, PathBuf};
if let Some(p) = std::env::var_os("PUNKTFUNK_ZEROCOPY_RENDER_NODE").filter(|s| !s.is_empty()) {
return PathBuf::from(p);
}
// No NVIDIA node found — sysfs may simply not be mounted (a container without /sys). Keep the
// historical guess; the CUDA context below is what actually fails if this host has no NVIDIA.
nvidia_render_node_in(Path::new("/dev/dri"), Path::new("/sys/class/drm"))
.unwrap_or_else(|| PathBuf::from("/dev/dri/renderD128"))
}
/// The scan half of [`nvidia_render_node`], parameterized on its two roots so a test can pin the
/// sysfs shape it depends on (`<sys_class_drm>/renderD129/device/vendor` holding `0x10de`). Nodes
/// are visited in name order so the choice is stable across boots.
fn nvidia_render_node_in(
dri: &std::path::Path,
sys_class_drm: &std::path::Path,
) -> Option<std::path::PathBuf> {
let mut nodes: Vec<std::ffi::OsString> = std::fs::read_dir(dri)
.map(|rd| {
rd.filter_map(|e| e.ok())
.map(|e| e.file_name())
.filter(|n| n.as_encoded_bytes().starts_with(b"renderD"))
.collect()
})
.unwrap_or_default();
nodes.sort();
nodes
.into_iter()
.find(|node| {
std::fs::read_to_string(sys_class_drm.join(node).join("device").join("vendor"))
.ok()
.and_then(|s| u32::from_str_radix(s.trim().trim_start_matches("0x"), 16).ok())
== Some(PCI_VENDOR_NVIDIA)
})
.map(|node| dri.join(node))
}
/// Per-size GL machinery to blit a dmabuf EGLImage into a CUDA-registrable `GL_RGBA8` texture.
struct GlBlit {
program: u32,
@@ -528,13 +584,15 @@ impl EglImporter {
// GBM platform on the NVIDIA render node: this ties the EGLDisplay (and its GL contexts)
// to the same DRM device CUDA-GL interop associates with, which the EGL device platform
// did not (cuGraphicsGLRegisterImage rejected device-platform GL textures).
let path = std::ffi::CString::new("/dev/dri/renderD128").unwrap();
// SAFETY: `path` is a live local `CString` (built from a string with no interior NUL, so it
let node = nvidia_render_node();
let path = std::ffi::CString::new(node.as_os_str().as_encoded_bytes())
.with_context(|| format!("render node path {} has an interior NUL", node.display()))?;
// SAFETY: `path` is a live local `CString` (its constructor rejected interior NULs, so it
// is NUL-terminated); `path.as_ptr()` is a valid pointer to that buffer which outlives this
// synchronous `open`. `open` only reads the path and returns a new fd (or -1); it neither
// retains the pointer nor writes through it, so there is no aliasing or lifetime hazard.
let render_fd = unsafe { libc::open(path.as_ptr(), libc::O_RDWR | libc::O_CLOEXEC) };
ensure!(render_fd >= 0, "open /dev/dri/renderD128 for GBM");
ensure!(render_fd >= 0, "open {} for GBM", node.display());
// SAFETY: `render_fd` is the live DRM render-node fd just returned by `open` and checked
// `>= 0`. `gbm_create_device` (libgbm, linked above) builds a `gbm_device` over that fd and
// returns a `*mut gbm_device` (or null); it borrows but does not take ownership of the fd,
@@ -546,7 +604,7 @@ impl EglImporter {
// and no `EglImporter` exists yet to close it again, so this `close` runs exactly once on
// the live `render_fd`, releasing it before the error return. No double-close.
unsafe { libc::close(render_fd) };
anyhow::bail!("gbm_create_device failed");
anyhow::bail!("gbm_create_device failed on {}", node.display());
}
// SAFETY: `Egl::load_required` dlopens the system libEGL and binds its entry points,
@@ -568,7 +626,7 @@ impl EglImporter {
&[egl::ATTRIB_NONE],
)
}
.context("eglGetPlatformDisplay(GBM) on the NVIDIA render node")?;
.with_context(|| format!("eglGetPlatformDisplay(GBM) on {}", node.display()))?;
egl.initialize(display).context("eglInitialize")?;
let exts = egl
@@ -590,20 +648,46 @@ impl EglImporter {
egl.bind_api(egl::OPENGL_API)
.context("eglBindAPI(OpenGL)")?;
// The default EGL_SURFACE_TYPE in eglChooseConfig is WINDOW_BIT, which a headless device
// display has none of — request a pbuffer-capable config (we run surfaceless anyway).
let config = egl
.choose_first_config(
display,
&[
egl::SURFACE_TYPE,
egl::PBUFFER_BIT,
egl::RENDERABLE_TYPE,
egl::OPENGL_BIT,
egl::NONE,
],
)
// display has none of — ask for a pbuffer-capable config first (NVIDIA's GBM platform has
// them, and this is the request every working host has been served for a year).
//
// Then fall back to no surface-type constraint at all, because a config's surface type is
// irrelevant to us: we never create an EGLSurface, we `eglMakeCurrent` surfaceless. Mesa's
// GBM platform advertises ONLY window-capable configs (gbm_surface is the whole point of
// the platform there), so the pbuffer request finds nothing on any Mesa device — which is
// how a hybrid host that landed on the iGPU reported "no EGL config for OpenGL" while its
// NVIDIA EGL stack was fine. The node is picked correctly now; this keeps the failure from
// being fatal (and unreadable) if some other driver makes the same choice Mesa did.
let want_pbuffer = [
egl::SURFACE_TYPE,
egl::PBUFFER_BIT,
egl::RENDERABLE_TYPE,
egl::OPENGL_BIT,
egl::NONE,
];
let any_surface = [egl::RENDERABLE_TYPE, egl::OPENGL_BIT, egl::NONE];
let config = match egl
.choose_first_config(display, &want_pbuffer)
.context("eglChooseConfig")?
.context("no EGL config for OpenGL")?;
{
Some(c) => c,
None => {
tracing::debug!(
node = %node.display(),
"no pbuffer-capable OpenGL EGL config — retrying without a surface-type \
constraint (we run surfaceless)"
);
egl.choose_first_config(display, &any_surface)
.context("eglChooseConfig (no surface-type constraint)")?
.with_context(|| {
format!(
"no EGL config for OpenGL on {} — this display serves no \
OpenGL-renderable config at all",
node.display()
)
})?
}
};
let gl_ctx = egl
.create_context(
display,
@@ -637,6 +721,7 @@ impl EglImporter {
// `eglCreateImage(EGL_LINUX_DMA_BUF_EXT)` requires as its context argument later.
let no_ctx = unsafe { egl::Context::from_ptr(egl::NO_CONTEXT) };
tracing::info!(
node = %node.display(),
"zero-copy EGL importer ready (GBM platform + GL texture interop, dma_buf_import + modifiers)"
);
Ok(EglImporter {
@@ -1087,3 +1172,69 @@ impl Drop for EglImporter {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::{Path, PathBuf};
/// A fake `/dev/dri` + `/sys/class/drm` pair: every `(node, vendor)` gets a device node file
/// and, when `vendor` is `Some`, the sysfs `device/vendor` that identifies it.
fn fixture(nodes: &[(&str, Option<&str>)]) -> (tempfile::TempDir, PathBuf, PathBuf) {
let tmp = tempfile::tempdir().unwrap();
let dri = tmp.path().join("dev/dri");
let sys = tmp.path().join("sys/class/drm");
std::fs::create_dir_all(&dri).unwrap();
for (node, vendor) in nodes {
std::fs::write(dri.join(node), b"").unwrap();
if let Some(v) = vendor {
let dev = sys.join(node).join("device");
std::fs::create_dir_all(&dev).unwrap();
std::fs::write(dev.join("vendor"), v).unwrap();
}
}
(tmp, dri, sys)
}
/// The hybrid-laptop case this selection exists for: the iGPU is bound first, so the NVIDIA
/// GPU is NOT renderD128 — which is what the old hardcoded path assumed.
#[test]
fn picks_the_nvidia_node_not_the_first_one() {
let (_t, dri, sys) = fixture(&[
("renderD128", Some("0x8086\n")),
("renderD129", Some("0x10de\n")),
]);
assert_eq!(
nvidia_render_node_in(&dri, &sys),
Some(dri.join("renderD129"))
);
}
/// No NVIDIA GPU (or no readable sysfs) ⇒ no answer, and the caller keeps the historical
/// `/dev/dri/renderD128` guess rather than inventing one.
#[test]
fn no_nvidia_node_yields_nothing() {
let (_t, dri, sys) = fixture(&[("renderD128", Some("0x8086\n")), ("renderD129", None)]);
assert_eq!(nvidia_render_node_in(&dri, &sys), None);
// Missing roots entirely (a container without /sys, or without /dev/dri).
assert_eq!(
nvidia_render_node_in(Path::new("/nonexistent/dri"), &sys),
None
);
}
/// Ignore card/control nodes, and visit render nodes in name order so a two-NVIDIA host picks
/// the same one every boot.
#[test]
fn scans_render_nodes_only_and_in_order() {
let (_t, dri, sys) = fixture(&[
("card0", Some("0x10de\n")),
("renderD130", Some("0x10de\n")),
("renderD129", Some("0x10de\n")),
]);
assert_eq!(
nvidia_render_node_in(&dri, &sys),
Some(dri.join("renderD129"))
);
}
}
+42
View File
@@ -232,6 +232,48 @@ pub fn gpu_import_disabled() -> bool {
GPU_IMPORT_DISABLED.load(Ordering::Relaxed)
}
/// The same idea for the OTHER zero-copy half: consecutive failures to import a raw dmabuf in the
/// encoder (VAAPI's libva import, PyroWave's Vulkan one) with no successful frame in between.
///
/// That import can fail for reasons no retry can fix — a driver that will not take what the
/// compositor allocates. The encode-stall recovery above it cannot tell the difference, so it
/// rebuilt the identical failing encoder five times and then ended the video session, on every
/// connection, forever: a hybrid Intel/NVIDIA laptop reporting `vaCreateSurfaces` →
/// `VA_STATUS_ERROR_ALLOCATION_FAILED` on its first frame could not stream at all until its
/// operator found `PUNKTFUNK_ZEROCOPY=0` by hand. The host already knows how to encode that
/// machine — capture just has to stop handing it dmabufs. Latching here is what makes the next
/// session negotiate CPU frames on its own.
static RAW_DMABUF_FAILURE_STREAK: AtomicU32 = AtomicU32::new(0);
static RAW_DMABUF_DISABLED: AtomicBool = AtomicBool::new(false);
/// Below the encoder's own rebuild budget, so the latch is set before the session it doomed ends.
const RAW_DMABUF_FAILURE_LATCH: u32 = 3;
/// Record an encoder-side raw-dmabuf import failure. Latches the process-wide disable after
/// [`RAW_DMABUF_FAILURE_LATCH`] consecutive failures.
pub fn note_raw_dmabuf_import_failure(reason: &str) {
let streak = RAW_DMABUF_FAILURE_STREAK.fetch_add(1, Ordering::Relaxed) + 1;
if streak >= RAW_DMABUF_FAILURE_LATCH && !RAW_DMABUF_DISABLED.swap(true, Ordering::Relaxed) {
tracing::error!(
streak,
reason,
"zero-copy raw-dmabuf passthrough disabled for this host process: the encoder failed \
to import the compositor's dmabuf {streak} times in a row captures fall back to the \
CPU path (slower, but this host could not stream at all otherwise)"
);
}
}
/// Record a raw dmabuf that imported and encoded — resets the failure streak.
pub fn note_raw_dmabuf_import_ok() {
RAW_DMABUF_FAILURE_STREAK.store(0, Ordering::Relaxed);
}
/// True once repeated encoder import failures latched the raw-dmabuf passthrough off (see
/// [`note_raw_dmabuf_import_failure`]).
pub fn raw_dmabuf_import_disabled() -> bool {
RAW_DMABUF_DISABLED.load(Ordering::Relaxed)
}
/// DRM FourCC for a packed 32-bit format name (little-endian, e.g. `b"XR24"`).
const fn fourcc(c: &[u8; 4]) -> u32 {
(c[0] as u32) | ((c[1] as u32) << 8) | ((c[2] as u32) << 16) | ((c[3] as u32) << 24)
+156 -5
View File
@@ -870,6 +870,94 @@ impl PunktfunkRichInputEx {
}
}
/// [`PunktfunkPenSample::state`] bit: the pen hovers in range (implied by `TOUCHING`).
pub const PUNKTFUNK_PEN_IN_RANGE: u8 = 0x01;
/// [`PunktfunkPenSample::state`] bit: the tip is in contact.
pub const PUNKTFUNK_PEN_TOUCHING: u8 = 0x02;
/// [`PunktfunkPenSample::state`] bit: primary barrel button (or squeeze mapping) held.
pub const PUNKTFUNK_PEN_BARREL1: u8 = 0x04;
/// [`PunktfunkPenSample::state`] bit: secondary barrel button (or double-tap mapping) held.
pub const PUNKTFUNK_PEN_BARREL2: u8 = 0x08;
/// [`PunktfunkPenSample::tool`]: the pen tip.
pub const PUNKTFUNK_PEN_TOOL_PEN: u8 = 0;
/// [`PunktfunkPenSample::tool`]: the eraser (a client-side mode — Apple Pencil has no
/// hardware eraser end; the squeeze/double-tap mapping usually drives this).
pub const PUNKTFUNK_PEN_TOOL_ERASER: u8 = 1;
/// Most samples one [`punktfunk_connection_send_pen`] call accepts (one wire batch).
pub const PUNKTFUNK_PEN_BATCH_MAX: u32 = 8;
/// [`PunktfunkPenSample::tilt_deg`] sentinel: no tilt reading.
pub const PUNKTFUNK_PEN_TILT_UNKNOWN: u8 = 0xFF;
/// [`PunktfunkPenSample::azimuth_deg`] / `roll_deg` sentinel: no reading.
pub const PUNKTFUNK_PEN_ANGLE_UNKNOWN: u16 = 0xFFFF;
/// [`PunktfunkPenSample::distance`] sentinel: no hover-distance reading.
pub const PUNKTFUNK_PEN_DISTANCE_UNKNOWN: u16 = 0xFFFF;
/// One complete stylus state at one instant ([`punktfunk_connection_send_pen`];
/// design/pen-tablet-input.md). STATE-FULL, never an edge event: fill every field on every
/// sample (unknown axes take their `*_UNKNOWN` sentinel) — the host diffs consecutive samples
/// and synthesizes down/up/button transitions itself, which is what makes a lost datagram
/// self-heal. `x`/`y` are normalized `0.0..=1.0` in VIDEO-FRAME space (map your letterbox
/// before filling, exactly like wire touches).
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkPenSample {
/// Normalized `0.0..=1.0` across the video frame. Must be finite.
pub x: f32,
/// Normalized `0.0..=1.0` across the video frame. Must be finite.
pub y: f32,
/// Tip force, `0..=65535` full scale (`0` while hovering).
pub pressure: u16,
/// Hover distance `0..=65534` (0 = at the hover floor), or `PUNKTFUNK_PEN_DISTANCE_UNKNOWN`.
pub distance: u16,
/// Tilt azimuth, degrees `0..=359` clockwise from north, or `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
pub azimuth_deg: u16,
/// Barrel roll (Apple Pencil Pro `rollAngle`), degrees `0..=359`, or
/// `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
pub roll_deg: u16,
/// µs since the previous sample in the same call (`0` for the first) — the coalesced
/// capture spacing.
pub dt_us: u16,
/// Bitfield of `PUNKTFUNK_PEN_*` state bits. Unknown bits are rejected (`InvalidArg`).
pub state: u8,
/// `PUNKTFUNK_PEN_TOOL_PEN` or `PUNKTFUNK_PEN_TOOL_ERASER`.
pub tool: u8,
/// Tilt from the surface normal, degrees `0..=90`, or `PUNKTFUNK_PEN_TILT_UNKNOWN`.
pub tilt_deg: u8,
/// Set to 0.
pub _reserved: [u8; 3],
}
#[cfg(feature = "quic")]
impl PunktfunkPenSample {
/// `None` = invalid field (non-finite coordinate, unknown state bit, unknown tool) —
/// embedder input is validated strictly, unlike the loss-tolerant wire decode.
fn to_sample(self) -> Option<crate::quic::PenSample> {
use crate::quic as q;
let known = q::PEN_IN_RANGE | q::PEN_TOUCHING | q::PEN_BARREL1 | q::PEN_BARREL2;
if !self.x.is_finite() || !self.y.is_finite() || self.state & !known != 0 {
return None;
}
let tool = match self.tool {
PUNKTFUNK_PEN_TOOL_PEN => q::PenTool::Pen,
PUNKTFUNK_PEN_TOOL_ERASER => q::PenTool::Eraser,
_ => return None,
};
Some(q::PenSample {
state: self.state,
tool,
x: self.x,
y: self.y,
pressure: self.pressure,
distance: self.distance,
tilt_deg: self.tilt_deg,
azimuth_deg: self.azimuth_deg,
roll_deg: self.roll_deg,
dt_us: self.dt_us,
})
}
}
/// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8.
#[cfg(feature = "quic")]
unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> {
@@ -988,6 +1076,12 @@ pub const PUNKTFUNK_HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host supports the shared
/// clipboard, so a client may offer the toggle. (Mirrors `quic::HOST_CAP_CLIPBOARD`.)
pub const PUNKTFUNK_HOST_CAP_CLIPBOARD: u8 = 0x02;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host injects full-fidelity
/// stylus input, so a capable client splits pen contacts out of its touch path and sends them
/// via [`punktfunk_connection_send_pen`]; without the bit that call returns `Unsupported` and
/// the client keeps its pen-as-touch fallback. (Mirrors `quic::HOST_CAP_PEN`;
/// design/pen-tablet-input.md.)
pub const PUNKTFUNK_HOST_CAP_PEN: u8 = 0x10;
// Keep the ABI cap bits in lockstep with the wire constants (compile-time guard against drift).
#[cfg(feature = "quic")]
@@ -1001,6 +1095,15 @@ const _: () = {
assert!(PUNKTFUNK_CODEC_PYROWAVE == crate::quic::CODEC_PYROWAVE);
assert!(PUNKTFUNK_HOST_CAP_GAMEPAD_STATE == crate::quic::HOST_CAP_GAMEPAD_STATE);
assert!(PUNKTFUNK_HOST_CAP_CLIPBOARD == crate::quic::HOST_CAP_CLIPBOARD);
assert!(PUNKTFUNK_HOST_CAP_PEN == crate::quic::HOST_CAP_PEN);
assert!(PUNKTFUNK_PEN_IN_RANGE == crate::quic::PEN_IN_RANGE);
assert!(PUNKTFUNK_PEN_TOUCHING == crate::quic::PEN_TOUCHING);
assert!(PUNKTFUNK_PEN_BARREL1 == crate::quic::PEN_BARREL1);
assert!(PUNKTFUNK_PEN_BARREL2 == crate::quic::PEN_BARREL2);
assert!(PUNKTFUNK_PEN_BATCH_MAX as usize == crate::quic::PEN_BATCH_MAX);
assert!(PUNKTFUNK_PEN_TILT_UNKNOWN == crate::quic::PEN_TILT_UNKNOWN);
assert!(PUNKTFUNK_PEN_ANGLE_UNKNOWN == crate::quic::PEN_ANGLE_UNKNOWN);
assert!(PUNKTFUNK_PEN_DISTANCE_UNKNOWN == crate::quic::PEN_DISTANCE_UNKNOWN);
};
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
@@ -1710,9 +1813,11 @@ pub unsafe extern "C" fn punktfunk_generate_identity(
return PunktfunkStatus::InvalidArg;
}
unsafe {
std::ptr::copy_nonoverlapping(cert.as_ptr(), cert_pem_out as *mut u8, cert.len());
// `.cast()`, not `as *mut u8`: `c_char` is i8 on x86_64 but u8 on aarch64, so the
// `as` form is a REQUIRED conversion on one and a no-op clippy rejects on the other.
std::ptr::copy_nonoverlapping(cert.as_ptr(), cert_pem_out.cast::<u8>(), cert.len());
*cert_pem_out.add(cert.len()) = 0;
std::ptr::copy_nonoverlapping(key.as_ptr(), key_pem_out as *mut u8, key.len());
std::ptr::copy_nonoverlapping(key.as_ptr(), key_pem_out.cast::<u8>(), key.len());
*key_pem_out.add(key.len()) = 0;
}
PunktfunkStatus::Ok
@@ -2763,6 +2868,51 @@ pub unsafe extern "C" fn punktfunk_connection_send_rich_input2(
})
}
/// Send one stylus sample batch — `count` (`1..=PUNKTFUNK_PEN_BATCH_MAX`) state-full
/// [`PunktfunkPenSample`]s, oldest first (a capture callback's coalesced samples) — as one
/// `0xCC/0x05` pen datagram (non-blocking enqueue; design/pen-tablet-input.md). Split longer
/// runs into consecutive calls. Gate on `punktfunk_connection_host_caps() &
/// PUNKTFUNK_HOST_CAP_PEN`: toward a host without the bit this returns
/// [`PunktfunkStatus::Unsupported`] — keep the pen-as-touch fallback there.
/// [`PunktfunkStatus::InvalidArg`] on a bad count or a bad sample (non-finite coordinate,
/// unknown state bit / tool).
///
/// # Safety
/// `c` is a valid connection handle; `samples` is null or points to `count` valid
/// [`PunktfunkPenSample`]s.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_send_pen(
c: *mut PunktfunkConnection,
samples: *const PunktfunkPenSample,
count: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if samples.is_null() {
return PunktfunkStatus::NullPointer;
}
if count == 0 || count > PUNKTFUNK_PEN_BATCH_MAX {
return PunktfunkStatus::InvalidArg;
}
let raw = unsafe { std::slice::from_raw_parts(samples, count as usize) };
let mut batch = [crate::quic::PenSample::default(); crate::quic::PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(raw) {
match s.to_sample() {
Some(v) => *slot = v,
None => return PunktfunkStatus::InvalidArg,
}
}
match c.inner.send_pen(&batch[..count as usize]) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// The currently active session mode — the Welcome's, until an accepted
/// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
///
@@ -2977,9 +3127,10 @@ fn build_clip_event(
}
/// The host capability bitfield the session's `Welcome` carried — a bitfield of
/// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD`. A client tests
/// `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide whether to offer the shared-clipboard toggle.
/// Safe any time after connect.
/// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD` /
/// `PUNKTFUNK_HOST_CAP_PEN`. A client tests `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide
/// whether to offer the shared-clipboard toggle, `caps & PUNKTFUNK_HOST_CAP_PEN` before
/// sending stylus batches. Safe any time after connect.
///
/// # Safety
/// `c` is a valid connection handle; `caps` is writable (NULL is skipped).
+549 -38
View File
@@ -13,7 +13,13 @@
//! its rolling baseline: standing queue growth, the *pre-loss* signature of a saturated link
//! (bufferbloat) — this is the early-warning signal loss-based control lacks;
//! - **a jump-to-live flush** — the pump discarded its backlog, the strongest "we were behind"
//! evidence there is.
//! evidence there is;
//! - **host-encode-latency rise** — the host's per-AU 0xCF `encode_us` climbing above its rolling
//! baseline: the ENCODER falling behind its frame budget (the compute knee), the one failure a
//! fat LAN never surfaces as loss/OWD/decode. Paired with the host's own climb refusal (a
//! behind-cadence host acks climbs at the current rate) and short-ack cap learning
//! ([`BitrateController::on_ack`]), this is what stops an Automatic session from driving the
//! encoder off a cliff the network could carry.
//!
//! AIMD shape: a SEVERE window (an unrecoverable frame, a flush, ≥6 % loss, or a decode-latency
//! excursion far past baseline) backs off ×0.7 immediately; ordinary congestion
@@ -94,6 +100,25 @@ const UTILIZATION_DEN: u64 = 4;
/// cap always leaves ≥ ~12 % of climbing room — the two gates can't deadlock.
const PROVEN_HEADROOM_NUM: u32 = 3;
const PROVEN_HEADROOM_DEN: u32 = 2;
/// How far the window's mean HOST-ENCODE latency (the 0xCF `HostStages::encode_us` the host
/// already ships per AU) may rise above its rolling baseline before the window is bad. This is
/// the down-driver for the ENCODER's compute knee — the failure loss/OWD/decode are all blind
/// to: on a fat LAN the controller can climb to a rate the link carries fine but the ASIC
/// can't encode inside the frame budget (4K120 HEVC at ~800 Mbps ≈ 9.3 ms against 8.33), and
/// the only symptom is encode time. Baseline-RELATIVE on purpose: an escalated host reports
/// encode_us inflated by its retrieve-queue depth (~a frame), so an absolute budget threshold
/// would read permanently-red and drive the rate to the floor; a rise above the session's own
/// baseline survives that offset. ~half a 120 Hz frame budget of standing rise is real.
const ENCODE_RISE_US: i64 = 4_000;
/// Host-encode latency this far above baseline (≈1.5 × a 120 Hz budget) is SEVERE — the encode
/// queue is growing past the knee; skip the two-window confirmation.
const ENCODE_SEVERE_US: i64 = 12_000;
/// Clean windows parked at the learned [`host cap`](BitrateController::host_cap_kbps) before
/// re-probing above it (~60 s at the 750 ms tick). A cadence-refusal cap is scene-dependent
/// evidence, not a spec limit — without a re-probe, one heavy scene would cap the whole
/// session. A still-standing limit just re-teaches itself in two short acks, which the host
/// pre-clamps without touching the encoder — the re-probe costs no rebuild, no IDR.
const CAP_REPROBE_WINDOWS: u32 = 80;
/// Rolling window (in 750 ms report windows, ~30 s) whose minimum mean is the OWD baseline.
/// Long enough to remember the uncongested floor, short enough to follow genuine path changes.
const BASELINE_WINDOWS: usize = 40;
@@ -121,6 +146,27 @@ pub(crate) struct BitrateController {
/// keeping-up baseline. Empty on embedders that don't report decode latency (the decode
/// signal is then simply absent — identical to the pre-decode-signal behavior).
decode_means: VecDeque<i64>,
/// Recent window mean host-encode latencies (µs, from the 0xCF datagrams); rolling-min
/// baseline like the decode signal. Cleared whenever OUR OWN rate decrease changes the
/// encode regime (see [`on_ack`](Self::on_ack)) and on a mode switch.
encode_means: VecDeque<i64>,
/// The host-taught rate cap (§ABR overdrive): latched when the host acks BELOW what we
/// asked twice consecutively at the same value — its encoder's codec-level ceiling, or a
/// climb refusal while host encode can't hold cadence. Kept apart from `ceiling_kbps` so
/// the probe-measured link authority survives a mode switch's reset. Slowly re-probed
/// ([`CAP_REPROBE_WINDOWS`]) so scene-dependent evidence can't cap the session forever.
host_cap_kbps: Option<u32>,
/// The rate the last [`request`](Self::request) asked for — the reference an ack is judged
/// short against. Taken (not kept) by the ack, so one request is judged at most once.
last_requested_kbps: Option<u32>,
/// Consecutive short-ack streak: the value and how many times in a row it was acked. Two
/// identical short acks latch [`host_cap_kbps`](Self::host_cap_kbps) — one can be a
/// transient (a failed host rebuild keeping the old rate); the host's resolves are
/// deterministic min()s, so a persistent limit reproduces exactly.
short_ack_kbps: u32,
short_acks: u32,
/// Clean windows spent parked at the learned cap (the re-probe clock).
cap_probe_windows: u32,
/// Proven throughput: the session's highest windowed ACTUAL delivered rate seen with flat
/// decode latency — the known-good high-water mark climbs are bounded against. Never decays;
/// shrinking capacity (thermals, a heavier scene) is the reactive decode signal's job. On
@@ -147,6 +193,12 @@ impl BitrateController {
probing: true,
owd_means: VecDeque::with_capacity(BASELINE_WINDOWS),
decode_means: VecDeque::with_capacity(BASELINE_WINDOWS),
encode_means: VecDeque::with_capacity(BASELINE_WINDOWS),
host_cap_kbps: None,
last_requested_kbps: None,
short_ack_kbps: 0,
short_acks: 0,
cap_probe_windows: 0,
proven_kbps: 0,
bad_windows: 0,
clean_windows: 0,
@@ -168,22 +220,68 @@ impl BitrateController {
/// The host's [`crate::quic::BitrateChanged`] ack: its clamp is authoritative for what the
/// encoder now targets, and any ack proves the host renegotiates (resets the silence counter).
///
/// A SHORT ack (below what we asked) is the host telling us about a limit the network
/// signals can't see — its encoder's codec-level ceiling, or a climb refusal while encode
/// can't hold cadence. Two consecutive short acks at the SAME value latch it as
/// [`host_cap_kbps`](Self::host_cap_kbps), stopping the AIMD sawtooth from re-poking a
/// limit the host already refused; ONE is not enough — a failed host rebuild also acks
/// short once, and latching a transient would cap the session on a hiccup.
pub(crate) fn on_ack(&mut self, kbps: u32) {
if kbps > 0 {
if kbps < self.current_kbps {
// Our own decrease changes the encode-time regime (less work per frame; on an
// escalated host the queue offset shifts too) — judging the new regime against
// the old baseline would train-fire the encode down-driver. Re-seed it.
self.encode_means.clear();
}
if let Some(req) = self.last_requested_kbps.take() {
if kbps < req {
if self.short_ack_kbps == kbps {
self.short_acks += 1;
} else {
self.short_ack_kbps = kbps;
self.short_acks = 1;
}
if self.short_acks >= 2 && self.host_cap_kbps.is_none_or(|c| kbps < c) {
tracing::info!(
cap_kbps = kbps,
"adaptive bitrate: host cap learned (encoder ceiling or cadence \
refusal) climbs stop here until it lifts"
);
self.host_cap_kbps = Some(kbps.max(self.floor_kbps));
self.cap_probe_windows = 0;
}
} else {
self.short_acks = 0;
}
}
self.current_kbps = kbps;
}
self.unacked = 0;
}
/// An accepted mode switch: the encoder's ceiling and compute knee are properties of the
/// MODE (4K120 caps where 1080p60 never would) — drop the mode-scoped learned state. The
/// probe-measured `ceiling_kbps` (a LINK property) survives.
pub(crate) fn on_mode_switch(&mut self) {
self.host_cap_kbps = None;
self.short_acks = 0;
self.cap_probe_windows = 0;
self.encode_means.clear();
}
/// Feed one report window; returns the rate to request now, if any. `dropped` = frames that
/// went FEC-unrecoverable in the window, `loss_ppm` the window's [`crate::quic::LossReport`]
/// figure, `owd_mean_us` the window's mean skew-corrected capture→received latency (`None`
/// without a clock handshake), `decode_mean_us` the window's mean client decode-stage latency
/// (`None` on an embedder that doesn't report it — the signal is then absent), `actual_kbps`
/// the window's ACTUAL delivered throughput (wire bytes received ÷ window — what the pipeline
/// really carried, as opposed to the target it was allowed; feeds the utilization climb gate
/// and the proven-throughput high-water mark), `flushed` = the pump's jump-to-live fired in
/// the window.
/// (`None` on an embedder that doesn't report it — the signal is then absent),
/// `encode_mean_us` the window's mean HOST encode-stage latency (from the per-AU 0xCF
/// datagrams; `None` on an old host that doesn't send them), `actual_kbps` the window's
/// ACTUAL delivered throughput (wire bytes received ÷ window — what the pipeline really
/// carried, as opposed to the target it was allowed; feeds the utilization climb gate and
/// the proven-throughput high-water mark), `flushed` = the pump's jump-to-live fired in the
/// window.
#[allow(clippy::too_many_arguments)]
pub(crate) fn on_window(
&mut self,
@@ -192,6 +290,7 @@ impl BitrateController {
loss_ppm: u32,
owd_mean_us: Option<i64>,
decode_mean_us: Option<i64>,
encode_mean_us: Option<i64>,
actual_kbps: u32,
flushed: bool,
) -> Option<u32> {
@@ -242,6 +341,23 @@ impl BitrateController {
}
None => (false, false),
};
// Host-encode latency: the same rolling-min-baseline treatment, measuring the HOST'S
// encoder — the compute-knee down-driver (see [`ENCODE_RISE_US`]). This is the only
// signal that can push an already-too-high rate back under the knee: the host refuses
// further climbs while behind cadence, but nothing else ever DESCENDS on a clean LAN.
let (encode_bad, encode_severe) = match encode_mean_us {
Some(mean) => {
let base = self.encode_means.iter().min().copied();
let bad = base.is_some_and(|b| mean > b + ENCODE_RISE_US);
let severe = base.is_some_and(|b| mean > b + ENCODE_SEVERE_US);
if self.encode_means.len() == BASELINE_WINDOWS {
self.encode_means.pop_front();
}
self.encode_means.push_back(mean);
(bad, severe)
}
None => (false, false),
};
// The proven-throughput high-water mark: this window's delivered rate is now demonstrably
// digestible (decode latency stayed flat while it was carried). Loss doesn't disqualify —
// the bytes that DID arrive still went through the decoder; what loss means for the rate
@@ -253,8 +369,9 @@ impl BitrateController {
// deep decode-latency excursion) or loss far past any blip — one window is enough.
// Ordinary congestion (heavy-but-recoverable loss, an OWD rise, a decode-latency rise)
// still needs two consecutive windows.
let severe = dropped > 0 || flushed || loss_ppm >= SEVERE_LOSS_PPM || decode_severe;
let bad = severe || loss_ppm >= HEAVY_LOSS_PPM || owd_bad || decode_bad;
let severe =
dropped > 0 || flushed || loss_ppm >= SEVERE_LOSS_PPM || decode_severe || encode_severe;
let bad = severe || loss_ppm >= HEAVY_LOSS_PPM || owd_bad || decode_bad || encode_bad;
if bad {
self.bad_windows += 1;
self.clean_windows = 0;
@@ -264,6 +381,29 @@ impl BitrateController {
self.clean_windows += 1;
self.bad_windows = 0;
}
// The learned host cap re-probe (see [`CAP_REPROBE_WINDOWS`]): after ~60 s of clean
// windows parked at the cap, lift it one step (+12.5 %, ceiling-bounded) so a
// scene-dependent refusal can't quietly cap the whole session — a still-standing limit
// just re-latches from the next pair of short acks, at zero encoder cost.
if let Some(cap) = self.host_cap_kbps {
if bad {
self.cap_probe_windows = 0;
} else if self.current_kbps >= cap.saturating_sub(cap / 16) {
self.cap_probe_windows += 1;
if self.cap_probe_windows >= CAP_REPROBE_WINDOWS {
self.cap_probe_windows = 0;
let lifted = cap.saturating_add(cap / 8).min(self.ceiling_kbps);
if lifted > cap {
tracing::debug!(
from_kbps = cap,
to_kbps = lifted,
"adaptive bitrate: re-probing above the learned host cap"
);
self.host_cap_kbps = Some(lifted);
}
}
}
}
let cooled = self
.last_change
.is_none_or(|t| now.duration_since(t) >= CHANGE_COOLDOWN);
@@ -284,10 +424,14 @@ impl BitrateController {
// utilized window after a long-enough clean run climbs immediately.
let utilized =
actual_kbps as u64 * UTILIZATION_DEN >= self.current_kbps as u64 * UTILIZATION_NUM;
let cap = self
// The effective ceiling folds in the host-taught cap: the probe measured the LINK, but
// the host's short acks measured the ENCODER — whichever binds first is the limit.
let eff_ceiling = self
.ceiling_kbps
.min(self.host_cap_kbps.unwrap_or(u32::MAX));
let cap = eff_ceiling
.min(self.proven_kbps.saturating_mul(PROVEN_HEADROOM_NUM) / PROVEN_HEADROOM_DEN);
if self.current_kbps < self.ceiling_kbps && utilized && cap > self.current_kbps {
if self.current_kbps < eff_ceiling && utilized && cap > self.current_kbps {
// Slow start: double on every cooled clean window until the first congestion signal
// (this is how an Automatic session reaches a probe-measured ceiling in seconds).
// Congestion avoidance: +~6 % after a sustained clean run.
@@ -308,6 +452,7 @@ impl BitrateController {
fn request(&mut self, kbps: u32, now: Instant) -> Option<u32> {
self.last_change = Some(now);
self.unacked += 1;
self.last_requested_kbps = Some(kbps);
// `current_kbps` is NOT updated here — the host's ack is authoritative. A lost/ignored
// request just recomputes from the same base next time (and counts toward MAX_UNACKED).
Some(kbps)
@@ -331,7 +476,16 @@ mod tests {
fn run_clean(c: &mut BitrateController, start: Instant, from: u32, n: u32) -> Option<u32> {
let mut out = None;
for i in from..from + n {
out = c.on_window(ticks(start, i), 0, 0, Some(10_000), None, 1_000_000, false);
out = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
None,
None,
1_000_000,
false,
);
if out.is_some() {
return out;
}
@@ -345,7 +499,7 @@ mod tests {
let mut c = BitrateController::new(0);
let now = Instant::now();
assert_eq!(
c.on_window(now, 5, 900_000, Some(500_000), None, 1_000_000, true),
c.on_window(now, 5, 900_000, Some(500_000), None, None, 1_000_000, true),
None
);
}
@@ -356,22 +510,58 @@ mod tests {
let start = Instant::now();
// Heavy-but-recoverable loss (26 %) is ORDINARY: one window is a blip — no reaction.
assert_eq!(
c.on_window(ticks(start, 0), 0, 25_000, None, None, 1_000_000, false),
c.on_window(
ticks(start, 0),
0,
25_000,
None,
None,
None,
1_000_000,
false
),
None
);
// The second consecutive bad window backs off ×0.7.
assert_eq!(
c.on_window(ticks(start, 1), 0, 25_000, None, None, 1_000_000, false),
c.on_window(
ticks(start, 1),
0,
25_000,
None,
None,
None,
1_000_000,
false
),
Some(14_000)
);
c.on_ack(14_000);
// Still bad after the cooldown → another ×0.7 step from the ACKED rate.
assert_eq!(
c.on_window(ticks(start, 6), 0, 25_000, None, None, 1_000_000, false),
c.on_window(
ticks(start, 6),
0,
25_000,
None,
None,
None,
1_000_000,
false
),
None
); // bad #1 again
assert_eq!(
c.on_window(ticks(start, 7), 0, 25_000, None, None, 1_000_000, false),
c.on_window(
ticks(start, 7),
0,
25_000,
None,
None,
None,
1_000_000,
false
),
Some(9_800)
);
}
@@ -382,19 +572,28 @@ mod tests {
let mut c = BitrateController::new(20_000);
let start = Instant::now();
assert_eq!(
c.on_window(ticks(start, 0), 1, 0, None, None, 1_000_000, false),
c.on_window(ticks(start, 0), 1, 0, None, None, None, 1_000_000, false),
Some(14_000)
);
// …and so does a jump-to-live flush.
let mut c = BitrateController::new(20_000);
assert_eq!(
c.on_window(ticks(start, 0), 0, 0, None, None, 1_000_000, true),
c.on_window(ticks(start, 0), 0, 0, None, None, None, 1_000_000, true),
Some(14_000)
);
// …and ≥6 % window loss.
let mut c = BitrateController::new(20_000);
assert_eq!(
c.on_window(ticks(start, 0), 0, 80_000, None, None, 1_000_000, false),
c.on_window(
ticks(start, 0),
0,
80_000,
None,
None,
None,
1_000_000,
false
),
Some(14_000)
);
}
@@ -404,18 +603,18 @@ mod tests {
let mut c = BitrateController::new(20_000);
let start = Instant::now();
assert_eq!(
c.on_window(ticks(start, 0), 1, 0, None, None, 1_000_000, false),
c.on_window(ticks(start, 0), 1, 0, None, None, None, 1_000_000, false),
Some(14_000)
);
c.on_ack(14_000);
// A severe window INSIDE the 1.5 s cooldown (tick 1 = 750 ms) → held; at the cooldown
// boundary (tick 2 = 1.5 s) it fires.
assert_eq!(
c.on_window(ticks(start, 1), 1, 0, None, None, 1_000_000, false),
c.on_window(ticks(start, 1), 1, 0, None, None, None, 1_000_000, false),
None
);
assert_eq!(
c.on_window(ticks(start, 2), 1, 0, None, None, 1_000_000, false),
c.on_window(ticks(start, 2), 1, 0, None, None, None, 1_000_000, false),
Some(9_800)
);
}
@@ -426,17 +625,17 @@ mod tests {
let start = Instant::now();
// ×0.7 of 6000 = 4200 < floor → clamped to 5000.
assert_eq!(
c.on_window(ticks(start, 0), 1, 0, None, None, 1_000_000, false),
c.on_window(ticks(start, 0), 1, 0, None, None, None, 1_000_000, false),
Some(5_000)
);
c.on_ack(5_000);
// At the floor, further bad windows request nothing.
assert_eq!(
c.on_window(ticks(start, 6), 1, 0, None, None, 1_000_000, false),
c.on_window(ticks(start, 6), 1, 0, None, None, None, 1_000_000, false),
None
);
assert_eq!(
c.on_window(ticks(start, 7), 1, 0, None, None, 1_000_000, false),
c.on_window(ticks(start, 7), 1, 0, None, None, None, 1_000_000, false),
None
);
}
@@ -446,7 +645,7 @@ mod tests {
let mut c = BitrateController::new(20_000);
let start = Instant::now();
assert_eq!(
c.on_window(ticks(start, 0), 1, 0, None, None, 1_000_000, false),
c.on_window(ticks(start, 0), 1, 0, None, None, None, 1_000_000, false),
Some(14_000)
);
c.on_ack(14_000);
@@ -469,9 +668,16 @@ mod tests {
// Every cooled clean window doubles until the ceiling caps the climb, then quiet.
let mut got = Vec::new();
for i in 0..14 {
if let Some(k) =
c.on_window(ticks(start, i), 0, 0, Some(10_000), None, 1_000_000, false)
{
if let Some(k) = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
None,
None,
1_000_000,
false,
) {
c.on_ack(k);
got.push(k);
}
@@ -485,20 +691,47 @@ mod tests {
c.set_ceiling(300_000);
let start = Instant::now();
assert_eq!(
c.on_window(ticks(start, 0), 0, 0, Some(10_000), None, 1_000_000, false),
c.on_window(
ticks(start, 0),
0,
0,
Some(10_000),
None,
None,
1_000_000,
false
),
Some(40_000)
);
c.on_ack(40_000);
// Severe window → immediate ×0.7, and slow start is over.
assert_eq!(
c.on_window(ticks(start, 2), 1, 0, Some(10_000), None, 1_000_000, false),
c.on_window(
ticks(start, 2),
1,
0,
Some(10_000),
None,
None,
1_000_000,
false
),
Some(28_000)
);
c.on_ack(28_000);
// Clean again — but the next climb is additive, after the 6-window clean run.
let mut next = None;
for i in 3..12 {
next = c.on_window(ticks(start, i), 0, 0, Some(10_000), None, 1_000_000, false);
next = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
None,
None,
1_000_000,
false,
);
if next.is_some() {
assert!(i >= 8, "additive climb must wait for the clean run");
break;
@@ -512,7 +745,7 @@ mod tests {
let mut c = BitrateController::new(0);
c.set_ceiling(1_000_000);
assert_eq!(
c.on_window(Instant::now(), 0, 0, None, None, 1_000_000, false),
c.on_window(Instant::now(), 0, 0, None, None, None, 1_000_000, false),
None
);
let mut c = BitrateController::new(20_000);
@@ -527,17 +760,44 @@ mod tests {
// Establish a ~10 ms baseline over a few clean windows.
for i in 0..4 {
assert_eq!(
c.on_window(ticks(start, i), 0, 0, Some(10_000), None, 1_000_000, false),
c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
None,
None,
1_000_000,
false
),
None
);
}
// Delay climbs 40 ms above baseline with ZERO loss — bufferbloat. Two windows → back off.
assert_eq!(
c.on_window(ticks(start, 4), 0, 0, Some(50_000), None, 1_000_000, false),
c.on_window(
ticks(start, 4),
0,
0,
Some(50_000),
None,
None,
1_000_000,
false
),
None
);
assert_eq!(
c.on_window(ticks(start, 5), 0, 0, Some(52_000), None, 1_000_000, false),
c.on_window(
ticks(start, 5),
0,
0,
Some(52_000),
None,
None,
1_000_000,
false
),
Some(14_000)
);
}
@@ -557,6 +817,7 @@ mod tests {
0,
Some(10_000),
Some(8_000),
None,
1_000_000,
false
),
@@ -572,6 +833,7 @@ mod tests {
0,
Some(10_000),
Some(38_000),
None,
1_000_000,
false
),
@@ -584,6 +846,7 @@ mod tests {
0,
Some(10_000),
Some(40_000),
None,
1_000_000,
false
),
@@ -605,6 +868,7 @@ mod tests {
0,
Some(10_000),
Some(8_000),
None,
1_000_000,
false
),
@@ -620,6 +884,7 @@ mod tests {
0,
Some(10_000),
Some(38_000),
None,
1_000_000,
false
),
@@ -634,6 +899,7 @@ mod tests {
0,
Some(10_000),
Some(40_000),
None,
1_000_000,
false
),
@@ -657,6 +923,7 @@ mod tests {
0,
Some(10_000),
Some(8_000),
None,
2_000,
false
),
@@ -673,6 +940,7 @@ mod tests {
0,
Some(10_000),
Some(8_000),
None,
18_000,
false
),
@@ -696,6 +964,7 @@ mod tests {
0,
Some(10_000),
Some(8_000),
None,
20_000,
false
),
@@ -710,6 +979,7 @@ mod tests {
0,
Some(10_000),
Some(8_000),
None,
30_000,
false
),
@@ -732,6 +1002,7 @@ mod tests {
0,
Some(10_000),
Some(8_000),
None,
20_000,
false
),
@@ -741,7 +1012,16 @@ mod tests {
// A long calm stretch (2 % utilization, decoder idle): the controller stays silent.
for i in 2..30 {
assert_eq!(
c.on_window(ticks(start, i), 0, 0, Some(10_000), Some(4_000), 600, false),
c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
Some(4_000),
None,
600,
false
),
None
);
}
@@ -761,6 +1041,7 @@ mod tests {
0,
Some(10_000),
Some(8_000),
None,
1_000_000,
false
),
@@ -776,6 +1057,7 @@ mod tests {
0,
Some(10_000),
Some(60_000),
None,
1_000_000,
false
),
@@ -783,6 +1065,235 @@ mod tests {
);
}
#[test]
fn two_identical_short_acks_latch_the_host_cap() {
// The 4K120 field failure: the encoder ceilings at ~794 Mbps while the link carries
// more — the host acks short. TWO identical short acks teach the cap; climbs then stop
// poking a limit the host already refused (the rebuild-storm driver).
let mut c = BitrateController::new(400_000);
c.set_ceiling(1_400_000);
let start = Instant::now();
assert_eq!(run_clean(&mut c, start, 0, 1), Some(800_000));
// First short ack: current follows (authoritative), but one short ack is not a cap.
c.on_ack(794_000);
assert!(c.host_cap_kbps.is_none());
// The next climb overshoots again and is short-acked at the SAME value: latch.
assert_eq!(run_clean(&mut c, start, 10, 1), Some(1_400_000));
c.on_ack(794_000);
assert_eq!(c.host_cap_kbps, Some(794_000));
// Parked AT the learned cap, nothing left to climb to — no more requests.
assert_eq!(run_clean(&mut c, start, 20, 12), None);
}
#[test]
fn one_short_ack_is_a_transient_not_a_cap() {
// A failed host rebuild acks short once (it kept the old rate) — latching THAT would
// cap the session on a driver hiccup. The streak must survive only identical repeats.
let mut c = BitrateController::new(400_000);
c.set_ceiling(1_400_000);
let start = Instant::now();
assert_eq!(run_clean(&mut c, start, 0, 1), Some(800_000));
c.on_ack(400_000); // rebuild failed, host kept the old rate
assert!(c.host_cap_kbps.is_none());
// The retry applies fully: streak broken, still no cap, full authority kept.
assert_eq!(run_clean(&mut c, start, 10, 1), Some(800_000));
c.on_ack(800_000);
assert!(c.host_cap_kbps.is_none());
}
#[test]
fn mode_switch_clears_the_learned_cap() {
let mut c = BitrateController::new(400_000);
c.set_ceiling(1_400_000);
let start = Instant::now();
assert_eq!(run_clean(&mut c, start, 0, 1), Some(800_000));
c.on_ack(794_000);
assert_eq!(run_clean(&mut c, start, 10, 1), Some(1_400_000));
c.on_ack(794_000);
assert_eq!(c.host_cap_kbps, Some(794_000));
// 4K120's ceiling means nothing at the new mode — the cap must not survive the switch
// (the probe-measured link ceiling does).
c.on_mode_switch();
assert!(c.host_cap_kbps.is_none());
assert_eq!(c.ceiling_kbps, 1_400_000);
}
#[test]
fn learned_cap_reprobes_after_a_sustained_clean_run() {
// A cadence-refusal cap is scene evidence, not a spec limit: after ~60 s parked clean
// at the cap, lift one step so a one-time heavy scene can't cap the session forever. A
// still-standing limit just re-latches from the next short-ack pair, at zero cost.
let mut c = BitrateController::new(400_000);
c.set_ceiling(1_400_000);
let start = Instant::now();
assert_eq!(run_clean(&mut c, start, 0, 1), Some(800_000));
c.on_ack(794_000);
assert_eq!(run_clean(&mut c, start, 10, 1), Some(1_400_000));
c.on_ack(794_000);
assert_eq!(c.host_cap_kbps, Some(794_000));
for i in 0..CAP_REPROBE_WINDOWS {
let _ = c.on_window(
ticks(start, 20 + i),
0,
0,
Some(10_000),
None,
None,
1_000_000,
false,
);
}
assert_eq!(c.host_cap_kbps, Some(794_000 + 794_000 / 8));
}
#[test]
fn host_encode_latency_rise_backs_off() {
// The compute knee: link pristine, client decoder fine — only HOST encode time moves
// (the 4K120 case: ~9.3 ms against an 8.33 ms budget shows up nowhere else). Two risen
// windows → ×0.7, exactly like an OWD/decode rise.
let mut c = BitrateController::new(20_000);
let start = Instant::now();
for i in 0..4 {
assert_eq!(
c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
None,
Some(7_000),
1_000_000,
false
),
None
);
}
assert_eq!(
c.on_window(
ticks(start, 4),
0,
0,
Some(10_000),
None,
Some(11_500),
1_000_000,
false
),
None
);
assert_eq!(
c.on_window(
ticks(start, 6),
0,
0,
Some(10_000),
None,
Some(12_000),
1_000_000,
false
),
Some(14_000)
);
}
#[test]
fn deep_encode_excursion_is_severe() {
// Encode time shooting ≈1.5 frame budgets over baseline = the queue is growing past
// the knee right now — no two-window confirmation.
let mut c = BitrateController::new(20_000);
let start = Instant::now();
for i in 0..4 {
assert_eq!(
c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
None,
Some(7_000),
1_000_000,
false
),
None
);
}
assert_eq!(
c.on_window(
ticks(start, 4),
0,
0,
Some(10_000),
None,
Some(20_000),
1_000_000,
false
),
Some(14_000)
);
}
#[test]
fn rate_decrease_rebases_the_encode_baseline() {
// After OUR OWN decrease the encode regime legitimately changes (less work per frame;
// an escalated host's reported encode_us also carries a queue offset) — the old
// baseline must not train-fire repeated backoffs down to the floor.
let mut c = BitrateController::new(20_000);
let start = Instant::now();
for i in 0..4 {
let _ = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
None,
Some(7_000),
1_000_000,
false,
);
}
let _ = c.on_window(
ticks(start, 4),
0,
0,
Some(10_000),
None,
Some(12_000),
1_000_000,
false,
);
assert_eq!(
c.on_window(
ticks(start, 6),
0,
0,
Some(10_000),
None,
Some(12_500),
1_000_000,
false
),
Some(14_000)
);
// The decrease applies → rebase. The new regime's ~15 ms means (an escalated host's
// queue offset) would be far over the OLD 7 ms baseline, but must now read clean.
c.on_ack(14_000);
for i in 8..11 {
assert_eq!(
c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
None,
Some(15_000),
1_000_000,
false
),
None
);
}
}
#[test]
fn ack_silence_disables_the_controller() {
let mut c = BitrateController::new(20_000);
@@ -791,7 +1302,7 @@ mod tests {
let mut i = 0;
// Keep every window bad and never ack: exactly MAX_UNACKED requests, then silence.
while i < 60 {
if c.on_window(ticks(start, i), 1, 0, None, None, 1_000_000, false)
if c.on_window(ticks(start, i), 1, 0, None, None, None, 1_000_000, false)
.is_some()
{
sent += 1;
@@ -224,6 +224,20 @@ pub(crate) struct DecodeLatAcc {
pub(crate) count: u32,
}
/// Host encode-stage latency accumulator — [`DecodeLatAcc`]'s mirror for the HOST side of the
/// pipeline. The datagram task adds one sample per 0xCF `HostStages::encode_us` (host encoder
/// submit → bitstream ready) and the pump drains a window mean into
/// [`crate::abr::BitrateController::on_window`]'s encode signal. Host encode time was measured,
/// shipped and drawn on the overlay, but never an ABR input — which is how a fat-LAN Automatic
/// session drove the encoder past its compute knee with nothing to stop it (§ABR overdrive).
/// Its own accumulator rather than the overlay's `host_timing` channel: that channel is a lossy
/// `try_send` the embedder may never drain, and the controller must not depend on it.
#[derive(Default)]
pub(crate) struct EncodeLatAcc {
pub(crate) sum_us: u64,
pub(crate) count: u32,
}
/// The pre-decode video hand-off from the data-plane pump to the embedder. Unlike the side planes
/// (self-contained samples that drop the newest on overflow), video AUs are reference-chained under the
/// host's infinite GOP: dropping ANY frame mid-stream corrupts every dependent frame until the next
+44 -5
View File
@@ -20,7 +20,7 @@ use crate::quic::{
RfiRequest, RichInput,
};
use crate::session::Frame;
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU32, AtomicU64, Ordering};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU16, AtomicU32, AtomicU64, Ordering};
use std::sync::mpsc::{Receiver, RecvTimeoutError};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
@@ -104,8 +104,11 @@ pub struct NativeClient {
/// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing
/// audio-latency (and memory) without limit — mic is best-effort end to end.
mic_tx: tokio::sync::mpsc::Sender<(u32, u64, Vec<u8>)>,
/// Outbound rich input (DualSense touchpad / motion) → 0xCC datagrams by the worker.
rich_input_tx: tokio::sync::mpsc::UnboundedSender<RichInput>,
/// Outbound 0xCC rich-input plane, PRE-ENCODED datagrams: [`RichInput`] touchpad/motion
/// (encoded in [`NativeClient::send_rich_input`]) and stylus [`crate::quic::PenBatch`]es
/// (encoded in [`NativeClient::send_pen`]) share the channel — the worker's task just
/// forwards bytes, so a new 0xCC kind never touches the pump.
rich_input_tx: tokio::sync::mpsc::UnboundedSender<Vec<u8>>,
/// Outbound control-stream requests (mode switch, speed test) → the worker's control task.
/// Bounded ([`CTRL_QUEUE`]) — the requests are sparse; a full queue means the control task
/// is wedged/dead, and callers treat it like a closed session.
@@ -121,6 +124,9 @@ pub struct NativeClient {
/// Monotonic id for outbound fetches ([`NativeClient::clip_fetch`]); stays below
/// [`crate::clipboard::INBOUND_REQ_FLAG`] so it never collides with an inbound serve `req_id`.
next_xfer_id: AtomicU32,
/// Wrapping per-connection [`crate::quic::PenBatch::seq`] counter, stamped by
/// [`NativeClient::send_pen`] (the host's reorder gate compares it).
pen_seq: AtomicU16,
/// The host capability bitfield ([`crate::quic::Welcome::host_caps`]) — see
/// [`NativeClient::host_caps`].
pub host_caps: u8,
@@ -344,7 +350,7 @@ impl NativeClient {
std::sync::mpsc::sync_channel::<crate::quic::HostTiming>(HOST_TIMING_QUEUE);
let (input_tx, input_rx) = tokio::sync::mpsc::unbounded_channel::<InputEvent>();
let (mic_tx, mic_rx) = tokio::sync::mpsc::channel::<(u32, u64, Vec<u8>)>(MIC_QUEUE);
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<RichInput>();
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<Vec<u8>>();
let (ctrl_tx, ctrl_rx) = tokio::sync::mpsc::channel::<CtrlRequest>(CTRL_QUEUE);
let (clip_event_tx, clip_event_rx) =
std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE);
@@ -473,6 +479,7 @@ impl NativeClient {
clip: Mutex::new(clip_event_rx),
clip_cmd_tx,
next_xfer_id: AtomicU32::new(1),
pen_seq: AtomicU16::new(0),
host_caps: negotiated.host_caps,
probe,
shutdown,
@@ -1073,7 +1080,39 @@ impl NativeClient {
/// loss like every datagram. No-op unless the host runs the DualSense gamepad backend.
pub fn send_rich_input(&self, rich: RichInput) -> Result<()> {
self.rich_input_tx
.send(rich)
.send(rich.encode())
.map_err(|_| PunktfunkError::Closed)
}
/// Queue one stylus sample batch for delivery as a `0xCC/0x05` pen datagram
/// (design/pen-tablet-input.md). `samples` are state-full and oldest-first (a capture
/// callback's coalesced samples), at most [`crate::quic::PEN_BATCH_MAX`] per call — split
/// longer runs into consecutive calls so the stamped wrapping `seq` keeps them ordered.
/// Best-effort like every datagram: a lost batch self-heals on the next one (the samples
/// carry full state, the host diffs — see [`crate::quic::PenTracker`]).
///
/// **Heartbeat contract**: while the pen is in range or touching, repeat the last sample
/// at least every ~100 ms even when nothing changed (capture APIs are silent for a
/// stationary pen) — the host force-releases the stroke after
/// [`crate::quic::PEN_TOUCH_TIMEOUT_MS`] of silence as its dead-client failsafe.
///
/// Requires the host to have advertised [`crate::quic::HOST_CAP_PEN`]; toward an older
/// host this returns `Unsupported` (embedders keep their pen-as-touch fallback instead of
/// spraying 240 Hz datagrams the host drops unread).
pub fn send_pen(&self, samples: &[crate::quic::PenSample]) -> Result<()> {
if self.host_caps & crate::quic::HOST_CAP_PEN == 0 {
return Err(PunktfunkError::Unsupported(
"host did not advertise HOST_CAP_PEN",
));
}
if samples.is_empty() || samples.len() > crate::quic::PEN_BATCH_MAX {
return Err(PunktfunkError::InvalidArg(
"pen batch must hold 1..=PEN_BATCH_MAX samples",
));
}
let seq = self.pen_seq.fetch_add(1, Ordering::Relaxed);
self.rich_input_tx
.send(crate::quic::PenBatch::new(seq, samples).encode())
.map_err(|_| PunktfunkError::Closed)
}
+22 -3
View File
@@ -36,6 +36,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
};
let handshake::HandshakeOut {
conn,
ep,
session,
ctrl_send,
ctrl_recv,
@@ -99,17 +100,24 @@ pub(super) async fn run_pump(args: WorkerArgs) {
}
});
// Rich-input task: embedder DualSense touchpad / motion → 0xCC uplink datagrams.
// Rich-input task: pre-encoded 0xCC uplink datagrams (DualSense touchpad / motion, pen
// batches — encoded at the NativeClient surface so new plane kinds never touch the pump).
let rich_conn = conn.clone();
tokio::spawn(async move {
while let Some(rich) = rich_input_rx.recv().await {
let _ = rich_conn.send_datagram(rich.encode().into());
while let Some(d) = rich_input_rx.recv().await {
let _ = rich_conn.send_datagram(d.into());
}
});
// Adaptive bitrate ack slot: the control task parks the latest BitrateChanged here; the
// pump's controller drains it on its report tick (`take()` — an ack is consumed once).
let bitrate_ack: Arc<Mutex<Option<u32>>> = Arc::new(Mutex::new(None));
// Host-encode-latency accumulator (the ABR encode signal, see [`EncodeLatAcc`]): the
// datagram task adds one sample per 0xCF; the pump drains a window mean per report tick.
let encode_lat = Arc::new(Mutex::new(super::frame_channel::EncodeLatAcc::default()));
// Bumped by the control task on every accepted mode switch (the `clock_gen` pattern): the
// pump resets the controller's mode-scoped learned state (host cap, encode baseline).
let mode_gen = Arc::new(AtomicU32::new(0));
// Control task (see [`control_task`]): the handshake stream stays open for mid-stream
// renegotiation, speed tests, clock re-sync, and clipboard metadata.
@@ -126,6 +134,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
clock_gen: clock_gen.clone(),
clip_event_tx: clip_event_tx.clone(),
cursor_shape_tx,
mode_gen: mode_gen.clone(),
}
.run(),
);
@@ -139,6 +148,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
hidout_tx,
hdr_meta_tx,
host_timing_tx,
encode_lat.clone(),
cursor_state_tx,
));
@@ -174,6 +184,8 @@ pub(super) async fn run_pump(args: WorkerArgs) {
clock_offset,
clock_gen,
decode_lat,
encode_lat,
mode_gen,
frames_dropped,
fec_recovered,
bitrate_ack,
@@ -191,4 +203,11 @@ pub(super) async fn run_pump(args: WorkerArgs) {
0
};
conn.close(close_code.into(), b"client closed");
// Flush the CONNECTION_CLOSE before the runtime is dropped (the same discipline as the pairing
// + probe paths). `close` only queues the frame — the endpoint driver puts it on the wire, and
// this fn is the body of a `block_on` whose runtime is dropped the instant it returns, so
// without this the driver could simply never be polled again. The host then saw a deliberate
// quit as silence: no `QUIT_CLOSE_CODE`, an 8 s idle timeout, and the keep-alive linger meant
// for an UNWANTED disconnect. Bounded — a host already gone must not delay the client's exit.
let _ = tokio::time::timeout(std::time::Duration::from_millis(300), ep.wait_idle()).await;
}
@@ -24,6 +24,10 @@ pub(super) struct ControlTask {
/// Host cursor shapes ([`CursorShape`], sent on pointer-bitmap change) → the embedder's
/// shape plane ([`NativeClient::next_cursor_shape`]).
pub(super) cursor_shape_tx: std::sync::mpsc::SyncSender<crate::quic::CursorShape>,
/// Bumped on every ACCEPTED mode switch (the `clock_gen` pattern): the pump watches it and
/// resets the bitrate controller's mode-scoped learned state — the encoder ceiling / compute
/// knee it was taught belong to the OLD mode.
pub(super) mode_gen: Arc<AtomicU32>,
}
impl ControlTask {
@@ -40,6 +44,7 @@ impl ControlTask {
clock_gen,
clip_event_tx,
cursor_shape_tx,
mode_gen,
} = self;
// Mid-stream clock re-sync (see [`ClockResync`]): a batch runs every
// CLOCK_RESYNC_INTERVAL and whenever the pump asks (CtrlRequest::ClockResync after
@@ -88,6 +93,7 @@ impl ControlTask {
if let Ok(ack) = Reconfigured::decode(&msg) {
if ack.accepted {
*mode_slot.lock().unwrap() = ack.mode;
mode_gen.fetch_add(1, Ordering::Relaxed);
tracing::info!(mode = ?ack.mode, "host accepted mode switch");
} else {
tracing::warn!(active = ?ack.mode, "host rejected mode switch");
+29 -4
View File
@@ -19,6 +19,12 @@ pub(super) struct DataPump {
pub(super) clock_offset: Arc<std::sync::atomic::AtomicI64>,
pub(super) clock_gen: Arc<AtomicU32>,
pub(super) decode_lat: Arc<Mutex<DecodeLatAcc>>,
/// Host encode-stage latency window accumulator (the ABR encode signal — see
/// [`super::super::frame_channel::EncodeLatAcc`]); fed by the datagram task.
pub(super) encode_lat: Arc<Mutex<super::super::frame_channel::EncodeLatAcc>>,
/// Accepted-mode-switch generation (control task bumps): a change resets the controller's
/// mode-scoped learned state ([`BitrateController::on_mode_switch`]).
pub(super) mode_gen: Arc<AtomicU32>,
pub(super) frames_dropped: Arc<std::sync::atomic::AtomicU64>,
pub(super) fec_recovered: Arc<std::sync::atomic::AtomicU64>,
pub(super) bitrate_ack: Arc<Mutex<Option<u32>>>,
@@ -41,6 +47,8 @@ impl DataPump {
clock_offset: pump_clock_offset,
clock_gen: pump_clock_gen,
decode_lat: pump_decode_lat,
encode_lat: pump_encode_lat,
mode_gen: pump_mode_gen,
frames_dropped,
fec_recovered,
bitrate_ack,
@@ -127,6 +135,7 @@ impl DataPump {
let mut clock_detector_armed = true;
let mut resync_wanted = false;
let mut seen_clock_gen = pump_clock_gen.load(Ordering::Relaxed);
let mut seen_mode_gen = pump_mode_gen.load(Ordering::Relaxed);
// Standing-latency bleed (see StandingLatency): the third detector, for the small,
// constant, loss-free OWD elevation the two jump-to-live detectors deliberately
// tolerate (< QUEUE_HIGH frames, < FLUSH_LATENCY behind) — a sub-frame standing
@@ -338,9 +347,15 @@ impl DataPump {
);
}
}
// Adaptive bitrate: drain any host ack first (its clamp is authoritative), then
// feed the controller this window's congestion signals; a decision becomes a
// SetBitrate on the control stream.
// Adaptive bitrate: an accepted mode switch first (it invalidates the
// mode-scoped learned state), then drain any host ack (its clamp is
// authoritative), then feed the controller this window's congestion signals; a
// decision becomes a SetBitrate on the control stream.
let mg = pump_mode_gen.load(Ordering::Relaxed);
if mg != seen_mode_gen {
seen_mode_gen = mg;
abr.on_mode_switch();
}
if let Some(acked) = bitrate_ack.lock().unwrap().take() {
abr.on_ack(acked);
}
@@ -356,6 +371,14 @@ impl DataPump {
*acc = DecodeLatAcc::default();
(count > 0).then(|| (sum / count as u64) as i64)
};
// Same drain for the host-encode window (0xCF `encode_us` via the datagram
// task) — `None` on an old host that doesn't send stage timings.
let encode_mean_us = {
let mut acc = pump_encode_lat.lock().unwrap();
let (sum, count) = (acc.sum_us, acc.count);
*acc = Default::default();
(count > 0).then(|| (sum / count as u64) as i64)
};
// The window's ACTUAL delivered throughput — what the pipeline really carried, vs
// the target it was allowed. Wire bytes (headers + FEC) slightly overstate the
// media rate the decoder ingests; acceptable for the climb gate / proven-mark
@@ -369,17 +392,19 @@ impl DataPump {
loss_ppm,
owd_mean_us,
decode_mean_us,
encode_mean_us,
actual_kbps,
flush_in_window,
) {
// Log the window's signals alongside the decision so an on-glass session can
// tell a decode-driven re-target (the new signal — decode_mean_us elevated with
// tell a decode-/encode-driven re-target (the new signals — elevated with
// loss/OWD flat) from a network-driven one.
tracing::info!(
kbps,
loss_ppm,
owd_mean_us = owd_mean_us.unwrap_or(-1),
decode_mean_us = decode_mean_us.unwrap_or(-1),
encode_mean_us = encode_mean_us.unwrap_or(-1),
actual_kbps,
flushed = flush_in_window,
"adaptive bitrate: requesting encoder re-target"
@@ -14,6 +14,9 @@ pub(super) async fn run(
hidout_tx: std::sync::mpsc::SyncSender<crate::quic::HidOutput>,
hdr_meta_tx: std::sync::mpsc::SyncSender<crate::quic::HdrMeta>,
host_timing_tx: std::sync::mpsc::SyncSender<crate::quic::HostTiming>,
// The ABR encode signal's accumulator (see [`EncodeLatAcc`]) — fed HERE, not off
// `host_timing_tx`: that channel is the overlay's, lossy and embedder-drained.
encode_lat: Arc<Mutex<super::super::frame_channel::EncodeLatAcc>>,
cursor_state_tx: std::sync::mpsc::SyncSender<crate::quic::CursorState>,
) {
// Per-pad reorder gate for v2 rumble envelopes (the seq analog of the host's gamepad-state
@@ -74,6 +77,11 @@ pub(super) async fn run(
}
Some(&crate::quic::HOST_TIMING_MAGIC) => {
if let Some(t) = crate::quic::decode_host_timing_datagram(&d) {
if let Some(s) = &t.stages {
let mut acc = encode_lat.lock().unwrap();
acc.sum_us += s.encode_us as u64;
acc.count += 1;
}
let _ = host_timing_tx.try_send(t);
}
}
@@ -3,12 +3,17 @@
//! up the data-plane [`Session`]. A typed application close from the host surfaces as
//! [`PunktfunkError::Rejected`] instead of the generic transport error.
use super::super::*;
use super::*;
/// Everything [`run_pump`](super::run_pump) needs from a successful connect + handshake.
pub(super) struct HandshakeOut {
pub(super) conn: quinn::Connection,
/// The dialing endpoint, kept alive for the session so the pump can FLUSH its
/// `CONNECTION_CLOSE` before the runtime is dropped ([`super::run_pump`]). Dropping it here
/// left nothing to drive the close onto the wire, so a deliberate quit reached the host as
/// silence — an 8 s idle timeout with no quit code, which the host reads as an unwanted
/// disconnect and lingers the display for.
pub(super) ep: quinn::Endpoint,
pub(super) session: Session,
pub(super) ctrl_send: quinn::SendStream,
pub(super) ctrl_recv: io::MsgReader,
@@ -238,6 +243,7 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
match handshake.await {
Ok((session, send, recv, negotiated, host_caps)) => Ok(HandshakeOut {
conn,
ep,
session,
ctrl_send: send,
ctrl_recv: recv,
@@ -9,7 +9,6 @@
//! Keyboard/mouse/touch events pass through unchanged; an older host (no caps bit) keeps
//! getting the legacy per-transition gamepad events.
use super::super::*;
use super::*;
pub(super) async fn run(
+3 -2
View File
@@ -5,7 +5,7 @@ use crate::clipboard::{ClipCommand, ClipEventCore};
use crate::config::{CompositorPref, GamepadPref, Mode};
use crate::error::Result;
use crate::input::InputEvent;
use crate::quic::{HdrMeta, HidOutput, RichInput};
use crate::quic::{HdrMeta, HidOutput};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU64};
use std::sync::mpsc::SyncSender;
use std::sync::{Arc, Mutex};
@@ -43,7 +43,8 @@ pub(crate) struct WorkerArgs {
pub(crate) cursor_state_tx: SyncSender<crate::quic::CursorState>,
pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>,
pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>,
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>,
/// Pre-encoded 0xCC datagrams (rich input AND pen batches — see `NativeClient.rich_input_tx`).
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<Vec<u8>>,
pub(crate) ctrl_rx: tokio::sync::mpsc::Receiver<CtrlRequest>,
pub(crate) ctrl_tx: tokio::sync::mpsc::Sender<CtrlRequest>,
/// Inbound clipboard event plane feed — the control task pushes ClipState/ClipOffer, the
+4 -1
View File
@@ -106,7 +106,10 @@ pub use stats::Stats;
/// (design/remote-desktop-sweep.md §8): the client's mouse-model chord tells the host who
/// renders the pointer. Additive; rides the existing control stream (a new message TYPE, which
/// pre-§8 hosts ignore), so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 12;
/// v13: added `punktfunk_connection_send_pen` — the stylus wire plane
/// (design/pen-tablet-input.md): a client sends `RICH_PEN` sample batches once the host
/// advertises `HOST_CAP_PEN`. Additive and capability-gated, so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 13;
/// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
/// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**

Some files were not shown because too many files have changed in this diff Show More