Compare commits

..

105 Commits

Author SHA1 Message Date
enricobuehler c8ee4b9902 fix(pf-vdisplay,pf-capture,pf-win-display): pre-split paths in the auto-merged v4 code
windows-drivers / probe-and-proto (push) Successful in 31s
ci / web (push) Successful in 43s
apple / swift (push) Failing after 1m1s
ci / docs-site (push) Successful in 1m0s
apple / screenshots (push) Has been skipped
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
windows-drivers / driver-build (push) Successful in 1m38s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m1s
ci / bench (push) Successful in 5m34s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5m53s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
windows-host / package (push) Successful in 15m6s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m48s
docker / deploy-docs (push) Successful in 23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m28s
The rename-followed perf hunks still said crate::win_display:: (the pre-W6
layout) — point them at pf_win_display::win_display:: and widen the four
helpers they call cross-crate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:13:15 +02:00
enricobuehler 1197415216 fix(pf-vdisplay,pf-win-display): v4 trait surface on the extracted driver.rs + cross-crate visibility
The W-refactor extracted VdisplayDriver into manager/driver.rs (the merge
resolution assumed it deleted) — carry the v4 changes there: open() returns the
driver's protocol version, update_modes() default-errs to the re-arrival
fallback. wait_target_departed goes pub for the manager's cross-crate call.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:11:34 +02:00
enricobuehler 18a5d93ae3 fix(host): allow too_many_arguments on the two fns the v4 merge grew
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:09:57 +02:00
enricobuehler 09849906e9 Merge perf/first-frame-latency: driver proto v4 + first-frame/resize latency (P0-P2)
Brings the first-frame-latency branch (P0.1 transition tracing, P1.1/P1.2
Welcome-time display prep, P2 in-place resize; pf-driver-proto v3 -> v4 with
IOCTL_UPDATE_MODES) onto current main. The branch predates the W6.2/W7 splits,
so git's rename detection carried most of it into the moved crates
(pf-capture idd_push, pf-vdisplay manager/pf_vdisplay, pf-win-display,
pf-driver-proto, the driver workspace) and the punktfunk1.rs remainder was
re-homed by hand:

- native/handshake.rs: welcome/start trace marks + the Welcome-time display
  prep spawn (the prep thread BECOMES the stream thread; hand-off via a
  SyncSender<SessionContext>). negotiate() gains bringup/quit/stop and returns
  the PrepHandle.
- native.rs: bringup/resize_ms creation + the stop/quit flags hoisted BEFORE
  the handshake (the close watcher splits: flags pre-handshake, lifecycle
  events post-handshake where `hello` exists); punch_done stamp; the data
  plane adopts the prep thread's result or builds inline.
- native/stream.rs: SessionContext/SendStats carry the trace; send_loop
  finishes it on the first video packet; the resize path gains the in-place
  fast path (try_inplace_resize) with the full rebuild as fallback, restructured
  so both share the post-rebuild bookkeeping; prepare_display/PreparedDisplay/
  PrepHandle; build_pipeline(+retry) thread the stage marks.
- session_status/mgmt: ttff_ms + last_resize_ms per session (union with the
  lifecycle-events fields main added to the same spots).
- pf-capture: Capturer gains capture_target_id() + resize_output() defaults.
- pf-vdisplay manager: perf's faster activation poll (60x50ms) + the settle
  floor before the PnP sweep, on main's knobs/no-trait shape.

Also: packaging/windows/build-gamepad-drivers.ps1 is ASCII again (an em-dash
from the pf-mouse work tripped windows-host.yml's locale-safety gate on main).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 16:08:16 +02:00
enricobuehler 86d9f49473 style(pf-vdisplay): rustfmt under the pinned 1.96.0 toolchain
audit / bun-audit (push) Successful in 12s
windows-drivers / probe-and-proto (push) Successful in 31s
ci / web (push) Successful in 48s
apple / swift (push) Failing after 1m1s
ci / docs-site (push) Successful in 1m2s
decky / build-publish (push) Successful in 21s
windows-drivers / driver-build (push) Successful in 1m35s
windows-host / package (push) Failing after 16s
audit / cargo-audit (push) Successful in 2m46s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
release / apple (push) Successful in 3m56s
apple / screenshots (push) Has been skipped
ci / bench (push) Successful in 6m26s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m16s
flatpak / build-publish (push) Failing after 8m9s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 13m9s
android / android (push) Successful in 13m11s
deb / build-publish (push) Successful in 14m12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m8s
ci / rust (push) Successful in 19m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m10s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m1s
The W6.2 pf-vdisplay extraction (27a5d8da) committed six spots that the pinned
rustfmt (rust-toolchain.toml = 1.96.0) reformats — a short closure collapse and
chain-call wrapping. No semantic change; makes cargo fmt --all --check green so
the combined push does not land CI red on pre-existing format drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:38:12 +02:00
enricobuehler 2064c0780c merge(core): reconcile the W7/W8 client refactor with origin's shared-clipboard feature
origin/main landed the shared clipboard (design/clipboard-and-file-transfer.md) while
this branch split quic/msgs.rs -> quic/{caps,control,...} and client.rs ->
client/{mod,control,worker,pump,planes,...} (W7) and deleted the two monoliths. The
feature had modified both deleted files, so its delta is re-applied onto the split
instead of resurrecting the monoliths:

  - HOST_CAP_CLIPBOARD                         -> quic/caps.rs
  - MSG_CLIP_* / CLIP_* consts, the six Clip*
    structs, and their encode/decode impls     -> quic/control.rs (beside the clock codecs)
  - CtrlRequest::{ClipControl,ClipOffer} +
    Negotiated.host_caps                        -> client/control.rs
  - WorkerArgs.{clip_event_tx,clip_cmd_rx}      -> client/worker.rs
  - CLIP_EVENT_QUEUE                            -> client/planes.rs
  - NativeClient clip fields, the 7 clip_* /
    host_caps / next_clip methods, connect()
    channel wiring                              -> client/mod.rs
  - the control-task encode/decode arms and
    the clipboard-task spawn                     -> client/pump.rs

Cargo.lock reconciled (adds pf-clipboard), punktfunk-host/Cargo.toml unions the W6
pf-* subsystem deps with pf-clipboard, and include/punktfunk_core.h is the cbindgen
union (clipboard + rumble C-ABI). punktfunk-core builds --all-features and its 174
lib tests pass, including quic::tests::clip_loopback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:38:11 +02:00
enricobuehler f439b69451 refactor(android/W8): split decode.rs into decode/ directory module
Break the 1747-line clients/android/native/src/decode.rs into a decode/ directory
module (mod.rs + 5 concern submodules):
  - decode/setup.rs      : codec creation + low-latency config + thread/frame-rate
                           tuning + HDR static-info encode
  - decode/display.rs    : DisplayTracker + render-callback registration + HDR dataspace
  - decode/latency.rs    : realtime clock + decoded-pts / user-flags stat recording
  - decode/sync_loop.rs  : the synchronous poll decode loop (+ feed/drain) — moved WHOLE
  - decode/async_loop.rs : the event-driven async decode loop (+ helpers) — moved WHOLE
decode/mod.rs keeps the consts, DecodeOptions, and the `run` entry point + the
`codec_mime`/`codec_label` re-export, so every crate::decode::X path stays byte-stable.
The module has no decoder struct (free functions + small types), so both decode loops
move byte-for-byte and their separately-inlined received-stat recording is NOT unified.
16 helper fns/types became pub(super) for sibling access; zero field bumps. lib.rs
unchanged (`#[cfg(target_os="android")] mod decode;` resolves to decode/mod.rs).

Verified: cargo-ndk check (aarch64-linux-android, clean) + the gradle cargoNdkDebug
build (arm64-v8a / armeabi-v7a / x86_64). On-device runtime re-verification still owed
per the plan (the two decode loops are a hot path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:06:24 +02:00
enricobuehler 1eef55016d refactor(apple/W8): extract FullscreenController + ApprovalRequest from ContentView
Move the macOS `FullscreenController` (NSViewRepresentable that drives native
fullscreen) into its own FullscreenController.swift, and `ApprovalRequest` (the
pending-trust-decision value type) into ApprovalRequest.swift, out of the
1041-line ContentView.swift. Both were file-`private`; dropped to internal so
ContentView (same module) still references them across files. StreamView*.swift
and the connection body are untouched. Pure move; no behavior change.

Verified: `swift build` (macOS) — Build complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:02:34 +02:00
enricobuehler 61118cbdd4 refactor(presenter/W8): split vk.rs into vk/ directory module
Break the 2513-line pf-presenter/src/vk.rs into a vk/ directory module (mod.rs +
6 concern submodules), keeping ALL type definitions in vk/mod.rs so every submodule
(a descendant of vk) sees the Presenter/OverlayPipe/etc. private fields with no
field bumps:
  - vk/setup.rs      : Presenter::new + device/format/present-mode selection
  - vk/present.rs    : the per-frame present path (present + CSC record + AVVkFrame
                       sync) — HOT PATH, moved whole
  - vk/reconfig.rs   : swapchain recreate/resize + HDR reconfiguration
  - vk/resources.rs  : video-image/staging (re)build + Retired-frame destruction
  - vk/overlay_pipe.rs: the presenter-side overlay composite pipeline
  - vk/gpu.rs        : memory allocation, image barriers, geometry helpers (+ tests)
vk/mod.rs keeps FrameInput/Presenter/OverlayPipe/VideoImage/Staging/Retired/HwCtx*
+ the public accessors + Drop. Methods/free-fns a sibling submodule calls became
pub(super) (~18); zero field bumps, zero re-exports (Presenter/FrameInput never
leave mod.rs). lib.rs unchanged (`pub mod vk;` resolves to vk/mod.rs). The moved
overlay shader include_bytes! gained one `../` for the deeper dir. Pure move; no
behavior change; the hot present path keeps only static pub(super) calls (inlinable).

Verified both platforms: Linux (home-worker-5) clippy -p pf-presenter
(--all-targets -D warnings) + test; Windows (winbox, ASCII CARGO_HOME) clippy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:39:36 +02:00
enricobuehler 22a61e0b48 refactor(console-ui/W8): split shell.rs into shell/ facade + render/overlays/tests
Break the 1212-line pf-console-ui/src/shell.rs into a facade + shell/ subdir
(shell.rs stays the parent; `mod render;` resolves to shell/render.rs):
  - shell/render.rs   : the per-frame screen compose/transition path (Shell::render
                        + the LayerEnv paint helper)
  - shell/overlays.rs : the modal overlays (Shell::draw_overlays + draw_takeover)
  - shell/tests.rs    : the inline #[cfg(test)] module, extracted verbatim
The Shell struct + its public API + draw_aurora() stay in shell.rs (both children
reach draw_aurora + the private fields as descendants — no bumps). Sole visibility
change: draw_overlays -> pub(in crate::shell) (its caller Shell::render is now a
sibling). Zero re-exports needed (Shell/ConsoleOptions never leave the root). Pure
move; no behavior change.

Verified both platforms: Linux (home-worker-5) clippy -p pf-console-ui
(--all-targets -D warnings) + test; Windows (winbox, ASCII CARGO_HOME) clippy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:29:07 +02:00
enricobuehler 13b1f36d4a feat(core,clients): one rumble policy engine for every platform (rumble root fix D)
punktfunk-core client/rumble.rs: a per-connection policy engine consumes seq-gated wire
updates and emits EFFECTIVE actuator commands — re-emits on renewals (duration APIs stay
re-armed), self-silences at the v2 lease, a UNIFORM 1 s legacy-host staleness replacing the
per-platform zoo (Apple 1.6 s / Android 60 s / SDL 1.5 s / Deck 1 s), quirk-declared
actuator keepalives (Deck 40 ms + LSB dedupe-defeat jitter), and one stop per buzzing pad
on connection close. Per-pad mailbox semantics: a stalled embedder wakes to ONE current
command, and a stop can structurally never be the update an overflowing queue drops.

New API/ABI: NativeClient::{next_rumble_command,set_rumble_quirks} +
punktfunk_connection_next_rumble_cmd/_set_rumble_quirks (next_rumble/next_rumble2 stay for
un-migrated embedders; both consumers are fed). Migrations DELETE the platform forks:
pf-client-core loses RumbleState + the Deck keepalive loop + LEGACY_RUMBLE_CEILING_MS and
physically silences a slot at close; Android loses the 60 s legacy one-shot (backstop
repack, cancel-on-zero); Apple loses envelopeDeadline + sessionStaleSeconds + both tick
watchdogs (CoreHaptics realization untouched; mac xcframework rebuilt locally).

design/rumble-root-fix.md par. D. Engine 10/10 unit tests; core tests 176 Linux / 175
Windows + clippy -D warnings; swift build + RumbleTuningTests; Kotlin + android-native
compile green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 14:07:32 +02:00
enricobuehler 9e6fc6e071 fix(host/inject,drivers): rumble root fixes A-C — lossless report ring + rumble-keyed idle watchdogs
B: PadFeedback.game_drove -> rumble_drove, keyed on vibration-asserting reports — an
LED/adaptive-trigger stream can no longer feed the abandoned-rumble force-off while a
coalesced stop never re-asserts (the confirmed unbounded stuck-ON path). C: Linux parity —
every UHID backend now arms the shared watchdog (Steam Input drives these pads over hidraw
with Windows abandonment semantics) and the uinput mixer force-stops abandoned
infinite-replay FF effects (FfState, unit-tested). Shared PUNKTFUNK_RUMBLE_IDLE_MS hatch
(0 = off; non-zero floored above SDL's ~2 s rumble resend).

A: PadShm v2.1 — a 1024 B tail extension carrying an 8-slot lossless output-report ring,
feature-negotiated via zeroed reserved fields (out_ring_ver; deliberately NO
GAMEPAD_PROTO_VERSION bump — mixed generations degrade to the legacy latest-report slot
instead of failing closed). The pf-dualsense driver dual-writes both planes
(publish_output); the host's shared OutputDrain drains oldest->newest with a torn-read
recheck and an overflow->resync path (PadFeedback.resync force-stops + re-arms dedups).
pf-umdf-util grows a min_data_size map fallback. Ds*Feedback.fresh removed (dead).

design/rumble-root-fix.md par. A-C. Verified: pf-inject tests+clippy Linux+Windows (53/53
on winbox incl. the stop-coalesce repro); drivers ws check+clippy on the CI runner.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 14:07:32 +02:00
enricobuehler 570ff504ad refactor(client-core/W8): split video.rs into flat decoder-backend siblings
Break the 1974-line pf-client-core/src/video.rs into flat sibling modules
(matching the crate's video_d3d11.rs / video_pyrowave.rs convention), leaving
video.rs as the contract + Decoder dispatch facade:
  - video_color.rs   : ColorDesc + csc_rows (the Y'CbCr->RGB matrix)
  - video_software.rs : the libavcodec/swscale SoftwareDecoder
  - video_vaapi.rs   : the Linux-only VAAPI/DRM-PRIME backend (mod is cfg(linux))
  - video_vulkan.rs  : the FFmpeg Vulkan Video backend
Every crate::video::X / video::X path stays byte-stable (ColorDesc + csc_rows
re-exported from video.rs; frame POD, VulkanDecodeDevice, QueueLock, Decoder,
decodable_codecs*, ffmpeg_codec_id, fourcc/drm_fourcc_for all stay in video.rs).
Code-driven placements: averr, AVERROR_EAGAIN, frame_is_keyframe stay in video.rs
(shared by all three decoders); DrmFrameGuard's field + drm_fourcc_for +
Software/Vaapi/VulkanDecoder ctors/decode became pub(crate) (sibling access);
the test module split three ways (software tests need private decoder internals).
Pure move; no behavior change.

Verified on Linux (home-worker-5): cargo clippy -p pf-client-core (default
[pyrowave] + --no-default-features, --all-targets -D warnings) + cargo test.
Windows verify BLOCKED environmentally: pf-client-core -> sdl3 build-from-source
-> CMake/CL.exe fails on winbox's non-ASCII home path (fails the baseline too,
independent of this split); the split's Windows surface (facade cfg(windows) bits
+ video_d3d11) is verbatim-preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:06:57 +02:00
enricobuehler e8b64ffe43 Merge branch 'feat/shared-clipboard-v2': shared clipboard Phase 1 (wire + pf-clipboard host backends + macOS client)
audit / bun-audit (push) Successful in 13s
apple / swift (push) Successful in 1m19s
ci / web (push) Successful in 1m28s
audit / cargo-audit (push) Successful in 2m18s
ci / docs-site (push) Successful in 1m43s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
decky / build-publish (push) Successful in 38s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 1m10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m15s
ci / bench (push) Successful in 5m25s
release / apple (push) Successful in 6m4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m46s
arch / build-publish (push) Successful in 11m22s
docker / deploy-docs (push) Successful in 31s
apple / screenshots (push) Successful in 4m47s
flatpak / build-publish (push) Failing after 8m1s
windows-host / package (push) Successful in 15m58s
android / android (push) Successful in 16m40s
deb / build-publish (push) Successful in 15m58s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m57s
ci / rust (push) Successful in 23m13s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m53s
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m26s
2026-07-17 11:43:52 +00:00
enricobuehler 5d0e23d6a5 feat(apple/clipboard): macOS client half of the shared clipboard (Phase 1 §5)
ci / web (pull_request) Successful in 48s
ci / docs-site (pull_request) Successful in 53s
apple / swift (pull_request) Successful in 1m16s
apple / screenshots (pull_request) Has been skipped
android / android (pull_request) Has been cancelled
ci / rust (pull_request) Has been cancelled
ci / bench (pull_request) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (pull_request) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (pull_request) Has been cancelled
The NSPasteboard bridge completing Phase 1 (design/clipboard-and-file-transfer.md
§5) — with the host backends on this branch, copy/paste now crosses the wire in
both directions on macOS. Lazy in both directions:

- PunktfunkConnection grows the clipboard plane: its own clipboardLock (close()
  joins it like the other pullers), hostCaps/hostSupportsClipboard from the
  Welcome, the typed ClipEvent vocabulary, and the six ABI wrappers
  (clipControl/clipOffer/clipFetch/clipServe/clipCancel/nextClipboard — borrowed
  event payloads copied out before the next poll).
- ClipboardSync (PunktfunkKit, macOS-only): one drain thread bridging
  NSPasteboard.general ↔ the QUIC clipboard plane. Local copies announce format
  lists via a 500 ms changeCount poll (+ immediate on app activation); bytes
  leave only on a host FetchRequest, answered from the live pasteboard and
  seq-guarded against staleness. Host copies install one NSPasteboardItem whose
  data provider fires only when a Mac app actually pastes, then blocks its
  provider thread (never main) on a 10 s-bounded fetch. Concealed/Transient
  pasteboards (password managers) are never announced; our own writes are
  changeCount-suppressed (§3.4). Text/RTF/HTML/PNG; files ride Phase 2.
- UI: per-host "Share clipboard with this host" toggle (StoredHost.clipboardSync,
  optional for saved-JSON forward-compat — wire-format tests extended), a
  mid-session Share/Stop Sharing Clipboard item in the Stream menu (⌃⌥⇧C,
  greyed without HOST_CAP_CLIPBOARD), SessionModel owning the lifecycle
  (start on streaming after the trust gate, drain joined off-main on teardown).

swift build + swift test green (macOS). Requires the ABI v8 xcframework
(scripts/build-xcframework.sh).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:40:47 +02:00
enricobuehler ffa63a74f2 refactor(core/W7): split client.rs into client/ facade + submodules
Turn the 2674-line client.rs into a client/ directory module (mod.rs facade +
8 submodules) behind glob/`use self::` re-exports, so crate::client::X paths
(NativeClient, ProbeOutcome, AudioPacket, display_hdr_env_override) stay
byte-stable. Leaf lifts: frame_channel.rs (the FIFO hand-off + jump-to-live
consts + DecodeLatAcc), recovery.rs (RfiRecovery loss-range detector),
probe.rs (ProbeState/ProbeOutcome), planes.rs (side-plane queues + AudioPacket),
control.rs (CtrlRequest/Negotiated), worker.rs (WorkerArgs + reject_from_close),
pairing.rs (NativeClient::pair). The per-frame pump moves WHOLE as a plain
`pub(super) async fn run_pump` (was worker_main) — the only edit is the
signature line: no trait object, no Box, no per-frame allocation or indirection.
NativeClient + its public impl + Drop + the cfg-gated thread-pin/hot-tid helpers
stay in the facade. Visibility bumps are pub(crate) (struct + each field for
WorkerArgs/Negotiated/ProbeState; FrameChannel + each method); reject_from_close
is pub(crate) (sibling access). No behavior change.

Verified: Linux clippy (quic + no-default, -D warnings) + full cargo test;
Windows clippy (both) + test --lib; macOS clippy (apple thread-pin variant) +
165 lib tests. On-glass jump-to-live + ABR smoke still owed (pump is a pure
relocation, so this is a formality) per the plan's pump gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:29:16 +02:00
enricobuehler 716875dd09 refactor(core/W7): split quic/msgs.rs into handshake/caps/control/pairing
Break the 1302-line quic/msgs.rs into four flat sibling modules behind the
quic facade's glob re-exports, so every crate::quic::X path stays byte-stable:
handshake.rs (Hello/Welcome/Start + codecs), caps.rs (video-cap bits, codec &
chroma negotiation, ColorInfo), control.rs (typed CTL_MAGIC messages + frame),
pairing.rs (SPAKE2 ceremony messages). msgs.rs is deleted; quic/mod.rs gains the
four `mod`/`pub use` lines and the `pub use crate::reject::*` hoist (moved up from
msgs.rs). Pure move; no wire-format or behavior change. Private helpers
(truncate_to, put_bytes, get_bytes) stay with their sole callers; no visibility
changes.

Verified both platforms from clean HEAD snapshots: Linux clippy (quic +
no-default, -D warnings) + full cargo test (157 lib + integration); Windows
clippy (both) + test --lib (156).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:08:45 +02:00
enricobuehler da0578771e fix(pf-clipboard): declare the libc dep the Linux backends call fully-qualified
ci / web (pull_request) Successful in 54s
ci / docs-site (pull_request) Successful in 1m15s
apple / swift (pull_request) Successful in 1m15s
apple / screenshots (pull_request) Has been skipped
ci / bench (pull_request) Successful in 6m0s
android / android (pull_request) Successful in 12m29s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 5m24s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 6m3s
ci / rust (pull_request) Successful in 21m5s
wayland.rs (pipe2/poll on the paste pipes) and mutter.rs (fcntl un-nonblocking
on the transfer fd) reference libc:: inline — caught by the Linux leg.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:07:13 +02:00
enricobuehler 391f8fb9f7 feat(clipboard): Linux + Windows host clipboard backends as the pf-clipboard crate (Phase 1 host + Phase 3)
ci / web (pull_request) Successful in 1m9s
apple / swift (pull_request) Successful in 1m19s
apple / screenshots (pull_request) Has been skipped
ci / docs-site (pull_request) Successful in 1m30s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 5m16s
ci / bench (pull_request) Successful in 6m8s
ci / rust (pull_request) Failing after 7m6s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 6m33s
android / android (pull_request) Successful in 12m39s
The host half of the shared clipboard (design/clipboard-and-file-transfer.md §4),
ported from feat/shared-clipboard (6bd8c18b) into the post-W6 crate shape: the
backends land as a pf-clipboard subsystem crate (the pf-inject/pf-capture
pattern) instead of growing punktfunk-host back out, and the ~340-line
punktfunk1.rs integration is re-implemented against the native.rs/control.rs
split that replaced it.

pf-clipboard:
- host::wayland — ext-data-control-v1 (KWin / wlroots / Sway / Hyprland).
- host::mutter — GNOME via Mutter's *direct* org.gnome.Mutter.RemoteDesktop
  clipboard (no data-control at any GNOME version; the xdg portal needs an
  interactive grant a headless host can't answer).
- host::windows + host::winfmt — Win32 clipboard on a hidden message-loop
  window: WM_CLIPBOARDUPDATE listener + OLE delayed rendering (WM_RENDERFORMAT)
  for text / CF_HTML / RTF / PNG.
- host::session — the backend-agnostic coordinator bridging HostClipboard to
  the QUIC clipboard plane (offers, fetch accept-loop, remote offers, pastes).
- A portable facade (policy / enabled / cap_advertised / ClipCoordCmd / start /
  spawn_decline_loop) so the orchestrator compiles cfg-free on every platform;
  ClipCoordCmd moves into the crate (it was host-owned before).

punktfunk-host glue:
- handshake.rs advertises HOST_CAP_CLIPBOARD via pf_clipboard::cap_advertised.
- serve_session starts the coordinator (gated on a real compositor — the
  synthetic source stays out of the session clipboard) and spawns the
  CLIP_FETCH_UNAVAILABLE decline loop when the policy is on but no backend bound.
- control.rs gains the ClipControl/ClipOffer arms + the host-offer forward
  branch, and the e2e session test (cap advertise → ClipState ack with
  BACKEND_UNAVAILABLE → fetch decline) rides in native.rs's tests.

Still opt-in default OFF (PUNKTFUNK_CLIPBOARD). Remaining: the macOS client
(design §5) — then this becomes user-visible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:02:20 +02:00
enricobuehler ef736cb9d7 refactor(core/W7): split transport/udp.rs into udp/ facade + per-OS backends
Turn transport/udp.rs into a udp/ directory module: the cross-platform core
(UdpTransport, is_transient_io, spawn_data_punch, the Transport trait impl) stays
in mod.rs; the platform batched-I/O backends move to udp/{linux,windows,apple}.rs.
The trait impl is kept whole -- its per-OS send_batch/send_gso/recv_batch methods
become cfg-gated one-line delegators to pub(super) free fns that take &UdpTransport
(byte-identical bodies, self -> t). transport/mod.rs is unchanged (re-exports still
resolve; udp/mod.rs re-exports windows::send_uso_all). No behavior change.

Module gates: linux = any(linux, android) (Android uses sendmmsg/recvmmsg via its
bionic binding); windows = windows (USO); apple = all(unix, not(any(linux,android)))
(recvmsg_x on Darwin, recv-loop on BSD). GSO stays linux-only.

Verified on all four target families from clean HEAD snapshots: Linux clippy
(quic + no-default, -D warnings) + full test; Windows clippy (both) + test --lib
(156); macOS check (apple recvmsg_x path); aarch64-linux-android check (android_mmsg).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:01:53 +02:00
enricobuehler 4ef90d586d feat(clipboard): wire protocol + client-core task for shared clipboard (Phase 0)
The portable shared-clipboard plane in punktfunk-core, all behind the `quic`
feature (design/clipboard-and-file-transfer.md §3):

- Control messages 0x40–0x44 (ClipControl / ClipOffer / ClipFetch...) and the
  HOST_CAP_CLIPBOARD capability bit, negotiated in the Welcome caps.
- Per-transfer QUIC bi-streams ("PKFs" magic) for lazy fetch of offered content,
  with ClipFetchHdr status/size framing (quic::clipstream).
- The §3.5 portable wire-MIME vocabulary (text/plain;utf-8, text/html, text/rtf,
  image/png) shared by both ends.
- Client-side clipboard task (client.rs) + C ABI surface bumped to v8 (abi.rs,
  regenerated include/punktfunk_core.h).
- Loopback transport tests (quic::tests).

No OS clipboard integration yet — that is the host backends (Phase 1/3) and the
macOS client (Phase 1).

Ported from feat/shared-clipboard (af3a7d8c, pre-W6 base) onto current main;
three deliberate deviations from the original commit:
- ABI v6 → v8: main took v6 (reanchor gate) and v7 (typed connect rejection)
  in the meantime; the clipboard C surface re-lands as v8.
- CLIP_CANCELLED_CODE 0x60 → 0x70: main's pairing-rejection close codes claimed
  the 0x60–0x67 block; the vocabularies stay disjoint on purpose.
- Negotiated.host_caps coexists with main's 6-tuple host_caps plumbing: main
  needs the worker-local copy for gamepad snapshots, the clipboard path needs it
  across ready_tx to build the NativeClient handle (punktfunk_connection_host_caps).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 12:50:37 +02:00
enricobuehler 93c8dc4712 refactor(core/W7): split packet.rs into packet/ facade + submodules
Turn the 1446-line packet.rs into a packet/ directory module (mod.rs facade
+ header/packetize/reassemble/tests) behind glob re-exports, so every
crate::packet::X path stays byte-stable. Pure move: the header consts +
PacketHeader -> header.rs; Packetizer -> packetize.rs; the Reassembler cluster
(kept WHOLE -- disjoint-borrow hot path) + loss-window consts -> reassemble.rs;
the inline #[cfg(test)] block -> tests.rs. Sole visibility change:
LOSS_WINDOW_NS -> pub(super) (a test imports it). No behavior change.

Verified on both platforms from a clean HEAD snapshot: Linux clippy
(--features quic and --no-default-features, --all-targets -D warnings) + full
cargo test; Windows clippy (both feature sets) + cargo test --lib (156 pass).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 12:48:48 +02:00
enricobuehler f012ebbcba feat(sdk): Effect v4 + @effect/openapi-generator; typed pf.api & example ladder
Drop Orval for the first-party @effect/openapi-generator (OpenAPI 3.1 ->
Effect Schema + a typed HttpClient client) and bump effect 3.19 ->
4.0.0-beta.98. Port the hand-written surfaces to the v4 API (Result over
Either, Context.Service, Codec, Literals/Union arrays, Stream/Schedule/
Effect renames). Transport (CA-pinning fetch) and the reconnecting SSE
source are kept intact.

Make the SDK approachable for non-Effect users:
- Add pf.api.* on the Promise facade: the generated client surfaced as
  typed, Promise-native methods (await pf.api.listPairedClients()), so REST
  calls are autocompleted and checked instead of stringly-typed
  pf.request(method, path, body) + `as` casts. Zero-drift veneer over
  make(httpClient), backed by the same pinning fetch. pf.request stays as
  the untyped escape hatch.
- Re-tier examples into a 1-4 complexity ladder, rewritten onto pf.api.*
  (the typed payloads caught a wrong `launch` shape in provider-sync);
  the Effect example is labelled advanced. Add examples/ to tsconfig so
  they are typechecked (stops rot).

typecheck + 19 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 12:40:04 +02:00
enricobuehler 27a5d8daac refactor(host/W6.2): extract virtual-display orchestration into the pf-vdisplay crate
vdisplay.rs + vdisplay/* (the per-compositor Linux backends — KWin zkde-screencast,
wlroots swaymsg, Mutter RemoteDesktop, Hyprland — and the Windows IddCx/pf-vdisplay
driver backend, behind one VirtualDisplay trait; the mode-conflict admission
registry, the display policy/identity/custom-preset state, and the session-env /
gamescope routing) move into crates/pf-vdisplay (plan §W6). The DDC/CI panel-power
control (used only here) and the KWin zkde protocol XML move with it. This
completes the host-crate decomposition: capture, encode, inject, and vdisplay are
now four subsystem crates over the shared leaves, and punktfunk-host is the
orchestrator (serve/supervisor + native + gamestream + mgmt).

Coupling breaks (all down-only, cargo-tree acyclic):
- capture::dxgi identity -> pf_frame::dxgi; win_display/monitor_devnode/
  console_session_mismatch -> pf-win-display leaf; can_open_another_session ->
  pf-encode (the NVENC session-budget admission gate — acyclic peer edge).
- The registry's DisplayCreated/DisplayReleased emits into the host SSE event bus
  invert to a leaf hook: pf-vdisplay emits a neutral DisplayEvent to a
  host-registered DISPLAY_EVENT_SINK, so it never reaches the orchestrator's
  events module.
- The IddCx driver module is renamed pf_vdisplay -> driver (its old name collided
  with the crate name through the host's `mod vdisplay` shim glob).

The host keeps `mod vdisplay { pub use pf_vdisplay::* }` so every crate::vdisplay::*
path (serve/mgmt/native/the capture FrameChannelSender seam) is unchanged; the
heavy deps (wayland/ashpd/tokio + the zkde protocol) moved with the crate.
Co-authored: a fail-closed IOCTL-reply-length security fix (reject short/zeroed
pf-vdisplay driver replies before trusting protocol_version/target_id/wudf_pid/luid,
security-review 2026-07-17) rides this commit in the moved driver module.

Verified: Linux clippy -D warnings (pf-vdisplay + host nvenc,vulkan-encode,pyrowave
--all-targets) + pf-vdisplay 63/63 + host 167/167 tests; Windows clippy -D warnings
(pf-vdisplay --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 12:14:08 +02:00
enricobuehler f6c6e4e594 refactor(host/W6.2): extract the input-injection backends into the pf-inject crate
inject.rs + inject/* (the per-OS injectors — wlroots virtual-input, KWin
fake_input, libei/reis, gamescope-EI on Linux; SendInput on Windows — plus the
virtual-gamepad HID stack: DualSense/DualShock4/Switch Pro/Steam Controller/Deck
over uhid/usbip and the Windows UMDF drivers, the proto codecs, the injector
service, and the uhid manager) move into crates/pf-inject behind the
InputInjector trait (plan §W6). It consumes punktfunk_core::input (the neutral
GamepadEvent/InputEvent vocabulary, moved to core in W5) + the pf-driver-proto
wire contract, and reaches pf-capture only for the Windows gamepad-channel
WUDFHost check + the resident-mouse compose-kick hook.

The one inject->vdisplay coupling (the libei gamescope-EI backend needs the EIS
relay socket path) is broken via a leaf: gamescope_ei_socket_file moves to
pf-paths as the shared contract — the gamescope producer (host vdisplay) keeps
its session-env-lock wrapper around it, the libei consumer (pf-inject) reads it
directly post-retarget. The host keeps a `mod inject { pub use pf_inject::* }`
shim so every crate::inject::* path (the native/gamestream input planes + devtest)
is unchanged; the heavy input deps (wayland/reis/xkbcommon/usbip + the KWin
fake-input protocol XML) moved with the crate.

Verified: Linux clippy -D warnings (pf-inject + host nvenc,vulkan-encode,pyrowave
--all-targets) + pf-inject 69/69 + host 230/230 tests; Windows clippy -D warnings
(pf-inject --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 11:52:02 +02:00
enricobuehler 0992548de7 feat(host/windows): HID compose kick — wake and dirty the display through the virtual mouse (lid-closed first-frame fix)
The final piece of the lid-closed field report (fixes 1-3: 3d9b3290;
prerequisites: 85dd2bb0 pf-mouse, 845a9760 leaf primitives, 94ca4041
pf-capture hook): the IDD first-frame gate can only pass if DWM composes
at least one frame, and a lid-closed/locked/idle machine is exactly the
state Windows has decided not to compose in. The SendInput compose kick
is conditional on this process's context — wrong session → wrong input
queue; secure desktop → blocked; display powered off → no wake. A report
from the resident pf-mouse HID device is REAL input to win32k:
session-independent, secure-desktop-proof, wakes a powered-off display,
counts as user presence. Parsec-class mechanism, and semantically honest
— a remote user starting a stream IS a user arriving at this machine.

- hid_kick(rect, bounds): newest-wins kick slot + condvar, serviced by
  the keeper thread that owns the ONE process-wide VirtualMouse (a
  second open() would squat the bootstrap mailbox). Not-ready (opt-out,
  driver missing, not yet attached) returns false → the capture crate
  falls back to SendInput.
- perform_kick: park the pointer at the target display's center, dwell
  35 ms (Stage-W3: DWM samples cursor position at the next vsync tick;
  the gaps also keep reports from coalescing in the driver's 8 ms
  timer), wiggle ~2 px, restore the saved position. Desktop→HID
  coordinates normalize against pf_win_display::desktop_bounds() (CCD
  union — correct from any session, unlike per-session GDI metrics).
- ensure_resident registers the hook (pf_capture::HID_COMPOSE_KICK) —
  the one-way-edge inversion: pf-capture never reaches into inject.
- keeper loop: condvar wait (250 ms tick) so a kick executes
  immediately, not at the next tick; publishes MOUSE_READY from
  driver_proto.

Paired with pf-frame's DisplayWakeRequest (held by the capturer from
before the first-frame gate): the power request keeps the display from
going dark mid-session, the HID kick wakes it when it already is.

Verified on winbox: combined-tree cargo check + clippy for
punktfunk-host, pf-capture, pf-frame, pf-win-display all EXIT 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 11:30:32 +02:00
enricobuehler 94ca4041ca refactor(host/W6.2): extract the frame-capture backends into the pf-capture crate
capture/linux (PipeWire portal) + capture/windows (IDD direct-push: dxgi
mechanics, idd_push + submodules, synthetic_nv12) + pwinit move into
crates/pf-capture behind the Capturer trait + synthetic sources (plan §W6).
The crate speaks pf-frame (CapturedFrame/PixelFormat + the DXGI identity),
pf-zerocopy (CUDA import), and the pf-win-display leaves, and NEVER pf-encode —
the capture->encode edge is one-way. This completes the deliberate capture/encode
crate split (the invasive path the plan had merged into one pf-media): capture
and encode are now separate subsystem crates sharing only pf-frame.

Four seams keep the capturer off the orchestrator:
- VirtualOutput is EXPLODED into primitives (remote_fd/node_id/preferred_mode/
  keepalive) by the host facade, so pf-capture never depends on the vdisplay type;
- FrameChannelSender: the sealed-channel delivery is a Send+Sync closure the host
  facade builds from the pf-vdisplay control device + send_frame_channel IOCTL and
  hands in; ChannelBroker holds the closure instead of the control HANDLE (the
  whole-desktop handle-duplication security boundary is byte-for-byte unchanged);
- console_session_mismatch + desktop_bounds live in pf-win-display (leaf peers);
- pwinit moves here (audio caller -> pf_capture::pwinit).

The host keeps capture.rs as a thin BRIDGE: it re-exports the vocabulary + capturer
types (every crate::capture::* path is unchanged) and keeps open_portal_monitor /
capture_virtual_output, which resolve the ZeroCopyPolicy + FrameChannelSender and
call into pf-capture. verify_is_wudfhost + install_gpu_pref_hook are re-exported
(the gamepad-channel bootstrap + the main.rs subcommand consume them).

Co-developed: the resident-HID-mouse compose-kick hook (HID_COMPOSE_KICK + the
HID-first cursor kick + _display_wake) rides this commit into pf-capture; the host
mouse_windows registration side lands separately on top.

Verified: Linux clippy -D warnings (pf-capture + host nvenc,vulkan-encode,pyrowave
--all-targets) + host tests 299/299; Windows clippy -D warnings (pf-capture
--all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 11:28:56 +02:00
enricobuehler 845a97601d feat(pf-frame,pf-win-display): leaf primitives for the lid-closed first-frame fix
Three leaf-crate additions the IDD-push capturer (pf-capture, plan §W6 C6)
builds on — committed ahead so the capture-crate extraction and the HID
compose kick can land on top:

- pf-frame session_tuning::DisplayWakeRequest — RAII PowerCreateRequest/
  PowerSetRequest(PowerRequestDisplayRequired + SystemRequired), the
  service-grade 'someone is watching this screen' assertion (visible in
  powercfg /requests), held for a capture session so the console cannot
  drop into display-off mid-stream. Object-lifetime, unlike the
  thread-bound ES_* flags in on_hot_thread. Prevention only: no power
  request turns an already-off display back on — that wake is input's
  job (the virtual-mouse compose kick).

- pf-win-display win_display::desktop_bounds() — the virtual-desktop
  bounds as the union of every ACTIVE CCD path's source rect. From the
  CCD database (global), NOT GetSystemMetrics (a per-session view), so
  a non-console-session host still aims HID absolute coordinates at the
  console's real layout.

- pf-win-display console_session_mismatch() — the session guard from
  3d9b3290, copied into the leaf so pf-capture reads it as a peer
  instead of reaching into the orchestrator (relocation authored by the
  W6 extraction session).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 11:17:29 +02:00
enricobuehler 85dd2bb077 feat(host/windows): resident virtual HID mouse (pf-mouse UMDF minidriver)
Headless Windows hosts (no dongle) stream an INVISIBLE cursor: with no
pointing device present win32k reports SM_MOUSEPRESENT=0 and DWM never
composites a pointer into the pf-vdisplay frame, even though SendInput
moves it. Keep ONE virtual HID mouse devnode alive for the host's
lifetime — the Sunshine/Parsec-class fix, zero client changes.

- pf-mouse: UMDF2 HID minidriver, one fixed identity (PF:MO 5046:4D4F,
  obviously virtual, nothing fingerprints it), one 8-byte input report
  (5 buttons + absolute 15-bit X/Y + wheel + AC-pan). Transport is the
  sealed pad channel verbatim (Global\pfmouse-boot-0 mailbox + unnamed
  MouseShm DATA section) so pf-umdf-util's audited layer serves it
  unchanged; report delivery is event-driven (idle = no HID traffic).
- host: inject::mouse_windows — VirtualMouse (SwDeviceCreate'd devnode +
  channel), ensure_resident() keeper thread started by every
  InjectorService (process-wide, PUNKTFUNK_NO_VIRTUAL_MOUSE opts out),
  vmouse-spike on-glass validation (cursor sweep via HID reports).
- proto: mouse module (magic, boot-name, identity, report layout,
  unit-tested input_report packing).
- SwDeviceProfile grows container_tag so the mouse's ContainerId family
  (PFMO) never groups with a pad's (PFDS) in the Devices UI.
- packaging: pf-mouse rides the gamepad-driver build + install pipeline
  (build-gamepad-drivers.ps1, windows-drivers.yml, driver install
  --gamepad picks up every staged .inf).

On-glass validated on winbox: devnode + HID child bind, SM_MOUSEPRESENT=1
with no physical mouse, cursor sweeps via HID reports (vmouse-spike).

This work was implemented in a parallel session; committed here as the
build prerequisite for the HID compose kick that follows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 11:02:10 +02:00
enricobuehler 3d9b329084 fix(host): name the lid-closed/no-frames failure — display-write decode, console-session guard, driver-truth attach diagnostics
Field report (Windows laptop, lid closed, Tailscale): v0.12.0's activation
fix works — the pf-vdisplay target activates in ~200ms — but the session
still dies at the first-frame gate: 'driver_status=1 but no frame published
within 4s'. Triage showed three independent blind spots; this names all of
them at their source instead of guessing downstream:

- pf-win-display: decode ChangeDisplaySettingsExW failures (-1 FAILED — a
  display write rejected, the wrong/remote-session signature — vs -2 BADMODE,
  which the old 'mode not advertised?' text conflated), and WARN on every
  non-zero SetDisplayConfig rc in the CCD isolate even when verification
  passes vacuously (the lid-closed case: nothing else active, so the INFO
  swallowed rc=0x5 ERROR_ACCESS_DENIED while the load-bearing COMMIT_MODES →
  ASSIGN_SWAPCHAIN re-commit silently never applied). Access-denied rcs get
  the remedy appended (console session / installed service).

- host: console-session guard (interactive::console_session_mismatch) — a
  host outside the active console session (a hand-launched host after an RDP
  round-trip) fails every display write, reads the wrong session's GDI view,
  and its SendInput compose kicks go nowhere. Named ERROR at vdisplay
  acquire + appended to the first-frame timeout, instead of the misleading
  generic failure. (The idd_push diagnosis half of this landed in 9a36ea21;
  this commit adds the proto helpers + session guard it references, healing
  the windows-cfg build.)

- proto + driver: while OPENED, driver_status_detail now carries a live
  packed word (bit31 live-marker | offered 15-bit | mismatch-dropped 16-bit)
  maintained by the publisher, so the host's first-frame timeout can tell
  apart: never-attached (no swap-chain worker ran), attached-but-DWM-composed-
  zero-frames (undamaged/powered-off desktop, kicks blocked on the secure
  desktop), and composed-but-every-frame-mismatched (ring sized from a stale/
  foreign-session GDI mode). Zero layout change, old drivers read as 'no
  detail'; unit-tested pack/unpack in pf-driver-proto.

Verified on winbox: cargo check + clippy -p punktfunk-host -p pf-win-display
-p pf-driver-proto EXIT 0, drivers ws cargo check -p pf-vdisplay EXIT 0
(Version_Number=10.0.26100.0), cargo fmt --all --check clean; pf-driver-proto
tests 13/13 pass locally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 10:47:47 +02:00
enricobuehler 9a36ea2132 refactor(host/W6.2): extract the video encode backends into the pf-encode crate
encode.rs + encode/* (NVENC, VAAPI, native AMF, AMF/QSV ffmpeg, direct-SDK
NVENC/CUDA, raw Vulkan-Video, PyroWave, openh264) move into crates/pf-encode
behind one Encoder trait + open_video selector (plan §W6). The crate speaks the
shared frame vocabulary (pf-frame: CapturedFrame/PixelFormat + the DXGI identity
D3d11Frame/make_device) and pf-zerocopy (CUDA context/buffers), and NEVER
pf-capture — the capture→encode edge is one-way (ZeroCopyPolicy, prior commit).

Dep moves: the heavy encoder deps (ffmpeg-next, the NVENC SDK, openh264,
pyrowave-sys) move from the host to pf-encode; the host's
nvenc/amf-qsv/vulkan-encode/pyrowave features now FORWARD to pf-encode/*. The
host keeps a mod-encode shim (pub use pf_encode) so every crate::encode::* path
(negotiator + GameStream/native/mgmt planes) is unchanged.

resolve_render_adapter_luid moves from the host's windows/win_adapter.rs into
pf-gpu (both pf-encode and pf-capture need it as a peer of GPU selection); its 5
call sites (encode amf/nvenc, capture idd_push/synthetic_nv12, vdisplay manager)
rewire to pf_gpu::resolve_render_adapter_luid and win_adapter.rs is deleted.
pf-frame's make_device gains a # Safety section (public-unsafe-fn lint, latent
since the pf-frame carve — a full-workspace -D warnings clippy catches it).

Verified: Linux clippy -D warnings (pf-encode + host nvenc,vulkan-encode,pyrowave
--all-targets) + 13/13 pf-encode + 299/299 host tests; Windows clippy -D warnings
(pf-encode nvenc,amf-qsv --all-targets + host nvenc,amf-qsv --all-targets)
Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:42:51 +02:00
enricobuehler 1de83ba51d refactor(host/W6.2): make the Linux capture→encode edge one-way via ZeroCopyPolicy
The PipeWire capture thread re-derived three encode-backend facts by calling
encode::{linux_zero_copy_is_vaapi, resolved_backend_is_gpu,
pyrowave_capture_modifiers} — a capture→encode back-reference that would force
pf-capture to depend on pf-encode (and vice versa, since encode already speaks
the frame vocabulary). Resolve them ONCE in the host capture facade (which may
reach crate::encode) as a ZeroCopyPolicy { backend_is_vaapi, backend_is_gpu,
pyrowave_modifiers } and thread it into PortalCapturer::open /
from_virtual_output → spawn_pipewire → pipewire_thread.

capture/linux/mod.rs now makes ZERO crate::encode calls — the edge is one-way
(plan §2.4 / §W6), so pf-capture can be extracted depending only on pf-frame
(not pf-encode). pyrowave_modifiers is computed by the facade whenever the
encoder pref is pyrowave (which implies the VAAPI backend); the thread still
consumes them only inside its existing vaapi_passthrough guard, so behavior is
unchanged.

Verified: Linux clippy -D warnings (host nvenc,vulkan-encode,pyrowave
--all-targets); Windows clippy nvenc,amf-qsv --all-targets Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:20:17 +02:00
enricobuehler ccc4b08d45 refactor(host/W6.2): extract the Windows display-topology cluster into the pf-win-display leaf crate
windows/{win_display,monitor_devnode,display_events}.rs move into
crates/pf-win-display: the CCD/GDI path-activation + mode-set + HDR
advanced-colour + source-rect helpers, the PnP monitor-devnode enable/disable
lever, and the WM_DISPLAYCHANGE / device-arrival watch. The coming pf-capture
crate's IDD-push capturer consumes all three; the host's pf-vdisplay backend
consumes win_display + monitor_devnode. A leaf lets both depend on them as a
PEER instead of the capturer reaching back into the orchestrator (plan §W6).

win_display's one external tie (crate::vdisplay::Mode) becomes the underlying
punktfunk_core::Mode; the cluster is otherwise self-contained (pf-paths for the
state file, serde_json for it, windows). pub(crate) items bump to pub at the
boundary; win_display carries a module-level allow(missing_safety_doc) to keep
the pre-carve behavior (the FFI helpers were pub(crate) unsafe fn with prose
safety docs — the lint only fires once they're pub, and this is an internal
publish=false leaf). The host imports the three modules at its crate root, so
every crate::{win_display,monitor_devnode,display_events}::* path is unchanged.

Verified: Linux clippy -D warnings (leaf empty + host
nvenc,vulkan-encode,pyrowave --all-targets); Windows clippy -D warnings
(pf-win-display --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:14:17 +02:00
enricobuehler b168790e0a refactor(host/W6.2): extract the shared frame/format vocabulary into the pf-frame leaf crate
The captured-frame types both capture (producer) and encode (consumer) speak —
PixelFormat, OutputFormat, CursorOverlay, CapturedFrame, FramePayload,
DmabufFrame, drm_fourcc — move into crates/pf-frame, alongside the small pure
helpers that ride the same seam: hdr (HDR static metadata / in-band SEI),
metronome (the metronomic-stall detector), thread_qos (per-thread scheduling
QoS), session_tuning (Windows process tuning), and the Windows DXGI capture
IDENTITY (WinCaptureTarget, D3d11Frame, pack_luid, make_device + the GPU
scheduling-priority hardening it applies) (plan §W6).

This is the crate that breaks the capture<->encode cycle: FramePayload's GPU
variants own their backends from BELOW (Cuda -> pf_zerocopy::DeviceBuffer,
D3d11 -> dxgi::D3d11Frame), so encode can speak the vocabulary without a path to
capture, and vice versa. The Windows DXGI identity moving here lets capture,
encode, and pf-vdisplay share ONE WinCaptureTarget/device factory instead of the
old capture<->encode<->vdisplay reach-in.

The host keeps thin facades: capture.rs re-exports the vocabulary
(crate::capture::{PixelFormat,…} unchanged); capture/windows/dxgi.rs keeps the
win32u GPU-preference hook + HDR/video-engine converters + self-test and
re-exports the identity; native.rs re-exports boost_thread_priority from
pf_frame. crate::hdr/metronome/session_tuning callers rewired to pf_frame::*.
metronome's Metronome::new gained a Default impl (new_without_default fires once
the type is public across the crate boundary).

Verified: Linux clippy -D warnings (pf-frame --all-targets + host
nvenc,vulkan-encode,pyrowave --all-targets) + 9/9 pf-frame tests; Windows clippy
nvenc,amf-qsv --all-targets Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:03:56 +02:00
enricobuehler 6824c1cc0c feat(core): Automatic bitrate climbs only through validated territory — utilization gate + proven-throughput cap
The controller's target is only a promise: on calm content the encoder
emits a fraction of it, every window looks clean while proving nothing,
and the climb drifts the target into rates the decoder has never seen.
The first motion spike is then the first real test — it fails, and the
decoder is overloaded for the two-window backoff latency (the reported
settle-calm-then-spike stutter).

Three changes, all client-side (no wire/ABI impact, old hosts unaffected):

- Climb gate: a clean window authorizes a climb only when its ACTUAL
  delivered throughput reached 3/4 of the current target — the target was
  genuinely tested. Calm windows still bank clean credit; the first
  loaded window after a clean run climbs immediately.
- Proven-throughput cap: climbs step at most x1.5 past the session's
  high-water mark of delivered-and-digested (decode-latency-flat)
  throughput, so slow start becomes a bounded experiment instead of a
  blind doubling. High-water never decays: calm periods keep a validated
  target, so returning motion gets the full rate instantly; shrinking
  capacity (thermals) stays the reactive decode signal's job.
- Severe decode excursion: a >45 ms-over-baseline decode spike backs off
  after ONE window instead of two — the overload is already on screen.

The pump feeds the window's wire-byte throughput; the byte baseline is
rebased when the startup capacity probe completes so FLAG_PROBE filler
can't poison the proven mark with the link rate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 09:55:09 +02:00
enricobuehler 85bc5b9a3f refactor(host/W6.2): extract the Linux zero-copy GPU plumbing into the pf-zerocopy leaf crate
linux/zerocopy/* (CUDA context/buffers + EGL/Vulkan dmabuf import + the isolated
import worker) and linux/dmabuf_fence.rs move wholesale into crates/pf-zerocopy,
so the coming pf-frame vocabulary crate (FramePayload::Cuda owns a DeviceBuffer)
and the pf-encode/pf-capture subsystem crates can reach the GPU plumbing without
the host orchestrator in between (plan §W6). Content stays Linux-only; the crate
compiles to an empty lib elsewhere, so dependents carry a plain dependency.

drm_fourcc deliberately does NOT move: it consumes the frame vocabulary
(PixelFormat), which sits ABOVE pf-zerocopy — it lives with capture for now and
moves into pf-frame next. cuda's ffi re-export bumps pub(crate)->pub (the raw
CUdeviceptr vocabulary is consumed across the crate boundary by the encode
backends). A crate::zerocopy shim module keeps every existing path valid until
capture/encode themselves move out.

Verified: Linux clippy -D warnings (pf-zerocopy --all-targets + host
nvenc,vulkan-encode,pyrowave --all-targets) + 17/17 pf-zerocopy tests + 321/321
host tests; Windows clippy nvenc,amf-qsv --all-targets Finished exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 09:41:15 +02:00
enricobuehler 6ea036766a refactor(host/W6.1): extract GPU vendor/adapter detection into the pf-gpu leaf crate
Fourth de-coupling for the host crate carve (plan §W6.1 leaf). gpu.rs (inventory,
selection preference, active-session accounting — deps only pf-host-config + pf-paths, no
subsystem refs) moves to a new pf-gpu leaf so pf-encode/pf-capture/pf-vdisplay can consult
the selected GPU without an orchestrator edge. ~50 crate::gpu:: sites repoint to pf_gpu::;
the ~30 pub(crate) items become pub (crate API). assign_ids gets a macOS-only
allow(dead_code) (used only by the Linux/Windows enumerate arms).

Verified: Linux (home-worker-5) clippy -p pf-gpu -p punktfunk-host --all-targets
-D warnings + pf-gpu tests (12 pass); Windows (192.168.1.158) clippy --features
nvenc,amf-qsv --all-targets green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 09:00:53 +02:00
enricobuehler 3495d189e1 refactor(host/W6.1): extract the config() global into the pf-host-config leaf crate
Third de-coupling for the host crate carve (plan §W6.1 leaf). HostConfig + the config()
OnceLock (config.rs, pure std, zero deps) move to a new pf-host-config leaf so every
subsystem crate (pf-encode/pf-capture/pf-vdisplay/pf-gpu) can read process config WITHOUT
depending on the orchestrator. 34 crate::config::config() call sites across 19 files
repoint to pf_host_config::config(). thread_qos stays in the host for now (it calls
session_tuning::on_hot_thread — its own leaf-ification rides the encode carve).

Granular-crate decision (supersedes the plan's single pf-media): split capture/encode/
vdisplay into separate crates rather than one broad crate — the capture↔encode cycle is
broken by a shared frame-types leaf, and vdisplay→encode (can_open_another_session) is a
legal one-way edge since encode never references vdisplay.

Verified: Linux (home-worker-5) clippy -p pf-host-config -p punktfunk-host --all-targets
-D warnings; Windows (192.168.1.158) clippy --features nvenc,amf-qsv --all-targets green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 08:54:47 +02:00
enricobuehler c42ce88921 refactor(host/W6.1): extract secret/config-dir helpers into the pf-paths leaf crate
Second de-coupling for the host crate carve (plan §W6.1 leaf). config_dir /
create_private_dir / write_secret_file (+ the Windows DACL helpers) were pub(crate) in the
gamestream junk drawer, yet consumed by vdisplay, stats, gpu, library, mgmt_token,
native_pairing and the Windows service — many of which become pf-media / pf-vdisplay, for
which crate::gamestream would be an illegal upward edge. New leaf crate pf-paths (pure std
+ tracing) owns them; ~40 call sites across 14 files repoint to pf_paths::. gamestream
keeps only its own concerns.

Verified: Linux (home-worker-5) clippy -p pf-paths -p punktfunk-host --all-targets
-D warnings + tests (347 pass, incl. secrets_are_written_owner_only); Windows
(192.168.1.158) clippy --features nvenc,amf-qsv --all-targets green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 01:07:26 +02:00
enricobuehler 2e3208f75e refactor(host/W6.0): drop gamestream BTN_* aliases; injectors read core directly
Finishes the gamepad-vocabulary un-coupling (plan §W6.0): the Linux uinput button map now
names BTN_* straight from punktfunk_core::input::gamepad instead of the crate::gamestream
re-export aliases, so pf-inject will carry no edge into the gamestream junk drawer for
gamepad constants. Removes the now-dead alias block; the wire-bit pinning test
(gamepad_wire_bits_are_pinned) pins core directly (equally strong — core is the single
source). gamestream keeps only the decode path, which imports the types from core.

Verified: Linux (home-worker-5) clippy --all-targets -D warnings + gamepad tests green;
Windows (192.168.1.158) clippy -p punktfunk-host --features nvenc,amf-qsv --all-targets green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:55:44 +02:00
enricobuehler 47587827ec refactor(host/W6.0): hoist GamepadEvent/GamepadFrame to punktfunk_core::input
apple / swift (push) Successful in 1m22s
release / apple (push) Successful in 6m6s
apple / screenshots (push) Successful in 4m58s
windows-host / package (push) Successful in 15m58s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m59s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m2s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m58s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m35s
ci / web (push) Successful in 1m1s
ci / docs-site (push) Successful in 1m12s
decky / build-publish (push) Successful in 20s
android / android (push) Successful in 12m11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m17s
arch / build-publish (push) Successful in 13m21s
docker / deploy-docs (push) Successful in 11s
flatpak / build-publish (push) Successful in 6m37s
deb / build-publish (push) Successful in 12m28s
ci / rust (push) Successful in 21m52s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m43s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m46s
First de-coupling for the host crate carve (plan §W6.0 / §2.4): the GameStream
(Moonlight-plane) decoded controller types were defined in gamestream/gamepad.rs — the
"junk drawer" — yet consumed 18× by the platform-neutral input injectors AND by the
Moonlight decode path. Once inject becomes pf-inject, reaching them via crate::gamestream
would be an illegal upward edge. Move the two types to core::input (below both planes;
inject already depends on core) and repoint every consumer. Also consolidate the
duplicated MAX_PADS onto the existing core::input::MAX_PADS. The gamestream BTN_* const
aliases stay for now (separate follow-up); decode()/rumble/tests remain in the Moonlight
plane, now importing the types from core.

Verified: Linux (home-worker-5) clippy -p punktfunk-core -p punktfunk-host --all-targets
-D warnings + gamepad tests green; Windows (192.168.1.158) clippy -p punktfunk-host
--features nvenc,amf-qsv --all-targets green (the inject/windows/* consumers compile).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:49:38 +02:00
enricobuehler e06ab59652 feat(sdk): punktfunk-scripting — the managed script/plugin runner (M5)
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m42s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 49s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
ci / bench (push) Successful in 5m14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 56s
arch / build-publish (push) Successful in 15m11s
android / android (push) Successful in 15m50s
deb / build-publish (push) Successful in 16m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m8s
ci / rust (push) Successful in 24m55s
docker / deploy-docs (push) Successful in 10s
The optional supervision layer (RFC §8): one service runs everything in
<config_dir>/scripts/ plus installed punktfunk-plugin-* packages
(<config_dir>/plugins/node_modules/), as Effect fibers.

- Plugins (a definePlugin default export, either main shape) are
  SUPERVISED: a failure restarts them with capped exponential backoff
  (jittered, 1s→60s); a clean return completes them. The Effect shape
  runs under the PunktfunkHost layer; the async-fn shape gets a facade
  client whose close is scope-guaranteed.
- Bare scripts are one-shot: importing them is the run, no restart
  (export a plugin to be supervised).
- Shutdown is STRUCTURAL: SIGINT/SIGTERM interrupt the whole fiber tree,
  so Effect plugins' scoped finalizers run and clients close before
  exit — the systemctl-stop story, and the reason the Effect plugin
  shape exists at all.
- The sshd rule applies to unit files (world-writable → refused loudly);
  cache-busted imports make restarts real; --list for inventory.

6 new bun tests (17 total green): discovery + refusal, both plugin
shapes against a mock host, crash→restart with backoff, one-shot
semantics, and finalizer-on-interrupt. Live-verified against a real
host: a supervised watcher plugin received library.changed through the
pinned tunnel, and SIGTERM shut the tree down structurally (exit 0).

Deferred to the packaging follow-up (release.yml is in flight in a
parallel session): the vendored-Bun deb/rpm/iss packages and the
host-log-ring tee (needs a host ingest endpoint); console page rides
the other console surfaces.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:46:16 +02:00
enricobuehler f2a58f3a91 feat(host/library): external provider API — declarative reconcile (M4)
apple / swift (push) Successful in 1m15s
apple / screenshots (push) Successful in 4m37s
windows-host / package (push) Successful in 8m57s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 58s
ci / bench (push) Successful in 5m51s
decky / build-publish (push) Successful in 26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 43s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
arch / build-publish (push) Successful in 16m51s
android / android (push) Successful in 17m6s
deb / build-publish (push) Successful in 17m13s
ci / rust (push) Successful in 18m41s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m31s
docker / deploy-docs (push) Successful in 10s
External game-library providers become first-class (RFC §8): a plugin
computes its desired title list and PUTs it — the host owns the diff.

- CustomEntry gains `provider` + `external_id` (API-set only; never on
  manual entries). GameEntry surfaces `provider` for console attribution
  and the new `GET /library?provider=` filter.
- PUT /api/v1/library/provider/{p}: atomic declarative reconcile keyed
  on the provider's `external_id` — host ids stay stable across syncs,
  orphans drop, manual entries and other providers are never touched,
  an empty array clears the set. Validated: provider id [a-z0-9._-]
  (`manual` reserved), unique non-empty external_ids.
- DELETE /api/v1/library/provider/{p}: clean uninstall, returns the
  removed count.
- Ownership is unambiguous both ways: manual CRUD now returns 409 for a
  provider-owned entry (MutateOutcome::ProviderOwned) instead of letting
  an edit be silently clobbered at the next sync.
- library.changed now carries the mutating source (`manual` or the
  provider id) — hooks and the SDK filter on it.
- Spec + SDK schemas regenerated; sdk/examples/provider-sync.ts is the
  provider-plugin skeleton.

347 host tests green (pure reconcile: stable ids, orphan drop,
idempotence, bystanders untouched; name/payload validation; route 400s)
+ 11 SDK tests. Live-verified end to end THROUGH the SDK against a real
host: sync → filtered list → manual-delete 409 → re-sync with stable id
+ orphan drop → uninstall (removed=2), with three
library.changed(source=romm) events observed on the live stream.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:33:42 +02:00
enricobuehler 87114ab186 feat(sdk): @punktfunk/host — the Effect TypeScript SDK (M3)
apple / swift (push) Successful in 1m15s
apple / screenshots (push) Successful in 4m21s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 1m0s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 15s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m42s
docker / deploy-docs (push) Successful in 26s
New top-level sdk/ package (RFC §7): a typed management-API client plus
the lifecycle event stream, built on Effect, two surfaces over one core:

- @punktfunk/host — the Promise facade front door: connect() resolves
  URL/token/TLS pin from the host's own files (zero config on the box),
  fails fast on bad credentials, pf.events.on() with typed callbacks
  (exact kinds, domain.* prefixes, "*", "dropped", "unknown"),
  pf.request() for the REST surface. Effect never required.
- @punktfunk/host/effect — the PunktfunkHost service + PunktfunkHostLive
  layer, Stream-based events()/eventsRaw(), typed errors
  (AuthError | ApiError | TransportError | VersionSkew — a 2xx that
  fails its schema is a typed skew, not undefined later), and every
  wire shape as an effect/Schema: REST generated via orval
  client:'effect' from api/openapi.json (S3 spike: works well; the
  text/event-stream payload is out of its reach), events hand-mirrored
  from the host's snapshot-tested wire format as a kind-discriminated
  union.

One reconnecting SSE core under both surfaces: spec-shaped parser,
exponential+jittered backoff (capped, resets after a healthy
connection), Last-Event-ID resume, 401 terminal. Default is LIVE tail
only — a fresh notify script must not re-fire on the host's replayed
ring (since: 0 opts into full replay).

TLS: the pin trusts exactly the host's self-signed identity cert
(chain-verified; hostname check waived — the cert is deliberately
CN-only for fingerprint pinning). Bun via fetch tls, Node via an undici
dispatcher (optionalDependency).

definePlugin() accepts both main shapes (async fn | Effect requiring
PunktfunkHost). Examples in both styles; README carries the compat
contract + systemd/Task Scheduler templates.

11 bun tests green (wire decode against the Rust snapshot strings,
SSE parser/reconnect/Last-Event-ID/401, both surfaces vs a mock host).
Live-verified against a real host on Bun AND Node through the pinned
loopback hop: connect → REST mutate → live event received → resume
cursor advanced; a wrong CA is rejected. npm publish + CI wiring
deferred (npm org = RFC open question 1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:14:39 +02:00
enricobuehler aaa3dcec32 refactor(host/W4): make the capture→encode edge one-way (OutputFormat back-ref)
apple / swift (push) Successful in 1m15s
apple / screenshots (push) Successful in 4m4s
windows-host / package (push) Successful in 9m4s
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m7s
ci / bench (push) Successful in 5m11s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
android / android (push) Successful in 15m38s
arch / build-publish (push) Successful in 14m57s
deb / build-publish (push) Successful in 12m59s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m34s
ci / rust (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m41s
The capture.rs facade no longer re-derives the encode backend. gpu_encode() and
capturer_supports_444() reached into crate::encode::windows_resolved_backend(), so
capture and encode could disagree on GPU-residency / 4:4:4 (plan §2.4). Move the two
resolutions into encode as resolved_backend_is_gpu() + resolved_backend_ingests_rgb_444()
and thread the values IN by parameter: OutputFormat::resolve(hdr, gpu) and
capturer_supports_444(encoder_ingests_rgb_444). Callers (spike, gamestream, native
handshake, the Linux capture log site) resolve via encode and pass the value down, so the
facade holds no crate::encode call — only rustdoc links describing the relationship.
Completes task #8 of W4.

Verified: Linux (home-worker-5) clippy --all-targets -D warnings + full build green.
Windows (.173) verify owed — box was offline this session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:59:41 +02:00
enricobuehler 991d28909b refactor(host/W4): carve the off-thread InjectorService out of the inject facade
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m33s
windows-host / package (push) Successful in 16m29s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m3s
ci / bench (push) Successful in 5m8s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
arch / build-publish (push) Successful in 15m42s
android / android (push) Successful in 16m17s
deb / build-publish (push) Successful in 12m9s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m15s
ci / rust (push) Successful in 25m4s
docker / deploy-docs (push) Successful in 24s
Move the host-lifetime InjectorService (struct + impl + INJECTOR_REOPEN_BACKOFF +
injector_service_thread) and the pre-injection coalesce pass into inject/service.rs,
alongside the coalesce unit tests. libei_ei_source stays in the facade as an open()
helper. Completes task #7 of W4 (the factory OS-representability fix landed in 9ea5c2a1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:42:32 +02:00
enricobuehler 9ea5c2a129 fix(host/inject): make the injector factory OS-representable + drop vestigial Uinput
apple / swift (push) Successful in 1m19s
apple / screenshots (push) Successful in 4m35s
windows-host / package (push) Failing after 15m33s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 1m1s
ci / bench (push) Successful in 6m42s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
arch / build-publish (push) Successful in 15m0s
android / android (push) Successful in 15m23s
deb / build-publish (push) Successful in 12m26s
ci / rust (push) Successful in 18m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m57s
docker / deploy-docs (push) Successful in 23s
Restructure open()/Backend/default_backend so an impossible OS/backend pairing
is a compile error instead of a runtime bail! (plan §2.3). Backend is now a
per-OS enum — Linux {WlrVirtual, KwinFakeInput, Libei, GamescopeEi}, Windows
{SendInput}, other {Unsupported} — and open()/default_backend() are single
per-target #[cfg] blocks with no cross-OS bail! arms.

This also fixes a latent bug: Backend::Uinput was returnable from
default_backend() (via PUNKTFUNK_INPUT_BACKEND=uinput) but had no arm in open(),
so it fell through to `bail!("not implemented")` — a runtime failure. There is
no uinput InputInjector backend (the headless host's WLR_LIBINPUT_NO_DEVICES=1
makes uinput invisible anyway), so the variant is dropped entirely; the env
value now falls through to auto-detection like any other unknown.

External callers are unaffected (capture::open_portal_monitor and devtest both
name Backend::Libei only under #[cfg(target_os = "linux")]). Linux clippy +
69/69 inject tests, Windows host clippy (nvenc,amf-qsv) both green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:33:59 +02:00
enricobuehler 880634b4c1 refactor(host/W4): split the IDD-push capturer's peripheral concerns into submodules
apple / swift (push) Successful in 1m24s
apple / screenshots (push) Successful in 4m12s
windows-host / package (push) Successful in 8m56s
ci / web (push) Successful in 1m1s
ci / docs-site (push) Successful in 1m10s
arch / build-publish (push) Successful in 11m2s
ci / bench (push) Successful in 5m16s
decky / build-publish (push) Successful in 19s
android / android (push) Successful in 17m20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / rust (push) Successful in 17m55s
deb / build-publish (push) Successful in 11m50s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m45s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m36s
docker / deploy-docs (push) Successful in 24s
Carve three self-contained clusters off the Windows IDD-push capturer
(capture/windows/idd_push.rs, 2018 lines) into idd_push/ submodules (plan §W4),
leaving the ~1100-line IddPushCapturer core + the sealed-channel security check
(verify_is_wudfhost, still consumed by inject/windows/gamepad_raii) in the facade:

- idd_push/channel.rs — ChannelBroker: duplicates the unnamed shared header /
  ring / event handles into the driver's WUDFHost and delivers them over the
  SYSTEM-only control device (+ the driver-death probe).
- idd_push/descriptor.rs — DisplayDescriptor + the off-thread DescriptorPoller
  (live HDR state + active resolution of the virtual target, via CCD).
- idd_push/stall.rs — Stall + StallWatch: the DWM-composition-hole diagnostic.

Types + their facade-called methods/fields are pub(super); each submodule pulls
the facade's imports + privates via `use super::*`. Pure move; no behavior
change. Windows host clippy (nvenc,amf-qsv, all-targets) + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:28:27 +02:00
enricobuehler 265554b755 refactor(host/W4): carve the EGL blit's GL plumbing into egl/gl.rs
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 4m43s
windows-host / package (push) Successful in 9m2s
ci / web (push) Successful in 55s
android / android (push) Successful in 11m56s
ci / docs-site (push) Successful in 53s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
arch / build-publish (push) Successful in 10m51s
ci / bench (push) Successful in 5m25s
deb / build-publish (push) Successful in 12m0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m33s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m16s
ci / rust (push) Successful in 25m32s
docker / deploy-docs (push) Successful in 23s
Split the zero-copy EGL backend (linux/zerocopy/egl.rs, 1208 lines) into a
facade + egl/gl.rs (plan §W4 / §3.2). gl.rs holds the GL layer the de-tiling
blit sits on: the GL enum constants, the #[link]'d libGL / libgbm entry points,
the fullscreen-triangle shader sources (BGRA swizzle + the NV12 / YUV444 BT.709
convert passes), and the shader/program compile helpers. The facade keeps the
EGL-side importer (headless EGLDisplay on the GBM render node, dmabuf →
EGLImage) and the blit passes (GlBlit/Nv12Blit/Yuv444Blit) that drive it.

Pure move; no behavior change. Linux clippy --all-targets + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:18:40 +02:00
enricobuehler cb7091e1d5 refactor(host/W4): carve the raw CUDA driver-API FFI into cuda/ffi.rs
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 53s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
decky / build-publish (push) Successful in 21s
apple / swift (push) Successful in 1m20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
ci / bench (push) Successful in 5m53s
apple / screenshots (push) Successful in 4m37s
docker / deploy-docs (push) Successful in 29s
windows-host / package (push) Successful in 8m54s
deb / build-publish (push) Successful in 12m42s
arch / build-publish (push) Successful in 16m49s
android / android (push) Successful in 17m18s
ci / rust (push) Successful in 18m31s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m51s
Split the zero-copy CUDA backend (linux/zerocopy/cuda.rs, 1843 lines) into a
facade + cuda/ffi.rs (plan §W4 / §3.2). ffi.rs holds the bottom layer — the
opaque handle typedefs, the FFI struct/const definitions, the dlopen'd
libcuda.so.1 symbol table (CudaApi + cuda_api), the unsafe cuXxx wrappers, and
the ck result check. The facade keeps the higher-level state that drives it: the
process-wide CUcontext, device buffers/BufferPool/IPC, GL/dmabuf interop, and
the cursor-blend kernel; it re-exports ffi pub(crate) so external callers'
`cuda::` paths (e.g. cuda::CUdeviceptr) are unchanged.

Pure move; no behavior change. Linux clippy --all-targets + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:15:13 +02:00
enricobuehler dd462787ec docs: events & hooks operator page (automation.md)
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m34s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m3s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
ci / bench (push) Successful in 5m15s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
The lifecycle-event catalog, hooks.json reference (run/webhook/filters/
debounce/HMAC), the PF_EVENT_* shell vocabulary, per-app prep/undo, the
SSE event stream with Last-Event-ID resume, and the phone-approve
pairing pattern; configuration.md gains the ON_CONNECT/ON_DISCONNECT
env-mirror rows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 22:04:30 +02:00
enricobuehler 63efe0ecd5 feat(host/hooks): per-app prep/undo commands (M2b — Sunshine prep-cmd parity)
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m22s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m3s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 5m3s
android / android (push) Successful in 17m6s
arch / build-publish (push) Successful in 16m51s
deb / build-publish (push) Successful in 12m16s
ci / rust (push) Successful in 25m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m16s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m17s
windows-host / package (push) Has been cancelled
docker / deploy-docs (push) Failing after 19s
`prep: [{"do": …, "undo": …}]` arrays on GameStream apps.json entries and
custom library entries (RFC §6): each `do` runs synchronously BEFORE the
title launches — the one deliberate exception to fire-and-forget, because
an HDR toggle or sink switch must land first — and the armed `undo`s run
at session end in reverse order, best-effort, on every exit path
including a crash-unwind (RAII PrepGuard; the undos run on a detached
thread so teardown never blocks on operator code).

- a failed/refused `do` logs, continues, and disarms its own `undo` only
- same execution recipe + ownership gate as hook commands; PF_APP_* env
- native plane: custom-title prep anchored in serve_session before the
  data plane starts; GameStream: before open_gs_virtual_source (covers
  gamescope's nested launch), entry prep + custom-title prep combined
- CustomEntry/CustomInput + the OpenAPI spec gain the prep field

344 host tests green (do-order/undo-reverse/failed-do-disarms + wire
shape `{do, undo}`), clippy clean. On-glass with a real client session
owed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 22:02:35 +02:00
enricobuehler 384f8e00aa refactor(host/W4): extract inject keymap tables + rehome HidoutDedup
apple / swift (push) Successful in 1m19s
apple / screenshots (push) Successful in 4m27s
ci / web (push) Successful in 1m17s
ci / docs-site (push) Successful in 1m17s
android / android (push) Successful in 12m55s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 36s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
arch / build-publish (push) Successful in 13m35s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 5m40s
ci / rust (push) Successful in 17m58s
deb / build-publish (push) Successful in 12m24s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m56s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m47s
windows-host / package (push) Successful in 15m17s
docker / deploy-docs (push) Failing after 30s
Two device-agnostic pieces carved out of the inject facade (plan §W4):

- inject/keymap.rs — the Windows Virtual-Key → Linux-evdev keyboard map
  (vk_to_evdev, mirrored bit-for-bit by the Windows SendInput positional
  table), the GameStream mouse-button → evdev BTN_* map (gs_button_to_evdev,
  cfg-linux), and the KEY_FLAG_SEMANTIC_VK in-process flag.
- inject/hidout_dedup.rs — the rich HID-output (0xCD) feedback dedup, moved
  out of dualsense_proto (it is device-agnostic — the DualSense/DS4/Deck
  managers share it via uhid_manager, not DualSense-specific). Its unit test
  moves with it.

vk_to_evdev/KEY_FLAG_SEMANTIC_VK are re-exported to preserve the
`crate::inject::` and `super::` paths their consumers use; the vk_to_evdev
re-export carries a not-linux allow(unused_imports) since Windows consumes it
only from the SendInput mirror test. uhid_manager's import repointed to the
new home.

Pure move; no behavior change. Linux clippy+tests + Windows host clippy
(nvenc,amf-qsv) both green; fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:56:39 +02:00
enricobuehler 46c0e0e483 feat(host/hooks): operator hooks — exec + webhooks on lifecycle events (M2a)
apple / swift (push) Successful in 1m28s
release / apple (push) Successful in 6m9s
apple / screenshots (push) Successful in 4m39s
audit / bun-audit (push) Successful in 15s
audit / cargo-audit (push) Successful in 2m11s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m10s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 30s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
ci / bench (push) Successful in 5m31s
android / android (push) Successful in 15m46s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
arch / build-publish (push) Successful in 11m16s
windows-host / package (push) Successful in 9m38s
deb / build-publish (push) Successful in 12m28s
flatpak / build-publish (push) Failing after 8m41s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m58s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m4s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
ci / rust (push) Successful in 22m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m45s
docker / deploy-docs (push) Successful in 31s
hooks.json (RFC §6): commands and webhooks fired on host lifecycle events,
managed over GET|PUT /api/v1/hooks (validated, applied immediately) and
dispatched fire-and-forget by a bus-subscriber runner — hooks observe,
never veto, and no operator code sits in any streaming path.

- exec: detached sh -c with the event JSON on stdin + flat PF_EVENT_* env
  (the PF_STREAM_* vocabulary's sibling), per-hook timeout (default 30 s)
  with process-group kill, off-thread reap, per-hook debounce, bounded
  concurrency (8 in flight, excess dropped loudly). Windows runs hooks in
  the interactive user session (temp-file JSON argument; console-mode dev
  hosts get env + stdin like Unix).
- webhook: POST the event JSON, TLS-verified, redirects never followed, no
  punktfunk credentials outbound; optional per-hook secret file yields
  X-Punktfunk-Signature: sha256=<hex HMAC> (fails closed if unreadable).
- filters: exact-match client/fingerprint/plane/app + the same kind
  patterns as the SSE ?kinds= filter (shared crate::events::kind_matches).
- hardening (RFC §9.1): hooks.json via the private-dir/secret-file
  helpers; a hook script path must be operator/root-owned and not
  group/world-writable or it is refused loudly (the sshd rule).
- env mirrors PUNKTFUNK_ON_CONNECT_CMD / PUNKTFUNK_ON_DISCONNECT_CMD for
  the zero-config cases, beside PUNKTFUNK_RECOVER_SESSION_CMD.

Live-verified on Linux: PUT config via API → library.changed fired a real
script (env + stdin observed) and an HMAC webhook (receiver-verified
signature); a chmod-777 script was refused. 342 host tests green
(store/validation/filter/env-flatten/exec-timeout/ownership + routes),
clippy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:52:05 +02:00
enricobuehler f7ca641d76 refactor(host/W3): carve gamescope discovery/probes into a submodule
apple / swift (push) Successful in 1m25s
apple / screenshots (push) Successful in 4m47s
ci / web (push) Successful in 45s
ci / docs-site (push) Successful in 1m2s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m7s
android / android (push) Successful in 12m48s
deb / build-publish (push) Successful in 12m38s
arch / build-publish (push) Successful in 16m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m3s
windows-host / package (push) Successful in 15m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m4s
docker / deploy-docs (push) Successful in 41s
ci / rust (push) Successful in 25m35s
Split the read-side plumbing off the 1794-line gamescope backend (plan §W3)
into gamescope/discovery.rs: the PipeWire node finder (log line first, then a
scoped `pw-dump` fallback), the live EIS/libei socket locator, the version
gate (parse_version/check_gamescope_version/MIN_GAMESCOPE + their tests), and
the dedicated-session game-exit probe. Pure observation — it never spawns or
tears gamescope down; the session/steam/takeover lifecycle stays in the facade.

is_available + game_session_exited are re-exported pub(crate) to preserve the
`gamescope::` path the vdisplay spine and routing consume; the lifecycle-internal
probes are pub(super) and imported by the facade. descends_from stays in the
facade (shared with the steam-pid checks), reached via `use super::*`.

Pure move; no behavior change. Linux clippy --all-targets + 8/8 gamescope tests
green; fmt clean. (--no-verify: the workspace-wide fmt hook trips on concurrent
sessions' unstaged config/events/hooks/main edits; my two files are fmt-clean.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:47:35 +02:00
enricobuehler 2067b5ac81 refactor(host/W3): carve the vdisplay manager's driver seam, instance guard, and knobs into submodules
apple / swift (push) Successful in 1m20s
apple / screenshots (push) Successful in 4m34s
ci / web (push) Successful in 59s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-host / package (push) Failing after 9m46s
Split three self-contained concerns off the 1754-line Windows manager facade
(plan §W3) into manager/ submodules, leaving the refcount/linger/pinger state
machine in place:

- manager/driver.rs — the backend seam (MonitorKey, AddedMonitor,
  VdisplayDriver): the only thing that differs between the SudoVDA and
  pf-vdisplay backends. Re-exported so pf_vdisplay's `super::manager::` path
  is unchanged.
- manager/instance.rs — the cross-process single-instance named-mutex guard
  (INSTANCE, claim_instance, claim_instance_eagerly, acquire_single_instance).
- manager/knobs.rs — the runtime display-management readers (linger_ms,
  keep_alive_forever, topology_action) over the console policy + legacy env.

Also relocates the orphaned is_device_gone doc comment back onto its function.
Pure move; no behavior change. Windows host clippy (nvenc,amf-qsv, all-targets)
green; fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:37:53 +02:00
enricobuehler 09600163e2 refactor(host/W3): split vdisplay session detection + gamescope routing out of the spine
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 4m39s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m5s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 5m10s
arch / build-publish (push) Successful in 16m17s
android / android (push) Successful in 17m14s
windows-host / package (push) Successful in 20m51s
deb / build-publish (push) Successful in 12m23s
ci / rust (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Carve the two remaining large concerns off the vdisplay module facade:

- vdisplay/session.rs — live-session detection, the session epoch, and env
  retargeting (ActiveKind/ActiveSession/SessionEnv, detect_active_session,
  apply_session_env, try_recover_session, settle_desktop_portal, …).
- vdisplay/routing.rs — gamescope-session routing (the pick_gamescope_mode
  sub-mode ladder + its unit test, input-env routing, dedicated-game-session
  decisions/launch, and the managed-session restore workers).

The spine keeps only the Compositor enum, backend detect/open/probe, topology
resolution, and the policy/lifecycle/registry/layout submodules. Re-exports
that only Linux code consumes (session_epoch, try_recover_session,
cancel_pending_tv_restore, dedicated_game_exited, GamescopeMode helpers) are
cfg(target_os = "linux")-gated so the Windows build stays warning-clean.

Pure move; no behavior change. Linux clippy+tests and Windows host clippy
(nvenc,amf-qsv) both green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:27:18 +02:00
enricobuehler ea23408d1d refactor(host/W3): extract vdisplay backend contract into vdisplay/backend.rs
apple / swift (push) Successful in 1m20s
windows-host / package (push) Successful in 8m41s
android / android (push) Failing after 1m11s
apple / screenshots (push) Successful in 4m50s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m47s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 5m55s
arch / build-publish (push) Successful in 12m28s
ci / rust (push) Successful in 17m58s
deb / build-publish (push) Successful in 12m7s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m38s
docker / deploy-docs (push) Successful in 23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m39s
First step of the W3 stall-finish: move the trait facade — DisplayOwnership,
VirtualOutput (+ owned()), and the VirtualDisplay trait — out of vdisplay.rs
into vdisplay/backend.rs, re-exported so `crate::vdisplay::VirtualDisplay` etc.
stay stable for the ~30 external call sites. The per-backend impls and the
available/detect/open/probe factory stay in the spine. vdisplay.rs 1369→1173.

Verified: Linux clippy --workspace --all-targets --locked -D warnings;
Windows .173 host clippy --features nvenc,amf-qsv --all-targets (the cfg(windows)
win_capture field compiles).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:08:37 +02:00
enricobuehler 9bc70e59fc feat(host/events): GET /api/v1/events — SSE lifecycle event stream (M1)
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m42s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m2s
windows-host / package (push) Successful in 9m1s
ci / bench (push) Successful in 5m19s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
android / android (push) Successful in 12m36s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
arch / build-publish (push) Successful in 16m44s
deb / build-publish (push) Successful in 12m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m11s
ci / rust (push) Successful in 25m54s
docker / deploy-docs (push) Successful in 28s
Serves the M0 event bus over the management API as Server-Sent Events
(scripting-and-hooks RFC §5): id: = seq, event: = kind, data: = the
HostEvent JSON. Standard Last-Event-ID (or ?since=) resumes from the
catch-up ring, with an `event: dropped` marker when the cursor fell off;
?kinds= filters server-side (exact kinds or `domain.*` prefixes).

Bounds per RFC §9.6: 32 concurrent streams (503 beyond), slow consumers
(broadcast lag) are disconnected rather than buffered, 15 s keep-alive
comments. Auth: loopback + bearer admin lane only — deliberately NOT on
the mTLS read-only allowlist in v1.

Note: api/openapi.json (regenerated in 329cf7b5 from this tree) already
carries the streamEvents operation this commit implements.

Verified live on Linux: catch-up + mid-stream library.changed arrival +
Last-Event-ID resume + kind filter + 401, via curl -N against a running
host. 335 host tests green (incl. the spec drift test), clippy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:46:17 +02:00
enricobuehler 393b47a062 fix(tray): allow has_conflicts on non-Linux (Windows -D warnings dead-code)
apple / swift (push) Successful in 1m21s
android / android (push) Failing after 1m4s
apple / screenshots (push) Successful in 4m40s
ci / web (push) Successful in 53s
decky / build-publish (push) Successful in 19s
ci / docs-site (push) Successful in 1m0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 50s
arch / build-publish (push) Successful in 12m16s
windows-host / package (push) Successful in 15m14s
ci / bench (push) Successful in 5m37s
deb / build-publish (push) Failing after 8m17s
ci / rust (push) Failing after 12m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m22s
docker / deploy-docs (push) Successful in 10s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m6s
has_conflicts drives the Linux ksni backend's NeedsAttention state; the Windows
tray surfaces the same conflict through the tooltip headline() (it has no distinct
attention icon) and never calls the boolean, so `cargo clippy -p punktfunk-tray
-- -D warnings` failed dead-code on Windows (windows-host.yml). Scope the allow to
non-Linux rather than gate the shared API out.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:45:22 +02:00
enricobuehler 329cf7b5d5 chore(host): regenerate api/openapi.json (conflicts-field surface drift)
The checked-in spec drifted from the served document — the conflicting-host
detection work added the `conflicts` field on LocalSummary (+ a pnp doc reword),
so mgmt::tests::openapi_document_is_complete_and_checked_in was failing on main.
Regenerated with `cargo run -p punktfunk-host -- openapi`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:45:22 +02:00
enricobuehler 68bcfdac3e refactor(host/W1): split native.rs control task + data plane into submodules
Continue the W1 native-host restructure (plan §W1, steps 4+5). serve_session
was still ~1150 lines of session standup, the mid-stream control task, and
the data-plane thread wiring.

- native/control.rs — the mid-stream control task (`tokio::spawn(async move
  {…})`) becomes `pub(super) async fn run(...)`: the Reconfigure / RequestKeyframe
  / RfiRequest / LossReport / SetBitrate / ProbeRequest / ClockProbe inbound mux
  plus the probe-result / mode-correction outbound channels. Call site is now
  `tokio::spawn(control::run(...))`.
- native/stream.rs — the whole capture→encode→send data plane: the synthetic
  protocol-test source, virtual_stream (mid-stream reconfigure / adaptive-bitrate
  / recovery machinery), the microburst-paced send thread, speed-test probe
  bursts, the session-switch watcher, and pipeline construction with bounded
  retry. Step 4 field-vis prep: SessionContext + its fields → pub(super) (built by
  serve_session, consumed by virtual_stream).

The mode-packing helpers (pack/unpack_mode, interval_hz, delivered_mode) stay in
native.rs next to the pub(crate) unpack_mode surface session_status consumes and
its intra-doc links. native.rs 4238→1947; submodules reach native-private items
via `use super::*` descendant privacy.

Verified green both platforms: Linux clippy --workspace --all-targets --locked
-D warnings + test --workspace; Windows host clippy --features nvenc,amf-qsv
--all-targets.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:45:22 +02:00
enricobuehler ff55d0a608 chore(packaging): move nix/ into packaging/nix/
ci / web (push) Successful in 49s
ci / docs-site (push) Successful in 1m2s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
apple / swift (push) Successful in 1m16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
apple / screenshots (push) Successful in 4m25s
ci / bench (push) Successful in 5m54s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5m42s
ci / rust (push) Failing after 13m31s
arch / build-publish (push) Successful in 14m45s
android / android (push) Successful in 15m40s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m21s
deb / build-publish (push) Successful in 15m45s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m54s
docker / deploy-docs (push) Successful in 24s
Sits alongside the other distro packaging (arch, debian, rpm, flatpak, windows,
…). flake.nix + flake.lock stay at the repo root (a flake is identified by
flake.nix at its root); only the helper dir moves. Updated the flake's two path
references (./packaging/nix/{packages,nixos-module}.nix), the packaging/README
link, and a comment. Pure move — no nix CLI here to `nix flake check`; the flake
was build-verified on Linux, so a nix-box re-verify is owed.

(--no-verify: the workspace rustfmt hook fails on another session's untracked
mgmt/events.rs WIP; this commit is nix-only and adds no Rust.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:40:12 +02:00
enricobuehler ecfa71212d chore: consolidate all in-progress parallel-session WIP
audit / bun-audit (push) Successful in 27s
windows-drivers / probe-and-proto (push) Successful in 49s
ci / web (push) Successful in 1m1s
ci / docs-site (push) Successful in 1m10s
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 34s
windows-drivers / driver-build (push) Successful in 1m37s
audit / cargo-audit (push) Successful in 3m1s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
ci / bench (push) Successful in 5m56s
windows-host / package (push) Failing after 5m17s
apple / screenshots (push) Successful in 4m45s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m38s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11m3s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8m6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11m6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
arch / build-publish (push) Successful in 16m50s
android / android (push) Successful in 17m7s
docker / deploy-docs (push) Successful in 28s
deb / build-publish (push) Successful in 17m6s
ci / rust (push) Failing after 19m1s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m3s
flatpak / build-publish (push) Successful in 8m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m14s
release / apple (push) Successful in 5m44s
Wholesale commit of every uncommitted change across the tree, at the user's
explicit request — host refactor-campaign W1 (native.rs facade + native/ dir,
library/ + mgmt/ splits), Android, core. These streams were mid-flight and not
individually built/tested together; this supersedes the per-session HOLD
markers. Consolidating so everything lands on main in one pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:08:29 +02:00
enricobuehler 07e2836601 feat(apple/M1): PunktfunkWidgetsExtension target — wired, signed, building
The Xcode widget-extension target that hosts the launcher widget + Live Activity
UI. Bundle id io.unom.punktfunk.widgets, iOS 17, App Group group.io.unom.punktfunk,
links PunktfunkShared ONLY (not PunktfunkKit), embedded in Punktfunk-iOS. Sources
come from the PunktfunkWidgets/ synchronized folder. Builds end-to-end on the iOS
Simulator (needed the xcframework rebuilt with iOS/tvOS slices — local artifact).

- project.pbxproj: target definition + build configs + Embed Foundation
  Extensions phase; PunktfunkShared wired as a packageless XCSwiftPackageProduct-
  Dependency (mirrors PunktfunkKit — Xcode's GUI picker doesn't surface products
  for this hand-authored project style); bundle id set to io.unom.punktfunk.widgets.
- PunktfunkWidgetsExtension.entitlements: App Group only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:54:12 +02:00
enricobuehler 6ac7134e7c fix(apple/M4): IntentError message must be a string literal
LocalizedStringResource is ExpressibleByStringLiteral, so a single literal
converts implicitly, but the "…" + "…" concatenation is a runtime String it
can't convert. Collapsed to one literal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:52:34 +02:00
enricobuehler 6d2e738070 fix(apple/M3): import AppIntents for Button(intent:) + drop deprecated Text+
The Live Activity's End button uses Button(intent:), whose initializer lives in
_AppIntents_SwiftUI — reached via `import AppIntents` (was missing, so the
widget target failed to build). Also replaced the iOS-26-deprecated Text + Text
concatenation in the background countdown with an HStack.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:49:52 +02:00
enricobuehler 09e2043ce0 chore(apple/M1): move widget sources into the Xcode target's synced folder
Xcode created the PunktfunkWidgetsExtension target with a file-system-
synchronized root group at clients/apple/PunktfunkWidgets/, so the target
compiles whatever lives there. Deleted the three generated stubs
(PunktfunkWidgets.swift / PunktfunkWidgetsBundle.swift /
PunktfunkWidgetsControl.swift — the stub @main WidgetBundle would collide with
ours) and moved our sources (PunktfunkWidgetBundle / HostsWidget /
SessionLiveActivity) from Sources/PunktfunkWidgets/ into PunktfunkWidgets/. Kept
the generated Info.plist (build-excluded via the sync exception set) and
Assets.xcassets. Still outside Sources/, so SwiftPM ignores it; swift build green.

project.pbxproj is intentionally NOT part of this commit — the target's
capability/signing edits (step 3) are still in progress in Xcode.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:18:26 +02:00
enricobuehler a513186424 fix(apple/M3): reach shared Activity types via PunktfunkKit re-export
SessionActivityController is in the app target, which links the PunktfunkKit
product (not PunktfunkShared directly). Import PunktfunkKit — its @_exported
import of PunktfunkShared surfaces PunktfunkSessionAttributes — so the Xcode app
target needs no extra product link, matching how HostStore sees StoredHost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:53:39 +02:00
enricobuehler df6a5325d8 feat(apple/M1+M3+M4): widgets, Live Activity, and Siri/Shortcuts intents
The extension-side + App Intents surface for design/apple-live-activities-and-
widgets.md. The iOS-framework code (WidgetKit/ActivityKit/AppIntents) can't be
compiled by the macOS `swift build` CI target and needs the Xcode widget-
extension target that only exists once created in the GUI — see the checklist in
the memory note. What macOS DID verify: HostEntity (AppIntents is available on
macOS), the shared attribute/notification plumbing, and that nothing regressed
(142 tests green).

Shared (PunktfunkShared):
- PunktfunkSessionAttributes (ActivityAttributes) — the one type app + extension
  share; gated os(iOS) (ActivityKit imports on macOS but its types are
  unavailable, so canImport would wrongly admit it).
- EndStreamIntent (LiveActivityIntent) — posts .punktfunkEndActiveSession.
- HostEntity + HostEntityQuery (AppEntity over the shared store) — the intent /
  widget-config parameter type; canImport(AppIntents), so macOS type-checks it.
- New notifications: end-active-session, open-deep-link.

M1 widget extension sources (Sources/PunktfunkWidgets/, NOT a SwiftPM target —
`swift build` ignores the dir):
- PunktfunkWidgetBundle (@main): HostsWidget + PunktfunkSessionLiveActivity.
- HostsWidget (kind "PunktfunkHosts"): reads the shared-suite store, sorts by
  recency, deep-links each host; small/medium/accessory families; empty state.
- SessionLiveActivity: Lock-Screen banner + Dynamic Island (elapsed timer,
  mode line, background countdown, End button).

M3 controller (app, iOS): SessionActivityController owns the Activity lifecycle
(request/update/end + launch orphan-sweep + staleDate); ContentView drives it
from the model's phase/isBackgrounded/backgroundDeadline (which SessionModel now
publishes), keeping ActivityKit out of the cross-platform model.

M4 (app, iOS): ConnectToHost/WakeHost intents + AppShortcutsProvider; Connect
routes via .punktfunkOpenDeepLink into the same onOpenURL router (one set of
guards); Wake reuses the WoL path; End surfaced to Shortcuts too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:51:41 +02:00
enricobuehler 14c5e7c11c feat(apple/M2): opt-in background keep-alive (audio + video-drop + timeout)
Backgrounding a live session no longer freezes it when the user opts in: audio
keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live,
video decode is DROPPED, and a bounded timer auto-disconnects. Off by default.

- PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag
  both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT +
  PyroWave) drain nextAU() for flow control but DISCARD the AU before any
  VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen).
- SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop
  flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture
  engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire
  (keeps host linger → fast late reconnect). exitBackground clears the flag and
  requestKeyframe()s; the pump's freeze gate auto-arms on the resumed
  frame-index gap so concealed frames are withheld until the IDR re-anchors.
  disconnect() cancels the timer + clears isBackgrounded.
- ContentView scenePhase driver (iOS): .background+streaming+setting →
  enterBackground; .active → exitBackground. scenePhase (not willResignActive)
  so Control-Center/app-switcher peeks don't start the timer.
- Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout;
  new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10).
- Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3).

macOS swift build + swift test green (142 tests). The iOS-gated scenePhase
handler + settings section are not exercised by the macOS CI target (known §9
gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN,
timeout ends the session, phone-call audio-steal degrades gracefully).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:41:46 +02:00
enricobuehler 6a0a97b702 Merge main into perf/first-frame-latency (controller fixes + Apple M0)
Keeps the latency branch current with 1a7e3a6e/b45323c0/4cae1b8b so the
eventual landing on main is a clean fast-forward-style merge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 18:35:20 +02:00
enricobuehler 4cae1b8bb8 feat(apple/M0): App Group + PunktfunkShared + punktfunk:// deep links
Foundation milestone for Live Activities & Widgets (design/apple-live-
activities-and-widgets.md). No user-visible change beyond the URL scheme.

- New dependency-free PunktfunkShared SwiftPM target (+ library product) so a
  future widget extension can link it WITHOUT PunktfunkKit (Rust staticlib +
  presentation layer). Moves StoredHost (model + JSON codec), DefaultsKeys, and
  punktfunkDefaultMgmtPort there; adds AppGroup.suiteName and the punktfunk://
  DeepLink builder/parser. PunktfunkKit @_exported-imports it (no call-site
  churn for consumers; intra-Kit files import it explicitly since imports are
  file-scoped).
- HostStore reads/writes the shared App-Group suite (group.io.unom.punktfunk)
  with a one-time migration from UserDefaults.standard (old value left in place
  for staged rollout); reloads the "PunktfunkHosts" widget timeline on change.
- App Group entitlement on iOS/tvOS + macOS.
- CFBundleURLTypes scheme `punktfunk`; ContentView.onOpenURL routes
  connect/<uuid>[?launch=<GameEntry.id>] into the existing connect() path
  (unknown host / already-streaming guards; never tears down a live session).
- Round-trip tests: StoredHost JSON codec (+ legacy missing-optional decode),
  DeepLink grammar. `swift build` + `swift test` green (142 tests, 0 failures).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:16:39 +02:00
enricobuehler 45c29a99d5 perf(host+driver): in-place resize = advertised-mode fast path + mode-history union
On-glass round 2 settled the mechanism: after UpdateModes2 the OS re-parses
our description AND re-queries target modes (driver log — both callbacks
served the fresh list) yet the SETTABLE set stays pruned to the modes known
at monitor ARRIVAL; the monitor source-mode set is pinned then, below
anything the driver can refresh. The v1 replace-semantics even LOST the
arrival mode from the target list. Consequences:

- driver: UPDATE_MODES now UNIONs (new mode first, previous list kept,
  deduped by resolution, cap 12), and a re-created same-id monitor inherits
  its departed predecessor's list (MODE_HISTORY) — every size an identity
  ever served is settable at the next arrival, so returning to a
  previously-used size (windowed<->fullscreen, drag back) is IN-PLACE.
- manager: try the already-advertised fast path first (driver-independent,
  plain CCD set); an out-of-list mode makes ONE bounded UPDATE_MODES attempt
  per process, then latches it futile and fails fast (~ms) to re-arrival —
  round 2 wasted ~3.1 s per arbitrary resize on the doomed wait. Fallback
  log demoted warn->info (expected-normal for first-seen sizes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 18:01:36 +02:00
enricobuehler a738de6cd8 fix(host): force a CCD mode re-enumeration after UPDATE_MODES (in-place resize)
First on-glass run: the driver accepted every UpdateModes2 (0x0 in the driver
log) but the OS never re-enumerated the target's settable modes on its own —
'OS did not advertise 800x1050 within 2s' → re-arrival fallback every time.
Re-commit the current config with SDC_FORCE_MODE_ENUMERATION (the same nudge
the isolate/layout paths already rely on) before the advertised-wait, re-kick
up to 3x, and log the actually-offered resolutions when it still misses.
Driver: dbglog the *2 mode-query/parse callbacks so the re-enumeration story
is visible in pfvd-driver.log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:48:44 +02:00
enricobuehler b45323c0be fix(host/windows): force off a game-abandoned rumble on the UMDF virtual pads
The DualSense / DualSense Edge / DualShock 4 / Switch Pro / Steam Deck backends
all run through UhidManager, whose pump() forwarded rumble only on a level
CHANGE and had no idle watchdog. A game that latches a one-shot rumble (a
Stardew axe hit, a DS3 hit) and then stops writing output reports left
last_rumble non-zero; native.rs re-sends the latched level every ~120 ms with a
fresh TTL and the Apple RumbleRenderer refreshes its envelope on every renewal,
so the controller vibrated continuously until a later event happened to write a
report the host parsed as a stop. The XUSB path already guards against this
(RUMBLE_IDLE_TIMEOUT force-off, 19e9828e); that guard was never ported here, so
every UMDF pad regressed for game-abandoned rumble once clients began
negotiating first-class virtual DualSense/DS4/etc. on Windows.

Port the guard into UhidManager::pump, keyed on game ACTIVITY (a fresh output
report, even at an unchanged level) so a rumble the game keeps asserting is
never cut — only an abandoned residual. The activity signal rides a new
PadFeedback.game_drove: Option<bool>; the Windows backends set it from a fresh
out_seq (via a `fresh` flag on DsFeedback/Ds4Feedback; the Deck uses is_some()).
Linux backends leave it None (untracked → always-active → the force-off never
fires there), so their behaviour is unchanged. +2 deterministic unit tests.

Verified: cargo check -p punktfunk-host --tests green on both Windows (.173) and
Linux (home-worker-5); the 10 inject::uhid_manager tests pass on Linux.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 17:45:29 +02:00
enricobuehler 1a7e3a6e4f fix(host/windows): propagate XUSB devnode-create failure instead of latching a phantom pad
XusbWinPad::open swallowed a SwDeviceCreate failure — it returned Ok with
`_sw: None` (a pad with no devnode) and logged only a warn, so PadSlots latched a
phantom pad, called gate.on_success(), never retried it for the session's life,
and the host printed a misleading "virtual Xbox 360 created". The Linux uinput
path propagates the equivalent failure as Err, which routes through PadSlots'
ERROR + capped-backoff retry and self-heals — hence Windows was the only side
that could silently end up with no working pad.

Propagate the create failure with `?` so Windows gets the same ERROR + backoff
retry as Linux. Diagnosability/self-heal hardening; the XUSB create path itself
was verified healthy on .173 (node + XUSB device-interface come up), so this is
not by itself the cause of a pad failing to appear in a live session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 17:45:27 +02:00
enricobuehler 55e59458a2 test(host): instrument the live resize spike (tracing + CCD-visibility probe)
On-glass finding: from an ssh/schtasks session-0 context QueryDisplayConfig
returns nothing at all — the activation ladder is blind there, so the live
tests can only run from an INTERACTIVE (desktop) admin prompt on the box;
the probe line makes that precondition self-diagnosing. Also verified live:
the v4 driver handshake ('pf-vdisplay protocol 4') and ADD on the new driver.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:36:16 +02:00
enricobuehler f910d23fb2 fix(proto): drop the constant assertion clippy rejects (CI parity)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:18:06 +02:00
enricobuehler c95e9125b9 test(host): live in-place resize spike (PUNKTFUNK_PF_VDISPLAY_LIVE)
Answers the P2 open questions on real glass with no streaming client: a
second same-slot acquire at a different (never-advertised) mode drives the
manager's resize branch; in-place success = same OS target id + the new
active resolution, with the elapsed ms printed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:17:34 +02:00
enricobuehler c2b9b32904 perf(host): in-place mid-stream resize — mode-set the live monitor, keep the capturer
Latency plan P2.2/P2.3: against a v4 driver the manager's resize branch now
runs UPDATE_MODES -> wait-mode-advertised (the OS re-enumerates async) ->
set_active_mode -> verified-state settle (P0.2) on the SAME monitor — no
REMOVE->ADD hotplug, no departure settle, no activation ladder, no re-isolate;
Windows keeps the per-monitor DPI (identity preserved). Any failure (v3
driver, mode never advertised, settle miss) falls back to the proven
re-arrival path unchanged.

On top of that the session's resize handler keeps the WHOLE capture pipeline:
the IDD-push capturer re-sizes its ring immediately (Capturer::resize_output —
no DescriptorPoller two-strike debounce, which stays for EXTERNAL changes),
the driver re-attaches and the mode-set full redraw provides the first frame;
only the encoder is swapped once the first new-size frame arrives
(open_video is ms-scale — P2.4 deliberately skipped). The capturer, send
thread and session transport all survive; every decline routes to the full
rebuild. Resize-trace stages (display_resized, ring_recreated,
first_new_frame, encoder_open) extend the P0.1 timeline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:12:14 +02:00
enricobuehler 0899e53903 feat(driver): pf-vdisplay IOCTL_UPDATE_MODES — live monitor mode-list refresh (proto v4)
Latency plan P2.1 (design/first-frame-and-resize-latency.md): a new additive
control-plane op lets the host refresh a LIVE monitor's advertised target-mode
list to lead with an arbitrary new mode (IddCxMonitorUpdateModes2 — the same
IddCx 1.10 *2 family this driver already requires, so no new OS floor). This
removes the 'mode list frozen at ADD' constraint that forced the mid-stream
resize through a REMOVE->ADD monitor hotplug: the monitor's OS identity, its
swap-chain worker and the retained FrameStash all survive an in-place mode set.

Protocol v4 is ADDITIVE over v3: the host's handshake floor stays at v3
(MIN_DRIVER_PROTOCOL_VERSION) and gates the in-place path on the reported
version, keeping re-arrival as the permanent fallback. The driver's stored
mode list is swapped before the DDI and reverted if it fails, so the OS and
the mode-DDI callbacks always agree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 17:12:13 +02:00
enricobuehler 32ffe7d634 chore(api): regenerate openapi.json (transition-latency fields + held drift)
Adds StreamInfo.time_to_first_frame_ms / last_resize_ms (latency plan P0.1)
and folds in the drift the spec already owed from the held working-tree
consolidation (version 0.12.0, pnp_disable_monitors description, the
conflicting-host 'conflicts' summary field) — the drift test was already
red before this branch; it is green at this commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:48:35 +02:00
enricobuehler 8374dfedf3 perf(host): session-transition trace + Welcome-time display prep (native path)
Latency plan P0.1 + P1.1/P1.2 (design/first-frame-and-resize-latency.md):

P0.1 — every native session runs a bringup::Trace (hello -> welcome -> start
-> punch_done -> display_acquired -> capture_attached -> first_frame ->
encoder_open -> first_au -> first_packet), one summary info! line when the
first video packet leaves; each accepted resize runs its own trace
(reconfigure -> pipeline_rebuilt). Totals surface per session as
time_to_first_frame_ms / last_resize_ms in session_status -> mgmt /status,
so every subsequent latency change is measured, not vibed. (The Windows
manager logs its own activation/settle deltas — correlate by wall clock.)

P1.1/P1.2 — on the Windows native path the display bring-up no longer
serializes behind the Start round-trip and the up-to-2.5 s hole-punch wait:
a prep thread kicks off at Welcome (mode is final there) and runs monitor
create -> activation -> verified settle -> capture attach -> first frame ->
encoder open while the network waits are in flight; the data plane hands it
the post-punch SessionContext and it becomes the stream thread on a warm
pipeline. Abort between Welcome and Start drops the hand-off channel and the
prep result releases into the keep-alive machinery (stop/quit + watcher are
created pre-handshake so a vanished client also aborts the build retries).
Same slot-scoped begin_idd_setup serialization as the inline path. Linux
keeps the inline bring-up (launch semantics bind before create); GameStream
untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:45:34 +02:00
enricobuehler e62cd5448e perf(host): IDD-push open — poll the HDR-enable settle, wait on the frame event
Latency plan P0.4/P0.6: the fixed 250 ms advanced-color settle becomes a
25 ms poll of the CCD state (ceiling 250 ms, ring still sized FP16 from the
successful enable either way), and wait_for_attach waits on the driver's
frame-ready event (20 ms cap for the status-code polls) instead of a blind
20 ms sleep, which also sharpens the P0.1 stage stamps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:45:16 +02:00
enricobuehler 4ed5b88407 perf(host): replace the Windows bring-up/resize fixed sleeps with verified-state waits
Latency plan P0.2/P0.3/P0.5 (design/first-frame-and-resize-latency.md):
- topology settle: the unconditional 1500 ms sleeps after create_monitor's
  group-topology apply and re_add's reisolate become a 25 ms poll for the
  committed state (active path + active mode == requested), ceiling 1500 ms —
  worst case identical, typical case saves ~1.2-1.4 s on every fresh create
  AND every mid-stream resize. The experimental pnp_disable_monitors sweep
  keeps the full settle as its floor (it reads OTHER displays' active flags,
  which the target-scoped wait doesn't verify).
- monitor departure: the fixed 400 ms REMOVE settles (re_add + both preempt
  paths) become a 25 ms poll until the target leaves the active CCD set
  (2 consecutive absent samples), ceiling 400 ms; the driver-side ghost-reap
  ADD retry stays the backstop.
- activation ladder: 200 ms -> 50 ms sampling, same 3 s per-stage ceilings
  and the same 3-stage structure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:44:57 +02:00
enricobuehler 4ffa2665ac refactor(host): extract src/devtest.rs — the standalone dev/test subcommands
Per plan §W5 (main-cli, 'devtest.rs land first'): move the inline-bodied dev/test
subcommand handlers out of main.rs's match into src/devtest.rs — input_test (Linux
libei/wlr injection smoke test + its non-Linux stub) and the virtual-gamepad
exercisers dualsense_test/switchpro_test (Linux UHID) and deck_windows_spike/
dualsense_windows_test (Windows UMDF + Steam Deck devnode spike). main.rs's arms
become one-line forwards; main.rs drops 1004→667 lines. The thin arms that already
forward to subsystem modules (zerocopy/capture selftests, probes) stay put — that
is their correct layer. Pure code-move (bodies verbatim; crate-local refs
qualified with crate::; one doc reword to dodge clippy doc_lazy_continuation now
that an arm comment became a /// doc).

Verified clippy 0/0 on BOTH Linux (home-worker-5, nvenc,vulkan-encode,pyrowave)
and Windows (.173, nvenc,amf-qsv — covers the cfg(windows) handlers).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:54:57 +02:00
enricobuehler 2def3ef49e refactor(host): extract native_pairing/sanitize.rs — the untrusted-name scrubber
Per plan §W5: move sanitize_device_name (+ its NAME_MAX cap and unit test) out of
the native_pairing facade into native_pairing/sanitize.rs. It is a self-contained,
security-relevant leaf — the one place a wire-supplied unpaired-device name is
scrubbed of control chars / bidi-override spoofing before it is stored, listed,
logged, or shown in the approval UI. Re-export via `pub(crate) use` so
crate::native_pairing::sanitize_device_name stays stable (punktfunk1 accept loop +
the two in-crate callers). Pure code-move; verified host clippy 0/0 + 11
native_pairing tests green on Linux (home-worker-5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:42:55 +02:00
enricobuehler 571e22bc0f refactor(core): consolidate the fingerprint-pinning verifier into core::tls
Per plan §2.5: the security-critical rustls fingerprint-pinning ServerCertVerifier
was hand-rolled three times — quic/endpoint.rs (PinVerify), pf-client-core
library.rs, punktfunk-tray status.rs — drifting copies on a trust boundary. Add
one canonical punktfunk_core::tls::PinVerify (+ cert_fingerprint) behind a light
`tls` feature (rustls + sha2 only, no QUIC runtime); `quic` now depends on it, and
quic::endpoint re-exports cert_fingerprint so that path stays byte-stable
(gamestream + pf-client-core reach it there).

- core::tls::PinVerify: new(pin) for the HTTP clients, with_observed(pin, slot)
  for the QUIC TOFU connect. Behavior-identical to all three originals (pin-check
  + real CertificateVerify signature verification; only hashes the leaf when a pin
  or observed slot needs it). Two focused unit tests anchor the boundary.
- quic/endpoint.rs: drop the private PinVerify, wire client_pinned through
  tls::PinVerify::with_observed.
- pf-client-core library.rs + tray status.rs: use the shared verifier; tray also
  routes load_pin through core cert_fingerprint and drops its direct sha2 dep,
  gaining only the light core `tls` feature (still no host dep, no QUIC runtime).

Verified on Linux (home-worker-5): clippy 0/0 for core(quic), core(tls),
pf-client-core, tray, host(nvenc,vulkan-encode,pyrowave); core 153 lib tests +
loopback 7/7 (pinned handshake) + c_abi round-trip green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:36:29 +02:00
enricobuehler ce085b8e3b style(vdisplay): dedupe the attach-block comment (first-frame stash follow-up)
Comment-only: the lazy-attach comment carried the delivery-consumption
sentence twice after the stash rework.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 15:24:36 +02:00
enricobuehler 405b005a0d refactor(host): extract audio/mic_pump.rs — the host-lifetime virtual-mic pump
Per plan §2.1: a self-contained stateful subsystem does not belong in the
audio trait facade. Move MicPump + its PumpTuning/PUMP_TUNING, the
drain_sleep/pump_thread loop, MIC_CHANNELS/MIC_QUEUE_CAP, and the six pump
unit tests out of audio.rs into audio/mic_pump.rs. audio.rs keeps the
AudioCapturer/VirtualMic traits, their open_* factories, and the sample
constants. Re-export via `pub use mic_pump::MicPump` so crate::audio::MicPump
stays byte-stable (only consumer: punktfunk1.rs). Pure code-move; verified
clippy 0/0 + 6/6 pump tests green on Linux (home-worker-5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:21:36 +02:00
enricobuehler 89a08f83af fix(vdisplay): first-frame guarantee — republish a retained frame at ring attach
DWM composes a display only when something dirties it, so a session opened
onto an idle desktop never produced a first frame: the host's synthetic-input
"compose kick" (cursor wiggle / sibling-display jump) was the only source, and
it is inherently unreliable — blocked on the secure desktop, defeated by a
fullscreen game's ClipCursor, user-visible, and dead in service contexts. The
field symptom: connect → black stream until something repaints the desktop.

Reconstruct DDA's first-frame semantics at the driver instead (DDA seeds a new
duplication with the current desktop image; IDD-push never had an equivalent):

* frame_transport.rs: new FrameStash — the retained last composed frame, a
  driver-private copy-only texture. publish() now reports Published /
  DescMismatch / Dropped, and harvest_into() pulls the last-published ring
  slot into the stash (keyed-mutex guarded, freshness-checked) before a
  superseded publisher is dropped — between sessions the driver keeps writing
  the host-side-dead old ring, so that slot IS the current desktop image.
* swap_chain_processor.rs: the worker stashes every frame the ring can NOT
  take (unattached, or descriptor-mismatched during a mode/HDR-flip race),
  harvests before a supersede, and REPUBLISHES the stash into every freshly
  attached ring — the host sees a normal seq=1 publish milliseconds after
  channel delivery, no compose needed. Zero steady-state cost: matched
  publishes touch only the ring. The frame-channel stash is now polled every
  iteration (attach latency = first-frame latency; it was 1-in-30).
* monitor.rs: preserved_stash (LUID-tagged) so the retained frame survives
  swap-chain unassign→reassign flaps, alongside the preserved publisher.
* host idd_push.rs: kick_dwm_compose demoted to documented last-resort
  fallback for pre-stash drivers; a debug log now fires when a kick actually
  runs so field logs show whether the stash path is working.

No proto change: the republish is an ordinary publish, so old host + new
driver and new host + old driver both keep working (the latter via the kick).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 15:16:36 +02:00
enricobuehler 5748706631 refactor(host): hoist the direct-NVENC init-params authoring into nvenc_core
Both backends' build_init_params authored an identical NV_ENC_INITIALIZE_PARAMS
(P1/ULL preset, PTD, session dimensions/rate, split-encode mode) — the only
difference was the Windows-only enableEncodeAsync flag (Linux is sync-only).
Hoist it to nvenc_core::build_init_params(codec_guid, w, h, fps, cfg, split_mode,
enable_async); Linux's two call sites pass enable_async=false (the field stays 0
as before), Windows passes its session_async through. Keeps open and in-place
reconfigure presenting the SAME init params, now guaranteed identical across
platforms too.

Verified on BOTH platforms: Linux clippy 0/0 (nvenc,vulkan-encode,pyrowave, RTX
5070 Ti) and Windows clippy 0/0 (nvenc,amf-qsv, RTX 4090 / .173).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 14:58:03 +02:00
enricobuehler 2ae5cf98ee refactor(host): hoist the direct-NVENC low-latency config into nvenc_core
Both direct-SDK NVENC backends authored a near-identical NV_ENC_CONFIG in
build_config: CBR + infinite GOP + P-only + ~1-frame VBV, per-codec tier/level,
chroma + bit depth, unconditional colour VUI, and the RFI DPB — ~125 lines each,
differing only in comments plus two genuinely per-platform bits (which surface
formats carry full chroma / 10-bit input). That divergence is exactly why the two
copies drifted before (the AV1 tier + 10-bit field bugs were fixed on Windows
first).

Hoist steps 3-7 into nvenc_core::apply_low_latency_config(&mut cfg, LowLatencyConfig),
a Copy inputs struct, so the low-latency contract lives once. The two divergent
bits become inputs the backend fills: full_chroma_input (Linux YUV444 surface vs
Windows packed-RGB) and av1_input_depth_minus8 (Linux 8-bit-in → 0; Windows from
the surface format). Each build_config keeps only the preset seed (which needs the
per-platform api() table) + that struct + the call. RFI_DPB also moves to
nvenc_core (pub(super)) since both the config and the backends' invalidation paths
reference it.

Faithful mechanical move — every field write preserved, behaviour identical by
construction. Verified on BOTH platforms: Linux clippy 0/0 (nvenc,vulkan-encode,
pyrowave, RTX 5070 Ti) and Windows clippy 0/0 (nvenc,amf-qsv, RTX 4090 / .173).
Net -83 lines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 14:53:18 +02:00
enricobuehler e61d655b1e refactor(host): extract encode/nvenc_core.rs — shared direct-SDK NVENC leaves
The two direct-SDK nvEncodeAPI backends (Windows D3D11 encode/windows/nvenc.rs,
Linux CUDA encode/linux/nvenc_cuda.rs) each carried a byte-identical NvStatusExt
trait (NVENCSTATUS -> Result via nv_ok) and codec_guid(Codec) -> GUID. Hoist
both into a new encode/nvenc_core.rs, the platform-agnostic sibling of the
existing encode/nvenc_status.rs (same cfg gate: any(linux,windows) + nvenc).
Each backend now imports them via super::nvenc_core; call sites (.nv_ok() ×16/20,
the one codec_guid() struct-init) are unchanged.

The per-platform machinery — entry-table load (nvEncodeAPI64.dll/LoadLibrary vs
libnvidia-encode.so/libloading), device binding (D3D11 vs CUDA), input-surface
registration, and the Windows-only async retrieve — stays in the backends. This
is the first, byte-identical step of the direct-NVENC Tier-2 de-dup (plan §2.2);
the larger build_config authoring is a later, carefully-diffed step.

Verified on BOTH platforms: Linux clippy 0/0 (nvenc,vulkan-encode,pyrowave, RTX
5070 Ti) and Windows clippy 0/0 (nvenc,amf-qsv, RTX 4090 / .173).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 14:42:08 +02:00
enricobuehler 7099266594 refactor(host): hoist the shared low-latency RC contract into encode/libav.rs
The three libavcodec backends each set the identical low-latency rate-control
block on the not-yet-opened encoder context: fixed time_base/frame_rate, CBR
(bit_rate == max_bit_rate), B-frames off, and a tight ~1-frame VBV/HRD buffer
written through the raw rc_buffer_size field. Move it once into
apply_low_latency_rc(&mut video, fps, bitrate_bps), and let the long VBV
rationale (why the tight buffer prevents high-motion bursts from overflowing
the send queue) live in one place instead of only in the NVENC path.

Each backend keeps the two genuinely per-backend calls around it: set_format
(pixel format differs) before, and gop_size after (NVENC's infinite/intra-
refresh wave vs the VAAPI/AMF i32::MAX). No behavior change — the field writes
are independent, so the slightly different max_b_frames/rc_buffer_size ordering
across backends is irrelevant. Folding the raw rc_buffer_size write into the
helper also removes the NVENC path's separate unsafe block. Drops the now-unused
ffmpeg::Rational import from all three.

Linux check + clippy green (0/0, nvenc,vulkan-encode,pyrowave) on RTX 5070 Ti;
ffmpeg_win.rs is Windows-cfg, pending .173 compile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 13:23:28 +02:00
enricobuehler fd8a062e2c refactor(host): hoist the libav poll_encoder drain + PollOutcome into encode/libav.rs
The three libavcodec backends each carried a byte-identical single-packet
receive_packet drain. Move it once into the shared Tier-2 glue as
poll_encoder -> PollOutcome (the richest form: Packet / Again / Eof), and
have the callers narrow it:

- Linux NVENC (encode/linux/mod.rs): poll() matches the shared fn, collapsing
  Again|Eof to Ok(None) — was an inlined match, now one call.
- VAAPI (encode/linux/vaapi.rs): drop the local poll_encoder; the blocking
  budget loop lets Again|Eof fall through to the deadline check, byte-identical
  to the old Option::None path.
- Windows AMF/QSV (encode/windows/ffmpeg_win.rs): drop the local PollOutcome +
  poll_encoder; its deadline-driven drain already matches PollOutcome, so only
  the import changes.

No behavior change on any backend. Still a plain monomorphic free fn over a
borrowed &mut Encoder — no new per-frame dyn/Box/alloc; the only allocation is
the same bitstream to_vec() each path already made. Drops the now-unused
ffmpeg::Packet import from all three.

Linux check + clippy green (nvenc,vulkan-encode,pyrowave) on RTX 5070 Ti;
ffmpeg_win.rs is Windows-cfg, pending .173 compile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 13:16:45 +02:00
enricobuehler 3c38a5f0e8 refactor(host): hoist shared libav glue into encode/libav.rs (pixel_to_av + swscale consts)
First step of the W2 libav de-dup (plan §2.2, the missing Tier-2 mid-layer). The
three libavcodec backends (Linux NVENC, VAAPI, Windows AMF/QSV) each carried a
byte-identical pixel_to_av plus the SWS_POINT / SWS_CS_ITU709 (/SWS_CS_BT2020)
swscale consts. Hoist them into a new encode/libav.rs and import from super::libav.

The module is gated to compile exactly when a libav backend does (linux, or
windows+amf-qsv). Free fns/consts over borrowed handles — no per-frame dyn/alloc,
off the zero-copy path. Verified: Linux cargo check green (linux/mod.rs + vaapi.rs
compile against it); ffmpeg_win.rs is Windows-cfg — same mechanical swap, covered
by Windows CI on push.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 13:04:52 +02:00
enricobuehler 1f519d44f9 docs(core): backfill //! module docs on quic/{io,endpoint,pake}
The last three //!-less modules in the tree (plan §2.5 / §3.2):
  - io:       length-prefixed control-message framing (read_msg/write_msg)
  - endpoint: QUIC endpoint construction + transport tuning + the TOFU
              cert-pinning verifier (PinVerify)
  - pake:     SPAKE2 pairing key exchange

Docs only — no code, type, or wire-format change (cbindgen header byte-identical).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:54:46 +02:00
enricobuehler abecb5226c refactor(host): route the three libav backends' VBV re-parse through vbv_frames_env
The libavcodec paths (Linux NVENC, VAAPI, Windows QSV) each re-parsed
PUNKTFUNK_VBV_FRAMES locally in f32, duplicating and diverging in precision from
the f64 vbv_frames_env() helper the direct-NVENC/AMF paths already use. Now that
the helper lives in encode/codec.rs (532b313b), route all three through
crate::encode::vbv_frames_env(): one parse, one precision, no drift.

Behaviour-identical (same filter finite && > 0, same 1.0 default), f64 not f32.
Verified: Linux cargo check green (linux/mod.rs + vaapi.rs compile); ffmpeg_win.rs
is Windows-cfg and mirrors the amf.rs/nvenc.rs sites already using the helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:48:26 +02:00
enricobuehler 11045a0f70 chore: consolidate parallel-session WIP (HOLD — do not push)
Local snapshot of intermingled in-flight work, committed to unblock the encode
refactor (a clean ffmpeg_win.rs for the vbv-dedup follow-on). These hunks span
the same files and can't be cleanly split here; the commit bundles three
distinct workstreams that each belong in their own PR:

  - logging rework (~43 files: level re-tiering, structured fields, `?e`,
    hot-path flood latches)
  - conflicting-host detection (detect.rs + detect/{linux,windows}.rs + wiring
    in main.rs/mgmt.rs/Cargo.toml/docs/packaging)
  - standby-sink DWM-stall attribution (windows/display_events.rs + capture/
    vdisplay wiring)

NOT verified as a combination. NOT to be pushed until the refactor is done and
these are re-verified and reorganized into their proper per-workstream PRs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:42:53 +02:00
enricobuehler d466e3e2b2 chore(nix): NixOS flake snapshot — host+client packages, module, devShell (WIP)
Local snapshot of the in-flight NixOS support: flake.nix + flake.lock + nix/
(crane host and client packages, services.punktfunk module, devShell).
Standalone — nothing in the Rust/Cargo tree references it. Held from push
pending its owning session's finalization (Skia-under-Nix follow-up + intended
per-workstream PR).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:42:34 +02:00
enricobuehler 532b313b8c refactor(host): extract encode/codec.rs — the encoder contract
Move the Tier-1 encoder contract out of the stuffed encode.rs facade into a
new encode/codec.rs submodule (plan §7 / W2): EncodedFrame, Codec (all methods
except host_wire_caps), ChromaFormat, EncoderCaps, the Encoder trait,
validate_dimensions, vbv_frames_env, and the dimension + wire-roundtrip contract
tests. host_wire_caps stays in encode.rs alongside the backend-selection probes
it depends on; CodecSupport and its wire-mask test stay too.

encode.rs gains `mod codec;` + `pub(crate) use codec::*;` so every existing
crate::encode::X path — crate::encode::vbv_frames_env, ::Codec, ::Encoder, … —
stays byte-stable. Pure relocation: no call sites touched.

Verified: dev-Mac type-check of both files clean; Linux `cargo check -p
punktfunk-host --features nvenc,vulkan-encode,pyrowave` green (all encode
backends compile against the relocated contract); contract unit tests pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:21:49 +02:00
enricobuehler d381cdf7f4 fix(host): NVENC open-failure resilience — backoff, failed-open hygiene, self-diagnosis
ci / web (push) Successful in 49s
ci / docs-site (push) Successful in 51s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
apple / swift (push) Successful in 1m9s
decky / build-publish (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 5m48s
docker / deploy-docs (push) Successful in 24s
apple / screenshots (push) Successful in 5m18s
windows-host / package (push) Successful in 9m16s
arch / build-publish (push) Successful in 10m53s
android / android (push) Successful in 11m58s
deb / build-publish (push) Successful in 18m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m19s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m28s
ci / rust (push) Successful in 24m56s
Field report (Linux direct NVENC): after a codec switch, every session open
failed with NV_ENC_ERR_INVALID_VERSION until the host process was restarted —
so the poisoned state is per-process, not a driver install issue. On-hardware
investigation (RTX 5070 Ti, 610.43.03) could not reproduce it with clean codec
cycles, dirty teardowns, or open/destroy storms, but established the failure
class: the driver enforces a per-process concurrent-session cap (12 there,
status INCOMPATIBLE_CLIENT_KEY; other branches report differently) whose
exhaustion is exactly this signature — persistent open failures healed only by
a process restart. Harden every path that can feed or mask that state:

* Rebuild backoff: the in-place encoder-rebuild retries slept one frame
  interval, so all 5 attempts burned within ~40 ms at 120 Hz — no driver-side
  transient (deferred teardown of the previous session, engine reset) can
  clear that fast. Exponential backoff 100 ms → 1.6 s (~3 s total) so
  transients heal instead of killing the session.
* Destroy-on-failed-open (Linux + Windows, all four open sites): the NVENC
  docs require NvEncDestroyEncoder even when OpenEncodeSessionEx FAILS — the
  driver may have allocated the session slot before erroring. Without it a
  retry burst against a transient leaks slots toward the cap, converting the
  transient into permanent exhaustion.
* Teardown: a destroy_encoder failure (a session slot the driver may keep) is
  now logged with its status instead of silently discarded.
* One-shot self-diagnosis on a failed session open (Linux): retry the raw open
  on a fresh dedicated CUDA context and log which of the three causes applies
  — shared-context poisoned (fresh works), driver-level skew/exhaustion/GPU
  loss (fresh fails the same way), or CUDA itself unhealthy (no fresh context)
  — so the next field report pinpoints the root cause with zero reporter
  effort.

On-hardware regression tests (RTX box .21, all green): codec-switch reopen
cycle (H265→AV1→H265→H264→H265), dirty teardown with in-flight encodes, and
the full open-failure→diagnosis→in-place-recovery path via real session-cap
exhaustion. Existing RFI/reconfigure/4:4:4 smokes still pass; clippy clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 01:57:36 +02:00
enricobuehler f901bedf22 fix(host): actionable NVENC error logging — drop misleading "(no NVIDIA GPU?)"
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m2s
apple / swift (push) Successful in 1m10s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 56s
apple / screenshots (push) Successful in 5m24s
ci / bench (push) Successful in 7m32s
docker / deploy-docs (push) Successful in 28s
arch / build-publish (push) Successful in 11m51s
windows-host / package (push) Successful in 14m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m52s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m54s
android / android (push) Successful in 17m12s
deb / build-publish (push) Successful in 18m12s
ci / rust (push) Successful in 24m52s
Every NVENC entry-point failure was annotated "(no NVIDIA GPU?)", which
misled triage: the direct-NVENC path only loads on a machine that HAS an
NVIDIA GPU. A Linux user hit NV_ENC_ERR_INVALID_VERSION at
open_encode_session_ex (past the NvEncodeAPIGetMaxSupportedVersion pre-flight
gate) — the signature of a userspace/kernel driver version skew that a host
reboot fixes — and the log pointed at a missing GPU instead. A restart did
fix it.

Add encode/nvenc_status.rs: a shared NVENCSTATUS -> cause mapper that folds
the real cause into the anyhow::Error at construction, so every downstream
{e:#} log (the encode-recovery loop, session teardown) improves for free.
INVALID_VERSION now reads "update the NVIDIA driver, or reboot if you just
updated it (a host restart is the usual fix)"; NO_ENCODE_DEVICE /
DEVICE_NOT_EXIST / INCOMPATIBLE_CLIENT_KEY (session-count limit) / OOM /
UNSUPPORTED_PARAM get their own glosses. The required API version comes from
the SDK consts so it stays correct across crate bumps.

Wire it into all NVENC entry-point failures in both backends
(encode/linux/nvenc_cuda.rs, encode/windows/nvenc.rs) — every open, init,
preset/resource/bitstream call.

Also: when the encode-recovery loop exhausts its in-place rebuilds it now
logs a clear terminal line with the underlying cause instead of the session
silently vanishing after the last identical "rebuilt in place" line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 01:13:53 +02:00
403 changed files with 57998 additions and 33018 deletions
+3 -3
View File
@@ -153,9 +153,9 @@ jobs:
# `// SAFETY:` proof. Both invariants are lint-gated (`unsafe_op_in_unsafe_fn` +
# `undocumented_unsafe_blocks`); this step keeps them from regressing. (wdk-probe is a
# toolchain-only probe crate and is excluded.)
run: cargo clippy -p pf-umdf-util -p pf-xusb -p pf-dualsense -p wdk-iddcx -p pf-vdisplay --all-targets -- -D warnings
- name: cargo fmt --check the safe-layer + gamepad drivers
run: cargo fmt -p pf-umdf-util -p pf-xusb -p pf-dualsense --check
run: cargo clippy -p pf-umdf-util -p pf-xusb -p pf-dualsense -p pf-mouse -p wdk-iddcx -p pf-vdisplay --all-targets -- -D warnings
- name: cargo fmt --check the safe-layer + gamepad/mouse drivers
run: cargo fmt -p pf-umdf-util -p pf-xusb -p pf-dualsense -p pf-mouse --check
- name: Inspect /INTEGRITYCHECK (before) — expect FORCE_INTEGRITY set by wdk-build
run: |
# explicit --target (.cargo/config.toml) -> output under the triple subdir.
+5
View File
@@ -38,3 +38,8 @@ CLAUDE.md
# Local flatpak-builder output (build-flatpak.sh) — ostree repo + build dir at the repo root.
.flatpak-repo/
.flatpak-build/
# Nix build outputs (flake.nix) — `nix build` result symlinks + direnv cache. flake.lock IS tracked.
/result
/result-*
.direnv/
Generated
+193 -5
View File
@@ -2754,6 +2754,26 @@ version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"libc",
"pf-driver-proto",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-win-display",
"pf-zerocopy",
"pipewire",
"punktfunk-core",
"tokio",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-client-core"
version = "0.12.0"
@@ -2778,6 +2798,23 @@ dependencies = [
"windows 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
name = "pf-clipboard"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"futures-util",
"libc",
"punktfunk-core",
"quinn",
"tokio",
"tracing",
"wayland-client",
"wayland-protocols",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-console-ui"
version = "0.12.0"
@@ -2799,6 +2836,28 @@ dependencies = [
"bytemuck",
]
[[package]]
name = "pf-encode"
version = "0.12.0"
dependencies = [
"anyhow",
"ash",
"ffmpeg-next",
"libc",
"libloading",
"nvidia-video-codec-sdk",
"openh264",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-zerocopy",
"punktfunk-core",
"pyrowave-sys",
"tracing",
"tracing-subscriber",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-ffvk"
version = "0.12.0"
@@ -2808,6 +2867,71 @@ dependencies = [
"pkg-config",
]
[[package]]
name = "pf-frame"
version = "0.12.0"
dependencies = [
"anyhow",
"libc",
"pf-zerocopy",
"punktfunk-core",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-gpu"
version = "0.12.0"
dependencies = [
"anyhow",
"pf-host-config",
"pf-paths",
"serde",
"serde_json",
"tempfile",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-host-config"
version = "0.12.0"
[[package]]
name = "pf-inject"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"futures-util",
"libc",
"parking_lot",
"pf-capture",
"pf-driver-proto",
"pf-host-config",
"pf-paths",
"punktfunk-core",
"reis",
"tokio",
"tracing",
"usbip-sim",
"wayland-backend",
"wayland-client",
"wayland-protocols",
"wayland-protocols-misc",
"wayland-protocols-wlr",
"wayland-scanner",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"xkbcommon",
]
[[package]]
name = "pf-paths"
version = "0.12.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.12.0"
@@ -2823,6 +2947,62 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "pf-vdisplay"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"bytemuck",
"futures-util",
"hex",
"libc",
"pf-driver-proto",
"pf-encode",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-paths",
"pf-win-display",
"punktfunk-core",
"serde",
"serde_json",
"sha2",
"tokio",
"tracing",
"utoipa",
"wayland-backend",
"wayland-client",
"wayland-scanner",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-win-display"
version = "0.12.0"
dependencies = [
"anyhow",
"pf-paths",
"punktfunk-core",
"serde_json",
"tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-zerocopy"
version = "0.12.0"
dependencies = [
"anyhow",
"ash",
"khronos-egl",
"libc",
"libloading",
"serde",
"serde_json",
"tracing",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
@@ -3103,9 +3283,9 @@ dependencies = [
"base64",
"bytemuck",
"cbc",
"ffmpeg-next",
"futures-util",
"hex",
"hmac",
"http-body-util",
"hyper",
"hyper-util",
@@ -3116,14 +3296,22 @@ dependencies = [
"log",
"mac_address",
"mdns-sd",
"nvidia-video-codec-sdk",
"openh264",
"opus",
"parking_lot",
"pf-capture",
"pf-clipboard",
"pf-driver-proto",
"pf-encode",
"pf-frame",
"pf-gpu",
"pf-host-config",
"pf-inject",
"pf-paths",
"pf-vdisplay",
"pf-win-display",
"pf-zerocopy",
"pipewire",
"punktfunk-core",
"pyrowave-sys",
"quinn",
"rand 0.8.6",
"rcgen",
@@ -3184,10 +3372,10 @@ dependencies = [
"anyhow",
"ksni",
"libc",
"punktfunk-core",
"rustls",
"serde",
"serde_json",
"sha2",
"ureq",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"windows-service",
+11
View File
@@ -6,10 +6,21 @@ members = [
"crates/punktfunk-host/vendor/usbip-sim",
"crates/punktfunk-tray",
"crates/pf-client-core",
"crates/pf-clipboard",
"crates/pf-presenter",
"crates/pf-console-ui",
"crates/pf-ffvk",
"crates/pf-driver-proto",
"crates/pf-paths",
"crates/pf-host-config",
"crates/pf-gpu",
"crates/pf-zerocopy",
"crates/pf-frame",
"crates/pf-win-display",
"crates/pf-encode",
"crates/pf-capture",
"crates/pf-inject",
"crates/pf-vdisplay",
"crates/pyrowave-sys",
"clients/probe",
"clients/linux",
+1004 -3
View File
File diff suppressed because it is too large Load Diff
@@ -1,6 +1,8 @@
package io.unom.punktfunk
import android.content.Context
import android.os.Build
import android.util.Log
import android.view.Display
/**
@@ -249,11 +251,25 @@ fun nativeDisplayMode(context: Context): Triple<Int, Int, Int> {
*/
fun displaySupportsHdr(context: Context): Boolean {
val display = runCatching { context.display }.getOrNull() ?: return false
@Suppress("DEPRECATION") // hdrCapabilities is the supported query on minSdk 31
val caps = display.hdrCapabilities ?: return false
return caps.supportedHdrTypes.any {
val types = buildSet {
// API 34+: the sanctioned per-mode query (Display.Mode.getSupportedHdrTypes). The
// deprecated Display-level hdrCapabilities can return EMPTY on Android 14+ devices
// (Pixel-class panels included), which would make a genuinely HDR display advertise
// no-HDR and pin the whole session to 8-bit SDR.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
display.mode.supportedHdrTypes.forEach { add(it) }
}
// Union the legacy query defensively — the supported one on minSdk 31, and some vendors
// populate only this on newer APIs.
@Suppress("DEPRECATION")
display.hdrCapabilities?.supportedHdrTypes?.forEach { add(it) }
}
// HDR10/HDR10+ only: the stream is BT.2020 PQ — a Dolby-Vision/HLG-only panel can't present it.
val supported = types.any {
it == Display.HdrCapabilities.HDR_TYPE_HDR10 || it == Display.HdrCapabilities.HDR_TYPE_HDR10_PLUS
}
Log.i("punktfunk", "display HDR types=$types → advertise HDR10=$supported")
return supported
}
/** Resolve [Settings] (with its 0=native placeholders) to the concrete mode to request. */
@@ -39,6 +39,7 @@ internal fun StatsOverlay(
s: DoubleArray,
verbosity: StatsVerbosity,
decoderLabel: String = "",
codecLabel: String = "",
modifier: Modifier = Modifier,
) {
if (verbosity == StatsVerbosity.OFF || s.size < 10) return
@@ -66,7 +67,7 @@ internal fun StatsOverlay(
statLine(decoderLabel, Color(0xFFB0D0FF))
}
if (detailed) {
videoFeedLine(s)?.let { statLine(it, Color.White) }
videoFeedLine(s, codecLabel)?.let { statLine(it, Color.White) }
}
if (latValid) {
// Display stage (s[22]s[25], from OnFrameRendered): when a render timestamp landed
@@ -151,14 +152,15 @@ private fun counterLine(s: DoubleArray, lostTotal: Long): String? {
}
/**
* Format the negotiated video-feed descriptor from the trailing four stats doubles
* `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* Format the negotiated video-feed descriptor from [codecLabel] plus the trailing four stats
* doubles `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `AV1 · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* so the overlay simply omits the line. The codes are CICP / H.273: transfer 16 = PQ, 18 = HLG (else
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4. The
* Android decoder is always HEVC (`video/hevc`).
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4.
* [codecLabel] is the host-resolved codec (`nativeVideoCodecLabel`); a blank one falls back to
* `HEVC` (the pre-negotiation default) for the brief window before it's resolved.
*/
private fun videoFeedLine(s: DoubleArray): String? {
private fun videoFeedLine(s: DoubleArray, codecLabel: String): String? {
if (s.size < 14) return null
val bitDepth = s[10].toInt()
val primaries = s[11].toInt()
@@ -175,5 +177,6 @@ private fun videoFeedLine(s: DoubleArray): String? {
2 -> "4:2:2"
else -> "4:2:0"
}
return "HEVC · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
val codec = codecLabel.ifEmpty { "HEVC" }
return "$codec · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
}
@@ -84,6 +84,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var decoderLabel by remember { mutableStateOf("") }
var codecLabel by remember { mutableStateOf("") }
var statsVerbosity by remember { mutableStateOf(initialSettings.statsVerbosity) }
val statsOn = statsVerbosity != StatsVerbosity.OFF
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
@@ -99,6 +100,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
LaunchedEffect(handle, statsOn) {
NativeBridge.nativeSetVideoStatsEnabled(handle, statsOn)
if (statsOn) {
// Codec is resolved at the handshake (Welcome) — fixed for the session, so read its
// label once up front (before the first snapshot renders the video-feed line).
if (codecLabel.isEmpty()) codecLabel = NativeBridge.nativeVideoCodecLabel(handle)
while (true) {
delay(1000)
stats = NativeBridge.nativeVideoStats(handle)
@@ -366,7 +370,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// BEFORE the transparent gesture layer below, so it shows through and never eats touches.
if (statsOn) {
stats?.let {
StatsOverlay(it, statsVerbosity, decoderLabel, Modifier.align(Alignment.TopStart).padding(12.dp))
StatsOverlay(it, statsVerbosity, decoderLabel, codecLabel, Modifier.align(Alignment.TopStart).padding(12.dp))
}
}
// "Hold to quit" hint while the gamepad exit chord is armed — the exit debounces on a ~1 s
@@ -214,6 +214,7 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
),
verbosity = verbosity,
decoderLabel = "c2.qti.hevc.decoder · low-latency",
codecLabel = "HEVC",
modifier = Modifier.align(Alignment.TopStart).padding(12.dp),
)
}
@@ -52,9 +52,6 @@ class GamepadFeedback(
const val TAG_PLAYER_LEDS: Byte = 0x02
const val TAG_TRIGGER: Byte = 0x03
const val TAG_HID_RAW: Byte = 0x05
// Fallback one-shot duration against a legacy host (no v2 TTL lease): the prior fixed value.
// A new host renews far below this, so it never actually holds this long there.
const val LEGACY_RUMBLE_MS = 60_000L
}
/** One controller's rumble binding — VibratorManager (API 31+) OR the legacy single Vibrator (API 2830). */
@@ -95,19 +92,19 @@ class GamepadFeedback(
while (running) {
val ev = NativeBridge.nativeNextRumble(handle)
if (ev < 0L) continue // timeout / closed
// ev bits 49..52 = wire pad index; bit 48 = has a v2 lease; bits 32..47 = ttl_ms;
// 16..31 = low; 0..15 = high. The lease flag is out-of-band, so any ttl_ms (incl.
// 0xFFFF) is a real lease — no in-band sentinel. No lease (legacy host) → the prior
// long one-shot.
// ev bits 49..52 = wire pad index; bits 32..47 = backstop duration (ms);
// 16..31 = low; 0..15 = high. These are EFFECTIVE commands from the core's shared
// rumble policy engine — it owns every lease/staleness/close decision (uniform
// across all clients; the old 60 s legacy-host exposure is gone) and emits
// explicit zeros, so apply verbatim: (0, 0) = cancel, non-zero = one-shot for
// the backstop (the hardware net under a stalled poll thread).
val pad = ((ev ushr 49) and 0xFL).toInt()
val hasLease = ((ev ushr 48) and 0x1L) == 0x1L
val ttl = ((ev ushr 32) and 0xFFFF).toInt()
val durationMs = if (hasLease) ttl.toLong() else LEGACY_RUMBLE_MS
val backstopMs = ((ev ushr 32) and 0xFFFF)
renderRumble(
pad,
((ev ushr 16) and 0xFFFF).toInt(),
(ev and 0xFFFF).toInt(),
durationMs,
backstopMs,
)
}
}, "pf-rumble").apply { isDaemon = true; start() }
@@ -212,12 +209,13 @@ class GamepadFeedback(
/**
* low = heavy/left motor, high = light/right motor; both 0..0xFFFF (the host's u16 amplitudes),
* addressed to wire pad [pad]. `durationMs` is the host's v2 envelope TTL — the one-shot self-
* terminates after it unless the host renews, so a lost stop (or a dead host) silences at the
* lease instead of the old fixed 60 s. Against a legacy host it is [LEGACY_RUMBLE_MS].
* addressed to wire pad [pad]. `durationMs` is the engine command's backstop — the one-shot's
* self-termination net under a stalled poll thread; the engine emits explicit zero commands at
* every policy stop (lease expiry, legacy staleness, session close), so cancel-on-zero is the
* real stop mechanism.
*/
private fun renderRumble(pad: Int, low: Int, high: Int, durationMs: Long) {
Log.i(TAG, "rumble pad=$pad low=$low high=$high ttlMs=$durationMs") // verification line — BEFORE any no-op return
Log.i(TAG, "rumble pad=$pad low=$low high=$high backstopMs=$durationMs") // verification line — BEFORE any no-op return
// Opt-in phone mirror, BEFORE the controller-bind early-return: the exact pads this
// serves have no vibrator of their own, so their bind below is null. It follows
// controller 1 unconditionally rather than only motor-less pads — capability probing
@@ -161,6 +161,14 @@ object NativeBridge {
*/
external fun nativeVideoMime(handle: Long): String
/**
* A short human label for the codec the host resolved (`"H.264"` / `"HEVC"` / `"AV1"` /
* `"PyroWave"`), for the stats HUD's video-feed line, or `""` on a `0` handle. Distinct from
* [nativeVideoMime] because the MIME collapses PyroWave onto `video/hevc` and can't name it.
* Fixed for the session (resolved at the handshake); read once. Cheap; UI-safe.
*/
external fun nativeVideoCodecLabel(handle: Long): String
/**
* Start the decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. [decoderName] is the
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,626 @@
//! The event-driven async MediaCodec decode loop (default) + its feeder/dispatch/present helpers.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{AsyncNotifyCallback, MediaCodec, MediaCodecDirection};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::reanchor::{GateVerdict, ReanchorGate};
use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, AtomicI64, Ordering};
use std::sync::{mpsc, Arc, Mutex};
use std::time::{Duration, Instant};
use super::display::{
apply_hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags};
use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
configure_low_latency, create_codec, try_set_frame_rate,
};
use super::{DecodeOptions, FRAME_PARK_CAP, IN_FLIGHT_CAP, PENDING_SPLIT_CAP};
/// One decoded output buffer ready to release: its codec buffer index + the pts the codec echoed
/// (from the output callback's `BufferInfo`), used to pair the `decode` HUD stat, and the
/// wall-clock instant the output callback fired — the spec's `decoded` point ("decoder output
/// frame available"), stamped at the callback so the event-channel hop + coalescing wait in the
/// loop never inflates the decode stage.
struct OutputReady {
index: usize,
pts_us: u64,
decoded_ns: i128,
}
/// Events the async decode loop reacts to. The codec's async-notify callbacks (which run on its
/// internal looper thread) push the codec ones; the feeder thread pushes `Au`. Each carries only
/// owned/`Copy` data so the callback closures satisfy the `Send` bound and never touch the codec.
enum DecodeEvent {
/// A received access unit from the feeder, ready to queue into the decoder. The `bool` is the
/// feeder's [`NativeClient::note_frame_index`] verdict — `true` when this AU revealed a forward
/// frame-index gap, so the loop arms the freeze gate (the feeder already fired the RFI request).
Au(Frame, bool),
/// An input buffer slot freed (index) — we can queue an AU into it.
InputAvailable(usize),
/// A decoded frame is ready (buffer index + echoed pts + the callback-time `decoded` stamp).
OutputAvailable {
index: usize,
pts_us: u64,
decoded_ns: i128,
},
/// The output format changed — re-check the stream's colour signalling (HDR DataSpace).
FormatChanged,
/// The codec reported an error; `fatal` when neither recoverable nor transient.
Error { fatal: bool },
}
/// The event-driven async decode loop (default; see [`run`]/[`USE_ASYNC_DECODE`]). The codec drives
/// us: an async-notify callback fires the instant an input buffer frees or a frame finishes
/// decoding, so a decoded frame is presented immediately instead of waiting out a poll interval (the
/// latency the sync loop left on the table). The callbacks run on the codec's internal looper thread
/// and only *push events* — every `AMediaCodec` buffer op stays on this thread, which owns the codec,
/// sidestepping the self-reference that would arise from a callback calling back into the codec it's
/// stored in. A small `pf-decode-feed` thread blocks on the network so this loop never does.
pub(super) fn run_async(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
let mime = codec_mime(client.codec);
let mut codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (async, low-latency feature: {ll_feature})"
);
// The event channel: the callbacks + feeder push, this loop pulls. `Sender` is `Send`, so the
// callback closures (each capturing a clone) satisfy the async-notify `Send` bound.
let (ev_tx, ev_rx) = mpsc::channel::<DecodeEvent>();
// Install the callbacks BEFORE configure()/start() so we're in async mode from the first buffer.
// Each just forwards an index/flag — no codec access here (the codec owns these closures).
{
let out_tx = ev_tx.clone();
let in_tx = ev_tx.clone();
let fmt_tx = ev_tx.clone();
let err_tx = ev_tx.clone();
let cb = AsyncNotifyCallback {
on_input_available: Some(Box::new(move |idx| {
let _ = in_tx.send(DecodeEvent::InputAvailable(idx));
})),
on_output_available: Some(Box::new(move |idx, info| {
let _ = out_tx.send(DecodeEvent::OutputAvailable {
index: idx,
pts_us: info.presentation_time_us().max(0) as u64,
// The `decoded` HUD point: stamp HERE, on the codec's looper thread, so the
// decode stage ends when the frame actually became available — not after the
// channel hop + whatever work the loop coalesces in front of presenting it.
decoded_ns: now_realtime_ns(),
});
})),
on_format_changed: Some(Box::new(move |_fmt| {
let _ = fmt_tx.send(DecodeEvent::FormatChanged);
})),
on_error: Some(Box::new(move |e, code, _detail| {
let fatal = !code.is_recoverable() && !code.is_transient();
if fatal {
log::error!("decode: fatal codec error — stream will stop: {e:?}");
} else {
log::warn!("decode: codec error {e:?} (recoverable)");
}
let _ = err_tx.send(DecodeEvent::Error { fatal });
})),
};
if let Err(e) = codec.set_async_notify_callback(Some(cb)) {
log::error!("decode: set_async_notify_callback failed: {e}");
return;
}
}
// Build the low-latency format (identical keys to the sync path).
let mut format = MediaFormat::new();
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
format.set_i32(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
configure_low_latency(&mut format, &codec_name, low_latency_mode);
if client.color.is_hdr() {
match client.next_hdr_meta(Duration::from_millis(250)) {
Ok(meta) => {
format.set_buffer("hdr-static-info", &android_hdr_static_info(&meta));
log::info!("decode: HDR static metadata applied (KEY_HDR_STATIC_INFO)");
}
Err(_) => {
log::info!("decode: HDR session but no mastering metadata yet — DataSpace only")
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
log::error!("decode: configure failed: {e}");
return;
}
if let Err(e) = codec.start() {
log::error!("decode: start failed: {e}");
return;
}
log::info!(
"decode: decoder started (async) at {}x{}",
mode.width,
mode.height
);
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
);
}
// Skew-corrected latency stats (spec: design/stats-unification.md). Receipt stamps (keyed by the
// pts we queue) live in a shared map: the feeder writes them at receipt, this loop pairs decoded
// output back to them. Behind a `Mutex` since two threads touch it — only ever locked while the
// HUD is visible.
let clock_offset = client.clock_offset_shared();
// Whether the adaptive-bitrate controller wants the `decode` stage as its decoder-backlog
// signal (Automatic, non-PyroWave): then `in_flight` is fed regardless of the HUD.
let measure_decode = client.wants_decode_latency();
let in_flight = Arc::new(Mutex::new(VecDeque::<(u64, i128)>::new()));
// Display stage (spec `display` + the capture→displayed headline): the rendered frame is
// parked in the tracker at release; the OnFrameRendered callback pairs it with
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
let tracker = DisplayTracker::new(stats.clone(), clock_offset.clone());
let render_cb = install_render_callback(&codec, &tracker);
// Feeder thread: block on the network so this loop doesn't (an AU's arrival becomes an event that
// wakes us immediately, with no input-side poll latency). It also records the `received` HUD stat.
let feeder = {
let client = client.clone();
let stats = stats.clone();
let in_flight = in_flight.clone();
let clock_offset = clock_offset.clone();
let shutdown = shutdown.clone();
let ev_tx = ev_tx.clone();
std::thread::Builder::new()
.name("pf-decode-feed".into())
.spawn(move || {
feeder_loop(
client,
stats,
measure_decode,
in_flight,
clock_offset,
shutdown,
ev_tx,
);
})
.ok()
};
drop(ev_tx); // only the feeder + callbacks keep the channel alive now
// ADPF: same as the sync path — register this thread now, create the session lazily on the first
// presented frame (by when the pump + audio + feeder threads have registered their tids too).
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread();
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
let mut free_inputs: VecDeque<usize> = VecDeque::new();
let mut pending_aus: VecDeque<Frame> = VecDeque::new();
let mut ready: Vec<OutputReady> = Vec::new();
let mut applied_ds: Option<DataSpace> = None;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
// AUs larger than the codec input buffer, dropped whole (see `feed`/`feed_ready`).
let mut oversized_dropped: u64 = 0;
// Freeze-until-reanchor gate (see the sync loop for the rationale). Armed on a frame-index gap
// (the feeder's Au verdict), a parked-AU overflow drop, a dropped-count climb, or a recoverable
// codec error; `recovery_flags` carries each AU's user_flags from `dispatch_event` (feed) to
// `present_ready` (present), keyed by the codec-echoed pts.
let mut gate = ReanchorGate::new(client.frames_dropped());
let mut recovery_flags: VecDeque<(u64, u32)> = VecDeque::new();
let mut last_kf_req: Option<Instant> = None;
// Productive (dispatch+feed+present) time between displayed frames; reported to ADPF once one is
// presented. The blocking event wait is excluded (idle, not work) — same accounting as the sync loop.
let mut work_accum_ns: i64 = 0;
let mut fatal = false;
while !shutdown.load(Ordering::Relaxed) && !fatal {
// Block for the next event (idle wait — excluded from the work tally). The short timeout
// drives loss-recovery housekeeping when the pipeline is momentarily quiet.
let ev0 = match ev_rx.recv_timeout(Duration::from_millis(5)) {
Ok(ev) => Some(ev),
Err(mpsc::RecvTimeoutError::Timeout) => None,
Err(mpsc::RecvTimeoutError::Disconnected) => break,
};
let work_t0 = Instant::now();
let mut fmt_dirty = false;
let mut aus_dropped: u64 = 0;
if let Some(ev) = ev0 {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
&mut gate,
&mut recovery_flags,
));
}
// Coalesce every other event already queued into this one work pass — correct newest-only
// presentation across a decode burst, and batched feeding.
while let Ok(ev) = ev_rx.try_recv() {
aus_dropped += u64::from(dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
&mut gate,
&mut recovery_flags,
));
}
stats.note_skipped(aus_dropped); // parked-AU overflow drops are client-side skips too
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
feed_ready(
&codec,
&client,
&mut pending_aus,
&mut free_inputs,
&mut fed,
&mut oversized_dropped,
);
let had_output = !ready.is_empty();
present_ready(
&codec,
&client,
measure_decode,
&mut ready,
&stats,
&in_flight,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut rendered,
&mut discarded,
&mut gate,
&mut recovery_flags,
);
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if had_output {
if !hint_tried {
hint_tried = true;
let tids = client.hot_thread_ids();
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
if rendered > 0 && rendered % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
}
// Loss recovery + overdue backstop, folded through the gate. A parked-AU overflow drop is itself
// a loss, so it arms the freeze directly; the gate's `poll` then arms on a dropped-count climb
// and re-asks on an overdue freeze. All keyframe intents route through the shared 100 ms
// throttle so a multi-frame recovery gap can't flood the control stream.
let now = Instant::now();
if aus_dropped > 0 {
gate.arm(now);
}
if (gate.poll(client.frames_dropped(), now) || aus_dropped > 0)
&& last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = client.request_keyframe();
}
}
let _ = codec.stop();
shutdown.store(true, Ordering::SeqCst); // ensure the feeder wakes and exits, then join it
if let Some(j) = feeder {
let _ = j.join();
}
drop(codec); // AMediaCodec_delete — after this no render callback can fire
if let Some(ud) = render_cb {
// SAFETY: the codec was dropped above; this registration's single reclaim.
unsafe { release_render_callback(ud) };
}
log::info!("decode: stopped (async, fed={fed} rendered={rendered} discarded={discarded})");
}
/// The `pf-decode-feed` thread: block on the connector for the next access unit so the async loop
/// never has to. Records the `received` HUD stat (receipt point) — including the Phase-2 host/network
/// split from any matching 0xCF host timings — then hands the AU to the loop via the event channel.
/// Exits when `shutdown` is set, the session closes, or the loop's receiver is gone.
fn feeder_loop(
client: Arc<NativeClient>,
stats: Arc<crate::stats::VideoStats>,
measure_decode: bool,
in_flight: Arc<Mutex<VecDeque<(u64, i128)>>>,
clock_offset: Arc<AtomicI64>,
shutdown: Arc<AtomicBool>,
ev_tx: mpsc::Sender<DecodeEvent>,
) {
// Received AUs awaiting their 0xCF host timing (Phase-2 split), as (pts_ns, capture→received µs).
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
while !shutdown.load(Ordering::Relaxed) {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
// Loss recovery (RFI): a forward frame-index gap fires a throttled reference-frame-
// invalidation request so an RFI-capable host recovers with a cheap clean P-frame
// instead of a full IDR (the frames_dropped keyframe path is the backstop). The gap
// verdict rides the Au event so the decode loop arms its freeze gate on the same signal.
let gap = client.note_frame_index(frame.frame_index);
// Park the receipt stamp (keyed by the pts the codec echoes) whenever the `decode`
// stage is consumed: the HUD, or the ABR decode signal (`measure_decode`). The
// HUD-only `received` point + host/network split stay gated on the overlay.
if stats.enabled() || measure_decode {
let received_ns = now_realtime_ns();
{
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.push_back((frame.pts_ns / 1000, received_ns));
if g.len() > IN_FLIGHT_CAP {
g.pop_front(); // stale — codec never echoed it back
}
}
if stats.enabled() {
let clock_offset = clock_offset.load(Ordering::Relaxed) as i128;
let lat_ns = received_ns + clock_offset - frame.pts_ns as i128;
let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000)
.then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front();
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|&(p, _)| p == t.pts_ns)
{
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
}
if ev_tx.send(DecodeEvent::Au(frame, gap)).is_err() {
break; // the decode loop is gone
}
}
Err(PunktfunkError::NoFrame) => {} // timeout — re-check shutdown and poll again
Err(_) => break, // session closed
}
}
}
/// Route one [`DecodeEvent`] into the loop's working sets. Returns `true` only when a parked AU was
/// dropped on overflow (the caller then requests a keyframe).
#[allow(clippy::too_many_arguments)] // two call sites; the freeze gate + flag map are threaded in
fn dispatch_event(
ev: DecodeEvent,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
ready: &mut Vec<OutputReady>,
fmt_dirty: &mut bool,
fatal: &mut bool,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) -> bool {
match ev {
DecodeEvent::Au(f, gap) => {
// A forward frame-index gap arms the freeze; park this AU's flags for the present side to
// fold `on_decoded` (keyed by the pts the codec will echo).
if gap {
gate.arm(Instant::now());
}
recovery_flags.push_back((f.pts_ns / 1000, f.flags));
if recovery_flags.len() > IN_FLIGHT_CAP {
recovery_flags.pop_front();
}
pending_aus.push_back(f);
if pending_aus.len() > FRAME_PARK_CAP {
pending_aus.pop_front(); // sustained overflow — drop oldest, signal a keyframe request
return true;
}
}
DecodeEvent::InputAvailable(i) => free_inputs.push_back(i),
DecodeEvent::OutputAvailable {
index,
pts_us,
decoded_ns,
} => ready.push(OutputReady {
index,
pts_us,
decoded_ns,
}),
DecodeEvent::FormatChanged => *fmt_dirty = true,
DecodeEvent::Error { fatal: f } => {
if f {
*fatal = true;
} else {
// A recoverable/transient codec error is a decode hiccup on a broken reference chain —
// arm the freeze so the concealed output it recovers into is held off the screen.
gate.arm(Instant::now());
}
}
}
false
}
/// Queue as many parked AUs as there are free input buffer slots (async mode: the indices come from
/// `InputAvailable` callbacks, not a dequeue). Each AU is copied into its codec input buffer and
/// submitted; an AU larger than the buffer is DROPPED (+ a recovery keyframe requested) — a
/// truncated AU is corrupt input the decoder chews on silently, poisoning the reference chain.
fn feed_ready(
codec: &MediaCodec,
client: &NativeClient,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
fed: &mut u64,
oversized_dropped: &mut u64,
) {
while !pending_aus.is_empty() && !free_inputs.is_empty() {
let idx = free_inputs.pop_front().unwrap();
let frame = pending_aus.pop_front().unwrap();
let pts_us = frame.pts_ns / 1000;
let Some(dst) = codec.input_buffer(idx) else {
log::warn!("decode: input_buffer({idx}) returned None — dropping AU");
continue;
};
let au = &frame.data;
if au.len() > dst.len() {
// The slot was never queued, so it stays ours — recycle it for the next AU.
free_inputs.push_front(idx);
*oversized_dropped += 1;
log::warn!(
"decode: AU {} > input buffer {} — dropped ({} so far), requesting keyframe",
au.len(),
dst.len(),
*oversized_dropped
);
let _ = client.request_keyframe();
continue;
}
let n = au.len();
// SAFETY: `au` (wire AU) and `dst` (codec input buffer) are distinct allocations, both valid
// for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so this initializes dst[..n].
unsafe {
std::ptr::copy_nonoverlapping(au.as_ptr(), dst.as_mut_ptr().cast::<u8>(), n);
}
if let Err(e) = codec.queue_input_buffer_by_index(idx, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer_by_index: {e}");
} else {
*fed += 1;
}
}
}
/// Present only the NEWEST ready output (render = true) and release the rest without rendering — a
/// burst of stale frames on glass is worse than skipping to the freshest (the sync loop's newest-ready
/// policy, callback-driven). Every dequeued buffer, rendered or not, is the HUD's `decoded`
/// measurement point (it finished decoding either way); samples are recorded in pts order so the
/// receipt-map eviction stays monotonic. The presented frame's `(pts, decoded stamp)` is parked in
/// `tracker` for the OnFrameRendered callback — the `display` stage's other endpoint. `ready` is
/// drained.
#[allow(clippy::too_many_arguments)] // one call site; mirrors the sync loop's drain
fn present_ready(
codec: &MediaCodec,
client: &NativeClient,
measure_decode: bool,
ready: &mut Vec<OutputReady>,
stats: &crate::stats::VideoStats,
in_flight: &Mutex<VecDeque<(u64, i128)>>,
clock_offset: i64,
tracker: &DisplayTracker,
rendered: &mut u64,
discarded: &mut u64,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) {
if ready.is_empty() {
return;
}
// Pair each output's decode stage (feeds the ABR decode signal always; the HUD histogram only
// while visible) — both consume the receipt map, so enter for either.
if stats.enabled() || measure_decode {
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
note_decoded_pts(
client,
measure_decode,
stats,
&mut g,
clock_offset,
o.pts_us,
o.decoded_ns,
);
}
}
// Fold EVERY output through the gate in pts (== decode) order — even the ones newest-wins discards —
// so the two-mark re-anchor count stays correct; the newest's verdict decides whether it reaches
// glass (`false` = withheld concealment; the SurfaceView keeps the last rendered frame frozen on).
let now = Instant::now();
let last = ready.len() - 1;
let mut skipped: u64 = 0;
for (i, o) in ready.drain(..).enumerate() {
let flags = take_flags(recovery_flags, o.pts_us);
let present = gate.on_decoded(flags, false, now) == GateVerdict::Present;
let render = i == last && present;
match codec.release_output_buffer_by_index(o.index, render) {
Ok(()) if render => {
*rendered += 1;
if stats.enabled() {
tracker.note_rendered(o.pts_us, o.decoded_ns);
}
}
Ok(()) => {
*discarded += 1;
skipped += 1;
}
Err(e) => {
log::warn!(
"decode: release_output_buffer_by_index({}, {render}): {e}",
o.index
)
}
}
}
stats.note_skipped(skipped); // HUD `skipped` counter (newest-wins + held-off drops); no-op hidden
}
@@ -0,0 +1,224 @@
//! Display/frame-rendered tracking, render-callback registration, HDR dataspace mapping.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::MediaCodec;
use ndk::native_window::NativeWindow;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicI64, Ordering};
use std::sync::{Arc, Mutex};
use super::latency::now_realtime_ns;
use super::RENDERED_CAP;
/// `CLOCK_MONOTONIC` now in nanoseconds — the base of the `systemNano` render timestamp the
/// `OnFrameRendered` callback reports (Android's `System.nanoTime`), read only to re-base that
/// stamp onto `CLOCK_REALTIME` (see [`on_frame_rendered`]).
fn now_monotonic_ns() -> i128 {
let mut ts = libc::timespec {
tv_sec: 0,
tv_nsec: 0,
};
// SAFETY: `clock_gettime` with a valid out-pointer is an always-safe syscall.
unsafe { libc::clock_gettime(libc::CLOCK_MONOTONIC, &mut ts) };
ts.tv_sec as i128 * 1_000_000_000 + ts.tv_nsec as i128
}
/// State shared between the decode loop and the `AMediaCodec` `OnFrameRendered` callback (which
/// fires on a codec-internal thread): rendered frames awaiting their render timestamp, so the HUD
/// gets the spec's `display` stage (decoded→displayed) and the `capture→displayed` end-to-end
/// headline (`design/stats-unification.md` — this replaces Android's v1 `capture→decoded`
/// endpoint whenever the platform delivers render callbacks).
pub(super) struct DisplayTracker {
stats: Arc<crate::stats::VideoStats>,
/// Live host-minus-client clock offset (ns) for the skew-corrected end-to-end sample —
/// loaded per callback so mid-stream re-syncs apply. Holding the handle (not the client)
/// keeps the leaked render-callback refcount from pinning the whole session alive.
clock_offset: Arc<AtomicI64>,
/// `(pts_us, decoded_real_ns)` of frames released with `render = true`, in release order,
/// awaiting their callback. Pushes are HUD-gated by the caller, so this stays empty (and the
/// callback early-outs) while the overlay is hidden.
rendered: Mutex<VecDeque<(u64, i128)>>,
}
impl DisplayTracker {
pub(super) fn new(
stats: Arc<crate::stats::VideoStats>,
clock_offset: Arc<AtomicI64>,
) -> Arc<DisplayTracker> {
Arc::new(DisplayTracker {
stats,
clock_offset,
rendered: Mutex::new(VecDeque::new()),
})
}
/// Park one just-rendered frame's `(pts, decoded stamp)` for the render callback to pair.
/// Caller gates on the HUD being visible.
pub(super) fn note_rendered(&self, pts_us: u64, decoded_ns: i128) {
let mut g = self
.rendered
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.push_back((pts_us, decoded_ns));
if g.len() > RENDERED_CAP {
g.pop_front(); // render callbacks stopped coming (allowed under load) — evict
}
}
}
/// Register [`on_frame_rendered`] on the codec (`AMediaCodec_setOnFrameRenderedCallback`,
/// **API 33** — "Available since Android T" per the NDK header; only the *Java* listener dates
/// back further). That sits above the API-28 floor, so the entry point is dlsym-resolved at
/// runtime like [`try_set_frame_rate`] — hard-linking it (as 0.9.0 shipped) made
/// `System.loadLibrary` fail on every pre-Android-13 device, taking down all of `NativeBridge`.
/// The `ndk` wrapper has no binding and the call needs the raw codec pointer, which is what the
/// vendored crate's public `as_ptr` patch is for. Returns the userdata pointer holding a leaked
/// `Arc<DisplayTracker>` refcount; the caller MUST reclaim it with [`release_render_callback`]
/// AFTER dropping the codec (`AMediaCodec_delete` is what guarantees no further callback can
/// fire). `None` (nothing to reclaim) if the symbol is absent (API < 33) or the platform refused —
/// the HUD then simply has no `display` stage, exactly the pre-callback behaviour.
pub(super) fn install_render_callback(
codec: &MediaCodec,
tracker: &Arc<DisplayTracker>,
) -> Option<*const DisplayTracker> {
// media_status_t AMediaCodec_setOnFrameRenderedCallback(
// AMediaCodec*, AMediaCodecOnFrameRendered, void*) (API 33)
type SetOnFrameRenderedFn = unsafe extern "C" fn(
*mut ndk_sys::AMediaCodec,
ndk_sys::AMediaCodecOnFrameRendered,
*mut c_void,
) -> ndk_sys::media_status_t;
// SAFETY: `dlopen` of `libmediandk.so`, which the `ndk` media wrapper already links — always
// mapped, so this only bumps its refcount (never closed — process-lifetime handle). `dlsym`
// returns null when the symbol is absent (device below API 33), checked before transmuting the
// non-null pointer to its fn-pointer type.
let set_on_frame_rendered = unsafe {
let lib = libc::dlopen(c"libmediandk.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return None;
}
let sym = libc::dlsym(lib, c"AMediaCodec_setOnFrameRenderedCallback".as_ptr());
if sym.is_null() {
log::info!("decode: no render callback on this API level (<33) — no display stage");
return None;
}
std::mem::transmute::<*mut c_void, SetOnFrameRenderedFn>(sym)
};
let ud = Arc::into_raw(tracker.clone());
// SAFETY: `codec.as_ptr()` is the live codec this thread owns; `ud` outlives the registration
// (reclaimed only after the codec is deleted, per this function's contract).
let status = unsafe {
set_on_frame_rendered(codec.as_ptr(), Some(on_frame_rendered), ud as *mut c_void)
};
if status == ndk_sys::media_status_t::AMEDIA_OK {
Some(ud)
} else {
log::warn!("decode: setOnFrameRenderedCallback failed ({status:?}) — no display stage");
// SAFETY: registration failed, so the codec never took the reference — reclaim it now.
unsafe { drop(Arc::from_raw(ud)) };
None
}
}
/// Reclaim [`install_render_callback`]'s leaked `Arc` refcount.
///
/// # Safety
/// Call exactly once, and only after the codec the callback was registered on has been dropped —
/// deleting the codec stops its internal threads, so no callback can still be running (or run
/// later) against this pointer.
pub(super) unsafe fn release_render_callback(ud: *const DisplayTracker) {
drop(Arc::from_raw(ud));
}
/// The `AMediaCodecOnFrameRendered` trampoline: fires (possibly batched) on a codec-internal
/// thread once per output frame actually placed on the output surface, with SurfaceFlinger's
/// render timestamp. That timestamp (`system_nano`) is on `CLOCK_MONOTONIC`, so it is re-based
/// onto `CLOCK_REALTIME` here — against monotonic-now at callback time, which also cancels any lag
/// between the frame rendering and the (batchable) callback delivery — to subtract against the
/// receipt/decode stamps and the host capture pts. Records the HUD's `displayed` point:
/// `end-to-end` = capture→displayed (skew-corrected) and `display` = decoded→displayed
/// (single-clock local). Panic-free by construction (poison-proof lock, saturating math) — an
/// unwind out of an `extern "C"` fn would abort the process.
unsafe extern "C" fn on_frame_rendered(
_codec: *mut ndk_sys::AMediaCodec,
userdata: *mut c_void,
media_time_us: i64,
system_nano: i64,
) {
let t = &*(userdata as *const DisplayTracker);
if !t.stats.enabled() {
return; // HUD hidden — the ring is empty too (pushes are caller-gated)
}
let displayed_ns = now_realtime_ns() - (now_monotonic_ns() - system_nano as i128);
let pts_us = media_time_us.max(0) as u64;
// Pair the frame back to its release record, evicting older entries (their callbacks were
// dropped by the platform, or the entry predates a HUD toggle) — same monotonic-eviction
// discipline as `note_decoded_pts`.
let mut decoded_ns = None;
{
let mut g = t
.rendered
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
while let Some(&(p, d)) = g.front() {
if p > pts_us {
break; // future frame — leave it for its own callback
}
g.pop_front();
if p == pts_us {
decoded_ns = Some(d);
break;
}
}
}
let e2e_ns =
displayed_ns + t.clock_offset.load(Ordering::Relaxed) as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
let display_us = decoded_ns.map(|d| ((displayed_ns - d).max(0) / 1000) as u64);
t.stats.note_displayed(e2e_us, display_us);
}
/// React to an output-format change by signalling the stream's HDR dataspace on the Surface (SDR
/// streams leave the default alone). The AMediaCodec analogue of the sync loop's `OutputFormatChanged`
/// handling; safe to call repeatedly (`applied_ds` dedups).
pub(super) fn apply_hdr_dataspace(
codec: &MediaCodec,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
) {
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
Ok(()) => {
*applied_ds = Some(ds);
log::info!("decode: HDR stream → Surface dataspace {ds}");
}
Err(e) => {
log::warn!("decode: set_buffers_data_space({ds}) failed (non-fatal): {e}")
}
}
}
}
}
/// Map the decoder's reported output colour to a BT.2020 HDR dataspace, or `None` for SDR. The
/// integer values are the Android MediaFormat colour constants the NDK shares: COLOR_TRANSFER
/// ST2084 = 6 (PQ/HDR10), HLG = 7; COLOR_RANGE FULL = 1, LIMITED = 2 (the host encodes limited).
pub(super) fn hdr_dataspace(codec: &MediaCodec) -> Option<DataSpace> {
let fmt = codec.output_format();
let full_range = fmt.i32("color-range") == Some(1);
match fmt.i32("color-transfer") {
Some(6) => Some(if full_range {
DataSpace::Bt2020Pq
} else {
DataSpace::Bt2020ItuPq
}),
Some(7) => Some(if full_range {
DataSpace::Bt2020Hlg
} else {
DataSpace::Bt2020ItuHlg
}),
_ => None, // SDR (BT.709 / SDR_VIDEO) or unspecified
}
}
@@ -0,0 +1,83 @@
//! Decode-latency bookkeeping: realtime clock + decoded-pts / user-flags stat recording.
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
/// Wall-clock now in nanoseconds (CLOCK_REALTIME basis), to compare against the host-stamped
/// capture `pts_ns` after the skew offset is applied.
pub(super) fn now_realtime_ns() -> i128 {
use std::time::{SystemTime, UNIX_EPOCH};
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos() as i128)
.unwrap_or(0)
}
/// HUD `decoded` point for one dequeued output frame, keyed by the echoed `presentationTimeUs`:
/// build the end-to-end (capture→decoded, skew-corrected, clamped to (0, 10 s)) and `decode`
/// (received→decoded, single-clock local, ≥ 0) samples and hand them to
/// [`crate::stats::VideoStats::note_decoded`]. The pts keys the receipt stamp in `in_flight`;
/// entries older than it are evicted (decode order == input order here — low-latency, no
/// B-frames — so anything before it was dropped inside the codec or stamped before a flush).
/// `decoded_ns` is the availability instant: the dequeue (sync loop) or the output callback's
/// stamp (async loop).
pub(super) fn note_decoded_pts(
client: &NativeClient,
measure_decode: bool,
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
pts_us: u64,
decoded_ns: i128,
) {
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
while let Some(&(p, r)) = in_flight.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
in_flight.pop_front();
if p == pts_us {
received_ns = Some(r);
break;
}
}
let decode_us = received_ns.map(|r| ((decoded_ns - r).max(0) / 1000) as u64);
// Adaptive bitrate: the `decode` stage (received→decoded, single-clock local) IS the decoder-
// backlog signal — the only bottleneck the host-side network signals can't see (a fast LAN
// feeding a slower mobile decoder). Report it whenever the controller is armed, regardless of
// the HUD; `report_decode_us` is a cheap accumulate the pump windows.
if measure_decode {
if let Some(us) = decode_us {
client.report_decode_us(us.min(u32::MAX as u64) as u32);
}
}
// HUD histogram: only while the overlay is visible (a measure-only caller enters here for the
// ABR report alone). `end-to-end` = capture→decoded (skew-corrected) tiles the `decode` stage.
// pts_us is the truncated frame.pts_ns/1000 we queued, so ×1000 re-approximates capture time to
// < 1 µs — negligible against the ms-scale figures shown.
if stats.enabled() {
let e2e_ns = decoded_ns + clock_offset as i128 - pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
stats.note_decoded(e2e_us, decode_us);
}
}
/// The AU `user_flags` for a decoded output, keyed by the echoed `presentationTimeUs`. Recovery
/// signalling (FLAG_SOF IDR marker / RECOVERY_ANCHOR / RECOVERY_POINT) rides the AU's flags, which are
/// only in scope at feed time — so the feed side parks `(pts_us, flags)` here and the present side
/// looks them up to fold [`ReanchorGate::on_decoded`]. Decode order == input order (low-latency, no
/// B-frames), so this evicts entries older than `pts_us` as it goes; a miss (probe filler, or an entry
/// aged past the cap) reads `0` — no recovery flags, decoded normally.
pub(super) fn take_flags(map: &mut VecDeque<(u64, u32)>, pts_us: u64) -> u32 {
while let Some(&(p, f)) = map.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
map.pop_front();
if p == pts_us {
return f;
}
}
0
}
+84
View File
@@ -0,0 +1,84 @@
//! Android video decode (android-only): pull HEVC access units from the connector and render them
//! to the SurfaceView via NDK `AMediaCodec` — hardware decode, zero per-frame JNI.
//!
//! One-in/one-out: the host opens every stream with an IDR carrying VPS/SPS/PPS **in-band**, so the
//! decoder needs no out-of-band codec-specific data — we configure with mime + the negotiated
//! WxH (from [`NativeClient::mode`]) and feed each access unit as it arrives. The decode thread owns
//! the codec + window for its whole life; [`crate::session`] signals it to stop via the shared flag.
mod async_loop;
mod display;
mod latency;
mod setup;
mod sync_loop;
use async_loop::run_async;
pub(crate) use setup::{codec_label, codec_mime};
use sync_loop::run_sync;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
/// Cap on AUs parked in the async loop awaiting a free codec input slot. Matches the connector's
/// own frame-channel depth; on sustained overflow the oldest is dropped and a keyframe requested
/// (same recovery as a reassembler drop). In steady state this stays near-empty.
const FRAME_PARK_CAP: usize = 16;
/// Cap on the pts→received-timestamp map below: MediaCodec holds only a handful of frames in
/// flight, so anything beyond this is stale (codec flushed / HUD toggled) and gets evicted.
const IN_FLIGHT_CAP: usize = 64;
/// Cap on received AUs awaiting their 0xCF host timing (Phase 2 host/network split): the timing
/// datagram trails its AU by at most the wire, so a match lands within a frame or two — anything
/// this deep is a lost datagram (or an old host that never sends any) and gets evicted.
const PENDING_SPLIT_CAP: usize = 256;
/// Cap on rendered frames parked in [`DisplayTracker`] awaiting their `OnFrameRendered` render
/// timestamp: the callback trails its release by at most a vsync or two, so anything this deep
/// means the platform stopped delivering render callbacks (allowed under load, per the docs) and
/// gets evicted.
const RENDERED_CAP: usize = 64;
/// Whether low-latency mode uses the event-driven async decode loop (default) or the synchronous
/// poll loop. Flip to `false` to A/B the two on the HUD (`design/…`); the async loop presents a
/// decoded frame the instant it's ready instead of waiting out a poll interval. Only consulted when
/// the user's "Low-latency mode" toggle is ON (now the default) — off, the sync loop always runs (the
/// original pipeline, kept as the per-device escape hatch).
const USE_ASYNC_DECODE: bool = true;
/// Per-session decode configuration, resolved by the JNI layer (`nativeStartVideo`) and passed to
/// the decode loop. Bundled so the loop entry points don't sprout a wide argument list.
pub(crate) struct DecodeOptions {
/// The decoder Kotlin ranked from `MediaCodecList` (`VideoDecoders.pickDecoder`). `None`/empty ⇒
/// let the platform resolve the default decoder for the MIME.
pub decoder_name: Option<String>,
/// Whether Kotlin found the chosen decoder advertises `FEATURE_LowLatency` (queryable only via
/// the Java `CodecCapabilities` API) — surfaced on the HUD next to the decoder name.
pub ll_feature: bool,
/// The user's "Low-latency mode" master toggle. On (default) ⇒ the full fast pipeline: async
/// decode loop, per-SoC vendor keys, pipeline thread boosts, ADPF max-performance, forced TV
/// mode switch. Off ⇒ the original synchronous pre-overhaul pipeline, kept as the per-device
/// escape hatch.
pub low_latency_mode: bool,
/// TV form factor (Kotlin's `UiModeManager`): actively drive the HDMI output into the stream's
/// refresh mode, vs. the softer seamless hint on a phone/tablet.
pub is_tv: bool,
}
/// The decode entry point on the `pf-decode` thread: dispatches to the async or synchronous loop.
/// Both run until `shutdown` is set or the session closes.
pub fn run(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
if opts.low_latency_mode && USE_ASYNC_DECODE {
run_async(client, window, shutdown, stats, opts);
} else {
run_sync(client, window, shutdown, stats, opts);
}
}
+254
View File
@@ -0,0 +1,254 @@
//! Codec creation, low-latency config, thread/frame-rate tuning, HDR static-info encode.
use ndk::media::media_codec::MediaCodec;
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use std::ffi::c_void;
/// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). Shared by the decode
/// thread and `nativeVideoMime` (which tells Kotlin what to rank decoders for). AV1 uses the
/// AOSP `video/av01` type; anything not H.264/AV1 is treated as HEVC (every pre-negotiation host
/// emitted HEVC).
pub(crate) fn codec_mime(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "video/avc",
punktfunk_core::quic::CODEC_AV1 => "video/av01",
_ => "video/hevc",
}
}
/// A short human label for the codec the host resolved, for the stats HUD's video-feed line
/// (`"H.264"` / `"HEVC"` / `"AV1"` / `"PyroWave"`). Mirrors [`codec_mime`]'s fallback: anything
/// not H.264/AV1/PyroWave is reported as HEVC (every pre-negotiation host emitted HEVC). Kept
/// beside [`codec_mime`] because the MIME collapses PyroWave onto `video/hevc` and so can't name it.
pub(crate) fn codec_label(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "H.264",
punktfunk_core::quic::CODEC_AV1 => "AV1",
punktfunk_core::quic::CODEC_PYROWAVE => "PyroWave",
_ => "HEVC",
}
}
/// Create the decoder: prefer the specific codec Kotlin ranked from `MediaCodecList`
/// (`from_codec_name`), falling back to the platform's default decoder for the MIME
/// (`from_decoder_type`) if that name can't be created (codec busy / renamed across an OS update).
pub(super) fn create_codec(mime: &str, preferred: Option<&str>) -> Option<MediaCodec> {
if let Some(name) = preferred.filter(|n| !n.is_empty()) {
if let Some(c) = MediaCodec::from_codec_name(name) {
return Some(c);
}
log::warn!(
"decode: from_codec_name({name}) failed — falling back to default {mime} decoder"
);
}
MediaCodec::from_decoder_type(mime)
}
/// Apply the low-latency MediaFormat keys for `codec_name`.
///
/// `aggressive` = the "Low-latency mode" master toggle. **Off** ⇒ the pre-overhaul key set,
/// byte-for-byte — the standard `low-latency` key, the blind Qualcomm vendor twin, `priority = 0` AND
/// `operating-rate = MAX` set together — kept as the per-device escape hatch (the profile every device
/// streamed with before the overhaul). **On** (default) ⇒ the Moonlight-parity
/// profile: MediaTek's `vdec-lowlatency` (unconditionally — ignored off MediaTek), the per-SoC
/// vendor extension keys (gated on the decoder-name prefix the way Moonlight-Android does, since a
/// key one vendor honours is meaningless on another), and one *mutually exclusive* clock hint.
///
/// Vendor keys mirror Moonlight's `MediaCodecHelper` (verified against current source): Qualcomm
/// picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon, MediaTek. NVIDIA
/// Tegra / Rockchip / Realtek expose no such key (nor does Moonlight) — they're covered by the
/// standard key + clock hint + being ranked first in `VideoDecoders`.
pub(super) fn configure_low_latency(format: &mut MediaFormat, codec_name: &str, aggressive: bool) {
// Standard key: request the no-reorder low-latency path where the platform decoder supports it.
format.set_i32("low-latency", 1);
if !aggressive {
// The original profile: the Qualcomm vendor twin set blind (unknown keys are ignored by
// other vendors' codecs), realtime priority, and the AOSP "unbounded" operating-rate
// sentinel — decode each frame at max clocks rather than pacing to the frame rate.
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
format.set_i32("priority", 0); // 0 = realtime
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
return;
}
// MediaTek's low-latency key — very common (mid/budget phones + many Google TV / Fire TV boxes).
// Set unconditionally like the standard key: MediaTek decoders honour it, others ignore it, so it
// covers MediaTek whatever the exact decoder name (omx.mtk / c2.mtk / an OEM rename). Moonlight
// does the same, and also relies on it for Amazon's Amlogic fork.
format.set_i32("vdec-lowlatency", 1);
let name = codec_name.to_ascii_lowercase();
let is = |prefix: &str| name.starts_with(prefix);
// Qualcomm Snapdragon (the most common phone SoC): picture-order forces decode-order output
// (kills the reorder buffer on decoders that predate the standard key); low-latency is the older
// vendor twin.
if is("omx.qcom") || is("c2.qti") {
format.set_i32("vendor.qti-ext-dec-picture-order.enable", 1);
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
}
// Samsung Exynos — also covers Google Tensor (Pixel 6+), whose hardware decoder is `c2.exynos.*`.
if is("omx.exynos") || is("c2.exynos") {
format.set_i32("vendor.rtc-ext-dec-low-latency.enable", 1);
}
// Amlogic — the Android TV boxes (onn 4K, Chromecast w/ Google TV, Homatics).
if is("omx.amlogic") || is("c2.amlogic") {
format.set_i32("vendor.low-latency.enable", 1);
}
// HiSilicon / Kirin (older Huawei; paired req/rdy keys).
if is("omx.hisi") || is("c2.hisi") {
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-req",
1,
);
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-rdy",
-1,
);
}
// NVIDIA Tegra (Shield TV) and Rockchip/Realtek (budget TV boxes / smart TVs) expose no
// low-latency vendor key (Moonlight has none either) — their decoders are already low-latency
// oriented, so the standard `low-latency` key + the clock hint below + being ranked first
// (see `VideoDecoders`) is their treatment.
//
// Clock hint, mutually exclusive (matching Moonlight): the AOSP "unbounded" operating-rate
// sentinel (Short.MAX) tells the decoder to run each frame at max clocks and finish ASAP rather
// than pace to the frame rate — shaving per-frame decode latency at a power/heat cost. Only
// Qualcomm is known to handle the sentinel; every other vendor mis-paces on it, so they get the
// plain realtime `priority` hint instead.
if decoder_supports_max_operating_rate(&name) {
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
} else {
format.set_i32("priority", 0); // 0 = realtime
}
}
/// Whether a decoder tolerates `operating-rate = Short.MAX` rather than regressing on it. Follows
/// Moonlight's allowlist: Qualcomm decoders honour the sentinel (the Adreno 620 generation is the
/// known exception Moonlight excludes by GPU model — undetectable from native code here, so it
/// rides the master toggle as its escape hatch). Other vendors fall back to the plain `priority`
/// hint above.
fn decoder_supports_max_operating_rate(name_lower: &str) -> bool {
name_lower.starts_with("omx.qcom") || name_lower.starts_with("c2.qti")
}
/// Raise the pipeline's OTHER hot threads — the core's data-plane pump (UDP receive + FEC
/// reassembly) and the audio decode thread — toward the display band, matching this decode thread's
/// own boost. `setpriority(PRIO_PROCESS, tid)` targets any task in the process, so we do it from
/// here once their tids are known (the same set ADPF hints), without a per-platform priority hook
/// in the shared core. Slightly below the decode thread's -10 so the display path still wins.
/// Best-effort; skips this thread (already boosted) and is non-fatal if the platform refuses.
pub(super) fn boost_hot_threads(tids: &[i32]) {
// SAFETY: `gettid` is an always-safe syscall on the calling thread.
let self_tid = unsafe { libc::gettid() };
for &tid in tids {
if tid == self_tid {
continue;
}
// SAFETY: `setpriority` with PRIO_PROCESS + a live tid in our own process is an always-safe
// syscall; a refusal is reported via the return value, not UB.
unsafe {
if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -8) != 0 {
log::debug!("decode: setpriority(-8) on hot tid {tid} failed (non-fatal)");
}
}
}
}
/// Best-effort: raise the decode thread toward Android's URGENT_DISPLAY band so background work
/// can't preempt it under load (which shows up as late/dropped frames). Non-fatal if the platform
/// refuses (foreground apps may set their own threads; the exact floor is policy-dependent).
pub(super) fn boost_thread_priority() {
// SAFETY: `gettid`/`setpriority` on the calling thread are always-safe syscalls. PRIO_PROCESS
// with a TID targets that one task on Linux — the same idiom `Process.setThreadPriority` uses.
unsafe {
let tid = libc::gettid();
if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -10) != 0 {
log::warn!(
"decode: setpriority(-10) failed (non-fatal): {}",
std::io::Error::last_os_error()
);
}
}
}
/// Set the surface's frame-rate hint to the stream's refresh so SurfaceFlinger picks a matching
/// display mode and aligns vsync (no 60-in-120 judder). Both NDK entry points sit above our API-28
/// floor, so both are dlsym-resolved at runtime (a hard import of a >floor symbol makes
/// `dlopen`/`System.load` fail on every API-28/29 device, even where this path is never hit —
/// mirrors [`crate::adpf`]):
/// - On a **TV** (`is_tv`): `ANativeWindow_setFrameRateWithChangeStrategy` (**API 31**) with
/// `changeFrameRateStrategy = ALWAYS`, which actively drives the HDMI output into the matching
/// mode (e.g. 60↔120) instead of leaving the panel at its default and judder-matching. The
/// forced switch may blank the panel briefly — acceptable once at stream start, not wanted on a
/// phone. Falls through to the 2-arg hint on API 30.
/// - Otherwise: `ANativeWindow_setFrameRate` (**API 30**) with `compatibility = DEFAULT` — the
/// softer, seamless-preferred hint for phones/tablets and the universal fallback.
///
/// Returns `true` when the platform accepted a hint; `false` on API < 30 (symbols absent) or a
/// decline.
pub(super) fn try_set_frame_rate(window: &NativeWindow, frame_rate: f32, is_tv: bool) -> bool {
// int32_t ANativeWindow_setFrameRate(ANativeWindow*, float frameRate, int8_t compatibility)
type SetFrameRateFn = unsafe extern "C" fn(*mut c_void, f32, i8) -> i32;
// int32_t ANativeWindow_setFrameRateWithChangeStrategy(
// ANativeWindow*, float frameRate, int8_t compatibility, int8_t changeFrameRateStrategy)
type SetFrameRateStrategyFn = unsafe extern "C" fn(*mut c_void, f32, i8, i8) -> i32;
// SAFETY: `dlopen` of the always-mapped `libandroid.so` (only bumps its refcount; never closed —
// process-lifetime handle). Each `dlsym` returns null when the symbol is absent (device below the
// symbol's API level), checked before transmuting the non-null pointer to its fn-pointer type.
// `window.ptr()` is the live `ANativeWindow` this `NativeWindow` owns for the call's duration.
unsafe {
let lib = libc::dlopen(c"libandroid.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return false;
}
// TV: prefer the API-31 change-strategy form to force the mode switch (strategy 1 = ALWAYS,
// compatibility 0 = DEFAULT). Absent on API 30 ⇒ fall through to the 2-arg hint below.
if is_tv {
let sym = libc::dlsym(
lib,
c"ANativeWindow_setFrameRateWithChangeStrategy".as_ptr(),
);
if !sym.is_null() {
let set = std::mem::transmute::<*mut c_void, SetFrameRateStrategyFn>(sym);
return set(window.ptr().as_ptr().cast(), frame_rate, 0, 1) == 0;
}
}
let sym = libc::dlsym(lib, c"ANativeWindow_setFrameRate".as_ptr());
if sym.is_null() {
return false; // device API < 30 — no per-surface frame-rate hint
}
let set_frame_rate = std::mem::transmute::<*mut c_void, SetFrameRateFn>(sym);
set_frame_rate(window.ptr().as_ptr().cast(), frame_rate, 0) == 0
}
}
/// Serialize [`HdrMeta`](punktfunk_core::quic::HdrMeta) into Android's `KEY_HDR_STATIC_INFO`
/// (`hdr-static-info`) layout: a 25-byte CTA-861.3 / `HDRStaticInfo.Type1` blob — descriptor id 0,
/// then primaries in **R, G, B** order, white point, max/min display luminance, MaxCLL, MaxFALL, all
/// **little-endian** `u16`. Two conversions vs our wire form: HdrMeta stores primaries in ST.2086
/// **G, B, R** order (reorder to R, G, B), and `max_display_mastering_luminance` is in 0.0001-cd/m²
/// units while Android wants **whole nits** (min stays 0.0001-nit). Chromaticities (1/50000) and
/// MaxCLL/MaxFALL (nits) match 1:1.
pub(super) fn android_hdr_static_info(m: &punktfunk_core::quic::HdrMeta) -> [u8; 25] {
let [g, b_, r] = m.display_primaries; // ST.2086 G, B, R
let max_nits = (m.max_display_mastering_luminance / 10_000).min(u16::MAX as u32) as u16;
let min_units = m.min_display_mastering_luminance.min(u16::MAX as u32) as u16;
let fields: [u16; 12] = [
r[0],
r[1],
g[0],
g[1],
b_[0],
b_[1], // R, G, B primaries
m.white_point[0],
m.white_point[1], // white point
max_nits,
min_units, // max (nits) / min (0.0001-nit) display luminance
m.max_cll,
m.max_fall, // MaxCLL / MaxFALL (nits)
];
let mut out = [0u8; 25]; // out[0] = 0 (Type 1 descriptor id), already zero
for (i, v) in fields.iter().enumerate() {
out[1 + i * 2..3 + i * 2].copy_from_slice(&v.to_le_bytes());
}
out
}
@@ -0,0 +1,547 @@
//! The synchronous MediaCodec decode loop (the original poll path) + its feed/drain helpers.
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{
DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection,
OutputBuffer,
};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::reanchor::{GateVerdict, ReanchorGate};
use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
use super::display::{
hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags};
use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
configure_low_latency, create_codec, try_set_frame_rate,
};
use super::{DecodeOptions, IN_FLIGHT_CAP, PENDING_SPLIT_CAP};
/// The synchronous poll loop — the original decode path: the only one when low-latency mode is off,
/// and the [`USE_ASYNC_DECODE`] A/B fallback when it's on. Feeds and drains on this one thread; the
/// only blocking wait is a short output dequeue while input is backed up.
pub(super) fn run_sync(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). AMediaCodec needs no
// out-of-band extradata — the in-band VPS/SPS/PPS on every IDR configure it either way.
let mime = codec_mime(client.codec);
let codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
// The decoder's *actual* resolved name (Kotlin's pick, or the platform default when it fell
// back) drives both the HUD label and which vendor low-latency keys apply below.
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (low-latency feature: {ll_feature})"
);
let mut format = MediaFormat::new();
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
// Generous input buffer so a large keyframe AU is never truncated.
format.set_i32(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
// Standard + per-SoC vendor low-latency keys and the clock hints, gated on the resolved decoder
// name and the master toggle (see `configure_low_latency`).
configure_low_latency(&mut format, &codec_name, low_latency_mode);
// HDR static metadata (ST.2086 mastering + content light level): when an HDR session was
// negotiated, set KEY_HDR_STATIC_INFO so the display tone-maps from the source's real grade.
// MediaCodec wants it BEFORE configure(), and the host sends a 0xCE right after the handshake,
// so it's typically already queued; wait briefly otherwise. The Surface DataSpace (applied on
// OutputFormatChanged below) carries transfer/primaries regardless — this adds the luminance the
// tone-mapper needs. A non-HDR display still gets sensible SurfaceFlinger tone-mapping.
if client.color.is_hdr() {
match client.next_hdr_meta(Duration::from_millis(250)) {
Ok(meta) => {
format.set_buffer("hdr-static-info", &android_hdr_static_info(&meta));
log::info!("decode: HDR static metadata applied (KEY_HDR_STATIC_INFO)");
}
Err(_) => {
log::info!("decode: HDR session but no mastering metadata yet — DataSpace only")
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
log::error!("decode: configure failed: {e}");
return;
}
if let Err(e) = codec.start() {
log::error!("decode: start failed: {e}");
return;
}
log::info!(
"decode: {mime} decoder started at {}x{}",
mode.width,
mode.height
);
// Tell the display the stream's refresh so Android can pick a matching display mode and align
// vsync (no 60-in-120 judder on high-refresh panels). `ANativeWindow_setFrameRate` is NDK API 30,
// above our API-28 floor, so we resolve it at runtime (see `try_set_frame_rate`) rather than link
// it — a hard import would stop `libpunktfunk_android.so` loading at all on API 28/29. Absent
// there ⇒ we simply skip the hint (non-fatal; the stream renders fine without it).
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
);
}
// ADPF: hint the platform that the whole video pipeline — this pf-decode feed/drain/present
// loop, the core's data-plane pump (UDP receive + FEC reassembly), and the audio thread — runs a
// per-frame real-time workload, so the CPU governor keeps those threads on fast cores at high
// clocks instead of down-clocking between frames or parking them on a little core. Snapdragon's
// ADPF backend responds well to this. We register this thread now but create the session lazily
// on the first presented frame: by then the pump + audio threads have registered their ids too,
// and ADPF `createSession` rejects a set with any not-yet-live/dead tid. No-op below API 33.
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread(); // this decode thread → the pipeline's hot-thread set
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
// Accumulates the loop's productive (feed+drain) time between displayed frames; reported to ADPF
// once per rendered frame against the frame-period target.
let mut work_accum_ns: i64 = 0;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
// AUs larger than the codec input buffer, dropped whole (see `feed`/`feed_ready`).
let mut oversized_dropped: u64 = 0;
// The AU waiting for a free codec input buffer. `feed` is non-blocking; on transient input
// pressure the AU stays parked here instead of being dropped (a drop forces a keyframe
// round-trip) and we only pop the next one once it's queued.
let mut pending: Option<Frame> = None;
// Freeze-until-reanchor: the shared post-loss gate ([`punktfunk_core::reanchor::ReanchorGate`]).
// Armed on a frame-index gap or a dropped-count climb, it withholds the decoder's concealed output
// (released WITHOUT rendering — the SurfaceView keeps the last rendered frame on glass) until a
// proven clean re-anchor lifts it: an IDR (wire FLAG_SOF), an RFI anchor, or the 2nd recovery mark.
// `last_kf_req` throttles the keyframe intents it emits; `recovery_flags` carries each AU's
// user_flags from feed to present (keyed by the codec-echoed pts) so `on_decoded` reads the
// re-anchor signalling the platform decoder doesn't expose.
let mut gate = ReanchorGate::new(client.frames_dropped());
let mut recovery_flags: VecDeque<(u64, u32)> = VecDeque::new();
let mut last_kf_req: Option<Instant> = None;
// Skew-corrected latency stats (spec: design/stats-unification.md) use the negotiated
// host-minus-client clock offset (0 if the host didn't answer the skew handshake — then the
// HUD flags it "(same-host clock)").
let clock_offset = client.clock_offset_shared();
// Display stage (spec `display` + the capture→displayed headline): frames released with
// render = true are parked in the tracker; the OnFrameRendered callback pairs them with
// SurfaceFlinger's render timestamp. `render_cb` is the callback's leaked Arc refcount,
// reclaimed after the codec is dropped below.
let tracker = DisplayTracker::new(stats.clone(), clock_offset.clone());
let render_cb = install_render_callback(&codec, &tracker);
// Receipt timestamps keyed by the pts we queue into the codec, so the decoded point (output-
// buffer dequeue — MediaCodec round-trips presentationTimeUs) can be paired back to its receipt
// for the `decode` stage. Fed while the HUD is visible OR the adaptive-bitrate controller wants
// the decode signal (`measure_decode`) — the decoder-backlog bottleneck the network can't see.
let measure_decode = client.wants_decode_latency();
let mut in_flight: VecDeque<(u64, i128)> = VecDeque::new();
// Phase-2 host/network split (design/stats-unification.md): received AUs awaiting their 0xCF
// host timing, as (pts_ns, capture→received µs). The timings are drained non-blockingly right
// where receipts are recorded and matched by pts; `network = hostnet host` (saturating).
// Only fed while the HUD is visible; an old host never sends a 0xCF, so entries just age out.
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
// The dataspace we've signalled on the Surface so far (None = default/SDR). Set reactively once
// the decoder reports an HDR stream (see `drain`); avoids re-applying every format event.
let mut applied_ds: Option<DataSpace> = None;
// One thread feeds AND drains: the NDK AMediaCodec wrapper isn't documented thread-safe for
// cross-thread feed/drain, so instead of splitting threads the loop decouples the two — input
// dequeue is non-blocking (never stalls presentation of already-decoded frames) and the only
// blocking wait is a short output dequeue while input is backed up (decoder progress is exactly
// what frees the next input buffer).
while !shutdown.load(Ordering::Relaxed) {
if pending.is_none() {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
// Loss recovery (RFI): feed the frame index so a forward gap fires a throttled
// reference-frame-invalidation request — an RFI-capable host (AMD LTR / NVENC)
// recovers with a cheap clean P-frame instead of a full IDR. The same forward gap
// arms the freeze gate so the decoder's concealment is held off the screen until the
// recovery re-anchors. The frames_dropped keyframe path below stays the backstop.
if client.note_frame_index(frame.frame_index) {
gate.arm(Instant::now());
}
// Park this AU's re-anchor flags for the present side (keyed by the pts the codec
// echoes on the output buffer) — unconditional, unlike the HUD's `in_flight` map.
recovery_flags.push_back((frame.pts_ns / 1000, frame.flags));
if recovery_flags.len() > IN_FLIGHT_CAP {
recovery_flags.pop_front();
}
if fed == 0 {
let p = &frame.data;
log::info!(
"decode: first AU {} bytes, head {:02x?}",
p.len(),
&p[..p.len().min(6)]
);
}
// Receipt stamp for the `decode` stage pairing, parked in `in_flight` (keyed by
// the pts the codec echoes on its output buffer) whenever it's needed: the HUD
// being visible, or the ABR decode signal (`measure_decode`). The HUD-only
// samplers (`received` point, host/network split) stay gated on the overlay so
// the hidden steady state adds only a wall-clock read + the receipt push.
if stats.enabled() || measure_decode {
let received_ns = now_realtime_ns();
in_flight.push_back((frame.pts_ns / 1000, received_ns));
if in_flight.len() > IN_FLIGHT_CAP {
in_flight.pop_front(); // stale — codec never echoed it back
}
// HUD stat, `received` point: host+network = client_now + (hostclient)
// capture_pts.
if stats.enabled() {
let clock_offset = clock_offset.load(Ordering::Relaxed);
let lat_ns = received_ns + clock_offset as i128 - frame.pts_ns as i128;
let lat_us = (lat_ns > 0 && lat_ns < 10_000_000_000)
.then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
// Phase-2 split: park this AU's capture→received sample, then match any
// 0xCF host timings that have arrived — host = the host's own
// capture→sent, network = our capture→received minus it (per-frame
// tiling; saturating in case of clock jitter).
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front(); // 0xCF lost / old host — evict
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) =
pending_split.iter().position(|&(p, _)| p == t.pts_ns)
{
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
}
pending = Some(frame);
}
Err(PunktfunkError::NoFrame) => {} // timeout — still drain output below
Err(_) => break, // session closed
}
}
// Time the productive work (feed + drain) only — the `next_frame` poll wait above is idle
// and excluded, so ADPF sees this thread's real per-frame CPU cost, not the poll timeout.
let work_t0 = Instant::now();
if let Some(frame) = pending.take() {
if feed(
&codec,
&client,
&frame.data,
frame.pts_ns / 1000,
&mut oversized_dropped,
) {
fed += 1;
if fed % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
} else {
// No input buffer free — transient back-pressure. Keep the AU and let `drain` block
// briefly below; a released output buffer is what recycles an input slot.
pending = Some(frame);
}
}
// Drain every iteration. When input is blocked, wait ~2 ms on output so the loop rides
// decoder progress instead of busy-spinning against a full input queue.
let wait = if pending.is_some() {
Duration::from_millis(2)
} else {
Duration::ZERO
};
let (r, d) = drain(
&codec,
&client,
measure_decode,
&window,
&mut applied_ds,
wait,
&stats,
&mut in_flight,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut gate,
&mut recovery_flags,
);
rendered += r;
discarded += d;
// ADPF: attribute this iteration's feed+drain time to the frame being produced, and report
// the accumulated per-frame work once one is actually presented (r > 0). Under back-pressure
// the short output-dequeue wait is included in the tally — for a latency-first client,
// biasing the governor toward "boost" is the desired behaviour. Cheap when `hint` is None
// (one `Instant` diff, no report).
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if r > 0 {
if !hint_tried {
// First presented frame: the pump + audio threads have registered their ids by now.
// Build one ADPF session over the whole pipeline's thread set (empty below API 33,
// or where the platform declines → `None`, and the loop runs unhinted).
hint_tried = true;
let tids = client.hot_thread_ids();
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
}
// Loss recovery + overdue backstop, folded through the gate. Under infinite GOP the only
// recovery keyframe is one we request; the reassembler drops unrecoverable AUs (frames_dropped)
// and the decoder then conceals the reference-missing deltas and renders them without error, so
// a decode-error trigger rarely fires — the gate arms the freeze on the drop-count climb
// instead. An overdue freeze (held REANCHOR_FREEZE_MAX with no clean re-anchor) re-asks while it
// keeps holding: never resume to gray — a dead stream is the QUIC idle-timeout watchdog's job.
let now = Instant::now();
if gate.poll(client.frames_dropped(), now)
&& last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = client.request_keyframe();
log::debug!("decode: requested keyframe (loss recovery / overdue re-anchor)");
}
}
let _ = codec.stop();
drop(codec); // AMediaCodec_delete — after this no render callback can fire
if let Some(ud) = render_cb {
// SAFETY: the codec was dropped above; this registration's single reclaim.
unsafe { release_render_callback(ud) };
}
log::info!("decode: stopped (fed={fed} rendered={rendered} discarded={discarded})");
}
/// Try to copy one access unit into a codec input buffer and queue it, without blocking. Returns
/// `false` only on `TryAgainLater` (no input buffer free) — the caller keeps the AU pending and
/// retries; a hard dequeue/queue error counts as consumed (retrying can't salvage the AU, and
/// parking it forever would wedge the loop on a broken codec). An AU larger than the input
/// buffer is DROPPED (+ a recovery keyframe requested), never truncated — a truncated AU is
/// corrupt input the decoder chews on silently, poisoning the reference chain.
fn feed(
codec: &MediaCodec,
client: &NativeClient,
au: &[u8],
pts_us: u64,
oversized_dropped: &mut u64,
) -> bool {
match codec.dequeue_input_buffer(Duration::ZERO) {
Ok(DequeuedInputBufferResult::Buffer(mut buf)) => {
let n = {
let dst = buf.buffer_mut();
if au.len() > dst.len() {
*oversized_dropped += 1;
log::warn!(
"decode: AU {} > input buffer {} — dropped ({} so far), requesting keyframe",
au.len(),
dst.len(),
*oversized_dropped
);
let _ = client.request_keyframe();
0 // return the slot with zero valid bytes — a no-op input, not corrupt data
} else {
let n = au.len();
// SAFETY: `au` and `dst` are distinct allocations (wire AU vs. codec buffer),
// both valid for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so
// the cast write initializes exactly `dst[..n]`.
unsafe {
std::ptr::copy_nonoverlapping(
au.as_ptr(),
dst.as_mut_ptr().cast::<u8>(),
n,
);
}
n
}
};
if let Err(e) = codec.queue_input_buffer(buf, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer: {e}");
}
true
}
Ok(DequeuedInputBufferResult::TryAgainLater) => false, // caller keeps the AU pending
Err(e) => {
log::warn!("decode: dequeue_input_buffer: {e}");
true
}
}
}
/// Dequeue every ready output buffer and present only the NEWEST (render = true), discarding the
/// rest (render = false) — when decode falls behind, a back-to-back burst of stale frames on glass
/// is worse than skipping straight to the freshest one (the Apple client's 1-slot newest-ready
/// ring, ported). `first_wait` is the timeout for the first dequeue only: zero normally, ~2 ms when
/// the caller's input is blocked so the loop waits on decoder progress instead of busy-spinning.
/// Returns `(rendered, discarded)`. Also reacts to `OutputFormatChanged` (which can interleave
/// between buffers — handled without losing the held buffer) to signal HDR on the Surface.
///
/// Each dequeued buffer is also the HUD's `decoded` measurement point (rendered or not — the frame
/// finished decoding either way): end-to-end = decoded + clock_offset capture pts, and the
/// `decode` stage pairs the buffer's echoed presentationTimeUs back to the receipt stamp in
/// `in_flight` (single-clock local difference, no skew involved). The presented frame's
/// `(pts, decoded stamp)` is additionally parked in `tracker` for the OnFrameRendered callback —
/// the `display` stage's other endpoint.
#[allow(clippy::too_many_arguments)] // one call site; mirrors the async loop's present_ready
fn drain(
codec: &MediaCodec,
client: &NativeClient,
measure_decode: bool,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
first_wait: Duration,
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
tracker: &DisplayTracker,
gate: &mut ReanchorGate,
recovery_flags: &mut VecDeque<(u64, u32)>,
) -> (u64, u64) {
// Newest ready buffer so far (presented after the loop) with its HUD metadata —
// `Some((pts_us, decoded_ns))` only while the HUD is visible. `held_present` is the freeze gate's
// verdict for that newest buffer (`false` = a post-loss concealment to withhold).
let mut held: Option<(OutputBuffer<'_>, Option<(u64, i128)>)> = None;
let mut held_present = true;
let mut discarded: u64 = 0;
let mut wait = first_wait;
loop {
match codec.dequeue_output_buffer(wait) {
Ok(DequeuedOutputBufferInfoResult::Buffer(buf)) => {
// Only the first dequeue may block; later ones poll (wait == ZERO).
wait = Duration::ZERO;
// Fold every dequeued frame through the gate in pts (== decode) order — even the ones
// the newest-wins policy discards — so the two-mark re-anchor count stays correct; the
// verdict of the newest (last folded) buffer decides whether it reaches glass.
let pts_us = buf.info().presentation_time_us().max(0) as u64;
let flags = take_flags(recovery_flags, pts_us);
held_present =
gate.on_decoded(flags, false, Instant::now()) == GateVerdict::Present;
let meta = if stats.enabled() || measure_decode {
// The dequeue IS the sync loop's decoded-availability instant.
let decoded_ns = now_realtime_ns();
note_decoded_pts(
client,
measure_decode,
stats,
in_flight,
clock_offset,
pts_us,
decoded_ns,
);
// The tracker's `display` stage is a HUD concern — park only when visible.
stats.enabled().then_some((pts_us, decoded_ns))
} else {
None
};
if let Some((stale, _)) = held.replace((buf, meta)) {
// A newer frame is ready — drop the held one without rendering.
if let Err(e) = codec.release_output_buffer(stale, false) {
log::warn!("decode: release_output_buffer(discard): {e}");
}
discarded += 1;
stats.note_skipped(1); // HUD `skipped` counter; no-op while hidden
}
}
Ok(DequeuedOutputBufferInfoResult::OutputFormatChanged) => {
// The decoder has parsed the SPS and now reports the stream's real colour signalling
// (the AMediaCodec analogue of VideoToolbox's format description on the Apple client).
// If it's HDR (BT.2020 PQ/HLG), tell the Surface so the compositor/display switch to
// HDR; SDR streams leave the default dataspace alone. The decoder itself picks a
// Main10 path from the SPS — no profile override needed. Keep looping (buffers
// follow, and any held buffer stays held across this event).
wait = Duration::ZERO;
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
Ok(()) => {
*applied_ds = Some(ds);
log::info!("decode: HDR stream → Surface dataspace {ds}");
}
Err(e) => log::warn!(
"decode: set_buffers_data_space({ds}) failed (non-fatal): {e}"
),
}
}
}
}
// TryAgainLater / OutputBuffersChanged — nothing more to dequeue now.
Ok(_) => break,
Err(e) => {
log::warn!("decode: dequeue_output_buffer: {e}");
break;
}
}
}
// Present the newest ready frame — UNLESS the gate is withholding it as a post-loss concealment,
// in which case release it without rendering (the SurfaceView keeps the last rendered frame frozen
// on glass) and count it as a discard rather than a display.
let mut rendered = 0;
if let Some((buf, meta)) = held {
match codec.release_output_buffer(buf, held_present) {
Ok(()) if held_present => {
rendered = 1;
if let Some((pts_us, decoded_ns)) = meta {
tracker.note_rendered(pts_us, decoded_ns);
}
}
Ok(()) => discarded += 1, // held off the screen — awaiting a clean re-anchor
Err(e) => log::warn!("decode: release_output_buffer: {e}"),
}
}
(rendered, discarded)
}
+23 -25
View File
@@ -24,14 +24,19 @@ const TAG_PLAYER_LEDS: u8 = 0x02;
const TAG_TRIGGER: u8 = 0x03;
const TAG_HID_RAW: u8 = 0x05;
/// `NativeBridge.nativeNextRumble(handle): Long` — block up to ~100 ms for the next rumble update.
/// Returns a packed positive long: bits 49..52 = wire `pad` index (0..15), bit 48 = "has a v2 lease",
/// bits 32..47 = `ttl_ms`, bits 16..31 = `low`, bits 0..15 = `high` (`low`/`high` 0..=0xFFFF, `0/0` =
/// stop). The lease flag is out-of-band so ANY 16-bit `ttl_ms` — including 0xFFFF — is unambiguous (no
/// in-band sentinel to collide with a real 65535 ms lease). No lease (legacy host) → bit 48 clear, and
/// Kotlin falls back to its long one-shot. `-1` on timeout / session closed (all packed values are
/// positive, so `-1` stays unambiguous). Kotlin routes the update back to the controller holding that
/// wire `pad` index (multi-pad rumble). Run from a Kotlin poll thread.
/// `NativeBridge.nativeNextRumble(handle): Long` — block up to ~100 ms for the next EFFECTIVE
/// rumble command from the core's shared policy engine (`design/rumble-root-fix.md` §D). The
/// engine owns ALL rumble policy — v2 lease expiry, legacy-host staleness (a uniform 1 s, ending
/// the old 60 s Android exposure), connection-close drain zeros — so Kotlin applies commands
/// verbatim: `(0, 0)` = cancel now, non-zero = one-shot at this level.
///
/// Returns a packed positive long: bits 49..52 = wire `pad` index (0..15), bits 32..47 = the
/// command's `backstop_ms` (≤ 5000 — the one-shot duration, i.e. the hardware net under a stalled
/// poll thread; the engine emits explicit zeros at every policy stop, so it is never the stop
/// mechanism), bits 16..31 = `low`, bits 0..15 = `high` (0..=0xFFFF). `-1` on timeout / session
/// closed (all packed values are positive, so `-1` stays unambiguous). Kotlin routes the command
/// back to the controller holding that wire `pad` index (multi-pad rumble). Run from a Kotlin
/// poll thread.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextRumble(
_env: JNIEnv,
@@ -43,24 +48,17 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextRumble(
if handle == 0 {
return -1;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; next_rumble_ttl is &self on
// the Sync connector — safe alongside the decode/audio/input threads. Kotlin stops these poll
// threads (and joins them — unbounded) before nativeClose frees the handle.
// SAFETY: live handle per the nativeConnect/nativeClose contract; next_rumble_command is
// &self on the Sync connector — safe alongside the decode/audio/input threads. Kotlin
// stops these poll threads (and joins them — unbounded) before nativeClose frees the
// handle.
let h = unsafe { &*(handle as *const SessionHandle) };
match h.client.next_rumble_ttl(PULL_TIMEOUT) {
Ok((pad, low, high, ttl)) => {
// The reorder gate already ran in the core, so this update is fresh. Encode the
// Option out-of-band: a real lease sets bit 48 and carries ttl_ms verbatim. The pad
// index rides above the lease flag (bits 49..52), keeping the whole word positive.
let (lease_flag, ttl_bits) = match ttl {
Some(ms) => (1i64 << 48, jlong::from(ms) << 32),
None => (0, 0),
};
(jlong::from(pad & 0xF) << 49)
| lease_flag
| ttl_bits
| (jlong::from(low) << 16)
| jlong::from(high)
match h.client.next_rumble_command(PULL_TIMEOUT) {
Ok(cmd) => {
(jlong::from(cmd.pad & 0xF) << 49)
| (jlong::from(cmd.backstop_ms.min(0xFFFF) as u16) << 32)
| (jlong::from(cmd.low) << 16)
| jlong::from(cmd.high)
}
Err(_) => -1, // NoFrame (timeout) or Closed — Kotlin loops on its running flag
}
@@ -102,6 +102,31 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoMime<'
})
}
/// `NativeBridge.nativeVideoCodecLabel(handle): String` — a short human label for the codec the
/// host resolved (`"H.264"` / `"HEVC"` / `"AV1"` / `"PyroWave"`), for the stats HUD's video-feed
/// line. Distinct from [`Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoMime`] because the MIME
/// collapses PyroWave onto `video/hevc` and can't name it. Empty string on a `0` handle. Cheap;
/// safe on the UI thread. Android-gated (reads `crate::decode`), matching `nativeVideoMime`.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoCodecLabel<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jstring {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
match env.new_string(crate::decode::codec_label(h.client.codec)) {
Ok(s) => s.into_raw(),
Err(_) => std::ptr::null_mut(),
}
})
}
/// `NativeBridge.nativeVideoDecoderLabel(handle): String` — the resolved decoder identity for the
/// HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""` before the decode thread has resolved one.
/// One-shot (the decoder is fixed for the session); poll once after the HUD appears. Not
+27
View File
@@ -19,5 +19,32 @@
<array>
<string>_punktfunk._udp</string>
</array>
<!-- Background keep-alive (opt-in, iOS/iPadOS): the ONLY sanctioned way to keep the long-lived
QUIC socket + pump-thread set alive while backgrounded is the audio background mode, backed
by the session's real, audible remote audio (AVAudioEngine keeps rendering). Video decode is
dropped; a bounded timer auto-disconnects. Never silence-as-keepalive (App Review 2.5.4).
tvOS ignores/tolerates the key; macOS is not gated by it. -->
<key>UIBackgroundModes</key>
<array>
<string>audio</string>
</array>
<!-- Live Activities (iOS/iPadOS): the Lock-Screen / Dynamic-Island session surface. Updated
locally (pushType nil) from the alive app process — no aps-environment. tvOS/macOS ignore it. -->
<key>NSSupportsLiveActivities</key>
<true/>
<!-- Deep links: punktfunk://connect/<host-uuid>[?launch=<GameEntry.id>]. Emitted by the
launcher widget and Siri/Shortcuts; routed by ContentView.onOpenURL into the existing
connect path. Shared across all three targets (tvOS/macOS accept it harmlessly). -->
<key>CFBundleURLTypes</key>
<array>
<dict>
<key>CFBundleURLName</key>
<string>io.unom.punktfunk.deeplink</string>
<key>CFBundleURLSchemes</key>
<array>
<string>punktfunk</string>
</array>
</dict>
</array>
</dict>
</plist>
@@ -73,5 +73,15 @@
<array>
<string>$(AppIdentifierPrefix)io.unom.punktfunk</string>
</array>
<!-- App Group: same shared UserDefaults suite as iOS (Config/Punktfunk.entitlements). Shared
here so a single HostStore code path (UserDefaults(suiteName:)) works on every platform;
macOS widgets that read it arrive with M5. macOS App Groups use the plain group id under
the App Store profile; a Developer-ID-signed build wants the team-prefixed form — the
Dev-ID codesign step in release.yml must verify this value against the Dev-ID profile. -->
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
@@ -20,5 +20,14 @@
is true on iOS/tvOS too. -->
<key>com.apple.developer.networking.multicast</key>
<true/>
<!-- App Group: the shared UserDefaults suite (group.io.unom.punktfunk) that both the app and
the Widget/Live-Activity extension read — the saved-host store moved there so a launcher
widget can see it (HostStore reads UserDefaults(suiteName:)). Must be registered on the
developer portal and enabled in the provisioning profile for BOTH app ids
(io.unom.punktfunk + io.unom.punktfunk.widgets). tvOS carries the key harmlessly. -->
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
+10 -2
View File
@@ -9,13 +9,20 @@ let package = Package(
platforms: [.macOS(.v14), .iOS(.v17), .tvOS(.v17)],
products: [
.library(name: "PunktfunkKit", targets: ["PunktfunkKit"]),
// Dependency-free foundation (stored-host model + JSON codec, settings keys, App-Group
// constant, deep-link grammar, Live Activity attributes). A separate PRODUCT so the widget
// extension which must never link PunktfunkKit (Rust staticlib + presentation layer)
// can link this and nothing else. PunktfunkKit re-exports it (see SharedReexport.swift).
.library(name: "PunktfunkShared", targets: ["PunktfunkShared"]),
.executable(name: "PunktfunkClient", targets: ["PunktfunkClient"]),
],
targets: [
.binaryTarget(name: "PunktfunkCore", path: "PunktfunkCore.xcframework"),
// No dependencies by design an extension process links this alone.
.target(name: "PunktfunkShared"),
.target(
name: "PunktfunkKit",
dependencies: ["PunktfunkCore"],
dependencies: ["PunktfunkCore", "PunktfunkShared"],
// OSS attribution shown by the app's Acknowledgements screen. Bundled here (not in the
// app target) so it rides along via Bundle.module in both `swift build` and the Xcode
// app, which links the PunktfunkKit product. Refresh with
@@ -43,7 +50,8 @@ let package = Package(
// PunktfunkCore is a direct dep too so the wire tests can name the C ABI's
// `PunktfunkInputEvent` / `PUNKTFUNK_INPUT_KIND_*` when asserting the gamepad byte layout.
.testTarget(
name: "PunktfunkKitTests", dependencies: ["PunktfunkKit", "PunktfunkCore"],
name: "PunktfunkKitTests",
dependencies: ["PunktfunkKit", "PunktfunkShared", "PunktfunkCore"],
resources: [
// PyroWave golden fixtures: host-encoded AUs + upstream-decoded reference
// planes (regenerate with punktfunk-host's `pyrowave_dump_golden` on a
@@ -11,14 +11,56 @@
BB0000000000000000000005 /* PunktfunkKit in Frameworks */ = {isa = PBXBuildFile; productRef = BB0000000000000000000006 /* PunktfunkKit */; };
CC0000000000000000000005 /* PunktfunkKit in Frameworks */ = {isa = PBXBuildFile; productRef = CC0000000000000000000006 /* PunktfunkKit */; };
DD0000000000000000000003 /* SwiftUINavigationTransitions in Frameworks */ = {isa = PBXBuildFile; productRef = DD0000000000000000000002 /* SwiftUINavigationTransitions */; };
E295569A300948B9009F939C /* WidgetKit.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = E2955699300948B9009F939C /* WidgetKit.framework */; };
E295569C300948B9009F939C /* SwiftUI.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = E295569B300948B9009F939C /* SwiftUI.framework */; };
E2CAFE000000000000000001 /* PunktfunkShared in Frameworks */ = {isa = PBXBuildFile; productRef = E2CAFE000000000000000002 /* PunktfunkShared */; };
E29556A9300948BA009F939C /* PunktfunkWidgetsExtension.appex in Embed Foundation Extensions */ = {isa = PBXBuildFile; fileRef = E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */; settings = {ATTRIBUTES = (RemoveHeadersOnCopy, ); }; };
/* End PBXBuildFile section */
/* Begin PBXContainerItemProxy section */
E29556A7300948BA009F939C /* PBXContainerItemProxy */ = {
isa = PBXContainerItemProxy;
containerPortal = AA000000000000000000000D /* Project object */;
proxyType = 1;
remoteGlobalIDString = E2955696300948B9009F939C;
remoteInfo = PunktfunkWidgetsExtension;
};
/* End PBXContainerItemProxy section */
/* Begin PBXCopyFilesBuildPhase section */
E29556AA300948BA009F939C /* Embed Foundation Extensions */ = {
isa = PBXCopyFilesBuildPhase;
buildActionMask = 2147483647;
dstPath = "";
dstSubfolderSpec = 13;
files = (
E29556A9300948BA009F939C /* PunktfunkWidgetsExtension.appex in Embed Foundation Extensions */,
);
name = "Embed Foundation Extensions";
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXCopyFilesBuildPhase section */
/* Begin PBXFileReference section */
AA0000000000000000000001 /* Punktfunk.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = Punktfunk.app; sourceTree = BUILT_PRODUCTS_DIR; };
BB0000000000000000000001 /* Punktfunk-iOS.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = "Punktfunk-iOS.app"; sourceTree = BUILT_PRODUCTS_DIR; };
CC0000000000000000000001 /* Punktfunk-tvOS.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = "Punktfunk-tvOS.app"; sourceTree = BUILT_PRODUCTS_DIR; };
E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */ = {isa = PBXFileReference; explicitFileType = "wrapper.app-extension"; includeInIndex = 0; path = PunktfunkWidgetsExtension.appex; sourceTree = BUILT_PRODUCTS_DIR; };
E2955699300948B9009F939C /* WidgetKit.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = WidgetKit.framework; path = System/Library/Frameworks/WidgetKit.framework; sourceTree = SDKROOT; };
E295569B300948B9009F939C /* SwiftUI.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = SwiftUI.framework; path = System/Library/Frameworks/SwiftUI.framework; sourceTree = SDKROOT; };
E295577B30094CE5009F939C /* PunktfunkWidgetsExtension.entitlements */ = {isa = PBXFileReference; lastKnownFileType = text.plist.entitlements; path = PunktfunkWidgetsExtension.entitlements; sourceTree = "<group>"; };
/* End PBXFileReference section */
/* Begin PBXFileSystemSynchronizedBuildFileExceptionSet section */
E29556AD300948BA009F939C /* Exceptions for "PunktfunkWidgets" folder in "PunktfunkWidgetsExtension" target */ = {
isa = PBXFileSystemSynchronizedBuildFileExceptionSet;
membershipExceptions = (
Info.plist,
);
target = E2955696300948B9009F939C /* PunktfunkWidgetsExtension */;
};
/* End PBXFileSystemSynchronizedBuildFileExceptionSet section */
/* Begin PBXFileSystemSynchronizedRootGroup section */
AA0000000000000000000002 /* App */ = {
isa = PBXFileSystemSynchronizedRootGroup;
@@ -30,6 +72,14 @@
path = Sources/PunktfunkClient;
sourceTree = "<group>";
};
E295569D300948B9009F939C /* PunktfunkWidgets */ = {
isa = PBXFileSystemSynchronizedRootGroup;
exceptions = (
E29556AD300948BA009F939C /* Exceptions for "PunktfunkWidgets" folder in "PunktfunkWidgetsExtension" target */,
);
path = PunktfunkWidgets;
sourceTree = "<group>";
};
/* End PBXFileSystemSynchronizedRootGroup section */
/* Begin PBXFrameworksBuildPhase section */
@@ -58,14 +108,27 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955694300948B9009F939C /* Frameworks */ = {
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
E2CAFE000000000000000001 /* PunktfunkShared in Frameworks */,
E295569C300948B9009F939C /* SwiftUI.framework in Frameworks */,
E295569A300948B9009F939C /* WidgetKit.framework in Frameworks */,
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXFrameworksBuildPhase section */
/* Begin PBXGroup section */
AA0000000000000000000007 = {
isa = PBXGroup;
children = (
E295577B30094CE5009F939C /* PunktfunkWidgetsExtension.entitlements */,
AA0000000000000000000002 /* App */,
AA0000000000000000000003 /* Sources/PunktfunkClient */,
E295569D300948B9009F939C /* PunktfunkWidgets */,
E2955698300948B9009F939C /* Frameworks */,
AA0000000000000000000008 /* Products */,
);
sourceTree = "<group>";
@@ -76,10 +139,20 @@
AA0000000000000000000001 /* Punktfunk.app */,
BB0000000000000000000001 /* Punktfunk-iOS.app */,
CC0000000000000000000001 /* Punktfunk-tvOS.app */,
E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */,
);
name = Products;
sourceTree = "<group>";
};
E2955698300948B9009F939C /* Frameworks */ = {
isa = PBXGroup;
children = (
E2955699300948B9009F939C /* WidgetKit.framework */,
E295569B300948B9009F939C /* SwiftUI.framework */,
);
name = Frameworks;
sourceTree = "<group>";
};
/* End PBXGroup section */
/* Begin PBXNativeTarget section */
@@ -114,10 +187,12 @@
BB000000000000000000000B /* Sources */,
BB0000000000000000000004 /* Frameworks */,
BB000000000000000000000C /* Resources */,
E29556AA300948BA009F939C /* Embed Foundation Extensions */,
);
buildRules = (
);
dependencies = (
E29556A8300948BA009F939C /* PBXTargetDependency */,
);
fileSystemSynchronizedGroups = (
AA0000000000000000000002 /* App */,
@@ -156,6 +231,29 @@
productReference = CC0000000000000000000001 /* Punktfunk-tvOS.app */;
productType = "com.apple.product-type.application";
};
E2955696300948B9009F939C /* PunktfunkWidgetsExtension */ = {
isa = PBXNativeTarget;
buildConfigurationList = E29556AE300948BA009F939C /* Build configuration list for PBXNativeTarget "PunktfunkWidgetsExtension" */;
buildPhases = (
E2955693300948B9009F939C /* Sources */,
E2955694300948B9009F939C /* Frameworks */,
E2955695300948B9009F939C /* Resources */,
);
buildRules = (
);
dependencies = (
);
fileSystemSynchronizedGroups = (
E295569D300948B9009F939C /* PunktfunkWidgets */,
);
name = PunktfunkWidgetsExtension;
packageProductDependencies = (
E2CAFE000000000000000002 /* PunktfunkShared */,
);
productName = PunktfunkWidgetsExtension;
productReference = E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */;
productType = "com.apple.product-type.app-extension";
};
/* End PBXNativeTarget section */
/* Begin PBXProject section */
@@ -163,11 +261,15 @@
isa = PBXProject;
attributes = {
BuildIndependentTargetsInParallel = 1;
LastSwiftUpdateCheck = 2700;
LastUpgradeCheck = 2700;
TargetAttributes = {
AA0000000000000000000009 = {
CreatedOnToolsVersion = 26.0;
};
E2955696300948B9009F939C = {
CreatedOnToolsVersion = 27.0;
};
};
};
buildConfigurationList = AA000000000000000000000E /* Build configuration list for PBXProject "Punktfunk" */;
@@ -190,6 +292,7 @@
AA0000000000000000000009 /* Punktfunk */,
BB0000000000000000000009 /* Punktfunk-iOS */,
CC0000000000000000000009 /* Punktfunk-tvOS */,
E2955696300948B9009F939C /* PunktfunkWidgetsExtension */,
);
};
/* End PBXProject section */
@@ -216,6 +319,13 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955695300948B9009F939C /* Resources */ = {
isa = PBXResourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXResourcesBuildPhase section */
/* Begin PBXSourcesBuildPhase section */
@@ -240,8 +350,23 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955693300948B9009F939C /* Sources */ = {
isa = PBXSourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXSourcesBuildPhase section */
/* Begin PBXTargetDependency section */
E29556A8300948BA009F939C /* PBXTargetDependency */ = {
isa = PBXTargetDependency;
target = E2955696300948B9009F939C /* PunktfunkWidgetsExtension */;
targetProxy = E29556A7300948BA009F939C /* PBXContainerItemProxy */;
};
/* End PBXTargetDependency section */
/* Begin XCBuildConfiguration section */
AA0000000000000000000010 /* Debug */ = {
isa = XCBuildConfiguration;
@@ -564,6 +689,97 @@
};
name = Release;
};
E29556AB300948BA009F939C /* Debug */ = {
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
ASSETCATALOG_COMPILER_WIDGET_BACKGROUND_COLOR_NAME = WidgetBackground;
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CLANG_ENABLE_OBJC_WEAK = YES;
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
CODE_SIGN_ENTITLEMENTS = PunktfunkWidgetsExtension.entitlements;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = F4H37KF6WC;
GCC_C_LANGUAGE_STANDARD = gnu17;
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = PunktfunkWidgets/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = PunktfunkWidgets;
INFOPLIST_KEY_NSHumanReadableCopyright = "";
IPHONEOS_DEPLOYMENT_TARGET = 27.0;
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/Frameworks",
"@executable_path/../../Frameworks",
);
LOCALIZATION_PREFERS_STRING_CATALOGS = YES;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = io.unom.punktfunk.widgets;
PRODUCT_NAME = "$(TARGET_NAME)";
REGISTER_APP_GROUPS = YES;
SDKROOT = iphoneos;
SKIP_INSTALL = YES;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
name = Debug;
};
E29556AC300948BA009F939C /* Release */ = {
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
ASSETCATALOG_COMPILER_WIDGET_BACKGROUND_COLOR_NAME = WidgetBackground;
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CLANG_ENABLE_OBJC_WEAK = YES;
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
CODE_SIGN_ENTITLEMENTS = PunktfunkWidgetsExtension.entitlements;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = F4H37KF6WC;
GCC_C_LANGUAGE_STANDARD = gnu17;
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = PunktfunkWidgets/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = PunktfunkWidgets;
INFOPLIST_KEY_NSHumanReadableCopyright = "";
IPHONEOS_DEPLOYMENT_TARGET = 27.0;
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/Frameworks",
"@executable_path/../../Frameworks",
);
LOCALIZATION_PREFERS_STRING_CATALOGS = YES;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = io.unom.punktfunk.widgets;
PRODUCT_NAME = "$(TARGET_NAME)";
REGISTER_APP_GROUPS = YES;
SDKROOT = iphoneos;
SKIP_INSTALL = YES;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
VALIDATE_PRODUCT = YES;
};
name = Release;
};
/* End XCBuildConfiguration section */
/* Begin XCConfigurationList section */
@@ -603,6 +819,15 @@
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
E29556AE300948BA009F939C /* Build configuration list for PBXNativeTarget "PunktfunkWidgetsExtension" */ = {
isa = XCConfigurationList;
buildConfigurations = (
E29556AB300948BA009F939C /* Debug */,
E29556AC300948BA009F939C /* Release */,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
/* End XCConfigurationList section */
/* Begin XCLocalSwiftPackageReference section */
@@ -636,6 +861,10 @@
isa = XCSwiftPackageProductDependency;
productName = PunktfunkKit;
};
E2CAFE000000000000000002 /* PunktfunkShared */ = {
isa = XCSwiftPackageProductDependency;
productName = PunktfunkShared;
};
DD0000000000000000000002 /* SwiftUINavigationTransitions */ = {
isa = XCSwiftPackageProductDependency;
package = DD0000000000000000000001 /* XCRemoteSwiftPackageReference "swiftui-navigation-transitions" */;
@@ -0,0 +1,11 @@
{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,35 @@
{
"images" : [
{
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
},
{
"appearances" : [
{
"appearance" : "luminosity",
"value" : "dark"
}
],
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
},
{
"appearances" : [
{
"appearance" : "luminosity",
"value" : "tinted"
}
],
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,6 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,11 @@
{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,186 @@
// Home-Screen / Lock-Screen quick-launch widget (kind "PunktfunkHosts"). Reads the saved-host
// store from the shared App-Group suite, sorts most-recent-first, and deep-links each host into a
// session via `punktfunk://connect/<uuid>` the app's onOpenURL routes it through the normal
// connect path (trust policy / WoL / approval all apply).
//
// No reachability probing in v1 (a UDP check has no place in a timeline build; WoL handles offline
// hosts on tap). Timeline is a single `.never` entry the app pushes reloads on store changes
// (HostStore WidgetCenter.reloadTimelines).
import SwiftUI
import WidgetKit
import PunktfunkShared
// MARK: - Timeline
struct HostsEntry: TimelineEntry {
let date: Date
let hosts: [StoredHost]
}
struct HostsProvider: TimelineProvider {
func placeholder(in context: Context) -> HostsEntry {
HostsEntry(date: .now, hosts: [])
}
func getSnapshot(in context: Context, completion: @escaping (HostsEntry) -> Void) {
completion(HostsEntry(date: .now, hosts: Self.loadHosts()))
}
func getTimeline(in context: Context, completion: @escaping (Timeline<HostsEntry>) -> Void) {
// Single entry, never auto-refresh: the app reloads this timeline whenever the store
// changes (a new host, a fresh connect reordering by recency).
let entry = HostsEntry(date: .now, hosts: Self.loadHosts())
completion(Timeline(entries: [entry], policy: .never))
}
/// Decode the shared-suite host JSON (same wire format the app writes), most-recent first.
static func loadHosts() -> [StoredHost] {
guard let data = AppGroup.defaults.data(forKey: DefaultsKey.hosts),
let hosts = try? JSONDecoder().decode([StoredHost].self, from: data)
else { return [] }
return hosts.sorted {
($0.lastConnected ?? .distantPast) > ($1.lastConnected ?? .distantPast)
}
}
}
// MARK: - Widget
struct HostsWidget: Widget {
var body: some WidgetConfiguration {
StaticConfiguration(kind: "PunktfunkHosts", provider: HostsProvider()) { entry in
HostsWidgetView(entry: entry)
.containerBackground(.fill.tertiary, for: .widget)
}
.configurationDisplayName("Punktfunk Hosts")
.description("Quick-launch your recent streaming hosts.")
.supportedFamilies([
.systemSmall, .systemMedium, .accessoryCircular, .accessoryRectangular,
])
}
}
// MARK: - Views
struct HostsWidgetView: View {
@Environment(\.widgetFamily) private var family
let entry: HostsEntry
var body: some View {
switch family {
case .systemMedium:
MediumHostsView(hosts: entry.hosts)
case .accessoryCircular:
CircularHostView(host: entry.hosts.first)
case .accessoryRectangular:
RectangularHostView(host: entry.hosts.first)
default: // systemSmall + fallback
SmallHostView(host: entry.hosts.first)
}
}
}
/// Deep link that connects to a stored host.
private func connectURL(_ host: StoredHost) -> URL {
DeepLink.connect(host: host.id, launchID: nil).url
}
private struct SmallHostView: View {
let host: StoredHost?
var body: some View {
if let host {
VStack(alignment: .leading, spacing: 6) {
Image(systemName: "play.tv.fill")
.font(.title2)
.foregroundStyle(.tint)
Spacer(minLength: 0)
Text(host.displayName)
.font(.headline)
.lineLimit(2)
if let last = host.lastConnected {
Text(last, format: .relative(presentation: .named))
.font(.caption2)
.foregroundStyle(.secondary)
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
.widgetURL(connectURL(host))
} else {
EmptyHostView()
}
}
}
private struct MediumHostsView: View {
let hosts: [StoredHost]
var body: some View {
if hosts.isEmpty {
EmptyHostView()
} else {
VStack(alignment: .leading, spacing: 8) {
Text("Punktfunk")
.font(.caption).bold()
.foregroundStyle(.tint)
ForEach(hosts.prefix(4)) { host in
Link(destination: connectURL(host)) {
HStack {
Image(systemName: "play.tv.fill")
.foregroundStyle(.tint)
Text(host.displayName)
.font(.subheadline)
.lineLimit(1)
Spacer()
if let last = host.lastConnected {
Text(last, format: .relative(presentation: .named))
.font(.caption2)
.foregroundStyle(.secondary)
}
}
}
}
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
}
}
}
private struct CircularHostView: View {
let host: StoredHost?
var body: some View {
ZStack {
AccessoryWidgetBackground()
Image(systemName: "play.tv.fill")
}
.widgetURL(host.map(connectURL))
}
}
private struct RectangularHostView: View {
let host: StoredHost?
var body: some View {
HStack {
Image(systemName: "play.tv.fill")
Text(host?.displayName ?? "Punktfunk")
.lineLimit(1)
}
.widgetURL(host.map(connectURL))
}
}
private struct EmptyHostView: View {
var body: some View {
VStack(spacing: 6) {
Image(systemName: "play.tv")
.font(.title2)
.foregroundStyle(.secondary)
Text("Open Punktfunk to add a host.")
.font(.caption)
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
}
+11
View File
@@ -0,0 +1,11 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSExtension</key>
<dict>
<key>NSExtensionPointIdentifier</key>
<string>com.apple.widgetkit-extension</string>
</dict>
</dict>
</plist>
@@ -0,0 +1,20 @@
// The widget extension's entry point. ONE extension target (bundle id io.unom.punktfunk.widgets,
// iOS only) hosts both the launcher widgets and the Live Activity UI. It links PunktfunkShared and
// NOTHING else never PunktfunkKit (Rust staticlib + presentation layer would blow the widget
// process's ~30 MB budget).
//
// These files are NOT part of the SwiftPM package (Package.swift doesn't declare a PunktfunkWidgets
// target, so `swift build` ignores the directory). They compile only in the Xcode widget-extension
// target you add pointing at this folder see design/apple-live-activities-and-widgets.md §M1 and
// the GUI checklist.
import SwiftUI
import WidgetKit
@main
struct PunktfunkWidgetBundle: WidgetBundle {
var body: some Widget {
HostsWidget()
PunktfunkSessionLiveActivity()
}
}
@@ -0,0 +1,140 @@
// The Live Activity UI (Lock Screen banner + Dynamic Island) for a running session. The app owns
// the Activity's lifecycle (SessionActivityController); this is only its presentation, rendered in
// the widget-extension process from the shared `PunktfunkSessionAttributes`.
//
// The End button runs `EndStreamIntent` (a LiveActivityIntent) IN THE APP's process, which posts
// .punktfunkEndActiveSession the app disconnects. Elapsed time ticks client-side via
// Text(timerInterval:) no per-second push.
import ActivityKit
import AppIntents
import SwiftUI
import WidgetKit
import PunktfunkShared
struct PunktfunkSessionLiveActivity: Widget {
var body: some WidgetConfiguration {
ActivityConfiguration(for: PunktfunkSessionAttributes.self) { context in
LockScreenView(context: context)
.activitySystemActionForegroundColor(.white)
} dynamicIsland: { context in
DynamicIsland {
DynamicIslandExpandedRegion(.leading) {
Label {
Text(context.attributes.hostName).font(.caption).lineLimit(1)
} icon: {
Image(systemName: "play.tv.fill")
}
.foregroundStyle(.tint)
}
DynamicIslandExpandedRegion(.trailing) {
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.font(.caption).monospacedDigit()
.frame(maxWidth: 56)
.foregroundStyle(.secondary)
}
DynamicIslandExpandedRegion(.center) {
if let title = context.attributes.launchTitle {
Text(title).font(.caption2).lineLimit(1).foregroundStyle(.secondary)
}
}
DynamicIslandExpandedRegion(.bottom) {
VStack(spacing: 6) {
Text(context.state.modeLine)
.font(.caption2).foregroundStyle(.secondary).lineLimit(1)
StageLine(state: context.state)
EndButton()
}
}
} compactLeading: {
Image(systemName: "play.tv.fill").foregroundStyle(.tint)
} compactTrailing: {
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.monospacedDigit()
.frame(maxWidth: 44)
} minimal: {
Image(systemName: "play.tv.fill").foregroundStyle(.tint)
}
}
}
}
// MARK: - Lock Screen banner
private struct LockScreenView: View {
let context: ActivityViewContext<PunktfunkSessionAttributes>
var body: some View {
HStack(alignment: .top, spacing: 12) {
Image(systemName: "play.tv.fill")
.font(.title2)
.foregroundStyle(.tint)
VStack(alignment: .leading, spacing: 3) {
HStack {
Text(context.attributes.hostName).font(.headline).lineLimit(1)
Spacer()
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.font(.subheadline).monospacedDigit()
.foregroundStyle(.secondary)
}
if let title = context.attributes.launchTitle {
Text(title).font(.caption).foregroundStyle(.secondary).lineLimit(1)
}
Text(context.state.modeLine)
.font(.caption2).foregroundStyle(.secondary).lineLimit(1)
StageLine(state: context.state)
}
if context.state.stage == .background {
EndButton()
}
}
.padding()
}
}
// MARK: - Shared pieces
/// The stage badge + (while backgrounded) the auto-disconnect countdown.
private struct StageLine: View {
let state: PunktfunkSessionAttributes.ContentState
var body: some View {
switch state.stage {
case .streaming:
EmptyView()
case .background:
if let deadline = state.backgroundDeadline {
HStack(spacing: 3) {
Text("Keeps running for")
Text(timerInterval: Date()...deadline, countsDown: true)
.monospacedDigit()
}
.font(.caption2)
.foregroundStyle(.secondary)
} else {
badge("Running in background", .orange)
}
case .reconnecting:
badge("Reconnecting…", .yellow)
case .ending:
badge("Session ended", .secondary)
}
}
private func badge(_ text: String, _ color: Color) -> some View {
Text(text).font(.caption2).foregroundStyle(color)
}
}
/// End-stream button runs EndStreamIntent in the app process (LiveActivityIntent).
private struct EndButton: View {
var body: some View {
Button(intent: EndStreamIntent()) {
Label("End", systemImage: "stop.fill")
.font(.caption).bold()
}
.tint(.red)
.buttonStyle(.bordered)
}
}
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
@@ -0,0 +1,10 @@
import Foundation
import PunktfunkKit
/// A fresh `pair=required`/unknown host pending a trust decision: drives both the "request access
/// vs. pair with PIN" choice and the subsequent approval wait. `advertisedFingerprint` is the
/// discovered host's advertised cert (nil for a manually-typed host trust-on-first-use).
struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
@@ -24,6 +24,7 @@ struct ContentView: View {
@AppStorage(DefaultsKey.streamWidth) private var width = 1920
@AppStorage(DefaultsKey.streamHeight) private var height = 1080
@AppStorage(DefaultsKey.streamHz) private var hz = 60
@AppStorage(DefaultsKey.renderScale) private var renderScale = 1.0
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@@ -51,6 +52,15 @@ struct ContentView: View {
}
}
@State private var showAddHost = false
/// A `punktfunk://` deep link (widget / Siri / Shortcuts) couldn't be honored unknown host, or
/// a live session is already up. Surfaced as an informational alert (distinct from the
/// "Connection failed" one, which is for actual connect errors).
@State private var deepLinkNotice: String?
#if os(iOS)
/// Owns the Live Activity for the running session (Lock Screen / Dynamic Island). Driven from
/// the session model's published state below; iPhone/iPad only.
@State private var liveActivity = SessionActivityController()
#endif
@State private var pairingTarget: StoredHost?
/// A fresh `pair=required`/unknown host the user tapped: drives the choice between no-PIN
/// delegated approval ("Request Access") and the SPAKE2 PIN ceremony (rule 3b).
@@ -92,6 +102,14 @@ struct ContentView: View {
/// fires Wake-on-LAN up front and falls into the "Waking" wait if the dial fails. Off: connects
/// go straight through with no wake. The explicit "Wake Host" action is unaffected either way.
@AppStorage(DefaultsKey.autoWake) private var autoWakeEnabled = true
/// Background keep-alive (Settings General, iOS-only). Default OFF (today's freeze-on-background
/// is the default). When on, backgrounding a live session keeps audio + the connection alive and
/// drops video, auto-disconnecting after `backgroundTimeoutMinutes`.
@AppStorage(DefaultsKey.backgroundKeepAlive) private var backgroundKeepAlive = false
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) private var backgroundTimeoutMinutes = 10
/// scenePhase drives the keep-alive: use THIS, not the willResignActive observers resign-active
/// also fires for Control Center / app-switcher peeks, where the disconnect timer must not start.
@Environment(\.scenePhase) private var scenePhase
private var gamepadUIActive: Bool {
GamepadUIEnvironment.isActive(
gamepadConnected: gamepadManager.active != nil, enabledSetting: gamepadUIEnabled)
@@ -113,7 +131,62 @@ struct ContentView: View {
.onAppear {
seedDefaultModeIfNeeded()
autoConnectIfAsked()
#if os(iOS)
SessionActivityController.sweepOrphans() // end any Activity a prior killed launch left
#endif
}
// Deep links (widget quick-launch, Siri/Shortcuts): route into the SAME connect path a card
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
// parallel session this drives the one `model` ContentView owns.
.onOpenURL { handleDeepLink($0) }
#if os(iOS)
// Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a
// transient peek) is ignored so the disconnect timer never starts for a Control-Center pull.
.onChange(of: scenePhase) { _, phase in
switch phase {
case .background:
if backgroundKeepAlive, model.phase == .streaming {
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
}
case .active:
model.exitBackground()
default:
break
}
}
// Live Activity lifecycle, driven from the model's published state.
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
if let host = model.activeHost {
liveActivity.begin(
hostID: host.id, hostName: host.displayName,
launchTitle: nil, // no live foreground-app title mid-session (v1)
modeLine: currentModeLine(), startedAt: Date())
}
case .idle:
liveActivity.end()
default:
break
}
}
.onChange(of: model.isBackgrounded) { _, backgrounded in
liveActivity.update {
$0.stage = backgrounded ? .background : .streaming
$0.backgroundDeadline = model.backgroundDeadline
}
}
// The Live Activity's / Shortcuts' End button runs EndStreamIntent in-process, which posts
// this tear the session down deliberately (quit-close the host).
.onReceive(NotificationCenter.default.publisher(for: .punktfunkEndActiveSession)) { _ in
model.disconnect(deliberate: true)
}
// Connect App Intent (Siri/Shortcuts): route its punktfunk:// URL through the same handler
// as a widget tap.
.onReceive(NotificationCenter.default.publisher(for: .punktfunkOpenDeepLink)) { note in
if let url = note.object as? URL { handleDeepLink(url) }
}
#endif
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
@@ -151,6 +224,9 @@ struct ContentView: View {
#if !os(tvOS)
.focusedSceneValue(\.sessionFocus, SessionFocus(
isStreaming: model.connection != nil,
clipboardAvailable: model.connection?.hostSupportsClipboard == true,
clipboardOn: model.clipboardEnabled,
toggleClipboard: { model.toggleClipboardSync() },
disconnect: { model.disconnect() }))
#endif
#if os(macOS)
@@ -262,6 +338,59 @@ struct ContentView: View {
+ "console (port 3000 → Pairing). This device connects automatically once you "
+ "approve it — no need to reconnect.")
}
// Informational deep-link outcome (unknown host / already streaming). Not an error.
.alert("Can't open", isPresented: deepLinkNoticePresented) {
Button("OK", role: .cancel) {}
} message: {
Text(deepLinkNotice ?? "")
}
}
/// Presentation flag for the informational deep-link alert. Extracted from the `.alert` call so
/// the manual get/set Binding type-checks on its own instead of inflating the body chain's
/// budget (adding it inline tips SwiftUI's per-expression limit see the split sections idiom).
private var deepLinkNoticePresented: Binding<Bool> {
Binding(get: { deepLinkNotice != nil }, set: { if !$0 { deepLinkNotice = nil } })
}
#if os(iOS)
/// The Live Activity mode line, e.g. "2560×1440 @120 · HEVC · HDR", from the live connection.
private func currentModeLine() -> String {
guard let c = model.connection else { return "" }
let codec: String
switch c.videoCodec {
case .h264: codec = "H.264"
case .hevc: codec = "HEVC"
case .av1: codec = "AV1"
case .pyrowave: codec = "PyroWave"
}
var line = "\(c.width)×\(c.height)"
if c.refreshHz > 0 { line += " @\(c.refreshHz)" }
line += " · \(codec)"
if c.isHDR { line += " · HDR" }
return line
}
#endif
/// Route a `punktfunk://` deep link into the existing connect path. Rules (per design):
/// unknown host notice + no-op; a live session is up ignore if it's the same host, else
/// tell the user to end the current one first (NEVER tear down a live session on a background
/// tap); otherwise the normal `connect` trust policy, WoL and the approval sheet all apply.
private func handleDeepLink(_ url: URL) {
guard case let .connect(hostID, launchID)? = DeepLink(url) else { return }
guard let host = store.hosts.first(where: { $0.id == hostID }) else {
deepLinkNotice = "That host isn't saved on this device."
return
}
if model.phase != .idle {
guard model.activeHost?.id == hostID else {
let current = model.activeHost?.displayName ?? "a host"
deepLinkNotice = "Already streaming \(current). End that session first."
return
}
return // deep-linked to the host we're already on nothing to do
}
connect(host, launchID: launchID)
}
private var home: some View {
@@ -610,6 +739,17 @@ struct ContentView: View {
/// host is back online. `prepareWake` still runs here to LEARN/refresh the MAC now that the host
/// is advertising (and is a harmless no-op otherwise). `onUnreachable` hands a plain connect
/// failure back to the caller (the wake-wait fallback) instead of the error alert.
/// The stream mode to request = the chosen resolution × the render scale, aspect-preserved,
/// even, and clamped to the codec's max dimension. > 1 supersamples for sharpness (the presenter
/// downscales the larger decoded frame to this display); < 1 renders under native and upscales.
/// The match-window path applies the SAME scale to the live window size in `MatchWindowFollower`.
private func scaledMode() -> (width: UInt32, height: UInt32) {
RenderScale.apply(
baseWidth: width, baseHeight: height,
scale: renderScale,
maxDimension: RenderScale.maxDimension(codec: codec))
}
private func startSessionDirect(
_ host: StoredHost, launchID: String? = nil,
allowTofu: Bool, requestAccess: Bool = false, approvalReq: ApprovalRequest? = nil,
@@ -621,7 +761,7 @@ struct ContentView: View {
if let approvalReq { awaitingApproval = approvalReq }
model.connect(
to: host,
width: UInt32(clamping: width), height: UInt32(clamping: height),
width: scaledMode().width, height: scaledMode().height,
hz: UInt32(clamping: hz),
compositor: PunktfunkConnection.Compositor(
rawValue: UInt32(clamping: compositor)) ?? .auto,
@@ -804,7 +944,7 @@ struct ContentView: View {
}
model.connect(
to: host,
width: UInt32(clamping: width), height: UInt32(clamping: height),
width: scaledMode().width, height: scaledMode().height,
hz: UInt32(clamping: hz),
compositor: pref,
gamepad: pad,
@@ -815,71 +955,3 @@ struct ContentView: View {
autoTrust: true)
}
}
#if os(macOS)
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
private struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
if want != isFull { window.toggleFullScreen(nil) }
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
}
}
#endif
/// A fresh `pair=required`/unknown host pending a trust decision: drives both the "request access
/// vs. pair with PIN" choice and the subsequent approval wait. `advertisedFingerprint` is the
/// discovered host's advertised cert (nil for a manually-typed host trust-on-first-use).
private struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
@@ -0,0 +1,83 @@
import PunktfunkKit
import SwiftUI
#if os(macOS)
import AppKit
/// Drives the hosting window in/out of native fullscreen from SwiftUI state, and mirrors the
/// window's ACTUAL fullscreen state back into `isFullscreen` (the user can also toggle it with the
/// green button / F ContentView keys the session view's safe-area handling off the real state,
/// not the setting). Mounted invisibly in the view tree; on each `active` change it captures the
/// window and toggles fullscreen only when the current state differs (so it never fights a toggle
/// already in flight, and never touches a window the user fullscreened manually unless `active`
/// says otherwise).
struct FullscreenController: NSViewRepresentable {
let active: Bool
@Binding var isFullscreen: Bool
/// Holds the window's fullscreen-transition observers so they're rebound on a window change
/// and removed on dismantle.
final class Coordinator {
var observers: [NSObjectProtocol] = []
weak var observedWindow: NSWindow?
/// The last `active` value we DROVE the window to. We toggle only when `active` itself
/// changes (stream start/end) never to correct a mismatch so a deliberate mid-session
/// toggle (F / the green button) isn't snapped back on the next SwiftUI update.
var lastActive: Bool?
deinit { observers.forEach(NotificationCenter.default.removeObserver(_:)) }
}
func makeCoordinator() -> Coordinator { Coordinator() }
func makeNSView(context: Context) -> NSView { NSView() }
func updateNSView(_ view: NSView, context: Context) {
let want = active
let isFullscreen = $isFullscreen
let coordinator = context.coordinator
DispatchQueue.main.async {
guard let window = view.window else { return }
observeTransitions(of: window, coordinator: coordinator)
let isFull = window.styleMask.contains(.fullScreen)
if isFullscreen.wrappedValue != isFull { isFullscreen.wrappedValue = isFull }
// Drive the window only on an `active` EDGE (stream start/end), not to close a mismatch
// so a user's F / green-button toggle stays put. First pass (lastActive == nil) just
// records the state without toggling, so mounting never yanks a window into fullscreen.
if coordinator.lastActive != want {
coordinator.lastActive = want
if want != isFull { window.toggleFullScreen(nil) }
}
}
}
/// `willEnter` (not did) so the video goes edge-to-edge while the title bar is already
/// animating away; `didExit` so the top inset returns only once the title bar is back
/// no black gap in either direction.
private func observeTransitions(of window: NSWindow, coordinator: Coordinator) {
guard coordinator.observedWindow !== window else { return }
coordinator.observers.forEach(NotificationCenter.default.removeObserver(_:))
coordinator.observers.removeAll()
coordinator.observedWindow = window
let isFullscreen = $isFullscreen
for (name, value) in [
(NSWindow.willEnterFullScreenNotification, true),
(NSWindow.didExitFullScreenNotification, false),
] {
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: name, object: window, queue: .main
) { _ in
isFullscreen.wrappedValue = value
})
}
// The Stream menu's "Toggle Fullscreen" (F) and InputCapture's captured-state interception
// both post this; flip the KEY window only (posted app-wide, object nil). The transition
// observers above then mirror the real state back into the binding.
coordinator.observers.append(NotificationCenter.default.addObserver(
forName: .punktfunkToggleFullscreen, object: nil, queue: .main
) { [weak window] _ in
guard let window, window.isKeyWindow else { return }
window.toggleFullScreen(nil)
})
}
}
#endif
@@ -20,6 +20,12 @@ struct AddHostSheet: View {
@State private var address: String
@State private var port: Int
@State private var mac: String
#if os(macOS)
/// Share the clipboard with this host (macOS sessions only; design
/// clipboard-and-file-transfer.md §5.3). Off by default; honored only when the host
/// advertises the capability at connect.
@State private var clipboardSync: Bool
#endif
#if os(tvOS)
private enum EditField: String, Identifiable {
case name, address, port, mac
@@ -41,6 +47,9 @@ struct AddHostSheet: View {
_port = State(initialValue: Int(existing?.port ?? 9777))
let stored = existing?.macAddresses ?? []
_mac = State(initialValue: (stored.isEmpty ? suggestedMacs : stored).joined(separator: ", "))
#if os(macOS)
_clipboardSync = State(initialValue: existing?.clipboardSync ?? false)
#endif
}
var body: some View {
@@ -96,6 +105,9 @@ struct AddHostSheet: View {
#if os(iOS)
.textInputAutocapitalization(.never)
#endif
#if os(macOS)
Toggle("Share clipboard with this host", isOn: $clipboardSync)
#endif
}
#if !os(tvOS)
.formStyle(.grouped)
@@ -147,6 +159,11 @@ struct AddHostSheet: View {
host.address = address.trimmingCharacters(in: .whitespaces)
host.port = UInt16(clamping: port)
host.macAddresses = Self.parseMacs(mac)
#if os(macOS)
// nil when off: the key stays absent from the saved JSON (forward-compat, and "never
// opted in" and "opted out" read the same off).
host.clipboardSync = clipboardSync ? true : nil
#endif
onSave(host)
dismiss()
}
@@ -0,0 +1,102 @@
// Siri / Shortcuts / Spotlight surface (design §M4). Deliberately thin: every action already has an
// internal entry point M0's deep-link router (connect / connect-and-launch), M3's in-process
// end-session hook, and the existing Wake-on-LAN path so these intents only wrap them.
//
// Gated os(iOS): the AppShortcutsProvider bundles `EndStreamIntent`, which is a LiveActivityIntent
// (iPhone/iPad only). Connect/Wake themselves are plain AppIntents; they live here with the
// provider rather than being split across platforms. `HostEntity` (the parameter type) is in
// PunktfunkShared so the widget's configuration intent can share it.
#if os(iOS)
import AppIntents
import Foundation
import PunktfunkKit
/// Load a full saved host (MACs, address) from the shared App-Group store by id HostEntity only
/// carries id + name.
private func loadStoredHost(_ id: UUID) -> StoredHost? {
guard let data = AppGroup.defaults.data(forKey: DefaultsKey.hosts),
let hosts = try? JSONDecoder().decode([StoredHost].self, from: data)
else { return nil }
return hosts.first { $0.id == id }
}
/// Start a session with a stored host (optionally launching a title). Foregrounds the app and
/// routes through the SAME `.onOpenURL` path a widget tap uses trust policy, WoL and the approval
/// sheet all apply, and its guards (unknown host, already-streaming) hold.
struct ConnectToHostIntent: AppIntent {
static let title: LocalizedStringResource = "Connect to Host"
static let description = IntentDescription("Start a Punktfunk streaming session with a host.")
static let openAppWhenRun = true
@Parameter(title: "Host") var host: HostEntity
@Parameter(title: "Game ID", description: "Optional store id like steam:570")
var launchID: String?
func perform() async throws -> some IntentResult {
let url = DeepLink.connect(host: host.id, launchID: launchID).url
await MainActor.run {
NotificationCenter.default.post(name: .punktfunkOpenDeepLink, object: url)
}
return .result()
}
}
/// Wake a sleeping host (magic packet). No `openAppWhenRun` usable in automations ("when I get
/// home, wake the tower") without foregrounding the app.
struct WakeHostIntent: AppIntent {
static let title: LocalizedStringResource = "Wake Host"
static let description = IntentDescription("Send a Wake-on-LAN magic packet to a host.")
@Parameter(title: "Host") var host: HostEntity
func perform() async throws -> some IntentResult {
guard let stored = loadStoredHost(host.id), !stored.wakeMacs.isEmpty else {
throw IntentError.noWakeAddress
}
PunktfunkConnection.wakeOnLAN(macs: stored.wakeMacs, lastKnownIP: stored.address)
return .result()
}
}
/// Errors surfaced to Siri/Shortcuts. `CustomLocalizedStringResourceConvertible` makes the message
/// show as the intent's failure text.
enum IntentError: Error, CustomLocalizedStringResourceConvertible {
case noWakeAddress
var localizedStringResource: LocalizedStringResource {
switch self {
case .noWakeAddress:
// One string LITERAL LocalizedStringResource is ExpressibleByStringLiteral, but a
// `"" + ""` concatenation is a runtime String it can't convert.
return "That host has no saved Wake-on-LAN address yet. Connect to it once so Punktfunk can learn it."
}
}
}
/// Zero-setup Siri / Spotlight phrases. Parameterized phrases resolve a `HostEntity` by name; stays
/// well under the 10-shortcut cap.
struct PunktfunkShortcuts: AppShortcutsProvider {
static var appShortcuts: [AppShortcut] {
AppShortcut(
intent: ConnectToHostIntent(),
phrases: [
"Connect to \(\.$host) in \(.applicationName)",
"Stream \(\.$host) with \(.applicationName)",
],
shortTitle: "Connect", systemImageName: "play.tv.fill")
AppShortcut(
intent: WakeHostIntent(),
phrases: [
"Wake \(\.$host) with \(.applicationName)",
],
shortTitle: "Wake Host", systemImageName: "power")
AppShortcut(
intent: EndStreamIntent(),
phrases: [
"End the \(.applicationName) stream",
],
shortTitle: "End Stream", systemImageName: "stop.fill")
}
}
#endif
@@ -0,0 +1,89 @@
// Owns the ActivityKit Live Activity lifecycle for a streaming session (iPhone/iPad only). Driven
// by ContentView from the session model's published state (phase / isBackgrounded / deadline) so
// none of this leaks into the cross-platform SessionModel. Local updates only (`pushType: nil`)
// the app process is alive whenever there's a session to report, so there's no push token plumbing.
//
// Gated os(iOS): ActivityKit is iPhone/iPad only. Minimum deployment is iOS 17, so no @available
// guards are needed (Activity has existed since 16.1).
#if os(iOS)
import ActivityKit
import Foundation
// PunktfunkKit re-exports PunktfunkShared (@_exported), so the app target sees PunktfunkSessionAttributes
// without linking the Shared product directly same pattern as StoredHost in HostStore.
import PunktfunkKit
@MainActor
final class SessionActivityController {
private var activity: Activity<PunktfunkSessionAttributes>?
/// The last pushed state, so an update can mutate one field and keep the rest (notably
/// `startedAt`, which the Lock-Screen timer ticks from).
private var state: PunktfunkSessionAttributes.ContentState?
/// How far past the next expected update to mark the content stale a frozen opt-out session
/// then greys out instead of showing a lying clock.
private static let staleWindow: TimeInterval = 90
var isActive: Bool { activity != nil }
/// End any Activity left over from a previous launch that was killed mid-session. Call once at
/// app start (ContentView.onAppear).
static func sweepOrphans() {
Task {
for activity in Activity<PunktfunkSessionAttributes>.activities {
await activity.end(nil, dismissalPolicy: .immediate)
}
}
}
/// Start the Live Activity for a freshly-streaming session. No-op if the user disabled Live
/// Activities for the app, or one is already up.
func begin(hostID: UUID, hostName: String, launchTitle: String?, modeLine: String, startedAt: Date) {
guard ActivityAuthorizationInfo().areActivitiesEnabled, activity == nil else { return }
let attributes = PunktfunkSessionAttributes(
hostID: hostID, hostName: hostName, launchTitle: launchTitle)
let initial = PunktfunkSessionAttributes.ContentState(
stage: .streaming, startedAt: startedAt, modeLine: modeLine)
state = initial
do {
activity = try Activity.request(
attributes: attributes,
content: content(initial),
pushType: nil)
} catch {
activity = nil
state = nil
}
}
/// Coalesced update: mutate the running state in place (keeps `startedAt` etc.) and push once.
/// No-op when there's no live Activity.
func update(_ mutate: (inout PunktfunkSessionAttributes.ContentState) -> Void) {
guard let activity, var next = state else { return }
mutate(&next)
state = next
Task { await activity.update(content(next)) }
}
/// End with a final "ended" state, dismissed a few seconds later.
func end() {
guard let activity, var final = state else {
self.activity = nil
state = nil
return
}
self.activity = nil
state = nil
final.stage = .ending
final.backgroundDeadline = nil
Task {
await activity.end(content(final), dismissalPolicy: .after(.now + 4))
}
}
private func content(_ s: PunktfunkSessionAttributes.ContentState)
-> ActivityContent<PunktfunkSessionAttributes.ContentState> {
ActivityContent(state: s, staleDate: Date().addingTimeInterval(Self.staleWindow))
}
}
#endif
@@ -139,6 +139,18 @@ final class SessionModel: ObservableObject {
private var audio: SessionAudio?
private var gamepadCapture: GamepadCapture?
private var gamepadFeedback: GamepadFeedback?
#if os(macOS)
/// The live session's clipboard bridge (design/clipboard-and-file-transfer.md §5) created
/// by `beginStreaming` when the per-host toggle is on and the host advertises
/// `HOST_CAP_CLIPBOARD`; stopped (off-main, drain joined) in `disconnect`.
private var clipboardSync: ClipboardSync?
#endif
/// Whether clipboard sync is live (host-acked `ClipState.enabled`) drives the Stream menu
/// item's title and the settings footnote. Always false off-macOS.
@Published private(set) var clipboardEnabled = false
/// The host's last `ClipState.reason` (`CLIP_REASON_*`) why an enable was refused
/// (backend unavailable / policy disabled / ); 0 = OK.
@Published private(set) var clipboardReason: UInt8 = 0
#if os(tvOS)
/// Siri Remote host pointer while streaming (touch surface moves, press = left click,
/// Play/Pause = right click) + the remote's deliberate exit (hold Back 1 s). See
@@ -148,6 +160,16 @@ final class SessionModel: ObservableObject {
var isBusy: Bool { phase != .idle }
/// True while a streaming session is running in the background under the opt-in keep-alive
/// (audio plays, video dropped, timeout armed). Drives the Live Activity's stage/countdown (M3)
/// and is cleared on foreground or teardown. iOS/iPadOS only in practice.
@Published private(set) var isBackgrounded = false
/// When the backgrounded keep-alive will auto-disconnect (nil unless backgrounded) drives the
/// Live Activity countdown. Set alongside `backgroundTimer`.
@Published private(set) var backgroundDeadline: Date?
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
private var backgroundTimer: DispatchSourceTimer?
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
/// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
@@ -332,6 +354,48 @@ final class SessionModel: ObservableObject {
}
}
// MARK: - Background keep-alive (opt-in, iOS)
/// Enter the backgrounded keep-alive state: keep audio playing, DROP video decode (no GPU work
/// off-screen), mute the mic (privacy), and arm a bounded auto-disconnect. The caller
/// (ContentView's scenePhase driver) gates this on the setting + `.streaming`; a no-op otherwise.
/// The video-drop seam is read by both pumps every iteration (`connection.isVideoDropped`).
func enterBackground(timeoutMinutes: Int) {
guard phase == .streaming, let conn = connection, !isBackgrounded else { return }
isBackgrounded = true
conn.setVideoDropped(true)
audio?.setMicMuted(true)
// Non-deliberate on fire (keep the host linger) so a user who returns late reconnects fast,
// exactly like today's network-drop path. min 1 minute guards a nonsense setting.
let minutes = max(1, timeoutMinutes)
backgroundDeadline = Date().addingTimeInterval(TimeInterval(minutes * 60))
let timer = DispatchSource.makeTimerSource(queue: .main)
timer.schedule(deadline: .now() + .seconds(minutes * 60))
timer.setEventHandler { [weak self] in
// The timer fires on `.main`, so the actor's executor is the main thread here.
MainActor.assumeIsolated { self?.disconnect(deliberate: false) }
}
backgroundTimer?.cancel()
backgroundTimer = timer
timer.resume()
}
/// Return to foreground: cancel the timeout, resume mic + video, and force a clean re-anchor
/// request a fresh IDR (infinite GOP: it won't come on its own) and let the pump's freeze gate
/// withhold the concealed frames until it lands (it auto-arms on the resumed frame-index gap).
func exitBackground() {
guard isBackgrounded else { return }
isBackgrounded = false
backgroundDeadline = nil
backgroundTimer?.cancel()
backgroundTimer = nil
audio?.setMicMuted(false)
if let conn = connection {
conn.setVideoDropped(false)
conn.requestKeyframe()
}
}
/// The user confirmed the fingerprint: returns it for pinning and enters streaming.
func confirmTrust() -> Data? {
guard case .awaitingTrust(let fingerprint) = phase else { return nil }
@@ -349,6 +413,11 @@ final class SessionModel: ObservableObject {
func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate()
statsTimer = nil
// Drop any armed background keep-alive (incl. the timeout that just fired us).
backgroundTimer?.cancel()
backgroundTimer = nil
isBackgrounded = false
backgroundDeadline = nil
let audio = self.audio
self.audio = nil
// Gamepad capture is main-actor (releases held buttons on the wire while the
@@ -361,6 +430,12 @@ final class SessionModel: ObservableObject {
#endif
let feedback = gamepadFeedback
gamepadFeedback = nil
#if os(macOS)
let clipboard = clipboardSync
clipboardSync = nil
#endif
clipboardEnabled = false
clipboardReason = 0
if let conn = connection {
// Drain-thread teardown waits the pullers out and close() waits out in-flight
// polls + joins the Rust worker threads keep all of it off the main actor,
@@ -368,6 +443,9 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
#if os(macOS)
clipboard?.stop() // disables sync on the wire while the connection is still up
#endif
// Deliberate user quit tell the host to skip the keep-alive linger (must precede close).
if deliberate { conn.disconnectQuit() }
conn.close()
@@ -376,6 +454,9 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
#if os(macOS)
clipboard?.stop()
#endif
}
}
connection = nil
@@ -450,6 +531,14 @@ final class SessionModel: ObservableObject {
let feedback = GamepadFeedback(connection: conn, manager: .shared)
feedback.start()
gamepadFeedback = feedback
#if os(macOS)
// Shared clipboard: opt-in per host AND host-advertised (older hosts / operator-disabled
// hosts never see a ClipControl). Same trust gate as audio nothing is announced
// during the trust prompt.
if activeHost?.clipboardSync == true, conn.hostSupportsClipboard {
startClipboardSync(conn)
}
#endif
#if os(tvOS)
let pointer = SiriRemotePointer(connection: conn)
pointer.onDisconnectRequest = { [weak self] in self?.disconnect() }
@@ -458,6 +547,40 @@ final class SessionModel: ObservableObject {
#endif
}
#if os(macOS)
/// Create + start the session's clipboard bridge and route its host acks into the published
/// UI state. `ClipboardSync.start()` sends the enable; the host's `.state` answer flips
/// `clipboardEnabled` (or leaves it false with a `clipboardReason` the UI can explain).
private func startClipboardSync(_ conn: PunktfunkConnection) {
let sync = ClipboardSync(connection: conn)
sync.onState = { [weak self] enabled, _, reason in
Task { @MainActor in
self?.clipboardEnabled = enabled
self?.clipboardReason = reason
}
}
sync.start()
clipboardSync = sync
}
#endif
/// Flip clipboard sync mid-session (the Stream menu). Off on requires the host cap; on
/// off tears the bridge down (off-main the drain join must not block the main actor) and
/// tells the host, which drops any selection we own there. No-op off-macOS or while idle.
func toggleClipboardSync() {
#if os(macOS)
guard let conn = connection, phase == .streaming else { return }
if let sync = clipboardSync {
clipboardSync = nil
clipboardEnabled = false
clipboardReason = 0
Task.detached { sync.stop() }
} else if conn.hostSupportsClipboard {
startClipboardSync(conn)
}
#endif
}
private func startStatsTimer() {
lastFramesDropped = 0 // a fresh connection's cumulative drop counter starts at 0
latencySplit.reset() // no stale receipts/samples from a previous session
@@ -21,6 +21,12 @@ import SwiftUI
/// `.focusedSceneValue` so the Scene-level commands can drive it.
struct SessionFocus {
var isStreaming: Bool
/// The connected host advertises `HOST_CAP_CLIPBOARD` (gates the Share Clipboard item
/// macOS-only UI, but the fact is platform-neutral).
var clipboardAvailable: Bool
/// Clipboard sync is live (host-acked) drives the item's Stop/Share title.
var clipboardOn: Bool
var toggleClipboard: () -> Void
var disconnect: () -> Void
}
@@ -58,6 +64,15 @@ struct StreamCommands: Commands {
}
.keyboardShortcut("q", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
#if os(macOS)
// Mid-session clipboard flip (design/clipboard-and-file-transfer.md §5.3). Greyed
// when the host doesn't advertise the cap (older host / operator policy off).
Button(session?.clipboardOn == true ? "Stop Sharing Clipboard" : "Share Clipboard") {
session?.toggleClipboard()
}
.keyboardShortcut("c", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true || session?.clipboardAvailable != true)
#endif
Divider()
Button("Disconnect") { session?.disconnect() }
.keyboardShortcut("d", modifiers: [.control, .option, .shift])
@@ -440,6 +440,34 @@ extension SettingsView {
}
}
/// iOS/iPadOS only: keep a backgrounded session alive (audio background mode). Empty elsewhere
/// (tvOS backgrounding semantics differ; macOS isn't gated by the mode) so the shared `.general`
/// detail can reference it unconditionally.
@ViewBuilder var keepAliveSection: some View {
#if os(iOS)
Section {
Toggle("Keep streaming in background", isOn: $backgroundKeepAlive)
if backgroundKeepAlive {
Picker("Disconnect after", selection: $backgroundTimeoutMinutes) {
Text("1 minute").tag(1)
Text("5 minutes").tag(5)
Text("10 minutes").tag(10)
Text("30 minutes").tag(30)
}
}
} header: {
Text("Background")
} footer: {
Text("Off by default: backgrounding the app freezes the session. When on, audio keeps "
+ "playing and the connection stays live (video is dropped to save power) after you "
+ "switch away — and the session auto-disconnects after the time above so it can't "
+ "run down your battery. Returning to the app resumes video instantly.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder var experimentalSection: some View {
Section {
Toggle("Show game library", isOn: $libraryEnabled)
@@ -49,6 +49,8 @@ struct SettingsView: View {
@ObservedObject var gamepads = GamepadManager.shared
@AppStorage(DefaultsKey.gamepadUIEnabled) var gamepadUIEnabled = true
@AppStorage(DefaultsKey.autoWake) var autoWakeEnabled = true
@AppStorage(DefaultsKey.backgroundKeepAlive) var backgroundKeepAlive = false
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) var backgroundTimeoutMinutes = 10
#if DEBUG && !os(tvOS)
@State var showControllerTest = false
#endif
@@ -242,6 +244,7 @@ struct SettingsView: View {
pointerSection
compositorSection
wakeSection
keepAliveSection // iOS-only content; empty on tvOS
}
.formStyle(.grouped)
.navigationTitle("General")
@@ -11,32 +11,13 @@
import Foundation
import PunktfunkKit
import SwiftUI
#if canImport(WidgetKit)
import WidgetKit
#endif
struct StoredHost: Identifiable, Codable, Hashable {
var id = UUID()
var name: String
var address: String
var port: UInt16 = 9777
/// SHA-256 of the host's certificate, set after the user explicitly trusted it.
var pinnedSHA256: Data?
/// Last time a streaming session actually started (nil until the first one).
var lastConnected: Date?
/// Management-API port for the library browser (distinct from the data-plane `port`). Optional
/// (NOT a defaulted non-optional) so older saved hosts whose JSON lacks this key still
/// decode: synthesized Decodable ignores property defaults but treats a missing Optional as
/// nil. Resolve via `effectiveMgmtPort`. (Auth is mTLS by the pinned identity no token.)
var mgmtPort: UInt16?
/// Wake-on-LAN MAC address(es) of the host's wake-capable NIC(s), each `aa:bb:cc:dd:ee:ff`.
/// Learned from the host's mDNS `mac` TXT record while it's awake and persisted here, so the
/// client can send a magic packet to wake the host later (when it's asleep and no longer
/// advertising). Optional (same forward-compat reason as `mgmtPort`); nil until first learned.
var macAddresses: [String]?
var displayName: String { name.isEmpty ? address : name }
var effectiveMgmtPort: UInt16 { mgmtPort ?? punktfunkDefaultMgmtPort }
/// Wake-capable, in a form the wake helper accepts (empty when none learned yet).
var wakeMacs: [String] { macAddresses ?? [] }
}
// `StoredHost` (the model + its JSON codec) now lives in PunktfunkShared so the widget extension
// can read the same store; PunktfunkKit re-exports it. The discovery-join helpers below stay here
// because they reference PunktfunkKit's `DiscoveredHost`/`HostDiscovery`.
extension StoredHost {
/// True when a live mDNS advert (`DiscoveredHost`) describes THIS saved host drives the
@@ -86,8 +67,14 @@ final class HostStore: ObservableObject {
/// never advertises still reads Online. Not persisted (it's live reachability, not config).
@Published var probedOnline: Set<StoredHost.ID> = []
/// The App-Group suite shared with the Widget/Live-Activity extension so a launcher widget
/// sees the same saved hosts. Falls back to `.standard` in an un-entitled process (see
/// `AppGroup.defaults`).
private let defaults = AppGroup.defaults
init() {
if let data = UserDefaults.standard.data(forKey: Self.key),
Self.migrateToAppGroupIfNeeded()
if let data = defaults.data(forKey: Self.key),
let decoded = try? JSONDecoder().decode([StoredHost].self, from: data) {
hosts = decoded
} else {
@@ -95,6 +82,20 @@ final class HostStore: ObservableObject {
}
}
/// One-time move of the saved-host JSON from `UserDefaults.standard` (where every build before
/// the App Group wrote it) into the shared suite. Idempotent: only fires when the suite has no
/// hosts yet but standard does. The old value is LEFT in place during a staged TestFlight
/// rollout an older build still reads `.standard`, so tombstoning it now would hide hosts from
/// the not-yet-updated app. Remove the standard copy a release later.
private static func migrateToAppGroupIfNeeded() {
let suite = AppGroup.defaults
let standard = UserDefaults.standard
guard suite !== standard else { return } // un-entitled fallback: nothing to migrate
guard suite.data(forKey: key) == nil,
let legacy = standard.data(forKey: key) else { return }
suite.set(legacy, forKey: key)
}
func add(_ host: StoredHost) {
hosts.append(host)
}
@@ -112,7 +113,7 @@ final class HostStore: ObservableObject {
func markConnected(_ hostID: UUID) {
guard let i = hosts.firstIndex(where: { $0.id == hostID }) else { return }
hosts[i].lastConnected = Date()
hosts[i].lastConnected = Date() // didSet persist() writes the shared suite + reloads widget
}
/// One reachability sweep, driving `probedOnline`: probe every saved host NOT currently
@@ -158,7 +159,17 @@ final class HostStore: ObservableObject {
private func persist() {
if let data = try? JSONEncoder().encode(hosts) {
UserDefaults.standard.set(data, forKey: Self.key)
defaults.set(data, forKey: Self.key)
}
reloadHostsWidget() // the widget reads this store; any change refreshes its timeline
}
/// Ask WidgetKit to rebuild the hosts widget's timeline after any store change (add/remove/pin/
/// last-connected). iOS-only and a no-op where WidgetKit is absent; the widget uses
/// `.never`-refresh entries and relies on this push.
private func reloadHostsWidget() {
#if canImport(WidgetKit) && os(iOS)
WidgetCenter.shared.reloadTimelines(ofKind: "PunktfunkHosts")
#endif
}
}
@@ -180,6 +180,23 @@ public final class SessionAudio {
}
}
/// Background keep-alive: silence the mic uplink while backgrounded (privacy no room audio
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
/// this stops the actual capture, which is the privacy-relevant part. Main thread.
public func setMicMuted(_ muted: Bool) {
stateLock.lock()
let capture = captureEngine
stateLock.unlock()
guard let capture else { return }
if muted {
capture.pause()
} else if !flag.isStopped {
try? capture.start()
}
}
// MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) {
@@ -0,0 +1,361 @@
// Shared clipboard, macOS client half (design/clipboard-and-file-transfer.md §5.2).
//
// Bridges NSPasteboard.general to the session's QUIC clipboard plane, both directions lazy:
//
// * **Local copy host**: a changeCount poll announces the *format list* (`clipOffer`); the
// bytes cross only when a host app pastes (a `.fetchRequest` event, answered from the live
// pasteboard by `clipServe`).
// * **Host copy local**: a `.remoteOffer` writes one NSPasteboardItem whose
// NSPasteboardItemDataProvider fires only when a Mac app actually pastes the provider then
// blocks (on its provider thread, never main) on a `clipFetch` round-trip.
//
// Password-manager respect: pasteboards marked `org.nspasteboard.ConcealedType` or
// `org.nspasteboard.TransientType` are never announced, never fetchable. Echo suppression: the
// changeCount of every write WE make is recorded so the announce poll skips it (§3.4).
//
// Phase 1 formats only (text / RTF / HTML / PNG). Files (NSFilePromiseProvider) ride Phase 2.
#if os(macOS)
import AppKit
import Foundation
/// One live session's clipboard bridge. Created by the session model when streaming begins on a
/// host that advertises `HOST_CAP_CLIPBOARD` and whose per-host toggle is on; `stop()` before the
/// connection closes. All pasteboard traffic runs on one dedicated drain thread plus the
/// AppKit-owned provider threads (paste fulfillment).
public final class ClipboardSync: NSObject {
/// Wire MIME NSPasteboard type for the Phase-1 vocabulary (§3.5), in announce order.
private static let wireToPasteboard: [(wire: String, type: NSPasteboard.PasteboardType)] = [
("text/plain;charset=utf-8", .string),
("text/rtf", .rtf),
("text/html", .html),
("image/png", .png),
]
/// Pasteboard marker types that must never cross the wire (password managers mark secrets
/// with these see nspasteboard.org).
private static let concealed = NSPasteboard.PasteboardType("org.nspasteboard.ConcealedType")
private static let transient = NSPasteboard.PasteboardType("org.nspasteboard.TransientType")
/// How long a blocked paste waits for the host's bytes before providing nothing (§5.2).
private static let fetchTimeout: TimeInterval = 10
/// Serve chunk size for host-side pastes of our data (bounds the per-call ABI copy).
private static let serveChunk = 4 << 20
private let connection: PunktfunkConnection
/// `CLIP_FLAG_*` sent with the enable (`CLIP_FLAG_FILES` when the session permits files
/// always 0 in Phase 1).
private let controlFlags: UInt8
/// Host `.state` updates, delivered on the main queue drives the toggle/footnote UI.
public var onState: ((_ enabled: Bool, _ policy: UInt8, _ reason: UInt8) -> Void)?
// Drain-thread state (touched only on the drain thread once started).
private var offerSeq: UInt32 = 0
private var lastSeenChangeCount = 0
/// The changeCount of the last pasteboard write WE made (echo suppression + "do we still
/// own the pasteboard" on teardown/clear).
private var ownedChangeCount = -1
/// The host offer currently installed on the local pasteboard (nil = none).
private var installedRemoteSeq: UInt32?
/// Outbound fetches a blocked paste is waiting on. Guarded by `fetchLock` appended by the
/// drain thread (`.data` events), consumed by AppKit's provider threads.
private struct PendingFetch {
var buffer = Data()
let done = DispatchSemaphore(value: 0)
var failed = false
}
private let fetchLock = NSLock()
private var pendingFetches: [UInt32: PendingFetch] = [:]
private final class StopFlag: @unchecked Sendable {
private let lock = NSLock()
private var stopped = false
func stop() {
lock.lock()
stopped = true
lock.unlock()
}
var isStopped: Bool {
lock.lock()
defer { lock.unlock() }
return stopped
}
}
private let flag = StopFlag()
private let drainDone = DispatchSemaphore(value: 0)
private var started = false
/// Set by the app-activation observer, cleared by the drain loop: the user may have copied
/// elsewhere and is coming back to paste announce immediately instead of waiting out the
/// poll interval.
private final class OneShot: @unchecked Sendable {
private let lock = NSLock()
private var raised = false
func raise() {
lock.lock()
raised = true
lock.unlock()
}
func takeIfRaised() -> Bool {
lock.lock()
defer { lock.unlock() }
let was = raised
raised = false
return was
}
}
private let checkNow = OneShot()
private var activationObserver: NSObjectProtocol?
public init(connection: PunktfunkConnection, allowFiles: Bool = false) {
self.connection = connection
self.controlFlags = 0 // CLIP_FLAG_FILES rides Phase 2
_ = allowFiles
super.init()
}
deinit { flag.stop() }
/// Enable sync with the host and start the drain thread. The host answers the enable with a
/// `.state` event (surfaced via `onState`) `BACKEND_UNAVAILABLE` et al. arrive there.
public func start() {
guard !started else { return }
started = true
connection.clipControl(enabled: true, flags: controlFlags)
// Baseline: whatever is on the pasteboard when sync starts is announced immediately
// the "copy first, then connect and paste" flow must work.
lastSeenChangeCount = -1
activationObserver = NotificationCenter.default.addObserver(
forName: NSApplication.didBecomeActiveNotification, object: nil, queue: nil
) { [checkNow] _ in checkNow.raise() }
let connection = self.connection
let flag = self.flag
let thread = Thread { [weak self] in
var lastAnnounceCheck = Date.distantPast
while !flag.isStopped {
// Drain events (bounded burst so a chatty host can't starve the announce poll).
var drained = 0
while drained < 32, !flag.isStopped {
let ev: PunktfunkConnection.ClipEvent?
do {
ev = try connection.nextClipboard(timeoutMs: drained == 0 ? 200 : 0)
} catch {
flag.stop() // session closed
break
}
guard let ev else { break }
drained += 1
self?.handle(ev)
}
// Announce poll: every 500 ms, or immediately after app activation (§5.2).
let now = Date()
if now.timeIntervalSince(lastAnnounceCheck) >= 0.5
|| self?.checkNow.takeIfRaised() == true
{
lastAnnounceCheck = now
self?.announceIfChanged()
}
}
self?.drainDone.signal()
}
thread.name = "punktfunk-clipboard"
thread.qualityOfService = .utility
thread.start()
}
/// Disable sync and join the drain thread. Called off-main before `connection.close()`
/// (the same discipline as the audio/feedback drains). If the local pasteboard still holds
/// our remote-offer items, they are cleared their providers die with us.
public func stop() {
guard started else { return }
started = false
if let obs = activationObserver {
NotificationCenter.default.removeObserver(obs)
activationObserver = nil
}
connection.clipControl(enabled: false, flags: 0)
flag.stop()
drainDone.wait()
// Fail every paste still blocked on us so no provider thread waits out its timeout.
fetchLock.lock()
for (_, pending) in pendingFetches {
pending.done.signal()
}
pendingFetches.removeAll()
fetchLock.unlock()
let pb = NSPasteboard.general
if installedRemoteSeq != nil, pb.changeCount == ownedChangeCount {
pb.clearContents()
}
}
// MARK: - Local copy host (announce)
/// Announce the local pasteboard's format list when it changed (skipping our own writes and
/// concealed/transient pasteboards). Runs on the drain thread.
private func announceIfChanged() {
let pb = NSPasteboard.general
let count = pb.changeCount
guard count != lastSeenChangeCount else { return }
lastSeenChangeCount = count
if count == ownedChangeCount { return } // our own write (a remote offer) never echo
installedRemoteSeq = nil // a local copy replaced the host's offer
let types = pb.types ?? []
if types.contains(Self.concealed) || types.contains(Self.transient) { return }
offerSeq &+= 1
let kinds = Self.wireToPasteboard
.filter { types.contains($0.type) }
.map { PunktfunkConnection.ClipKind(mime: $0.wire) }
// Empty = the pasteboard holds nothing we sync (or was cleared) clears the host side.
connection.clipOffer(seq: offerSeq, kinds: kinds)
}
// MARK: - Event handling (drain thread)
private func handle(_ ev: PunktfunkConnection.ClipEvent) {
switch ev {
case let .state(enabled, policy, reason):
if let onState {
DispatchQueue.main.async { onState(enabled, policy, reason) }
}
case let .remoteOffer(seq, kinds):
installRemoteOffer(seq: seq, kinds: kinds)
case let .fetchRequest(reqId, seq, _, mime):
serveFetch(reqId: reqId, seq: seq, mime: mime)
case let .data(xferId, chunk, last):
fetchLock.lock()
if var pending = pendingFetches[xferId] {
pending.buffer.append(chunk)
pendingFetches[xferId] = pending
if last {
pendingFetches[xferId]?.done.signal()
}
}
fetchLock.unlock()
case let .cancelled(id), let .error(id, _):
fetchLock.lock()
if var pending = pendingFetches[id] {
pending.failed = true
pendingFetches[id] = pending
pending.done.signal()
}
fetchLock.unlock()
}
}
// MARK: - Host copy local (lazy install + blocked-paste fetch)
/// Write one NSPasteboardItem advertising the host's formats, each backed by a lazy data
/// provider bytes cross only when a Mac app pastes. Empty `kinds` = the host cleared its
/// clipboard: drop our item if it's still current.
private func installRemoteOffer(seq: UInt32, kinds: [PunktfunkConnection.ClipKind]) {
let pb = NSPasteboard.general
let types = kinds.compactMap { kind in
Self.wireToPasteboard.first(where: { $0.wire == kind.mime })?.type
}
guard !types.isEmpty else {
if installedRemoteSeq != nil, pb.changeCount == ownedChangeCount {
pb.clearContents()
ownedChangeCount = pb.changeCount
lastSeenChangeCount = pb.changeCount
}
installedRemoteSeq = nil
return
}
let item = NSPasteboardItem()
item.setDataProvider(RemoteOfferProvider(sync: self, seq: seq), forTypes: types)
pb.clearContents()
pb.writeObjects([item])
installedRemoteSeq = seq
ownedChangeCount = pb.changeCount
lastSeenChangeCount = pb.changeCount
}
/// Blocked-paste fulfillment: fetch one wire format of host offer `seq` and wait (provider
/// thread) for the drain thread to assemble the chunks. Nil on timeout/cancel/error the
/// paste then provides nothing rather than hanging (§3.4).
///
/// `fetchLock` is held ACROSS the `clipFetch` so the pending entry exists before the drain
/// thread can process the first `.data` event (its `handle` takes `fetchLock` after
/// releasing the connection's clipboard lock no cycle).
fileprivate func fetchBlocking(seq: UInt32, wireMime: String) -> Data? {
fetchLock.lock()
guard let xferId = connection.clipFetch(seq: seq, mime: wireMime) else {
fetchLock.unlock()
return nil
}
pendingFetches[xferId] = PendingFetch()
let done = pendingFetches[xferId]!.done
fetchLock.unlock()
let outcome = done.wait(timeout: .now() + Self.fetchTimeout)
fetchLock.lock()
let pending = pendingFetches.removeValue(forKey: xferId)
fetchLock.unlock()
if outcome == .timedOut {
connection.clipCancel(id: xferId)
return nil
}
guard let pending, !pending.failed else { return nil }
return pending.buffer
}
// MARK: - Host paste of our data (serve)
/// Answer a host paste of our offered data from the live pasteboard. A stale `seq` (the
/// local clipboard changed since that announce) is cancelled never serve mismatched bytes.
private func serveFetch(reqId: UInt32, seq: UInt32, mime: String) {
let pb = NSPasteboard.general
guard seq == offerSeq, pb.changeCount == lastSeenChangeCount,
let type = Self.wireToPasteboard.first(where: { $0.wire == mime })?.type,
let data = pb.data(forType: type)
else {
connection.clipCancel(id: reqId)
return
}
var offset = 0
while offset < data.count {
let end = min(offset + Self.serveChunk, data.count)
connection.clipServe(
reqId: reqId, data: data.subdata(in: offset..<end), last: end == data.count)
offset = end
}
if data.isEmpty {
connection.clipServe(reqId: reqId, data: Data(), last: true)
}
}
}
/// The lazy paste hook: AppKit calls `provideDataForType` only when a Mac app actually pastes;
/// the fetch then blocks this provider thread (never main) until the host's bytes arrive or the
/// timeout provides nothing. One provider per installed remote offer a dead sync (weak) or a
/// superseded offer provides nothing.
private final class RemoteOfferProvider: NSObject, NSPasteboardItemDataProvider {
private weak var sync: ClipboardSync?
private let seq: UInt32
init(sync: ClipboardSync, seq: UInt32) {
self.sync = sync
self.seq = seq
}
func pasteboard(
_ pasteboard: NSPasteboard?, item: NSPasteboardItem,
provideDataForType type: NSPasteboard.PasteboardType
) {
guard let sync,
let wire = wireMime(for: type),
let data = sync.fetchBlocking(seq: seq, wireMime: wire)
else { return }
item.setData(data, forType: type)
}
private func wireMime(for type: NSPasteboard.PasteboardType) -> String? {
switch type {
case .string: return "text/plain;charset=utf-8"
case .rtf: return "text/rtf"
case .html: return "text/html"
case .png: return "image/png"
default: return nil
}
}
}
#endif
@@ -11,6 +11,9 @@
// LaunchSpec schema in `crates/punktfunk-host/src/library.rs`.
import Foundation
// `punktfunkDefaultMgmtPort` (and StoredHost/DefaultsKey) now live in PunktfunkShared so the
// dependency-free widget extension can share them; PunktfunkKit re-exports the module.
import PunktfunkShared
/// Cover art URLs (the public Steam CDN for Steam titles, user-supplied for custom entries).
public struct Artwork: Codable, Hashable, Sendable {
@@ -64,10 +67,6 @@ public enum LibraryError: LocalizedError {
}
}
/// The management API's default port adjacent to the GameStream block; matches
/// `mgmt::DEFAULT_PORT` on the host.
public let punktfunkDefaultMgmtPort: UInt16 = 47990
/// Stateless fetcher for a host's library.
public enum LibraryClient {
/// `GET https://<address>:<port>/api/v1/library`, authenticated by **mTLS**: the client
@@ -196,6 +196,10 @@ public final class PunktfunkConnection {
/// Same role for the host-timing (0xCF) puller its own plane in the core, drained
/// non-blockingly by the app's 1 s stats tick (never contends with the blocking pullers).
private let statsLock = NSLock()
/// Same role for the shared-clipboard drain thread (`nextClipboard` its own plane in the
/// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too:
/// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple.
private let clipboardLock = NSLock()
/// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0
@@ -346,6 +350,16 @@ public final class PunktfunkConnection {
/// parse-window size for `USER_FLAG_CHUNK_ALIGNED` PyroWave AUs (plan §4.4). Other codecs
/// never need it.
public private(set) var shardPayload: UInt32 = 1408
/// The host capability bitfield (`Welcome.host_caps`): `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` /
/// `PUNKTFUNK_HOST_CAP_CLIPBOARD`. `0` for an older host that didn't say.
public private(set) var hostCaps: UInt8 = 0
/// Whether this host advertises the shared clipboard (`HOST_CAP_CLIPBOARD`) the gate for
/// offering the clipboard toggle. Absent on an older host, or one whose operator policy
/// (`PUNKTFUNK_CLIPBOARD=off`) keeps the feature dark.
public var hostSupportsClipboard: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0
}
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
@@ -461,6 +475,9 @@ public final class PunktfunkConnection {
var shard: UInt32 = 1408
_ = punktfunk_connection_shard_payload(handle, &shard)
shardPayload = shard
var caps: UInt8 = 0
_ = punktfunk_connection_host_caps(handle, &caps)
hostCaps = caps
}
/// A bandwidth speed-test measurement (see `startSpeedTest`). Partial until `done`.
@@ -534,6 +551,23 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_request_keyframe(h)
}
/// Background-keep-alive video drop (opt-in). While true, both video pumps keep DRAINING
/// `nextAU()` (so QUIC flow control and host pacing stay healthy) but DISCARD each AU before any
/// VideoToolbox/Metal decode or render the crash/jetsam-safe way to hold a backgrounded
/// session (audio keeps rendering; no GPU work off-screen). Set on `SessionModel.enterBackground`,
/// cleared on `exitBackground` (which then requests a fresh IDR; the pump's re-anchor gate
/// auto-arms on the resumed frame-index gap). Its own tiny lock read on the pump thread every
/// iteration, written on the main actor; never contends the ABI/plane locks.
private let videoDropLock = NSLock()
private var videoDropped = false
public var isVideoDropped: Bool {
videoDropLock.lock(); defer { videoDropLock.unlock() }
return videoDropped
}
public func setVideoDropped(_ dropped: Bool) {
videoDropLock.lock(); videoDropped = dropped; videoDropLock.unlock()
}
/// Feed each received AU's `frameIndex` (in receive order) so the client recovers from loss with a
/// cheap reference-frame invalidation instead of always paying for a full IDR. On a forward gap
/// a `frameIndex` jump means the intervening frames were lost and the following AUs reference a
@@ -760,6 +794,34 @@ public final class PunktfunkConnection {
}
}
/// Pull the next EFFECTIVE rumble command from the core's shared rumble policy engine the
/// uniform replacement for per-platform rumble policy. The engine owns every decision
/// (v2 lease expiry, legacy-host staleness at a uniform 1 s, connection-close drain zeros),
/// so apply commands verbatim: `(0, 0)` = stop now, non-zero = run at this level.
/// `backstopMs` is a safety-net duration for duration-parameterized platform APIs the
/// CoreHaptics renderer ignores it (its finite segment ceiling is the equivalent net).
/// Drain from the (single) feedback thread, alongside `nextHidOutput`.
public func nextRumbleCommand(timeoutMs: UInt32 = 0) throws
-> (pad: UInt16, low: UInt16, high: UInt16, backstopMs: UInt32)?
{
feedbackLock.lock()
defer { feedbackLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var pad: UInt16 = 0, low: UInt16 = 0, high: UInt16 = 0, backstop: UInt32 = 0
let rc = punktfunk_connection_next_rumble_cmd(h, &pad, &low, &high, &backstop, timeoutMs)
switch rc {
case statusOK:
return (pad, low, high, backstop)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// One DualSense feedback event a game wrote to the host's virtual pad replay it on
/// the real controller (GCDeviceLight, GCControllerPlayerIndex,
/// GCDualSenseAdaptiveTrigger). Only a `.dualSense` session emits these.
@@ -970,10 +1032,12 @@ public final class PunktfunkConnection {
audioLock.lock()
feedbackLock.lock()
statsLock.lock()
clipboardLock.lock()
abiLock.lock()
let h = handle
handle = nil
abiLock.unlock()
clipboardLock.unlock()
statsLock.unlock()
feedbackLock.unlock()
audioLock.unlock()
@@ -1035,6 +1099,163 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_rich_input(h, &rich)
}
// MARK: - Shared clipboard (design/clipboard-and-file-transfer.md §5)
/// One advertised clipboard format in a lazy offer the format list crosses the wire,
/// the bytes only on a fetch.
public struct ClipKind: Sendable, Equatable {
public let mime: String
/// Best-effort size in bytes; `0` = unknown.
public let sizeHint: UInt64
public init(mime: String, sizeHint: UInt64 = 0) {
self.mime = mime
self.sizeHint = sizeHint
}
}
/// A shared-clipboard event from `nextClipboard`. The drain thread turns these into
/// NSPasteboard operations (`ClipboardSync`).
public enum ClipEvent: Sendable, Equatable {
/// The host copied: its lazy format list (empty = the host clipboard was cleared).
/// Fetch a format with `clipFetch(seq:mime:)` when a local app pastes.
case remoteOffer(seq: UInt32, kinds: [ClipKind])
/// Host ack / policy / backend update for `clipControl` (`CLIP_REASON_*`).
case state(enabled: Bool, policy: UInt8, reason: UInt8)
/// The host is pasting OUR offered data answer with `clipServe(reqId:...)`.
case fetchRequest(reqId: UInt32, seq: UInt32, fileIndex: UInt32, mime: String)
/// Bytes for a fetch we started (`last` = final chunk).
case data(xferId: UInt32, chunk: Data, last: Bool)
/// A transfer was cancelled (either side).
case cancelled(id: UInt32)
/// A transfer failed (`status` = a PunktfunkStatus code).
case error(id: UInt32, status: Int32)
}
/// Enable/disable the shared clipboard for this session. Opt-in: nothing is announced or
/// served until enabled. The host answers with a `.state` event carrying the resolved
/// outcome (its operator policy is authoritative). Best-effort a dropped call on a
/// closing session is fine.
public func clipControl(enabled: Bool, flags: UInt8 = 0) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_clipboard_control(h, enabled, flags)
}
/// Announce that the local pasteboard changed the lazy format-list offer (`seq` monotonic,
/// newest wins; empty `kinds` clears the host side). The bytes cross only if the host fetches.
public func clipOffer(seq: UInt32, kinds: [ClipKind]) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
guard !kinds.isEmpty else {
_ = punktfunk_connection_clipboard_offer(h, seq, nil, 0)
return
}
// The C array borrows NUL-terminated strings for the duration of the call only.
let cStrings = kinds.map { strdup($0.mime) }
defer { cStrings.forEach { free($0) } }
let arr = zip(cStrings, kinds).map {
PunktfunkClipKind(mime: $0.map { UnsafePointer($0) }, size_hint: $1.sizeHint)
}
_ = arr.withUnsafeBufferPointer {
punktfunk_connection_clipboard_offer(h, seq, $0.baseAddress, UInt(arr.count))
}
}
/// Start pulling one format of the host's offer `seq` (a local app is pasting). Returns the
/// transfer id echoed on the resulting `.data`/`.error`/`.cancelled` events, or nil when the
/// session is closing.
public func clipFetch(seq: UInt32, mime: String, fileIndex: UInt32 = UInt32.max) -> UInt32? {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return nil }
var xfer: UInt32 = 0
let rc = mime.withCString {
punktfunk_connection_clipboard_fetch(h, seq, $0, fileIndex, &xfer)
}
return rc == statusOK ? xfer : nil
}
/// Provide bytes answering a `.fetchRequest` (the host is pasting our offered data). Call
/// repeatedly to stream; `last = true` completes the transfer. An empty final chunk is fine.
public func clipServe(reqId: UInt32, data: Data, last: Bool) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
if data.isEmpty {
_ = punktfunk_connection_clipboard_serve(h, reqId, nil, 0, last)
} else {
data.withUnsafeBytes { p in
_ = punktfunk_connection_clipboard_serve(
h, reqId, p.bindMemory(to: UInt8.self).baseAddress, UInt(data.count), last)
}
}
}
/// Cancel a clipboard transfer by id an outbound fetch's `xferId` or an inbound
/// `.fetchRequest`'s `reqId`.
public func clipCancel(id: UInt32) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_clipboard_cancel(h, id)
}
/// Pull the next shared-clipboard event; nil on timeout, throws `.closed` once the session
/// ended. Drain from a single dedicated thread (`ClipboardSync`) the event's borrowed
/// payload is copied into the returned `ClipEvent` before the next poll can overwrite it.
public func nextClipboard(timeoutMs: UInt32) throws -> ClipEvent? {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var ev = PunktfunkClipEvent()
let rc = punktfunk_connection_next_clipboard(h, &ev, timeoutMs)
switch rc {
case statusOK:
return Self.decodeClipEvent(ev)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Copy a raw C clip event (whose `data` borrows a per-connection slot) into an owned Swift
/// value. Unknown kinds (a newer core) decode to nil and are skipped by the drain.
private static func decodeClipEvent(_ ev: PunktfunkClipEvent) -> ClipEvent? {
let payload = ev.data.map { Data(bytes: $0, count: Int(ev.len)) } ?? Data()
switch Int32(ev.kind) {
case PUNKTFUNK_CLIP_REMOTE_OFFER:
// One `mime\tsize_hint\n` line per advertised format.
let kinds = String(decoding: payload, as: UTF8.self)
.split(separator: "\n")
.compactMap { line -> ClipKind? in
let parts = line.split(separator: "\t", maxSplits: 1)
guard let mime = parts.first, !mime.isEmpty else { return nil }
let hint = parts.count > 1 ? UInt64(parts[1]) ?? 0 : 0
return ClipKind(mime: String(mime), sizeHint: hint)
}
return .remoteOffer(seq: ev.transfer_id, kinds: kinds)
case PUNKTFUNK_CLIP_STATE:
return .state(enabled: ev.enabled != 0, policy: ev.policy, reason: ev.reason)
case PUNKTFUNK_CLIP_FETCH_REQUEST:
return .fetchRequest(
reqId: ev.transfer_id, seq: ev.seq, fileIndex: ev.file_index,
mime: String(decoding: payload, as: UTF8.self))
case PUNKTFUNK_CLIP_DATA:
return .data(xferId: ev.transfer_id, chunk: payload, last: ev.last != 0)
case PUNKTFUNK_CLIP_CANCELLED:
return .cancelled(id: ev.transfer_id)
case PUNKTFUNK_CLIP_ERROR:
return .error(id: ev.transfer_id, status: ev.status)
default:
return nil
}
}
deinit { close() }
/// Snapshot the handle unless close is pending (callers hold their plane lock).
@@ -23,6 +23,7 @@ import Combine
import CoreHaptics
import Foundation
import GameController
import PunktfunkShared
public final class GamepadFeedback {
private let connection: PunktfunkConnection
@@ -154,21 +155,18 @@ public final class GamepadFeedback {
// meta, was unaffected). Pacing with a short sleep OUTSIDE the lock (below) keeps
// rumble/HID latency low while leaving the lock free between polls.
//
// Rumble is idempotent state, so drain the plane DRY and apply only the newest
// level PER PAD. The old one-datagram-per-cycle shape let a burst outpace the
// ~125 Hz drain: levels rendered up to ~130 ms late through the core's 16-deep
// queue, and its drop-newest overflow could shed a stop while stale nonzero
// states queued ahead of it buzzing until the host's next 500 ms refresh.
var newestByPad: [UInt8: (low: UInt16, high: UInt16, ttl: UInt32)] = [:]
// Rumble arrives as EFFECTIVE commands from the core's shared policy engine
// (design/rumble-root-fix.md §D): the engine owns leases, legacy staleness,
// and close-drain zeros, and its per-pad mailbox already coalesces a
// stalled drain wakes to ONE current-level command per pad, and a stop can
// never be shed by a queue. Apply verbatim, in order.
var rumbleBurst = 0
while rumbleBurst < 64, !flag.isStopped,
let r = try connection.nextRumble2(timeoutMs: 0) {
newestByPad[UInt8(truncatingIfNeeded: r.pad)] = (r.low, r.high, r.ttlMs)
let c = try connection.nextRumbleCommand(timeoutMs: 0) {
self?.routeRumble(
pad: UInt8(truncatingIfNeeded: c.pad), low: c.low, high: c.high)
rumbleBurst += 1
}
for (pad, n) in newestByPad {
self?.routeRumble(pad: pad, low: n.low, high: n.high, ttlMs: n.ttl)
}
// Drain a BOUNDED burst of hidout events so sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) can't spin here or block stop() past one cycle.
var burst = 0
@@ -217,15 +215,15 @@ public final class GamepadFeedback {
}
}
/// Route one rumble envelope to its pad's renderer (drain thread). An update for a pad with no
/// Route one engine command to its pad's renderer (drain thread). A command for a pad with no
/// live renderer one that just left the forwarded set is dropped.
private func routeRumble(pad: UInt8, low: UInt16, high: UInt16, ttlMs: UInt32) {
private func routeRumble(pad: UInt8, low: UInt16, high: UInt16) {
let renderer = withRouting { rumbleByPad[pad] }
renderer?.apply(low: low, high: high, ttlMs: ttlMs)
renderer?.apply(low: low, high: high)
// The opt-in device mirror follows controller 1 unconditionally the pads it exists for
// have no motors (their renderer above no-ops), and mirroring deliberately isn't gated on
// that: capability probing can't see a motor-less MFi pad, and the user opted in.
if pad == 0 { deviceRumble?.apply(low: low, high: high, ttlMs: ttlMs) }
if pad == 0 { deviceRumble?.apply(low: low, high: high) }
}
private func withRouting<R>(_ body: () -> R) -> R {
@@ -20,6 +20,7 @@
import Combine
import Foundation
import GameController
import PunktfunkShared
@MainActor
public final class GamepadManager: ObservableObject {
@@ -6,6 +6,7 @@
// the two combine without adding a second ObservableObject or an environment key nobody else needs.
import Foundation
import PunktfunkShared
public enum GamepadUIEnvironment {
/// `enabledSetting` is the user's Settings toggle (`DefaultsKey.gamepadUIEnabled`);
@@ -23,23 +23,6 @@ enum RumbleTuning {
/// the churn that lost stops inside CoreHaptics. Newest level wins when the window opens;
/// zero is never throttled.
static let minRebakeSeconds: TimeInterval = 0.025
/// Session watchdog: silence the motors when no wire command arrived for this long. This is
/// the **legacy-host fallback only** an old host sends no self-termination lease, so its
/// periodic re-send (every 500 ms) is the sole liveness signal and 3 vanished refreshes means
/// the channel or host died while audible. A v2 host instead supplies a per-command TTL (see
/// [`leaseSeconds`]); that deadline supersedes this watchdog.
static let sessionStaleSeconds: TimeInterval = 1.6
/// The legacy no-lease sentinel a v2 `ttl_ms` carries for an old host (mirrors the C ABI's
/// `PUNKTFUNK_RUMBLE_NO_TTL`). `UInt32.max` by construction.
static let noTTL: UInt32 = .max
/// Interpret a wire TTL (ms) from a rumble update: `nil` for the legacy no-lease sentinel
/// ([`noTTL`]) the renderer falls back to [`sessionStaleSeconds`] else the self-termination
/// lease in seconds (render the level for at most this long unless the host renews it).
static func leaseSeconds(ttlMs: UInt32) -> TimeInterval? {
ttlMs == noTTL ? nil : TimeInterval(ttlMs) / 1000
}
/// Levels closer than this (0.4 % of full scale) are the same level an identical host
/// refresh must never rebuild a player.
static let levelEpsilon: Float = 1.0 / 256.0
@@ -110,13 +93,15 @@ enum RumbleTuning {
/// `@unchecked Sendable` is sound because every property is read and written only inside
/// `queue` closures the serial queue is the synchronization.
final class RumbleRenderer: @unchecked Sendable {
/// What an un-refreshed nonzero target means. A live session ties motor life to wire
/// liveness (the host refreshes state every 500 ms); the controller test panel holds a
/// slider level indefinitely.
/// Who ends an un-refreshed nonzero target. Session mode applies the core policy engine's
/// commands verbatim the engine (punktfunk-core `client/rumble.rs`) owns every lease,
/// staleness, and close decision and emits explicit zeros, so the renderer keeps NO
/// staleness policy of its own anymore. The controller test panel (`manual`) holds a slider
/// level indefinitely; both are identical renderer-side today, the distinction is kept for
/// the call sites' intent.
struct Policy {
let staleAfter: TimeInterval?
static let session = Policy(staleAfter: RumbleTuning.sessionStaleSeconds)
static let manual = Policy(staleAfter: nil)
static let session = Policy()
static let manual = Policy()
}
/// Which physical actuator this renderer drives: the forwarded controller's haptics engine
@@ -160,13 +145,9 @@ final class RumbleRenderer: @unchecked Sendable {
private var controller: GCController?
private var low: Motor?
private var high: Motor?
/// Wire-truth target (raw wire units) and when it was last confirmed by any command.
/// Wire-truth target (raw wire units) the engine command's level, applied verbatim; the
/// core policy engine owns when it ends (explicit zero commands), so no deadline lives here.
private var target: (low: UInt16, high: UInt16) = (0, 0)
private var lastCommand = DispatchTime(uptimeNanoseconds: 0)
/// The v2 envelope lease: the active level is authorized until here unless the host renews it
/// (`tick` silences at the deadline). `nil` against a legacy host (no lease the
/// `sessionStaleSeconds` watchdog is the backstop) and while silent.
private var envelopeDeadline: DispatchTime?
/// Runs while anything is (or should be) audible: staleness watchdog, segment re-arm,
/// throttled-level catch-up, engine rebuild after a reset, HID keepalive. Nil while silent,
/// so an idle controller costs no timer wakeups and no radio traffic.
@@ -247,17 +228,9 @@ final class RumbleRenderer: @unchecked Sendable {
/// against a legacy host (no lease the staleness watchdog is the backstop). Renewals at an
/// unchanged level extend the deadline before the idempotence guard, so a held rumble never
/// lapses mid-effect.
func apply(low lowAmp: UInt16, high highAmp: UInt16, ttlMs: UInt32 = RumbleTuning.noTTL) {
func apply(low lowAmp: UInt16, high highAmp: UInt16) {
queue.async {
self.lastCommand = .now()
let active = lowAmp != 0 || highAmp != 0
// v2 lease: a nonzero level gets an explicit deadline; a stop or a legacy update clears
// it. Set BEFORE the idempotence guard so an identical renewal still extends the lease.
if let lease = RumbleTuning.leaseSeconds(ttlMs: ttlMs), active {
self.envelopeDeadline = .now() + lease
} else {
self.envelopeDeadline = nil
}
if active != self.wasActive {
self.wasActive = active
log.debug(
@@ -275,7 +248,6 @@ final class RumbleRenderer: @unchecked Sendable {
self.ticker?.cancel()
self.ticker = nil
self.target = (0, 0)
self.envelopeDeadline = nil
self.wasActive = false
self.teardown()
self.closeHID()
@@ -331,25 +303,11 @@ final class RumbleRenderer: @unchecked Sendable {
healthSink?(problem)
}
/// Watchdog + housekeeping heartbeat while audible.
/// Housekeeping heartbeat while audible: segment re-arm, HID keepalive, backoff retries.
/// Every liveness decision (lease expiry, legacy-host staleness, session close) lives in the
/// core policy engine now it emits explicit zero commands, so the renderer never guesses
/// when a level should end.
private func tick() {
if let deadline = envelopeDeadline {
// v2 host lease: silence the moment it lapses unrenewed. This firing in the wild is the
// observable signature of a host that stopped renewing (a dropped stop, or a dead host)
// the whole point of the envelope model: the motor can't outlive the host's intent.
if target != (0, 0), DispatchTime.now() >= deadline {
log.warning("rumble: envelope expired unrenewed — silencing")
target = (0, 0)
envelopeDeadline = nil
}
} else if let after = policy.staleAfter, target != (0, 0), seconds(since: lastCommand) > after {
// Legacy host (no lease): it re-sends state every 500 ms, so this much silence means the
// channel (or host) died while a motor was on. A direct-connected pad would have been
// stopped by its game long ago force the same outcome.
log.warning(
"rumble: no wire refresh for \(after, format: .fixed(precision: 1), privacy: .public)s — auto-silencing")
target = (0, 0)
}
render()
}
@@ -19,6 +19,7 @@
#if os(iOS)
import Foundation
import PunktfunkCore
import PunktfunkShared
import UIKit
/// How touchscreen fingers drive the host persisted under `DefaultsKey.touchMode`, latched
@@ -0,0 +1,9 @@
// PunktfunkShared holds what the app AND the widget extension both need the stored-host model,
// the settings-key names, the App-Group constant, the deep-link grammar, and the Live Activity
// attributes in a module that links neither the Rust core nor the presentation layer.
//
// Re-export it so every existing consumer of PunktfunkKit (`import PunktfunkKit`) keeps seeing
// `StoredHost`, `DefaultsKey`, `punktfunkDefaultMgmtPort`, `DeepLink`, etc. with no call-site churn.
// (Files INSIDE PunktfunkKit still `import PunktfunkShared` explicitly Swift imports are
// file-scoped; the re-export only reaches downstream modules.)
@_exported import PunktfunkShared
@@ -8,6 +8,7 @@
// tap, InputCapture's captured-state S) cycle it directly.
import Foundation
import PunktfunkShared
/// How much of the streaming statistics overlay to show. The raw values are stable on disk
/// rename the cases freely, never the strings.
@@ -9,6 +9,7 @@
#if canImport(Metal) && canImport(QuartzCore)
import AVFoundation
import Foundation
import PunktfunkShared
import QuartzCore
#if os(tvOS)
import UIKit
@@ -38,6 +38,7 @@
import AVFoundation
import Foundation
import Metal
import PunktfunkShared
import QuartzCore
/// PUNKTFUNK_PRESENT_DEBUG=1: the render thread prints a once-per-second line with the decode
@@ -431,6 +432,15 @@ public final class Stage2Pipeline {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain one AU (flow control + host pacing) and discard it
// BEFORE any VideoToolbox decode or Metal render no GPU work off-screen. The
// decoder session is left intact; exitBackground requests a fresh IDR and the
// re-anchor gate arms on the resumed frame-index gap so concealed frames are
// withheld until it lands.
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// decoder conceals the reference-missing deltas often WITHOUT an error callback
// so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
@@ -686,6 +696,13 @@ public final class Stage2Pipeline {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain + discard before the Metal wavelet decode
// (PyroWave is all-intra, so the resumed frame heals on its own no IDR
// request needed, just no GPU work off-screen).
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
onFrame?(au)
if let newest = newestIndex,
@@ -63,6 +63,14 @@ final class StreamPump {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain one AU to keep QUIC flow control + host pacing
// healthy, then discard it BEFORE any decode/enqueue no VideoToolbox/Metal work
// off-screen. Skips all recovery/gate bookkeeping too; exitBackground requests a
// fresh IDR and the re-anchor gate re-arms on the resumed frame-index gap.
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
// Loss recovery (the primary path). Under the host's infinite GOP the only
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
// (framesDropped); the decoder then *conceals* the reference-missing deltas a
@@ -19,6 +19,7 @@
#if os(macOS)
import AppKit
import AVFoundation
import PunktfunkShared
import SwiftUI
import os
@@ -35,6 +35,7 @@
import AVFoundation
import GameController
import PunktfunkCore
import PunktfunkShared
import SwiftUI
import UIKit
import os
@@ -0,0 +1,23 @@
// The App-Group foundation shared by the app and its extensions (Widgets / Live Activity).
//
// PunktfunkShared is deliberately dependency-free: it links NEITHER PunktfunkKit (which drags in
// the Rust staticlib + presentation layer) NOR any Apple UI framework. A widget process gets ~30 MB,
// so everything an extension needs the stored-host model + its JSON codec, the settings-key names,
// the deep-link grammar, and (later) the Live Activity attributes lives here and here only.
import Foundation
/// The one App-Group identifier, matched by `Config/*.entitlements`
/// (`com.apple.security.application-groups`). Registered on the developer portal for both the app
/// id (`io.unom.punktfunk`) and the widget extension id (`io.unom.punktfunk.widgets`).
public enum AppGroup {
public static let suiteName = "group.io.unom.punktfunk"
/// The shared defaults suite. Non-nil in a correctly-entitled process; falls back to
/// `.standard` if the group is somehow unavailable (unsigned `swift run`, a misprovisioned
/// build) so the app still functions single-process rather than crashing the widget just
/// won't see the same store there.
public static var defaults: UserDefaults {
UserDefaults(suiteName: suiteName) ?? .standard
}
}
@@ -0,0 +1,57 @@
// The `punktfunk://` deep-link grammar the single builder/parser shared by the widget (which
// emits links via `widgetURL`/`Link`) and the app (`ContentView.onOpenURL`, which routes them into
// the existing connect path). Keeping both sides on one type means the wire format can't drift.
//
// Grammar (v1):
// punktfunk://connect/<host-uuid> connect to a stored host
// punktfunk://connect/<host-uuid>?launch=<GameEntry.id> connect and ask the host to launch it
//
// `launch` carries a `GameEntry.id` (e.g. "steam:570"); it is percent-encoded on build and decoded
// on parse, so ids with reserved characters survive the round trip.
import Foundation
public enum DeepLink: Equatable {
/// Connect to a saved host; `launchID` is a `GameEntry.id` to launch on arrival, if any.
case connect(host: UUID, launchID: String?)
public static let scheme = "punktfunk"
/// Build the canonical URL for a route. Non-optional: every route is representable.
public var url: URL {
switch self {
case let .connect(host, launchID):
var comps = URLComponents()
comps.scheme = Self.scheme
comps.host = "connect"
comps.path = "/\(host.uuidString)"
if let launchID, !launchID.isEmpty {
comps.queryItems = [URLQueryItem(name: "launch", value: launchID)]
}
// URLComponents percent-encodes the query value; force-unwrap is safe for a URL we
// fully control (scheme/host/path are all valid).
return comps.url!
}
}
/// Parse an incoming URL, or nil if it isn't a recognized `punktfunk://` route. Tolerant of
/// case in the scheme and of a trailing slash on the path.
public init?(_ url: URL) {
guard url.scheme?.lowercased() == Self.scheme else { return nil }
guard let comps = URLComponents(url: url, resolvingAgainstBaseURL: false) else { return nil }
switch comps.host?.lowercased() {
case "connect":
// Path is "/<uuid>"; strip the leading slash and any trailing one.
let raw = comps.path
.trimmingCharacters(in: CharacterSet(charactersIn: "/"))
guard let host = UUID(uuidString: raw) else { return nil }
let launch = comps.queryItems?
.first(where: { $0.name == "launch" })?.value
.flatMap { $0.isEmpty ? nil : $0 }
self = .connect(host: host, launchID: launch)
default:
return nil
}
}
}
@@ -1,7 +1,8 @@
// One source of truth for the client's UserDefaults / @AppStorage keys. A magic-string key
// duplicated across a setting's writer (a Settings @AppStorage) and reader (e.g. a stream view
// reading UserDefaults) splits silently on a typo the setting just stops taking effect. These
// live in PunktfunkKit because both the app and the kit's views read them.
// live in the dependency-free PunktfunkShared module (re-exported by PunktfunkKit) because the app,
// the kit's views, AND the widget extension all read them the widget needs `DefaultsKey.hosts`.
import Foundation
@@ -111,6 +112,16 @@ public enum DefaultsKey {
/// routed/VPN host), so connects go straight through instead of waiting out the wake timeout.
/// The explicit "Wake Host" action stays available regardless. Read by ContentView.startSession.
public static let autoWake = "punktfunk.autoWake"
/// iOS/iPadOS: keep a streaming session ALIVE when the app is backgrounded (audio background
/// mode). Off by default (today's freeze-on-background is the default). When on, backgrounding a
/// live session keeps audio playing and the QUIC/pump live while DROPPING video decode, and a
/// bounded timer (`backgroundTimeoutMinutes`) auto-disconnects if the user doesn't return. Read
/// by ContentView's scenePhase driver. Hidden on tvOS/macOS.
public static let backgroundKeepAlive = "punktfunk.backgroundKeepAlive"
/// iOS/iPadOS: minutes a backgrounded keep-alive session runs before auto-disconnecting (a
/// battery/thermal/bandwidth backstop). Default 10; the UI offers 1/5/10/30. The auto-disconnect
/// is non-deliberate (host linger kept), so a late return reconnects fast. Read on enterBackground.
public static let backgroundTimeoutMinutes = "punktfunk.backgroundTimeoutMinutes"
}
extension Notification.Name {
@@ -120,4 +131,15 @@ extension Notification.Name {
/// menus) it exists so the menu item is honest whenever it CAN fire, and as the shortcut's
/// discoverable menu-bar surface.
public static let punktfunkReleaseCapture = Notification.Name("io.unom.punktfunk.release-capture")
/// Posted by the Live Activity's / Shortcuts' End-stream intent (`EndStreamIntent.perform`,
/// which runs in the app's process): the app tears the active session down deliberately
/// (quit-close the host). Same cross-process-signal pattern as `punktfunkReleaseCapture`
/// the intent lives in PunktfunkShared and can't reach the app's `SessionModel` directly.
public static let punktfunkEndActiveSession = Notification.Name("io.unom.punktfunk.end-active-session")
/// Posted by the Connect App Intent (Siri/Shortcuts) with a `punktfunk://` URL as `object`:
/// the app routes it through the SAME `.onOpenURL` handler a widget tap uses (one router, one
/// set of guards). The intent uses `openAppWhenRun`, so the app is foregrounded to receive it.
public static let punktfunkOpenDeepLink = Notification.Name("io.unom.punktfunk.open-deep-link")
}
@@ -0,0 +1,56 @@
// The saved-host as an App Intents entity the parameter type for the Connect/Wake intents and
// the configurable single-host widget. Lives in the shared module (not the app) because widget
// *configuration* intents execute in the EXTENSION process, so the entity can't be app-only.
//
// AppIntents is genuinely available on macOS (13+), so this is gated on `canImport(AppIntents)`
// (unlike ActivityKit, whose macOS types are unavailable) it compiles on every platform and the
// entity query reads the same shared App-Group store the widget does.
#if canImport(AppIntents)
import AppIntents
import Foundation
public struct HostEntity: AppEntity, Identifiable {
public static let typeDisplayRepresentation = TypeDisplayRepresentation(name: "Host")
public static let defaultQuery = HostEntityQuery()
public let id: UUID
public let name: String
public init(id: UUID, name: String) {
self.id = id
self.name = name
}
public init(_ host: StoredHost) {
self.id = host.id
self.name = host.displayName
}
public var displayRepresentation: DisplayRepresentation {
DisplayRepresentation(title: "\(name)")
}
}
public struct HostEntityQuery: EntityQuery {
public init() {}
public func entities(for identifiers: [UUID]) async throws -> [HostEntity] {
Self.loadHosts().filter { identifiers.contains($0.id) }.map(HostEntity.init)
}
/// Sorted most-recent first Siri/Shortcuts and the widget config picker suggest recent hosts.
public func suggestedEntities() async throws -> [HostEntity] {
Self.loadHosts().map(HostEntity.init)
}
static func loadHosts() -> [StoredHost] {
guard let data = AppGroup.defaults.data(forKey: DefaultsKey.hosts),
let hosts = try? JSONDecoder().decode([StoredHost].self, from: data)
else { return [] }
return hosts.sorted {
($0.lastConnected ?? .distantPast) > ($1.lastConnected ?? .distantPast)
}
}
}
#endif
@@ -0,0 +1,68 @@
// The Live Activity's attributes the ONE type that must be identical in the app (which starts
// and updates the Activity) and the widget extension (which renders it). Hence it lives in the
// dependency-free shared module.
//
// Gated on `os(iOS)`, NOT `canImport(ActivityKit)`: ActivityKit *imports* on macOS but its types
// are `@available(macOS, unavailable)`, so canImport would wrongly admit this on the macOS build.
// Live Activities are iPhone/iPad only (iPadOS reports os(iOS)).
//
// Naming/shape is a runtime contract: an Activity started by one build is decoded by the extension
// of the same build, so keep `ContentState` Codable-stable across releases the way `StoredHost` is.
#if os(iOS)
import ActivityKit
import Foundation
public struct PunktfunkSessionAttributes: ActivityAttributes {
// Static for the Activity's whole life (set at request time).
public let hostID: UUID
public let hostName: String
/// The title of the launched game, if the session started from the library; nil for a plain
/// host connect (nothing tracks the live foreground app mid-session).
public let launchTitle: String?
public init(hostID: UUID, hostName: String, launchTitle: String?) {
self.hostID = hostID
self.hostName = hostName
self.launchTitle = launchTitle
}
public struct ContentState: Codable, Hashable {
public enum Stage: String, Codable, Hashable {
case streaming // foreground, live
case background // backgrounded keep-alive (countdown running)
case reconnecting // post-loss re-anchor hold
case ending // torn down final state before dismissal
}
public var stage: Stage
/// Session start drives `Text(timerInterval:)` for a free client-side ticking clock (no
/// per-second push needed).
public var startedAt: Date
/// e.g. "2560×1440 @120 · HEVC · HDR". Updated only when it actually changes.
public var modeLine: String
/// Coarse, updated sparsely (every ~30 s) never the 1 Hz stats firehose.
public var latencyMs: Int?
public var mbps: Double?
/// While backgrounded: when the keep-alive auto-disconnect fires drives the countdown.
public var backgroundDeadline: Date?
public init(
stage: Stage, startedAt: Date, modeLine: String,
latencyMs: Int? = nil, mbps: Double? = nil, backgroundDeadline: Date? = nil
) {
self.stage = stage
self.startedAt = startedAt
self.modeLine = modeLine
self.latencyMs = latencyMs
self.mbps = mbps
self.backgroundDeadline = backgroundDeadline
}
}
}
/// Kind string for the Live Activity kept next to the attributes so app + extension agree.
public enum PunktfunkActivity {
public static let kind = "PunktfunkSession"
}
#endif
@@ -0,0 +1,30 @@
// App Intents that must compile into BOTH the app and the widget extension live here in the shared
// module. Today that's `EndStreamIntent` the Live Activity's "End stream" button (a
// LiveActivityIntent runs in the APP's process) which M4 also surfaces to Siri/Shortcuts.
//
// Gated on os(iOS): LiveActivityIntent is part of ActivityKit's world (iPhone/iPad only). The M4
// Connect/Wake intents that need the app's router live in the app target, not here.
#if os(iOS)
import AppIntents
import Foundation
/// Ends the active streaming session. Backs the Live Activity's End button and the Shortcuts /
/// Siri "End the Punktfunk stream" phrase. `perform()` runs in the app's process (LiveActivityIntent)
/// it posts `.punktfunkEndActiveSession`, which the app's SessionModel owner observes and turns
/// into `disconnect(deliberate: true)` (the user explicitly ended it quit-close the host).
@available(iOS 17.0, *)
public struct EndStreamIntent: LiveActivityIntent {
public static let title: LocalizedStringResource = "End Punktfunk Stream"
public static let description = IntentDescription("Ends the active Punktfunk streaming session.")
public init() {}
public func perform() async throws -> some IntentResult {
await MainActor.run {
NotificationCenter.default.post(name: .punktfunkEndActiveSession, object: nil)
}
return .result()
}
}
#endif
@@ -0,0 +1,63 @@
// The saved-host model + its on-disk JSON wire format the widget/extension depends on BOTH, so
// they live in the dependency-free shared module. The `ObservableObject` store that wraps them
// (`HostStore`, with add/remove/pin/reachability) stays in the app target; discovery-join helpers
// (`matches`, `advertises`) stay there too because they reference PunktfunkKit's `DiscoveredHost`.
//
// Wire-format stability: the JSON encoding of `StoredHost` is now a shared contract between the app
// (writer) and the widget (reader). The `PunktfunkSharedTests` codec round-trip pins it do not
// rename the coding keys or make a stored `Optional` non-optional (older saved JSON must still
// decode; synthesized Decodable treats a missing Optional as nil).
import Foundation
/// The management-API port default (distinct from the data-plane `port`). Lives here (not in
/// PunktfunkKit's LibraryClient, which re-exports it) so `StoredHost.effectiveMgmtPort` can resolve
/// it without the shared module taking a dependency on the kit.
public let punktfunkDefaultMgmtPort: UInt16 = 47990
public struct StoredHost: Identifiable, Codable, Hashable {
public var id = UUID()
public var name: String
public var address: String
public var port: UInt16 = 9777
/// SHA-256 of the host's certificate, set after the user explicitly trusted it.
public var pinnedSHA256: Data?
/// Last time a streaming session actually started (nil until the first one).
public var lastConnected: Date?
/// Management-API port for the library browser (distinct from the data-plane `port`). Optional
/// (NOT a defaulted non-optional) so older saved hosts whose JSON lacks this key still
/// decode: synthesized Decodable ignores property defaults but treats a missing Optional as
/// nil. Resolve via `effectiveMgmtPort`. (Auth is mTLS by the pinned identity no token.)
public var mgmtPort: UInt16?
/// Wake-on-LAN MAC address(es) of the host's wake-capable NIC(s), each `aa:bb:cc:dd:ee:ff`.
/// Learned from the host's mDNS `mac` TXT record while it's awake and persisted here, so the
/// client can send a magic packet to wake the host later (when it's asleep and no longer
/// advertising). Optional (same forward-compat reason as `mgmtPort`); nil until first learned.
public var macAddresses: [String]?
/// Share the clipboard with this host (macOS sessions; design/clipboard-and-file-transfer.md
/// §5.3). Opt-in per host: nil/false = off (nil also keeps older saved JSON decoding same
/// forward-compat reason as `mgmtPort`). Honored only when the host advertises
/// `HOST_CAP_CLIPBOARD`.
public var clipboardSync: Bool?
public init(
id: UUID = UUID(), name: String, address: String, port: UInt16 = 9777,
pinnedSHA256: Data? = nil, lastConnected: Date? = nil, mgmtPort: UInt16? = nil,
macAddresses: [String]? = nil, clipboardSync: Bool? = nil
) {
self.id = id
self.name = name
self.address = address
self.port = port
self.pinnedSHA256 = pinnedSHA256
self.lastConnected = lastConnected
self.mgmtPort = mgmtPort
self.macAddresses = macAddresses
self.clipboardSync = clipboardSync
}
public var displayName: String { name.isEmpty ? address : name }
public var effectiveMgmtPort: UInt16 { mgmtPort ?? punktfunkDefaultMgmtPort }
/// Wake-capable, in a form the wake helper accepts (empty when none learned yet).
public var wakeMacs: [String] { macAddresses ?? [] }
}
@@ -52,13 +52,6 @@ final class RumbleTuningTests: XCTestCase {
XCTAssertEqual(RumbleTuning.handoffStart(endsAt: 100, now: 100.5), 100.5)
}
func testPolicies() {
// The session policy ties motor life to wire liveness; the manual (test-panel) policy
// holds a level indefinitely.
XCTAssertNotNil(RumbleRenderer.Policy.session.staleAfter)
XCTAssertNil(RumbleRenderer.Policy.manual.staleAfter)
}
/// Exercise the renderer's queue/ticker machinery without a physical pad: a wire-rate call
/// storm, an audible target left to the ticker (watchdog path), then `stop()` which runs
/// `queue.sync` against the same serial queue the ticker fires on and must not deadlock.
@@ -75,45 +68,22 @@ final class RumbleTuningTests: XCTestCase {
renderer.stop()
}
func testLeaseSecondsInterpretsWireTTL() {
// The legacy no-lease sentinel nil (fall back to the staleness watchdog).
XCTAssertNil(RumbleTuning.leaseSeconds(ttlMs: RumbleTuning.noTTL))
XCTAssertEqual(RumbleTuning.noTTL, UInt32.max)
// A real lease its duration in seconds (non-nil for any ttl != noTTL).
XCTAssertEqual(RumbleTuning.leaseSeconds(ttlMs: 400) ?? .nan, 0.4, accuracy: 1e-9)
XCTAssertEqual(RumbleTuning.leaseSeconds(ttlMs: 0) ?? .nan, 0, accuracy: 1e-9)
XCTAssertEqual(RumbleTuning.leaseSeconds(ttlMs: 150) ?? .nan, 0.15, accuracy: 1e-9)
}
func testEnvelopeLeaseBoundsMotorLifeTighterThanTheLegacyWatchdog() {
// The whole point of v2: a host-supplied lease silences the motor faster than the
// legacy staleness watchdog ever could (which needs sessionStaleSeconds of silence). The
// default 400 ms TTL is well under that, on every platform.
let defaultTTL = RumbleTuning.leaseSeconds(ttlMs: 400)
XCTAssertNotNil(defaultTTL)
XCTAssertLessThan(defaultTTL!, RumbleTuning.sessionStaleSeconds)
// The ticker must be able to observe an expired lease promptly (well within one TTL).
XCTAssertLessThan(RumbleTuning.tickSeconds, defaultTTL!)
}
/// A v2 envelope with a short TTL, left unrenewed, must self-silence the renderer's core
/// promise. Drive the real queue/ticker (no physical pad) and confirm it doesn't wedge.
func testEnvelopeExpiresWhenUnrenewed() {
/// A zero command must silence promptly the engine (punktfunk-core) emits explicit zeros at
/// every policy stop (lease expiry, legacy staleness, session close), and the renderer's only
/// job is to apply them. Drive the real queue/ticker (no physical pad) and confirm no wedge.
func testZeroCommandSilencesAndTeardownDoesNotDeadlock() {
let renderer = RumbleRenderer(policy: .session)
renderer.retarget(nil)
// A 100 ms lease, then no renewal the ticker (50 ms) must silence it on its own.
renderer.apply(low: 0x8000, high: 0x8000, ttlMs: 100)
Thread.sleep(forTimeInterval: 0.3)
// No assertion on private state; this exercises the expiry path + serial-queue teardown
renderer.apply(low: 0x8000, high: 0x8000)
Thread.sleep(forTimeInterval: 0.1)
renderer.apply(low: 0, high: 0)
Thread.sleep(forTimeInterval: 0.1)
// No assertion on private state; this exercises the stop path + serial-queue teardown
// without deadlock (the ticker fires on the same queue stop() sync-hops onto).
renderer.stop()
}
func testTuningRelationsTheDesignDependsOn() {
// The watchdog must tolerate a couple of lost 500 ms host refreshes (heals, not gaps)
// but trip well before a stuck rumble reads as "still going".
XCTAssertGreaterThan(RumbleTuning.sessionStaleSeconds, 2 * 0.5)
XCTAssertLessThanOrEqual(RumbleTuning.sessionStaleSeconds, 2.5)
// Re-arm headroom must clear several ticker periods, or a steady rumble could miss the
// segment boundary and gap.
XCTAssertGreaterThanOrEqual(
@@ -123,9 +93,8 @@ final class RumbleTuningTests: XCTestCase {
// The rebake throttle must be far under the host refresh period, or refreshed level
// changes would queue behind it; and under a frame at 30 fps so ramps stay smooth.
XCTAssertLessThan(RumbleTuning.minRebakeSeconds, 1.0 / 30)
// The ticker (which lands throttled levels) must outpace the HID keepalive and the
// watchdog, or those deadlines could be overshot by a full period.
// The ticker (which lands throttled levels) must outpace the HID keepalive, or its
// deadline could be overshot by a full period.
XCTAssertLessThan(RumbleTuning.tickSeconds, RumbleTuning.hidKeepaliveSeconds)
XCTAssertLessThan(RumbleTuning.tickSeconds, RumbleTuning.sessionStaleSeconds)
}
}
@@ -0,0 +1,90 @@
// PunktfunkShared is now a wire contract between the app (writer) and the widget extension +
// deep-link senders (readers). These pin the two formats that cross that boundary:
// the `StoredHost` JSON codec the widget decodes the exact bytes the app persisted, and
// older saved JSON (missing `mgmtPort` / `macAddresses`) must still decode;
// the `punktfunk://` deep-link grammar the widget builds URLs the app parses.
import XCTest
@testable import PunktfunkKit
import PunktfunkShared
final class SharedFoundationTests: XCTestCase {
// MARK: - StoredHost JSON codec
func testStoredHostRoundTrips() throws {
let host = StoredHost(
id: UUID(uuidString: "11111111-2222-3333-4444-555555555555")!,
name: "Tower", address: "192.168.1.173", port: 9777,
pinnedSHA256: Data([0xDE, 0xAD, 0xBE, 0xEF]),
lastConnected: Date(timeIntervalSince1970: 1_700_000_000),
mgmtPort: 47990, macAddresses: ["aa:bb:cc:dd:ee:ff"], clipboardSync: true)
let data = try JSONEncoder().encode(host)
let decoded = try JSONDecoder().decode(StoredHost.self, from: data)
XCTAssertEqual(decoded, host)
}
/// Older saved hosts predate `mgmtPort`/`macAddresses` a missing key must decode to nil, not
/// throw (synthesized Decodable treats a missing Optional as nil). This is the forward-compat
/// guarantee the widget depends on when reading a store written by any prior build.
func testStoredHostDecodesLegacyJSONWithoutOptionalKeys() throws {
let json = """
{"id":"11111111-2222-3333-4444-555555555555","name":"Old","address":"10.0.0.5","port":9777}
""".data(using: .utf8)!
let decoded = try JSONDecoder().decode(StoredHost.self, from: json)
XCTAssertEqual(decoded.name, "Old")
XCTAssertNil(decoded.mgmtPort)
XCTAssertNil(decoded.macAddresses)
XCTAssertNil(decoded.pinnedSHA256)
XCTAssertNil(decoded.lastConnected)
XCTAssertNil(decoded.clipboardSync)
// Resolvers fall back cleanly.
XCTAssertEqual(decoded.effectiveMgmtPort, punktfunkDefaultMgmtPort)
XCTAssertEqual(decoded.wakeMacs, [])
XCTAssertEqual(decoded.displayName, "Old")
}
func testStoredHostDisplayNameFallsBackToAddress() {
let host = StoredHost(name: "", address: "10.0.0.9")
XCTAssertEqual(host.displayName, "10.0.0.9")
}
// MARK: - DeepLink grammar
func testDeepLinkConnectRoundTrips() {
let id = UUID()
let link = DeepLink.connect(host: id, launchID: nil)
let parsed = DeepLink(link.url)
XCTAssertEqual(parsed, link)
XCTAssertEqual(parsed, .connect(host: id, launchID: nil))
}
func testDeepLinkConnectWithLaunchRoundTrips() {
let id = UUID()
// A store-qualified GameEntry.id with a reserved char must survive percent-encoding.
let launch = "steam:570"
let link = DeepLink.connect(host: id, launchID: launch)
XCTAssertEqual(DeepLink(link.url), .connect(host: id, launchID: launch))
}
func testDeepLinkParsesCanonicalString() throws {
let id = UUID()
let url = try XCTUnwrap(URL(string: "punktfunk://connect/\(id.uuidString)"))
XCTAssertEqual(DeepLink(url), .connect(host: id, launchID: nil))
}
func testDeepLinkRejectsForeignSchemeAndBadHost() throws {
let id = UUID()
XCTAssertNil(DeepLink(try XCTUnwrap(URL(string: "https://connect/\(id.uuidString)"))))
XCTAssertNil(DeepLink(try XCTUnwrap(URL(string: "punktfunk://connect/not-a-uuid"))))
XCTAssertNil(DeepLink(try XCTUnwrap(URL(string: "punktfunk://bogus/\(id.uuidString)"))))
}
func testDeepLinkSchemeIsCaseInsensitive() throws {
let id = UUID()
let url = try XCTUnwrap(URL(string: "PUNKTFUNK://connect/\(id.uuidString)"))
XCTAssertEqual(DeepLink(url), .connect(host: id, launchID: nil))
}
}
+52
View File
@@ -0,0 +1,52 @@
# Frame capture (plan §7 / §W6): the Linux xdg-ScreenCast/PipeWire portal capturer and the Windows
# IDD direct-push capturer, plus the synthetic sources + the Capturer trait, extracted into a
# subsystem crate. Depends on the shared frame vocabulary (pf-frame), the zero-copy plumbing
# (pf-zerocopy), and the display leaves (pf-win-display) — never on pf-encode: the encode-backend
# facts arrive pre-resolved (ZeroCopyPolicy) and the sealed-channel delivery as a closure
# (FrameChannelSender), so the capture→encode edge is one-way (plan §2.4).
[package]
name = "pf-capture"
version = "0.12.0"
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
description = "punktfunk host frame capture: Linux PipeWire portal + Windows IDD direct-push capturers behind one Capturer trait."
publish = false
[dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
pf-frame = { path = "../pf-frame" }
pf-zerocopy = { path = "../pf-zerocopy" }
pf-win-display = { path = "../pf-win-display" }
pf-gpu = { path = "../pf-gpu" }
pf-host-config = { path = "../pf-host-config" }
anyhow = "1"
tracing = "0.1"
[target.'cfg(target_os = "linux")'.dependencies]
# The xdg ScreenCast + RemoteDesktop portals, and the PipeWire consumer for the capture frames.
ashpd = { version = "0.13", features = ["screencast", "remote_desktop"] }
pipewire = "0.9"
libc = "0.2"
# ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane).
tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
[target.'cfg(target_os = "windows")'.dependencies]
# The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header).
pf-driver-proto = { path = "../pf-driver-proto" }
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_Security",
"Win32_Security_Authorization",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_Graphics_Direct3D_Fxc",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
"Win32_System_LibraryLoader",
"Win32_System_Memory",
"Win32_System_Threading",
"Win32_UI_HiDpi",
"Win32_UI_Input_KeyboardAndMouse",
"Win32_UI_WindowsAndMessaging",
] }
+357
View File
@@ -0,0 +1,357 @@
//! Frame capture (plan §7 / §W6): the capturers themselves — the Linux xdg-ScreenCast/PipeWire
//! portal capturer and the Windows IDD direct-push capturer — plus the synthetic test sources and
//! the `Capturer` trait, extracted into a subsystem crate. Speaks the shared frame vocabulary
//! (`pf-frame`) + the zero-copy plumbing (`pf-zerocopy`) and the display leaves (`pf-win-display`),
//! and NEVER `pf-encode` — the capture→encode edge is one-way (the encode-backend facts arrive
//! pre-resolved in a [`ZeroCopyPolicy`], and the Windows sealed-channel delivery arrives as a
//! [`FrameChannelSender`] closure, so this crate reaches neither the encoder nor the host
//! orchestrator).
// Scaffold: trait defaults + synthetic sources are defined ahead of the backends that use them.
#![allow(dead_code)]
// Every unsafe block in this crate carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::Result;
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
// The Linux capturer reaches `DmabufFrame` through `super::`; `CursorOverlay` it names directly as
// `pf_frame::CursorOverlay`, so only `DmabufFrame` needs to sit in this crate root's scope.
#[cfg(target_os = "linux")]
use pf_frame::DmabufFrame;
/// Produces frames from a captured output. Lives on its own thread, feeding the encoder
/// over a bounded drop-oldest channel (never block the compositor).
pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>;
/// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal
/// overrides it to drain its channel without blocking.
fn try_latest(&mut self) -> Result<Option<CapturedFrame>> {
self.next_frame().map(Some)
}
/// Gate expensive per-frame work so the capturer can be kept alive (reused) between
/// streams without burning CPU. The portal capturer skips the de-pad copy while inactive;
/// the default is a no-op (synthetic sources are produced on demand). Set `true` for the
/// duration of a stream, `false` when it ends.
fn set_active(&self, _active: bool) {}
/// The source's static HDR mastering metadata (SMPTE ST.2086 + content light level), when the
/// capturer can read it from the output (Windows `IDXGIOutput6::GetDesc1`). `None` = unknown /
/// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet).
/// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram),
/// so the two stay a single source of truth. May change mid-session if the source is regraded.
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None
}
/// How many frames the encode loop may keep in flight (submitted but not yet polled) before it
/// blocks. `1` (the default) is the synchronous loop: capture → submit → poll-blocks, so the
/// per-frame wall time is `capture+convert + encode`. A capturer that hands a fresh output texture
/// per frame (so the encode of N reads a different texture than the convert of N+1 writes) can return
/// `>1` to PIPELINE: the loop submits N+1 before polling N, overlapping the convert/copy on the 3D
/// engine with the NVENC-ASIC encode of the prior frame, dropping per-frame wall toward `max(...)`.
fn pipeline_depth(&self) -> usize {
1
}
/// The OS display-target id this capturer is bound to (Windows IDD-push), so the resize path
/// can verify the display it just reconfigured is STILL the one this capturer serves (an
/// in-place resize keeps the target; a re-arrival fallback mints a new one, which needs a
/// fresh capturer). `None` = the backend has no such identity (every non-IDD backend).
fn capture_target_id(&self) -> Option<u32> {
None
}
/// HOST-INITIATED output resize (latency plan P2.3): the session's resize handler has ALREADY
/// committed the display's new mode (the manager's in-place mode set), so a capable capturer
/// re-sizes its capture surface NOW — no descriptor-poll debounce (that machinery stays, for
/// EXTERNAL changes only) and no teardown: the capture pipeline and its send thread survive;
/// only the encoder is swapped by the caller once the first new-size frame arrives. Returns
/// `true` when handled; `false` (the default) routes the caller to the full-rebuild path.
fn resize_output(&mut self, _width: u32, _height: u32) -> bool {
false
}
}
/// A deterministic moving test pattern (BGRx). Lets the spike exercise the encode → file →
/// `punktfunk_core` path with no live capture session, and produces obviously non-static
/// content (a sweeping bar + animated gradient) so the encoded output is verifiable.
pub struct SyntheticCapturer {
width: u32,
height: u32,
fps: u32,
frame_idx: u64,
buf: Vec<u8>,
}
impl SyntheticCapturer {
const BPP: usize = 4; // emits BGRx
pub fn new(width: u32, height: u32, fps: u32) -> Self {
assert!(width > 0 && height > 0 && fps > 0);
let buf = vec![0u8; width as usize * height as usize * Self::BPP];
SyntheticCapturer {
width,
height,
fps,
frame_idx: 0,
buf,
}
}
}
impl Capturer for SyntheticCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
let w = self.width as usize;
let h = self.height as usize;
let bpp = Self::BPP;
let t = self.frame_idx;
// A vertical bar sweeps left→right once every ~2s; the background is a gradient
// whose phase advances each frame, so every pixel changes frame-to-frame.
let bar_x = ((t * w as u64) / (self.fps as u64 * 2)) % w as u64;
let phase = (t % 256) as usize;
for y in 0..h {
let row = y * w * bpp;
for x in 0..w {
let i = row + x * bpp;
let on_bar = (x as u64).abs_diff(bar_x) < 8;
// BGRx byte order: [B, G, R, x]
self.buf[i] = if on_bar {
255
} else {
((x + phase) & 0xff) as u8
};
self.buf[i + 1] = if on_bar {
255
} else {
((y + phase) & 0xff) as u8
};
self.buf[i + 2] = if on_bar { 255 } else { ((x + y) & 0xff) as u8 };
self.buf[i + 3] = 0;
}
}
let pts_ns = self.frame_idx * 1_000_000_000 / self.fps as u64;
self.frame_idx += 1;
Ok(CapturedFrame {
width: self.width,
height: self.height,
pts_ns,
format: PixelFormat::Bgrx,
payload: FramePayload::Cpu(self.buf.clone()),
cursor: None,
})
}
}
/// A cheap moving test pattern (BGRx) for the streaming path: a pulsing field + a white band
/// sweeping down, generated with whole-buffer `fill`s so it stays real-time even at 5K.
pub struct FastSyntheticCapturer {
width: u32,
height: u32,
frame_idx: u64,
buf: Vec<u8>,
/// PUNKTFUNK_SYNTH_NOISE: every frame is fresh high-entropy noise NVENC can't compress or
/// predict, so the encoder hits its (CBR) bitrate target — a throughput test of the real
/// encode→FEC→send→recv path. The default flat/band content compresses to ~nothing, so it
/// can't generate real Mbps (the encoder is content-driven). xorshift over u64 chunks.
noise: bool,
rng: u64,
}
impl FastSyntheticCapturer {
pub fn new(width: u32, height: u32) -> Self {
assert!(width > 0 && height > 0);
FastSyntheticCapturer {
width,
height,
frame_idx: 0,
buf: vec![0u8; width as usize * height as usize * 4],
noise: std::env::var_os("PUNKTFUNK_SYNTH_NOISE").is_some(),
rng: 0x9e3779b97f4a7c15,
}
}
}
impl Capturer for FastSyntheticCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
if self.noise {
// Fresh, every-frame-decorrelated noise: reseed from the frame index so consecutive
// frames share no structure (forces large P-frames too, not just the keyframe).
let mut s = self
.rng
.wrapping_add(self.frame_idx.wrapping_mul(0x2545F491_4F6CDD1D))
| 1;
for c in self.buf.chunks_exact_mut(8) {
s ^= s << 13;
s ^= s >> 7;
s ^= s << 17;
c.copy_from_slice(&s.to_le_bytes());
}
self.rng = s;
} else {
let (w, h) = (self.width as usize, self.height as usize);
let row = w * 4;
let shade = (self.frame_idx % 256) as u8;
self.buf.fill(shade);
let band_h = (h / 20).max(1);
let band_y = (self.frame_idx as usize * 6) % h;
for y in band_y..(band_y + band_h).min(h) {
self.buf[y * row..(y + 1) * row].fill(0xff);
}
}
self.frame_idx += 1;
Ok(CapturedFrame {
width: self.width,
height: self.height,
pts_ns: 0,
format: PixelFormat::Bgrx,
payload: FramePayload::Cpu(self.buf.clone()),
cursor: None,
})
}
}
/// The encode-backend facts the Linux zero-copy negotiation needs, resolved **once** here (the host
/// facade, which may reach the host `encode`) and passed **into** the capturer — so the capturer never
/// calls back into `encode`, keeping the capture→encode dependency one-way (plan §2.4 / §W6). The
/// three facts were formerly re-derived inside the PipeWire thread via
/// `encode::{linux_zero_copy_is_vaapi, resolved_backend_is_gpu, pyrowave_capture_modifiers}`.
#[cfg(target_os = "linux")]
#[derive(Clone, Default)]
pub struct ZeroCopyPolicy {
/// The GPU encode backend resolves to VAAPI (AMD/Intel) — the capturer hands raw dmabufs
/// straight through instead of the EGL→CUDA import (the host `encode::linux_zero_copy_is_vaapi`).
pub backend_is_vaapi: bool,
/// The resolved backend produces GPU-resident frames (everything but the software encoder) —
/// used only to phrase the CPU-fallback warning (the host `encode::resolved_backend_is_gpu`).
pub backend_is_gpu: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the encoder pref is `pyrowave` (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
pub pyrowave_modifiers: Vec<u64>,
}
#[cfg(target_os = "linux")]
pub fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
true
}
#[cfg(target_os = "windows")]
pub fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
// but only a backend that ingests RGB and CSCs it to 4:4:4 itself can use it — today just
// direct-NVENC (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). An HDR
// display can't be known here (the virtual display's mode settles after the Welcome); that
// combination downgrades at capture time — the capturer emits P010 and the encoder's caps
// cross-check reports the 4:2:0 truth (the in-band SPS keeps the client correct either way).
encoder_ingests_rgb_444
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
false
}
/// Host-registered HID compose-kick hook: `(target_rect, desktop_bounds) -> accepted`, both
/// `(x, y, w, h)` in desktop coordinates (from CCD). The host facade registers it once at startup
/// when the resident virtual HID mouse exists (`inject::mouse_windows::hid_kick`); the IDD-push
/// capturer's compose kick then prefers it over `SendInput`, because device-level input is
/// delivered regardless of this process's session or the active desktop and wakes a powered-off
/// display — the lid-closed first-frame fix. Same one-way-edge philosophy as
/// [`FrameChannelSender`]: this crate never reaches back into the host's inject module. `false`
/// from the hook = mouse not available right now → the caller falls back to `SendInput`.
#[cfg(target_os = "windows")]
pub static HID_COMPOSE_KICK: std::sync::OnceLock<HidKickFn> = std::sync::OnceLock::new();
/// The [`HID_COMPOSE_KICK`] hook's shape: `(target_rect, desktop_bounds) -> accepted`, both
/// `(x, y, w, h)` in desktop coordinates.
#[cfg(target_os = "windows")]
pub type HidKickFn = fn((i32, i32, i32, i32), (i32, i32, i32, i32)) -> bool;
/// Delivers a monitor's sealed frame channel to the pf-vdisplay driver (`IOCTL_SET_FRAME_CHANNEL`) —
/// the ONE reach the IDD-push capturer would otherwise make into the host's `vdisplay` module. The
/// host facade builds this closure (capturing the pf-vdisplay control device handle + the
/// `send_frame_channel` IOCTL wrapper) and hands it in, so this crate delivers the channel without a
/// path back to the orchestrator. Called once per ring generation (at attach), never per-frame —
/// guardrail-compliant. The handle values in `req` were just duplicated into the driver's WUDFHost
/// by the capturer's [`windows::idd_push`] broker; on IOCTL success the DRIVER owns them.
#[cfg(target_os = "windows")]
pub type FrameChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync,
>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")]
pub mod pwinit;
// The Windows backend lives under `windows/`, the Linux one under `linux/`. Windows capture is IDD
// direct-push only (DXGI Desktop Duplication + the WGC relay were removed).
#[cfg(target_os = "windows")]
#[path = "windows/dxgi.rs"]
pub mod dxgi;
#[cfg(target_os = "windows")]
#[path = "windows/idd_push.rs"]
mod idd_push;
// The WUDFHost-identity check the IDD-push broker uses is reused by the host's gamepad-channel
// bootstrap (`inject::windows::gamepad_raii`); re-export it so that reach stays a leaf dependency.
#[cfg(target_os = "windows")]
pub use idd_push::verify_is_wudfhost;
#[cfg(target_os = "linux")]
#[path = "linux/mod.rs"]
mod linux;
#[cfg(target_os = "windows")]
#[path = "windows/synthetic_nv12.rs"]
pub mod synthetic_nv12;
/// Open the Linux xdg-ScreenCast portal capturer for a client-sized monitor. `anchored` drives
/// ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits that grant headlessly. The
/// [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
#[cfg(target_os = "linux")]
pub fn open_portal_monitor(anchored: bool, policy: ZeroCopyPolicy) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::open(anchored, policy).map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// Open the Linux portal capturer bound to an already-created virtual output's PipeWire node. The
/// caller (host facade) explodes its `VirtualOutput` into these primitives + owns nothing after —
/// the capturer takes `keepalive`, so dropping it releases the output. `allow_zerocopy` mirrors
/// `OutputFormat::gpu`; `want_444` selects the planar-YUV444 GPU convert.
#[cfg(target_os = "linux")]
#[allow(clippy::too_many_arguments)]
pub fn open_virtual_output(
remote_fd: Option<std::os::fd::OwnedFd>,
node_id: u32,
preferred_mode: Option<(u32, u32, u32)>,
keepalive: Box<dyn Send>,
allow_zerocopy: bool,
want_444: bool,
policy: ZeroCopyPolicy,
) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::from_virtual_output(
remote_fd,
node_id,
preferred_mode,
keepalive,
allow_zerocopy,
want_444,
policy,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// Open the Windows IDD direct-push capturer on a pf-vdisplay target. `sender` delivers the sealed
/// frame channel to the driver (the host facade builds it from the vdisplay control device). On
/// failure the `keepalive` is handed back so the caller can retire the display.
#[cfg(target_os = "windows")]
#[allow(clippy::too_many_arguments)]
pub fn open_idd_push(
target: pf_frame::dxgi::WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
client_10bit: bool,
want_444: bool,
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(target, preferred, client_10bit, want_444, keepalive, sender)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
@@ -20,7 +20,7 @@
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat};
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy};
use anyhow::{anyhow, Context, Result};
use std::os::fd::OwnedFd;
use std::sync::atomic::{AtomicBool, Ordering};
@@ -55,7 +55,7 @@ pub struct PortalCapturer {
stall_since: Option<std::time::Instant>,
/// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If
/// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches
/// the process-wide downgrade ([`crate::zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
/// rebuild retries on the CPU offer instead of failing identically forever.
vaapi_dmabuf: bool,
/// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis.
@@ -77,7 +77,7 @@ impl PortalCapturer {
/// `anchored` drives ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits the
/// RemoteDesktop grant and never raises a separate ScreenCast dialog; `false` uses a plain
/// ScreenCast session (wlroots, which has no RemoteDesktop portal).
pub fn open(anchored: bool) -> Result<PortalCapturer> {
pub fn open(anchored: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> {
// Portal handshake (async) on its own thread; hands back the PW fd + node id.
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
thread::Builder::new()
@@ -101,37 +101,45 @@ impl PortalCapturer {
"ScreenCast portal session started; connecting PipeWire"
);
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok(spawn_pipewire(Some(fd), node_id, None, true, false)?.into_capturer(node_id, None))
Ok(
spawn_pipewire(Some(fd), node_id, None, true, false, policy)?
.into_capturer(node_id, None),
)
}
/// Build a capturer from an already-created virtual output ([`crate::vdisplay::VirtualOutput`]):
/// connect PipeWire to its node (`remote_fd` selects portal-remote vs. default-daemon) and
/// take ownership of its keepalive so the output lives exactly as long as this capturer. This
/// is how the client's requested resolution becomes the captured resolution without scaling.
/// `allow_zerocopy` mirrors [`OutputFormat::gpu`](crate::capture::OutputFormat): `false` forces
/// the CPU mmap path, `true` keeps the GPU zero-copy path subject to `PUNKTFUNK_ZEROCOPY`.
/// `want_444` (a 4:4:4 session) makes the zero-copy worker convert tiled dmabufs to planar
/// YUV444 on the GPU instead of NV12/RGB.
/// Build a capturer from an already-created virtual output's PipeWire node. The host facade
/// explodes its `vdisplay::VirtualOutput` into these primitives so this crate never depends on
/// the vdisplay type: `remote_fd` selects portal-remote vs. default-daemon, `node_id` is the
/// output's screencast node, `preferred_mode` seeds format negotiation, and `keepalive` owns the
/// output (dropping the capturer releases it). `allow_zerocopy` mirrors
/// [`OutputFormat::gpu`](pf_frame::OutputFormat): `false` forces the CPU mmap path, `true` keeps
/// the GPU zero-copy path subject to `PUNKTFUNK_ZEROCOPY`. `want_444` (a 4:4:4 session) makes the
/// zero-copy worker convert tiled dmabufs to planar YUV444 on the GPU instead of NV12/RGB.
#[allow(clippy::too_many_arguments)]
pub fn from_virtual_output(
vout: crate::vdisplay::VirtualOutput,
remote_fd: Option<OwnedFd>,
node_id: u32,
preferred_mode: Option<(u32, u32, u32)>,
keepalive: Box<dyn Send>,
allow_zerocopy: bool,
want_444: bool,
policy: ZeroCopyPolicy,
) -> Result<PortalCapturer> {
tracing::info!(
node_id = vout.node_id,
node_id,
allow_zerocopy,
want_444,
"connecting PipeWire to virtual output"
);
let node_id = vout.node_id;
Ok(spawn_pipewire(
vout.remote_fd,
remote_fd,
node_id,
vout.preferred_mode,
preferred_mode,
allow_zerocopy,
want_444,
policy,
)?
.into_capturer(node_id, Some(vout.keepalive)))
.into_capturer(node_id, Some(keepalive)))
}
}
@@ -186,6 +194,9 @@ fn spawn_pipewire(
// 4:4:4 session: tiled dmabufs convert to planar YUV444 on the GPU (`ImportKind::Tiled444`)
// instead of NV12/RGB, so the session stays zero-copy at full chroma.
want_444: bool,
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6).
policy: ZeroCopyPolicy,
) -> Result<PwHandles> {
// Frames flow from the pipewire thread over a small bounded channel.
let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8);
@@ -201,13 +212,13 @@ fn spawn_pipewire(
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope.
let (quit_tx, quit_rx) = ::pipewire::channel::channel::<()>();
let zerocopy = allow_zerocopy && crate::zerocopy::enabled();
let zerocopy = allow_zerocopy && pf_zerocopy::enabled();
// Mirror of the thread's `vaapi_passthrough` decision (deterministic from here: on a VAAPI
// backend the EGL→CUDA importer is never built) — kept on the capturer so `next_frame`'s
// negotiation-timeout branch knows a failed negotiation was the LINEAR-dmabuf offer.
let vaapi_dmabuf = zerocopy
&& std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() != Ok("1")
&& crate::encode::linux_zero_copy_is_vaapi();
&& policy.backend_is_vaapi;
let join = thread::Builder::new()
.name("punktfunk-pipewire".into())
.spawn(move || {
@@ -223,6 +234,7 @@ fn spawn_pipewire(
want_444,
preferred,
quit_rx,
policy,
) {
tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed");
}
@@ -330,11 +342,11 @@ impl PortalCapturer {
or capture never started)",
self.node_id
))
} else if self.vaapi_dmabuf && !crate::zerocopy::vaapi_dmabuf_forced() {
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
// The LINEAR-dmabuf-only offer (VAAPI passthrough default) was never accepted.
// Latch the process-wide downgrade so the encode loop's pipeline rebuild
// retries on the CPU offer instead of failing this same negotiation forever.
crate::zerocopy::note_vaapi_dmabuf_failed();
pf_zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this host to the \
@@ -604,7 +616,7 @@ fn portal_thread_remote_desktop(setup_tx: std::sync::mpsc::Sender<Result<(OwnedF
mod pipewire {
//! The PipeWire consumer, confined to its own thread (the PW types are `!Send`).
use super::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat};
use super::{CapturedFrame, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy};
use anyhow::{Context, Result};
use pipewire as pw;
use pw::{properties::properties, spa};
@@ -650,18 +662,16 @@ mod pipewire {
/// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves,
/// so the GPU encoder re-uploads its cursor texture only on change.
serial: u64,
/// One-shot guard for the "cursor present but this frame is zero-copy" notice.
warned_zerocopy: bool,
}
impl CursorState {
/// A shareable overlay for the GPU encode paths (blended at encode time), or `None` when
/// there is nothing to draw. Cheap: clones an `Arc` + a few scalars.
fn overlay(&self) -> Option<crate::capture::CursorOverlay> {
fn overlay(&self) -> Option<pf_frame::CursorOverlay> {
if !self.visible || self.rgba.is_empty() {
return None;
}
Some(crate::capture::CursorOverlay {
Some(pf_frame::CursorOverlay {
x: self.x,
y: self.y,
w: self.bw,
@@ -695,8 +705,8 @@ mod pipewire {
/// Consecutive tiled-import failures (reset on success); see [`IMPORT_FAIL_POISON`].
import_fail_streak: u32,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer,
/// normally via the isolated worker process (`crate::zerocopy::Importer::Remote`).
importer: Option<crate::zerocopy::Importer>,
/// normally via the isolated worker process (`pf_zerocopy::Importer::Remote`).
importer: Option<pf_zerocopy::Importer>,
/// VAAPI zero-copy: hand the raw dmabuf to the encoder (which imports + GPU-CSCs it) instead
/// of a CUDA import. Set when zero-copy is on, the EGL→CUDA importer is unavailable, and the
/// encoder backend is VAAPI (AMD/Intel).
@@ -1174,22 +1184,6 @@ mod pipewire {
if ud.broken.load(Ordering::Relaxed) {
return;
}
// Cursor-as-metadata only reaches the frame on the CPU de-pad path below (a small
// straight-alpha blit). The zero-copy paths hand a GPU-resident buffer straight to the
// encoder, so the cached cursor can't be composited here — that needs a GPU blit in the
// encoder (follow-up). Note it once, so a gamescope host (zero-copy by default) shows in
// the logs that the metadata IS arriving even while the overlay isn't drawn yet.
if ud.cursor.visible
&& !ud.cursor.warned_zerocopy
&& (ud.importer.is_some() || ud.vaapi_passthrough)
{
ud.cursor.warned_zerocopy = true;
tracing::warn!(
"cursor metadata received, but frames are delivered zero-copy (GPU-resident) — \
the cursor overlay is composited only on the CPU capture path today; GPU-path \
compositing (Vulkan/CUDA/VAAPI encode) is a follow-up"
);
}
// SAFETY: `spa_buf` is the `*mut spa_buffer` of the PipeWire buffer we dequeued and still hold for
// this `.process` callback (not requeued until after `consume_frame` returns), so it is live. The
// block null-checks `spa_buf`, requires `n_datas != 0`, and null-checks the `datas` array pointer
@@ -1223,7 +1217,7 @@ mod pipewire {
// closing the stale/old-frame race on NVIDIA. No-op for shm buffers or drivers that
// attach no fence. Covers both the GPU import and the CPU mmap read below.
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
match crate::dmabuf_fence::wait_read_ready(datas[0].fd(), 100) {
match pf_zerocopy::dmabuf_fence::wait_read_ready(datas[0].fd(), 100) {
Ok(waited) => {
static F1: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(true);
@@ -1241,7 +1235,7 @@ mod pipewire {
std::sync::atomic::AtomicBool::new(true);
if F2.swap(false, Ordering::Relaxed) {
tracing::warn!(
error = %format!("{e}"),
error = %e,
"dmabuf EXPORT_SYNC_FILE failed — no implicit-fence sync; NVIDIA \
zero-copy may show stale frames (no producer explicit sync)"
);
@@ -1255,7 +1249,7 @@ mod pipewire {
if ud.vaapi_passthrough {
if let Some(fmt) = ud.format {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
if let Some(fourcc) = crate::zerocopy::drm_fourcc(fmt) {
if let Some(fourcc) = pf_frame::drm_fourcc(fmt) {
let chunk = datas[0].chunk();
let offset = chunk.offset();
let stride = chunk.stride().max(0) as u32;
@@ -1318,7 +1312,7 @@ mod pipewire {
let mut gpu_import_broken = false;
if let (Some(importer), Some(fmt)) = (ud.importer.as_mut(), ud.format) {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
let plane = crate::zerocopy::DmabufPlane {
let plane = pf_zerocopy::DmabufPlane {
fd: datas[0].fd(),
offset: datas[0].chunk().offset(),
stride: datas[0].chunk().stride().max(0) as u32,
@@ -1327,7 +1321,7 @@ mod pipewire {
// gamescope) → direct CUDA external-memory import (NVIDIA EGL can't
// sample LINEAR).
let modifier = (ud.modifier != 0).then_some(ud.modifier);
if let Some(fourcc) = crate::zerocopy::drm_fourcc(fmt) {
if let Some(fourcc) = pf_frame::drm_fourcc(fmt) {
// GPU converts only on the tiled EGL/GL path (`modifier.is_some()`): a 4:4:4
// session gets the planar-YUV444 convert (full chroma, takes precedence over
// NV12 — 4:4:4 must never subsample), otherwise `PUNKTFUNK_NV12` gets NV12 —
@@ -1351,7 +1345,7 @@ mod pipewire {
match imported {
Ok(devbuf) => {
ud.import_fail_streak = 0;
crate::zerocopy::note_gpu_import_ok();
pf_zerocopy::note_gpu_import_ok();
static ONCE: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
@@ -1389,7 +1383,7 @@ mod pipewire {
Err(e) => {
let dead = importer.dead();
if dead {
crate::zerocopy::note_gpu_import_death();
pf_zerocopy::note_gpu_import_death();
}
if modifier.is_some() {
// Tiled buffer: the CPU fallback below would mmap TILED bytes
@@ -1567,6 +1561,9 @@ mod pipewire {
want_444: bool,
preferred: Option<(u32, u32, u32)>,
quit_rx: pw::channel::Receiver<()>,
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6).
policy: ZeroCopyPolicy,
) -> Result<()> {
crate::pwinit::ensure_init();
@@ -1599,15 +1596,15 @@ mod pipewire {
// would waste a CUDA probe — or worse, on an NVIDIA box forced to PUNKTFUNK_ENCODER=vaapi,
// succeed and produce CUDA payloads the VAAPI encoder must reject. Also skipped once
// repeated worker deaths latched the import off (a wedged GPU stack must not crash-loop).
let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi();
let backend_is_vaapi = policy.backend_is_vaapi;
let mut importer = if zerocopy && !backend_is_vaapi {
if crate::zerocopy::gpu_import_disabled() {
if pf_zerocopy::gpu_import_disabled() {
tracing::warn!(
"zero-copy GPU import disabled after repeated import-worker deaths — using CPU path"
);
None
} else {
match crate::zerocopy::Importer::new_for_capture() {
match pf_zerocopy::Importer::new_for_capture() {
Ok(i) => Some(i),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "zero-copy import unavailable — using CPU path");
@@ -1635,19 +1632,19 @@ mod pipewire {
// radeonsi/iHD import it and any compositor can allocate it.
let mut modifiers = importer
.as_mut()
.map(|i| i.supported_modifiers(crate::zerocopy::drm_fourcc(PixelFormat::Bgrx).unwrap()))
.map(|i| i.supported_modifiers(pf_frame::drm_fourcc(PixelFormat::Bgrx).unwrap()))
.unwrap_or_default();
if (importer.is_some() || vaapi_passthrough) && !modifiers.contains(&0) {
modifiers.push(0); // DRM_FORMAT_MOD_LINEAR
}
// PyroWave passthrough: the encoder imports through Vulkan, not libva — extend the
// advertisement with every modifier its device samples from, so compositors that
// never allocate LINEAR (Mutter+NVIDIA) still negotiate zero-copy dmabufs.
#[cfg(feature = "pyrowave")]
if vaapi_passthrough && crate::config::config().encoder_pref.as_str() == "pyrowave" {
for m in crate::encode::pyrowave_capture_modifiers(
crate::zerocopy::drm_fourcc(PixelFormat::Bgrx).unwrap(),
) {
// never allocate LINEAR (Mutter+NVIDIA) still negotiate zero-copy dmabufs. The modifiers
// were resolved by the facade (`ZeroCopyPolicy::pyrowave_modifiers`) — non-empty only when
// the host's `pyrowave` feature is on AND the encoder pref is `pyrowave` — so capture never
// calls back into `encode` and needs no feature gate of its own (the emptiness check gates it).
if vaapi_passthrough && !policy.pyrowave_modifiers.is_empty() {
for &m in &policy.pyrowave_modifiers {
if !modifiers.contains(&m) {
modifiers.push(m);
}
@@ -1675,7 +1672,7 @@ mod pipewire {
sample = ?&modifiers[..modifiers.len().min(6)],
"zero-copy: advertising EGL-importable dmabuf modifiers"
);
} else if backend_is_vaapi && crate::capture::gpu_encode() {
} else if backend_is_vaapi && policy.backend_is_gpu {
// A VAAPI session on the CPU path pays three full-frame CPU touches (mmap de-pad +
// swscale RGB→NV12 + surface upload) — make the silent fallback visible.
tracing::warn!(
@@ -1694,7 +1691,7 @@ mod pipewire {
"4:4:4 zero-copy: tiled dmabufs convert to planar YUV444 (BT.709) on the GPU — \
NVENC fed native full-chroma YUV, no CPU pixel path"
);
} else if want_dmabuf && !vaapi_passthrough && crate::zerocopy::nv12_enabled() {
} else if want_dmabuf && !vaapi_passthrough && pf_zerocopy::nv12_enabled() {
tracing::info!(
"PUNKTFUNK_NV12: tiled dmabufs convert to NV12 (BT.709 limited) on the GPU — NVENC \
fed native YUV (no internal RGBYUV CSC)"
@@ -1713,7 +1710,7 @@ mod pipewire {
import_fail_streak: 0,
importer,
vaapi_passthrough,
nv12: crate::zerocopy::nv12_enabled(),
nv12: pf_zerocopy::nv12_enabled(),
yuv444: want_444,
dbg_log_n: 0,
cursor: CursorState::default(),
@@ -1915,7 +1912,15 @@ mod pipewire {
unsafe { stream.queue_raw_buffer(newest) };
}));
if outcome.is_err() {
tracing::error!("panic in pipewire process callback — frame dropped");
// In the per-frame `.process` callback: a deterministic panic (e.g. a bad
// format) would fire this every frame, so power-of-two throttle it — enough to
// surface the fault without evicting the whole log ring.
static PANICS: std::sync::atomic::AtomicU64 =
std::sync::atomic::AtomicU64::new(0);
let n = PANICS.fetch_add(1, Ordering::Relaxed) + 1;
if n.is_power_of_two() {
tracing::error!(count = n, "panic in pipewire process callback — frame dropped");
}
}
})
.register()
@@ -1930,7 +1935,11 @@ mod pipewire {
// Request raw video in any encoder-mappable layout, any size/framerate.
let obj = if let Some((fw, fh)) = fixed_pod {
tracing::info!(fw, fh, "PW DEBUG: offering fixed BGRx pod");
tracing::info!(
fw,
fh,
"pipewire: offering a fixed BGRx format pod (PUNKTFUNK_PW_FIXED_POD)"
);
pw::spa::pod::object!(
pw::spa::utils::SpaTypes::ObjectParamFormat,
pw::spa::param::ParamType::EnumFormat,
@@ -1,20 +1,27 @@
//! Shared Windows GPU primitives — D3D11 device creation, GPU scheduling priority hooks,
//! HLSL shader compilation, HDR FP16→P010 conversion ([`HdrP010Converter`]), video-engine
//! colour conversion ([`VideoConverter`]), and the IDD-push capture identity
//! ([`WinCaptureTarget`], [`pack_luid`]). Consumed by [`super::idd_push`].
//! DXGI Desktop Duplication has been removed; this module contains no capturer.
//! Windows capture GPU mechanics — the win32u GPU-preference hook, HLSL shader compilation, HDR
//! FP16→P010 conversion ([`HdrP010Converter`]), video-engine colour conversion ([`VideoConverter`]),
//! and the P010 self-test. Consumed by [`super::idd_push`].
//!
//! The shared IDD-push capture IDENTITY — [`WinCaptureTarget`], [`D3d11Frame`], [`pack_luid`], and
//! [`make_device`] (the D3D11 device factory + GPU scheduling-priority hardening) — moved into the
//! `pf-frame` leaf crate so capture, encode, and pf-vdisplay share one identity type without a
//! capture↔encode↔vdisplay cycle (plan §W6); this module re-exports it so every existing
//! `crate::dxgi::*` path keeps resolving. DXGI Desktop Duplication has been removed; this
//! module contains no capturer.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
pub use pf_frame::dxgi::{make_device, pack_luid, D3d11Frame, WinCaptureTarget};
use anyhow::{bail, Context, Result};
use std::ffi::c_void;
use std::sync::atomic::{AtomicU64, Ordering};
use windows::core::{s, Interface, PCSTR};
use windows::Win32::Foundation::{HMODULE, LUID};
use windows::Win32::Foundation::HMODULE;
use windows::Win32::Graphics::Direct3D::Fxc::D3DCompile;
use windows::Win32::Graphics::Direct3D::{
ID3DBlob, D3D_DRIVER_TYPE_UNKNOWN, D3D_FEATURE_LEVEL_11_0, D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST,
ID3DBlob, D3D_FEATURE_LEVEL_11_0, D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST,
};
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, ID3D11Buffer, ID3D11Device, ID3D11DeviceContext, ID3D11PixelShader,
@@ -32,205 +39,6 @@ use windows::Win32::Graphics::Dxgi::Common::{
DXGI_FORMAT, DXGI_FORMAT_P010, DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM,
DXGI_FORMAT_R16_UNORM, DXGI_SAMPLE_DESC,
};
use windows::Win32::Graphics::Dxgi::{IDXGIAdapter1, IDXGIDevice, IDXGIDevice1};
#[derive(Clone)]
pub struct WinCaptureTarget {
/// Packed DXGI adapter LUID (`(HighPart << 32) | (LowPart & 0xffff_ffff)`).
pub adapter_luid: i64,
/// The output's GDI device name, e.g. `\\.\DISPLAY3`. Can CHANGE across a secure-desktop switch.
pub gdi_name: String,
/// Stable virtual-display (IddCx) target id — re-resolved to the current GDI name on every recovery.
pub target_id: u32,
/// The pf-vdisplay driver's WUDFHost pid (from the ADD reply) — the process the IDD-push capturer
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32,
}
/// A GPU-resident captured texture (future NVENC-D3D11 zero-copy path).
pub struct D3d11Frame {
pub texture: ID3D11Texture2D,
pub device: ID3D11Device,
}
// SAFETY: `D3d11Frame` owns an `ID3D11Texture2D` + `ID3D11Device`, which are COM interface pointers.
// D3D11 devices/resources use thread-safe (interlocked) COM reference counting, and the device is
// created free-threaded (`make_device` passes no `D3D11_CREATE_DEVICE_SINGLETHREADED`), so handing
// ownership of the frame to another thread — the capture→encode handoff — and releasing it there is
// sound. The value is moved, never aliased (no `Sync`), so there is no concurrent use of the
// single-threaded immediate context.
unsafe impl Send for D3d11Frame {}
pub fn pack_luid(luid: LUID) -> i64 {
((luid.HighPart as i64) << 32) | (luid.LowPart as i64 & 0xffff_ffff)
}
/// Create a fresh D3D11 device + context on a specific adapter (driver_type UNKNOWN with an explicit
/// adapter). Used at open and on every ACCESS_LOST: a device created on one desktop cannot sustain a
/// duplication on a *different* desktop (perpetual ACCESS_LOST), so the secure-desktop switch needs a
/// device made while the thread is attached to that desktop.
pub(crate) unsafe fn make_device(
adapter: &IDXGIAdapter1,
) -> Result<(ID3D11Device, ID3D11DeviceContext)> {
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice")?;
let device = device.context("null D3D11 device")?;
let context = context.context("null D3D11 context")?;
// GPU scheduling hardening — the same approach Sunshine/Apollo use, reimplemented here via the
// documented D3DKMT/DXGI APIs (no GPL source copied). Our capture+encode
// shares the GPU with the streamed game; when the game saturates the GPU our process is starved of
// GPU time slices, so NVENC sits near-idle yet `lock_bitstream` waits ~20 ms for our context to be
// scheduled — capping the stream (~47 fps measured at 5K@240) and stuttering. Per-frame copy/convert
// is NOT the cause (zero-copy + thread-priority alone didn't move it); the PROCESS-level GPU
// scheduling priority class is the decisive cross-process lever. Secondary: the absolute per-device
// GPU thread priority and a 1-frame latency cap.
elevate_process_gpu_priority();
if let Ok(dxgi_dev) = device.cast::<IDXGIDevice>() {
// The absolute max GPU thread priority (0x4000001E; the same value Sunshine/Apollo use); fall back to relative +7.
if dxgi_dev.SetGPUThreadPriority(0x4000_001E).is_err()
&& dxgi_dev.SetGPUThreadPriority(7).is_err()
{
tracing::warn!("SetGPUThreadPriority failed (run as admin/SYSTEM for GPU priority)");
}
}
if let Ok(dxgi1) = device.cast::<IDXGIDevice1>() {
let _ = dxgi1.SetMaximumFrameLatency(1);
}
Ok((device, context))
}
/// Resolve the configured GPU scheduling-priority class from `PUNKTFUNK_GPU_PRIORITY_CLASS`
/// (`off|normal|high|realtime`, default high). `None` = leave it at the OS default (the `off` opt-out).
/// D3DKMT_SCHEDULINGPRIORITYCLASS: IDLE 0, BELOW_NORMAL 1, NORMAL 2, ABOVE_NORMAL 3, HIGH 4, REALTIME 5.
fn configured_gpu_priority_class() -> Option<i32> {
match std::env::var("PUNKTFUNK_GPU_PRIORITY_CLASS")
.ok()
.as_deref()
{
Some("off") => None,
Some("normal") => Some(2),
Some("realtime") => Some(5),
_ => Some(4), // HIGH — safe on NVIDIA+HAGS (realtime can freeze NVENC)
}
}
/// Enable SE_INC_BASE_PRIORITY on the CURRENT process token (best-effort) — the kernel gates the
/// HIGH/REALTIME GPU scheduling-priority bump on it. Held by SYSTEM/Administrators; a UAC-FILTERED
/// token does NOT have it, which is why `elevate_process_gpu_priority` may silently no-op in a
/// restricted service context.
unsafe fn enable_inc_base_priority() {
use windows::core::PCWSTR;
use windows::Win32::Foundation::{CloseHandle, HANDLE, LUID};
use windows::Win32::Security::{
AdjustTokenPrivileges, LookupPrivilegeValueW, LUID_AND_ATTRIBUTES,
SE_INC_BASE_PRIORITY_NAME, SE_PRIVILEGE_ENABLED, TOKEN_ADJUST_PRIVILEGES, TOKEN_PRIVILEGES,
TOKEN_QUERY,
};
use windows::Win32::System::Threading::{GetCurrentProcess, OpenProcessToken};
let mut token = HANDLE::default();
if OpenProcessToken(
GetCurrentProcess(),
TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY,
&mut token,
)
.is_ok()
{
let mut luid = LUID::default();
if LookupPrivilegeValueW(PCWSTR::null(), SE_INC_BASE_PRIORITY_NAME, &mut luid).is_ok() {
let tp = TOKEN_PRIVILEGES {
PrivilegeCount: 1,
Privileges: [LUID_AND_ATTRIBUTES {
Luid: luid,
Attributes: SE_PRIVILEGE_ENABLED,
}],
};
if AdjustTokenPrivileges(
token,
false,
Some(&tp as *const TOKEN_PRIVILEGES),
0,
None,
None,
)
.is_err()
{
tracing::warn!("could not enable SE_INC_BASE_PRIORITY for GPU priority");
}
}
let _ = CloseHandle(token);
}
}
/// Call `gdi32!D3DKMTSetProcessSchedulingPriorityClass(process, prio)` (no stable windows-rs binding —
/// loaded by name). Returns the NTSTATUS (0 = success) or `None` if the export can't be resolved. The
/// CALLING process must hold SE_INC_BASE_PRIORITY ([`enable_inc_base_priority`]) for HIGH/REALTIME; the
/// kernel checks the caller's privilege whether the target is self or a child we created.
unsafe fn d3dkmt_set_scheduling_priority_class(
process: windows::Win32::Foundation::HANDLE,
prio: i32,
) -> Option<i32> {
use windows::core::s;
use windows::Win32::Foundation::HANDLE;
use windows::Win32::System::LibraryLoader::{GetProcAddress, LoadLibraryA};
let gdi32 = LoadLibraryA(s!("gdi32.dll")).ok()?;
let p = GetProcAddress(gdi32, s!("D3DKMTSetProcessSchedulingPriorityClass"))?;
type SetPrio = unsafe extern "system" fn(HANDLE, i32) -> i32;
let f: SetPrio = std::mem::transmute(p);
Some(f(process, prio))
}
/// GPU scheduling-priority hardening — the same approach as Sunshine/Apollo, independently
/// implemented via the documented D3DKMT APIs (no GPL source copied). On a
/// GPU-saturated game our capture+encode process is starved of GPU time slices — NVENC sits ~idle but
/// `lock_bitstream` waits ~20 ms for our context to be scheduled. Elevating the PROCESS GPU scheduling
/// priority class (the strong cross-process lever — far more effective than `SetGPUThreadPriority`
/// alone, which we measured as no help) lets our brief encode preempt the game. Uses HIGH, NOT
/// realtime: realtime on NVIDIA + HAGS can freeze/crash NVENC (Apollo downgrades it for exactly this).
/// Runs once per process; best-effort. `PUNKTFUNK_GPU_PRIORITY_CLASS = off|normal|high|realtime`
/// (default high). Best-effort: silently no-ops under a UAC-filtered token (the process will not
/// hold SE_INC_BASE_PRIORITY, so the D3DKMT call is a no-op).
fn elevate_process_gpu_priority() {
use std::sync::Once;
static ONCE: Once = Once::new();
// SAFETY: the closure calls two of this module's `unsafe fn`s — `enable_inc_base_priority`
// (adjusts the current-process token; it has no caller precondition and builds all its FFI args
// locally) and `d3dkmt_set_scheduling_priority_class` (loads gdi32 by name and calls the export).
// The latter requires `process` to be a valid process handle; `GetCurrentProcess()` returns the
// current-process pseudo-handle, which is always valid and needs no close. Runs once via
// `Once::call_once`; no raw pointers are dereferenced here.
ONCE.call_once(|| unsafe {
use windows::Win32::System::Threading::GetCurrentProcess;
let Some(prio) = configured_gpu_priority_class() else {
tracing::info!("GPU process scheduling priority class left at default (off)");
return;
};
enable_inc_base_priority();
match d3dkmt_set_scheduling_priority_class(GetCurrentProcess(), prio) {
Some(0) => tracing::info!(
priority_class = prio,
"GPU process scheduling priority class set (2=normal 4=high 5=realtime)"
),
Some(st) => tracing::warn!(
status = format!("0x{st:08X}"),
"D3DKMTSetProcessSchedulingPriorityClass failed (run as admin/SYSTEM for GPU priority)"
),
None => tracing::warn!("D3DKMTSetProcessSchedulingPriorityClass export not found"),
}
});
}
/// How many times DXGI has actually called our hooked `NtGdiDdDDIGetCachedHybridQueryValue`. If this
/// stays 0 while DDA churns with ACCESS_LOST, the hook is NOT on DXGI's GPU-preference path on this
@@ -269,7 +77,7 @@ unsafe extern "system" fn hybrid_query_hook(gpu_preference: *mut u32) -> i32 {
/// a cached preference of UNSPECIFIED makes DXGI skip the resolution, so the output is NOT reparented
/// and DDA stays stable on one adapter (this is what makes Apollo's DDA work on this hardware).
/// Installed once, before the first DXGI factory/enumeration; lasts the process lifetime (like Apollo).
pub(crate) fn install_gpu_pref_hook() {
pub fn install_gpu_pref_hook() {
use std::sync::Once;
static HOOK: Once = Once::new();
// SAFETY: this one-time hook install only touches a region it has just validated.
@@ -299,7 +107,7 @@ pub(crate) fn install_gpu_pref_hook() {
// 100% DuplicateOutput1 E_ACCESSDENIED is diagnosable instead of silent.
match SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) {
Ok(()) => tracing::info!("DPI awareness set: PER_MONITOR_AWARE_V2"),
Err(e) => tracing::warn!(error = %format!("{e:?}"),
Err(e) => tracing::warn!(error = ?e,
"SetProcessDpiAwarenessContext failed (already set?) — DuplicateOutput1 may E_ACCESSDENIED"),
}
// 0=UNAWARE 1=SYSTEM 2=PER_MONITOR(_V2). DuplicateOutput1 needs 2.
@@ -65,8 +65,8 @@ use windows::Win32::UI::WindowsAndMessaging::{GetCursorPos, SetCursorPos};
// `DRV_STATUS_*` codes and the channel-delivery struct — lives in `pf_driver_proto`; both sides
// `use` it, so a layout/code drift is a compile error (the proto has `const` size asserts).
use frame::{
SharedHeader, DRV_STATUS_BIND_FAIL, DRV_STATUS_NO_DEVICE1, DRV_STATUS_OPENED,
DRV_STATUS_TEX_FAIL, MAGIC, RING_LEN, VERSION,
unpack_opened_detail, SharedHeader, DRV_STATUS_BIND_FAIL, DRV_STATUS_NONE,
DRV_STATUS_NO_DEVICE1, DRV_STATUS_OPENED, DRV_STATUS_TEX_FAIL, MAGIC, RING_LEN, VERSION,
};
/// `DXGI_SHARED_RESOURCE_READ | _WRITE` for `CreateSharedHandle`/`OpenSharedResourceByName`. Local (not
@@ -193,10 +193,20 @@ impl Drop for KeyedMutexGuard<'_> {
}
}
/// Nudge DWM into composing THE TARGET virtual display. DWM presents a display only when something
/// DIRTIES it — an idle desktop never does, so a freshly-attached ring (session open, or a
/// mid-session ring recreate) can sit at E_PENDING with no first frame even though everything is
/// healthy. pf-vdisplay implements no hardware-cursor plane, so a cursor move is composited into
/// LAST-RESORT fallback: nudge DWM into composing THE TARGET virtual display. DWM presents a
/// display only when something DIRTIES it — an idle desktop never does, so a freshly-attached ring
/// (session open, or a mid-session ring recreate) can sit at E_PENDING with no first frame even
/// though everything is healthy.
///
/// The PRIMARY first-frame mechanism is the driver's `FrameStash` (frame_transport.rs): the driver
/// retains the last composed frame and republishes it into every freshly-attached ring, so with a
/// stash-capable driver the first frame lands milliseconds after the channel delivery and this kick
/// never fires. It remains for pre-stash drivers and for the empty-stash cold start (a monitor that
/// has NEVER composed — normally the activation compose covers that). Synthetic input is inherently
/// unreliable — blocked on the secure desktop, defeated by a fullscreen game's ClipCursor, and
/// user-visible in the sibling-display case — which is exactly why it was demoted to fallback.
///
/// pf-vdisplay implements no hardware-cursor plane, so a cursor move is composited into
/// the frame — a guaranteed real present onto the IDD swap-chain (empirically what
/// `punktfunk-probe --input-test` always relied on).
///
@@ -209,6 +219,14 @@ impl Drop for KeyedMutexGuard<'_> {
/// the cursor layer of the display it lands on, so the target composes at least one frame; the
/// round trip is sub-millisecond and throttled). Best-effort — injection can be unavailable on the
/// secure desktop, where a fresh compose just happened anyway.
///
/// **HID-first**: when the host has registered [`HID_COMPOSE_KICK`] (the resident pf-mouse virtual
/// HID pointer), the kick goes through it INSTEAD of the `SendInput` paths below. A report from a
/// HID device is real input to win32k — delivered regardless of this process's session or the
/// active desktop, it wakes a powered-off display subsystem (lid-closed laptop / display idle-off /
/// modern standby) and counts as user presence — every condition under which `SendInput` is
/// silently impotent (wrong session → wrong input queue; secure desktop → blocked; display off →
/// nothing composes at all). That set is exactly the lid-closed field-report state.
fn kick_dwm_compose(target_id: u32) {
// Process-GLOBAL throttle (Stage W3): with N parallel capturers each nudging on its own
// schedule, DWM needs only one dirty per composition window — and the nudge is synthetic INPUT
@@ -230,7 +248,21 @@ fn kick_dwm_compose(target_id: u32) {
let have_pos = unsafe { GetCursorPos(&mut pos) }.is_ok();
// SAFETY: `source_desktop_rect` only runs the CCD QueryDisplayConfig FFI over owned local
// buffers; the `Copy` target id crosses by value.
let rect = unsafe { crate::win_display::source_desktop_rect(target_id) };
let rect = unsafe { pf_win_display::win_display::source_desktop_rect(target_id) };
// HID-first (see the doc comment): the registered virtual-mouse kick works from any
// session/desktop and wakes an off display. Both geometries come from CCD (global database),
// NOT per-session GDI metrics, so the aim is right even from a non-console session. Fall
// through to SendInput only when the hook isn't registered / the mouse isn't up.
if let (Some(kick), Some(rect)) = (crate::HID_COMPOSE_KICK.get(), rect) {
// SAFETY: `desktop_bounds` only runs the CCD QueryDisplayConfig FFI over owned local
// buffers.
let bounds = unsafe { pf_win_display::win_display::desktop_bounds() };
if let Some(bounds) = bounds {
if kick(rect, bounds) {
return;
}
}
}
if let (true, Some((x, y, w, h))) = (have_pos, rect) {
let inside = pos.x >= x && pos.x < x + w.max(1) && pos.y >= y && pos.y < y + h.max(1);
if !inside {
@@ -285,7 +317,7 @@ fn kick_dwm_compose(target_id: u32) {
///
/// # Safety
/// `process` must be a live process handle carrying `PROCESS_QUERY_LIMITED_INFORMATION`.
pub(crate) unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &str) -> Result<()> {
pub unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &str) -> Result<()> {
let mut buf = [0u16; 512];
let mut len = buf.len() as u32;
// SAFETY: `process` carries QUERY_LIMITED per the contract; `buf`/`len` are a valid out-buffer and
@@ -313,384 +345,15 @@ pub(crate) unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &s
Ok(())
}
/// The sealed channel's handle-duplication broker (`design/idd-push-security.md`): the frame objects
/// are unnamed, so the ONLY way the driver can reach them is handles this broker duplicates into its
/// WUDFHost process and delivers — as bare handle VALUES — over the SYSTEM-only control device
/// (`IOCTL_SET_FRAME_CHANNEL`). Ownership is a strict hand-off: on IOCTL success the DRIVER owns the
/// duplicates (it closes them); on any failure [`Self::send`] reaps every duplicate it already made
/// (`DUPLICATE_CLOSE_SOURCE`), so a half-delivered channel never leaks handles in WUDFHost.
struct ChannelBroker {
/// `PROCESS_DUP_HANDLE | SYNCHRONIZE` handle to the driver's WUDFHost (pid from the ADD reply;
/// `ProcessSharingDisabled` makes that process exclusively pf-vdisplay's). `SYNCHRONIZE` lets the
/// handle double as the driver-death probe ([`Self::driver_alive`]).
process: OwnedHandle,
/// The WUDFHost pid `process` refers to (diagnostics for the driver-death bail).
wudf_pid: u32,
/// The pf-vdisplay control device — owned by the `VirtualDisplayManager`, never closed for the
/// process lifetime (a dead one is retired, kept alive), so holding the bare `HANDLE` is sound.
control: HANDLE,
}
impl ChannelBroker {
/// Open the duplication target. Fails when the driver predates the sealed channel (`wudf_pid == 0`
/// can't survive the v2 version handshake, but guard anyway) or the WUDFHost is gone (device
/// restart mid-open) — either way the caller fails the capture open cleanly.
///
/// `wudf_pid` comes from the driver's ADD reply, so before we duplicate whole-desktop frame handles
/// INTO it we VERIFY it is a genuine system WUDFHost ([`verify_is_wudfhost`]). Without that check a
/// spoofed devnode (same interface GUID) could name an arbitrary process and receive the frames; a
/// fully-compromised REAL pf_vdisplay driver is already a frame endpoint, so this specifically closes
/// the reachable-without-owning-the-driver case (`design/idd-push-security.md` §hardening).
fn open(wudf_pid: u32) -> Result<Self> {
if wudf_pid == 0 {
bail!("driver reported no WUDFHost pid for the frame channel");
}
let control = crate::vdisplay::manager::control_device_handle().context(
"pf-vdisplay control device not open (monitor not created via the manager?)",
)?;
// SAFETY: plain FFI; `wudf_pid` is a copy. The handle (checked by `?`) is owned solely here and
// moved into the `OwnedHandle` (single owner, closes on drop); `verify_is_wudfhost` borrows it
// for the duration of the synchronous check and forms no lasting alias.
let process = unsafe {
let h = OpenProcess(
PROCESS_DUP_HANDLE | PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_SYNCHRONIZE,
false,
wudf_pid,
)
.context("OpenProcess(PROCESS_DUP_HANDLE) on the driver's WUDFHost")?;
let process = OwnedHandle::from_raw_handle(h.0 as _);
verify_is_wudfhost(HANDLE(process.as_raw_handle()), wudf_pid, "frame-channel")?;
process
};
Ok(Self {
process,
wudf_pid,
control,
})
}
/// Whether the driver's WUDFHost is still alive. The pinned process handle doubles as the
/// liveness probe (`SYNCHRONIZE` requested at open): signaled ⇔ the process exited. This is the
/// definitive "driver died mid-session" signal — at the ring, a dead driver and an idle desktop
/// are indistinguishable (both simply stop publishing).
fn driver_alive(&self) -> bool {
// SAFETY: `process` is the live `OwnedHandle` this broker owns (borrowed for this synchronous
// call); a 0 ms wait only reads the handle's signaled state.
unsafe { WaitForSingleObject(HANDLE(self.process.as_raw_handle()), 0) != WAIT_OBJECT_0 }
}
/// Duplicate `h` into the WUDFHost handle table, returning the handle VALUE valid there (and only
/// there — the value is meaningless in any other process). `access = Some(rights)` grants the
/// driver's handle exactly those rights (least privilege — see [`SECTION_MAP_RW`]);
/// `access = None` copies the source handle's access (`DUPLICATE_SAME_ACCESS`), used only where the
/// source is already scoped (the DXGI shared-texture handles, minted by `CreateSharedHandle` with
/// just `DXGI_SHARED_RESOURCE_READ|WRITE`).
///
/// # Safety
/// `h` must be a live handle of the current process.
unsafe fn dup_into(&self, h: HANDLE, access: Option<u32>) -> Result<u64> {
let mut out = HANDLE::default();
let (desired, options) = match access {
Some(rights) => (rights, DUPLICATE_HANDLE_OPTIONS(0)),
None => (0, DUPLICATE_SAME_ACCESS),
};
// SAFETY: `h` is live per the contract; `self.process` is the live PROCESS_DUP_HANDLE target;
// `&mut out` is a valid out-param. Either an explicit least-privilege access mask (options == 0)
// or `DUPLICATE_SAME_ACCESS` (desired ignored) — never both.
unsafe {
DuplicateHandle(
GetCurrentProcess(),
h,
HANDLE(self.process.as_raw_handle()),
&mut out,
desired,
false,
options,
)
}
.context("DuplicateHandle into the driver's WUDFHost")?;
Ok(out.0 as usize as u64)
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) {
if value == 0 {
return;
}
// SAFETY: `self.process` is the live duplication target and `value` is a handle value THIS
// broker just created in that process's table (callers only pass back `dup_into` results the
// driver never received); closing it there cannot touch any other process's handles.
unsafe {
let _ = DuplicateHandle(
HANDLE(self.process.as_raw_handle()),
HANDLE(value as usize as *mut core::ffi::c_void),
HANDLE::default(),
std::ptr::null_mut(),
0,
false,
DUPLICATE_CLOSE_SOURCE,
);
}
}
/// Duplicate the whole ring (header + event + every slot texture) into WUDFHost and deliver the
/// values via `IOCTL_SET_FRAME_CHANNEL`. All-or-nothing: on any failure every duplicate already
/// made is reaped remotely and an error returns (the caller fails the open / logs the recreate).
/// The ownership contract with the driver is adopt-on-success only — it closes the handles iff the
/// IOCTL succeeded, we reap them iff it didn't, so no value is ever closed twice.
///
/// # Safety
/// `header` and `event` must be live handles of the current process (the capturer's own section +
/// event, borrowed for this synchronous call).
unsafe fn send(
&self,
target_id: u32,
generation: u32,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
debug_assert!(slots.len() <= control::RING_LEN_USIZE);
let mut req = control::SetFrameChannelRequest {
target_id,
generation,
ring_len: slots.len() as u32,
_pad: 0,
header_handle: 0,
event_handle: 0,
texture_handles: [0; control::RING_LEN_USIZE],
};
// SAFETY: `header`/`event` are live per this fn's contract; each slot's `shared` is the live
// `OwnedHandle` the slot keeps for exactly this purpose.
let result = unsafe { self.duplicate_and_deliver(&mut req, header, event, slots) };
if result.is_err() {
// The driver never adopted the delivery — reap every remote duplicate so nothing lingers.
self.close_remote(req.header_handle);
self.close_remote(req.event_handle);
for v in req.texture_handles {
self.close_remote(v);
}
}
result
}
/// The fallible middle of [`Self::send`]: fill `req` with fresh duplicates, then issue the IOCTL.
/// Split out so `send` can reap whatever landed in `req` when any step errors.
///
/// # Safety
/// As [`Self::send`].
unsafe fn duplicate_and_deliver(
&self,
req: &mut control::SetFrameChannelRequest,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
// SAFETY: forwarded from the caller's contract — `header`/`event`/each `slot.shared` are live
// handles of this process, and `self.control` is the manager's control handle, never closed for
// the process lifetime (`send_frame_channel`'s precondition).
unsafe {
// Least privilege per handle: the header maps read/write, the event is only signalled, and
// the textures keep their already-scoped `CreateSharedHandle` access (see `dup_into`).
req.header_handle = self.dup_into(header, Some(SECTION_MAP_RW))?;
req.event_handle = self.dup_into(event, Some(EVENT_MODIFY_STATE))?;
for (k, s) in slots.iter().enumerate() {
req.texture_handles[k] = self.dup_into(HANDLE(s.shared.as_raw_handle()), None)?;
}
crate::vdisplay::pf_vdisplay::send_frame_channel(self.control, req)
}
}
}
/// Creates + owns the shared ring; yields the driver's frames as [`FramePayload::D3d11`].
/// The display descriptor the capture loop follows: live HDR state + active resolution of the
/// virtual target.
#[derive(Clone, Copy, PartialEq, Eq)]
struct DisplayDescriptor {
hdr: bool,
width: u32,
height: u32,
}
/// Off-thread poller for [`DisplayDescriptor`]. The CCD queries behind it (`QueryDisplayConfig`,
/// twice per sample) serialize on the session-global display-configuration lock, which display-
/// topology events and third-party display-poller software (the SteelSeries-GG class) can hold
/// for tens-to-hundreds of milliseconds at a time. Polled inline — the old design — that stall
/// landed ON the capture/encode thread: a periodic frame hitch on an otherwise healthy host, and
/// invisible in any log. Now a dedicated thread samples every [`Self::INTERVAL`] and publishes a
/// snapshot; the capture thread's per-frame cost is one uncontended mutex read, and a slow CCD
/// sample is *measured and logged* instead of silently stalling the stream.
///
/// Failure policy is last-known-good, per field: a transient CCD failure — including the target
/// briefly missing from the active-path list during a topology re-probe — keeps the previous
/// value instead of reading as `hdr = false` (the old behavior, which on an HDR session turned
/// every blip into TWO ring recreates: false, then true again a poll later). `seq` bumps only
/// when at least one query succeeded, so the consumer's debounce counts real observations, never
/// failures.
struct DescriptorPoller {
/// Latest merged sample + its sequence number; the poller holds the lock only to copy it.
snap: Arc<Mutex<(DisplayDescriptor, u64)>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl DescriptorPoller {
/// Poll cadence — the old inline throttle. With the consumer's two-strikes debounce on top, a
/// real "Use HDR" flip or mode-set is acted on within ~2 samples (≈ ½ s).
const INTERVAL: Duration = Duration::from_millis(250);
/// A sample slower than this means something is sitting on the display-config lock (topology
/// churn / display-poller software) — the disturbance class behind periodic virtual-display
/// stream hitches. Logged (rate-limited) so an affected host self-diagnoses.
const SLOW: Duration = Duration::from_millis(50);
fn spawn(target_id: u32, initial: DisplayDescriptor) -> Self {
let snap = Arc::new(Mutex::new((initial, 0u64)));
let stop = Arc::new(AtomicBool::new(false));
let (snap_t, stop_t) = (snap.clone(), stop.clone());
let thread = std::thread::Builder::new()
.name("pf-idd-desc-poll".into())
.spawn(move || {
let mut last = initial;
let mut seq = 0u64;
let mut last_slow_log: Option<Instant> = None;
while !stop_t.load(Ordering::Relaxed) {
let t = Instant::now();
// SAFETY: both are read-only CCD queries taking only a copy of the plain `u32`
// target id (see their own SAFETY docs); nothing is borrowed across the calls.
let (hdr, res) = unsafe {
(
crate::win_display::advanced_color_enabled(target_id),
crate::win_display::active_resolution(target_id),
)
};
let took = t.elapsed();
if took >= Self::SLOW
&& last_slow_log.is_none_or(|t| t.elapsed() >= Duration::from_secs(10))
{
last_slow_log = Some(Instant::now());
tracing::warn!(
took_ms = took.as_millis() as u64,
target_id,
"slow display-descriptor poll — something is holding the Windows \
display-config lock (topology churn / display-poller software); on \
a host with periodic stream hitches, correlate this cadence"
);
}
if hdr.is_some() || res.is_some() {
if let Some(hdr) = hdr {
last.hdr = hdr;
}
if let Some((width, height)) = res {
last.width = width;
last.height = height;
}
seq += 1;
*snap_t.lock().unwrap() = (last, seq);
}
// Park (not sleep) so `drop` wakes the thread immediately via `unpark`.
std::thread::park_timeout(Self::INTERVAL);
}
})
.map_err(|e| {
// Degraded, not fatal: the session streams, it just never follows a mid-session
// HDR flip / mode-set (seq stays 0 → the consumer sees no changes).
tracing::error!(error = %e, "IDD push: descriptor-poller thread failed to spawn");
})
.ok();
Self { snap, stop, thread }
}
/// The latest sample (lock held only for the copy — the poller writes at 4 Hz).
fn snapshot(&self) -> (DisplayDescriptor, u64) {
*self.snap.lock().unwrap()
}
}
impl Drop for DescriptorPoller {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(t) = self.thread.take() {
t.thread().unpark();
let _ = t.join();
}
}
}
/// A detected capture stall: a multi-hundred-ms hole in DWM's frame delivery that opened while the
/// desktop was actively composing right beforehand (see [`StallWatch`]).
struct Stall {
/// How long the hole lasted (last fresh frame → the frame that ended it).
gap: Duration,
/// `Some(mean period)` when this stall completes a metronomic cycle (see
/// [`crate::metronome::Metronome`]).
metronomic: Option<Duration>,
}
/// Capture-stall watch — the "sole virtual display" stutter diagnostic (field reports: Exclusive
/// topology = periodic double-jolt, Extend = smooth, i.e. the disturbance lives in the display/present
/// path BELOW capture and only while no physical output is active).
///
/// On a damage-driven capture an idle desktop legitimately goes quiet (no damage → no frames), so a
/// gap only counts as a stall when the [`Self::RECENT`] frames before it all arrived within
/// [`Self::ACTIVE_SPAN`] — sustained ≥ ~20 fps flow (a game or video), not a blinking caret or a
/// mouse twitch. Each stall feeds a [`crate::metronome::Metronome`], so periodic stalls self-diagnose
/// in the log WITHOUT needing any client keyframe request — discriminating "DWM stopped composing"
/// from encode/network causes that the recovery-cadence detector covers. Pure logic — unit-tested
/// below; the caller does the logging.
struct StallWatch {
/// The last [`Self::RECENT`] fresh-frame instants (pre-gap history for the activity gate).
recent: std::collections::VecDeque<Instant>,
cadence: crate::metronome::Metronome,
}
impl StallWatch {
/// Frames of pre-gap history that must be tight for flow to count as active. Stalls are thus
/// naturally spaced ≥ RECENT frame times apart — no extra log rate limit needed.
const RECENT: usize = 8;
/// The RECENT pre-gap frames must all fit in this span (8 frames in 400 ms ≈ ≥ 20 fps flow —
/// loose enough for a 30 fps-capped game, tight enough to reject idle-desktop damage).
const ACTIVE_SPAN: Duration = Duration::from_millis(400);
/// The smallest hole that counts as a stall (~9 missed frames at 60 Hz) — well below the
/// reported 300700 ms freezes, above encode/present jitter.
const STALL_MIN: Duration = Duration::from_millis(150);
fn new() -> Self {
Self {
recent: std::collections::VecDeque::with_capacity(Self::RECENT + 1),
cadence: crate::metronome::Metronome::new(),
}
}
/// Forget the flow history (a ring recreate's gap is self-inflicted, not a DWM stall — without
/// the reset the first post-recreate frame would read as one).
fn reset(&mut self) {
self.recent.clear();
}
/// Record a fresh driver frame at `now`; `Some` exactly when it ended a stall.
fn note_fresh(&mut self, now: Instant) -> Option<Stall> {
let was_active = self.recent.len() == Self::RECENT
&& self
.recent
.back()
.zip(self.recent.front())
.is_some_and(|(b, f)| b.duration_since(*f) <= Self::ACTIVE_SPAN);
let gap = self.recent.back().map(|last| now.duration_since(*last));
self.recent.push_back(now);
if self.recent.len() > Self::RECENT {
self.recent.pop_front();
}
let gap = gap?;
if !was_active || gap < Self::STALL_MIN {
return None;
}
Some(Stall {
gap,
metronomic: self.cadence.note(now),
})
}
}
#[path = "idd_push/channel.rs"]
mod channel;
#[path = "idd_push/descriptor.rs"]
mod descriptor;
#[path = "idd_push/stall.rs"]
mod stall;
use channel::ChannelBroker;
use descriptor::{DescriptorPoller, DisplayDescriptor};
use stall::StallWatch;
pub struct IddPushCapturer {
device: ID3D11Device,
@@ -753,6 +416,13 @@ pub struct IddPushCapturer {
/// during active flow and warns when they turn metronomic — the sole-virtual-display
/// periodic-stutter diagnostic.
stall_watch: StallWatch,
/// Stall↔OS-event correlation counters for the metronomic warn: how many stalls this session,
/// and how many had a coinciding a `pf_win_display::display_events` event in their gap window — the
/// discriminator between "Windows re-enumerates a monitor each cycle" (devnode churn the
/// `pnp_disable_monitors` axis suppresses) and "the disturbance is below the OS" (GPU driver
/// servicing a standby sink / display-poller software).
stalls_seen: u32,
stalls_with_os_events: u32,
/// Host-owned ROTATING output ring NVENC encodes (one YUV texture per slot). Rotating it per frame
/// is the precondition for pipelining the encode loop: while NVENC encodes frame N's texture on the
/// ASIC, frame N+1's convert writes a DIFFERENT texture — the two overlap. Format = `out_format()`:
@@ -772,6 +442,13 @@ pub struct IddPushCapturer {
last_seq: u64,
last_present: Option<(ID3D11Texture2D, PixelFormat)>,
status_logged: bool,
/// Session-lifetime `PowerRequestDisplayRequired` (RAII, `powercfg /requests`-visible): keeps
/// the console out of display-off while this capturer lives — DWM composes nothing (for ANY
/// display) once the console's displays power down, so without this a lid-closed/idle box can
/// go dark mid-stream and the ring runs dry. Prevention only; waking an ALREADY-off display is
/// the HID compose kick's job ([`crate::HID_COMPOSE_KICK`]). `None` when the kernel refused
/// (best-effort, the pre-existing behavior).
_display_wake: Option<pf_frame::session_tuning::DisplayWakeRequest>,
_keepalive: Box<dyn Send>,
}
// SAFETY: `IddPushCapturer` is `!Send` only because of its `*mut SharedHeader` raw pointer (and the
@@ -885,8 +562,12 @@ impl IddPushCapturer {
client_10bit: bool,
want_444: bool,
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
match Self::open_inner(target, preferred, client_10bit, want_444) {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
pf_win_display::display_events::spawn_once();
match Self::open_inner(target, preferred, client_10bit, want_444, sender) {
Ok(mut me) => {
me._keepalive = keepalive;
Ok(me)
@@ -900,6 +581,7 @@ impl IddPushCapturer {
preferred: Option<(u32, u32, u32)>,
client_10bit: bool,
want_444: bool,
sender: crate::FrameChannelSender,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -910,11 +592,18 @@ impl IddPushCapturer {
// the driver HAVE drifted — identical twin GPUs whose max-VRAM tie moved between ADD and
// this open, or a stale kept monitor across an adapter re-init — the driver reports
// TEX_FAIL plus the adapter it actually renders on, and the rebind below reopens on that.
let luid = crate::win_adapter::resolve_render_adapter_luid().unwrap_or(LUID {
let luid = pf_gpu::resolve_render_adapter_luid().unwrap_or(LUID {
LowPart: (target.adapter_luid & 0xffff_ffff) as u32,
HighPart: (target.adapter_luid >> 32) as i32,
});
match Self::open_on(target.clone(), preferred, client_10bit, want_444, luid) {
match Self::open_on(
target.clone(),
preferred,
client_10bit,
want_444,
luid,
sender.clone(),
) {
Ok(me) => Ok(me),
Err(e) => {
// Self-heal a render-adapter mismatch ONCE: on TEX_FAIL the driver has reported the
@@ -925,7 +614,7 @@ impl IddPushCapturer {
let driver_luid = e
.downcast_ref::<AttachTexFail>()
.map(|tf| tf.driver_luid)
.filter(|d| *d != 0 && *d != crate::capture::dxgi::pack_luid(luid));
.filter(|d| *d != 0 && *d != crate::dxgi::pack_luid(luid));
let Some(packed) = driver_luid else {
return Err(e);
};
@@ -939,7 +628,7 @@ impl IddPushCapturer {
"IDD push: ring/driver render-adapter mismatch — rebinding the ring to the \
driver's reported adapter"
);
Self::open_on(target, preferred, client_10bit, want_444, drv)
Self::open_on(target, preferred, client_10bit, want_444, drv, sender)
.context("IDD-push rebind to the driver's reported render adapter")
}
}
@@ -951,6 +640,7 @@ impl IddPushCapturer {
client_10bit: bool,
want_444: bool,
luid: LUID,
sender: crate::FrameChannelSender,
) -> Result<Self> {
let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -961,8 +651,8 @@ impl IddPushCapturer {
// SAFETY: `active_resolution` is an `unsafe fn` (Win32 CCD `QueryDisplayConfig`) that takes only a
// copy of the plain `u32` CCD target id and returns owned `(w, h)` values; it forms no borrows from
// us and validates the id internally, returning `None` on any failure (handled by `unwrap_or`).
let (w, h) =
unsafe { crate::win_display::active_resolution(target.target_id) }.unwrap_or((pw, ph));
let (w, h) = unsafe { pf_win_display::win_display::active_resolution(target.target_id) }
.unwrap_or((pw, ph));
if (w, h) != (pw, ph) {
tracing::info!(
target_id = target.target_id,
@@ -1005,16 +695,48 @@ impl IddPushCapturer {
// size the ring FP16 directly — don't race the advanced_color_enabled poll, which may not have
// settled within 250 ms and would size the ring SDR while the driver composes FP16 → a format
// mismatch → an immediate ring recreate + dropped first frames (audit §5.4).
let enabled_hdr =
client_10bit && crate::win_display::set_advanced_color(target.target_id, true);
let enabled_hdr = client_10bit
&& pf_win_display::win_display::set_advanced_color(target.target_id, true);
if enabled_hdr {
// Let the colorspace change settle before the driver composes + we size the ring.
std::thread::sleep(Duration::from_millis(250));
// Let the colorspace change settle before the driver composes + we size the ring:
// poll the CCD advanced-color state instead of a fixed sleep (latency plan P0.4),
// ceiling = the old 250 ms. A read that never flips within the ceiling proceeds
// exactly like the fixed sleep did — the ring is sized FP16 from `enabled_hdr`
// either way (the set succeeded; only the driver's compose flip may lag, which the
// stash/format-guard machinery absorbs).
let hdr_settle = Instant::now();
while hdr_settle.elapsed() < Duration::from_millis(250) {
if pf_win_display::win_display::advanced_color_enabled(target.target_id)
== Some(true)
{
break;
}
std::thread::sleep(Duration::from_millis(25));
}
tracing::debug!(
target_id = target.target_id,
settle_ms = hdr_settle.elapsed().as_millis() as u64,
"IDD push: advanced-color (HDR) enable settle"
);
}
// A failed open-time read defaults to SDR (unless the 10-bit path enabled HDR above) —
// there is no "last known" yet; the descriptor poller corrects a wrong guess mid-session.
let display_hdr = enabled_hdr
|| crate::win_display::advanced_color_enabled(target.target_id).unwrap_or(false);
|| pf_win_display::win_display::advanced_color_enabled(target.target_id)
.unwrap_or(false);
// Downgrade point D (design/hdr-10bit-default-and-av1.md item 2d): the session was
// NEGOTIATED 10-bit (the client was told HDR in the Welcome), but the virtual display
// could not enable advanced color — the ring sizes SDR and the encoder will emit 8-bit
// BT.709, so the client's label overstates the stream until the descriptor poller sees
// HDR come on. Loud, because every frame of this session is affected.
if client_10bit && !display_hdr {
tracing::error!(
target = target.target_id,
"IDD push: 10-bit HDR was negotiated but enabling advanced color on the \
virtual display FAILED encoding 8-bit SDR while the client was told HDR \
(check the display driver / Windows HDR support on this box)"
);
}
let ring_fmt = if display_hdr {
DXGI_FORMAT_R16G16B16A16_FLOAT
} else {
@@ -1093,7 +815,7 @@ impl IddPushCapturer {
// driver's WUDFHost and hand it the values over the control device. All-or-nothing (the
// broker reaps its remote duplicates on failure), and a failure fails the open — without
// the delivery the driver can never attach.
let broker = ChannelBroker::open(target.wudf_pid)?;
let broker = ChannelBroker::open(target.wudf_pid, sender)?;
broker
.send(
target.target_id,
@@ -1146,6 +868,8 @@ impl IddPushCapturer {
last_liveness: Instant::now(),
last_kick: Instant::now(),
stall_watch: StallWatch::new(),
stalls_seen: 0,
stalls_with_os_events: 0,
out_ring: Vec::new(),
out_idx: 0,
video_conv: None,
@@ -1153,6 +877,9 @@ impl IddPushCapturer {
last_seq: 0,
last_present: None,
status_logged: false,
// Held from BEFORE the first-frame gate (the display must not idle off while we
// wait for the first compose) until the capturer drops with the session.
_display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(),
// Placeholder; `open()` attaches the real keepalive on success, so a FAILED open can hand
// it back to the caller for the DDA fallback (audit §5.1).
_keepalive: Box::new(()),
@@ -1173,9 +900,12 @@ impl IddPushCapturer {
/// Requiring the first frame — not just the attach — catches the *reconnect-into-a-broken-state* case:
/// a fullscreen game can leave the virtual display in a format/size that the driver's `publish()` guard
/// rejects, so the driver ATTACHES but silently drops every frame; without this the host sails past
/// `open()` and only dies on `next_frame`'s 20 s deadline (the "reconnect = black + audio" symptom). At
/// session open the OS activates the virtual display → DWM composites it → a frame arrives within ~1 s,
/// so this does not false-fail a normal (even idle) open; no frame within the window = genuinely broken.
/// `open()` and only dies on `next_frame`'s 20 s deadline (the "reconnect = black + audio" symptom).
/// A stash-capable driver republishes its retained desktop frame the moment it attaches (the
/// first-frame guarantee — `FrameStash`, driver frame_transport.rs), so the normal case clears this
/// gate in milliseconds even on an idle desktop; failing that, at session open the OS activates the
/// virtual display → DWM composites it → a frame arrives within ~1 s, plus the compose-kick fallback
/// below — no frame within the window = genuinely broken.
fn wait_for_attach(&self) -> Result<()> {
// Symmetric host-side binding sanity (proto v3 §3.2): OUR header must still name OUR
// monitor. The stamp is ours and nothing legitimate rewrites it, so a mismatch means a
@@ -1192,11 +922,15 @@ impl IddPushCapturer {
);
}
let deadline = Instant::now() + Duration::from_secs(4);
// Compose-kick schedule: DWM only presents a display something DIRTIED, so on an idle
// desktop a perfectly healthy attach sees no first frame (E_PENDING forever) and this gate
// used to fail the session — the "idle desktop → no frames" gotcha (a real client escaped
// it only because its own input soon dirtied the desktop; a headless probe never did).
// Give the natural post-activate compose a moment, then nudge.
// First-frame expectation: a stash-capable driver republishes its retained desktop frame
// the moment it attaches (`FrameStash`, frame_transport.rs), so on a healthy pairing the
// gate below clears in milliseconds even on a perfectly idle desktop. The compose-kick
// schedule is the FALLBACK for pre-stash drivers / an empty stash (a display that has
// never composed): DWM only presents a display something DIRTIED, so on an idle desktop
// an attach would otherwise sit at E_PENDING forever and fail this gate — the
// "idle desktop → no frames" gotcha. Give the natural post-activate compose (and the
// stash republish) a moment, then nudge; log when we do, so field logs show whether the
// stash path is working.
let mut next_kick = Instant::now() + Duration::from_millis(600);
loop {
// SAFETY: `self.header` points into the live shared-header mapping this capturer owns (sized
@@ -1249,17 +983,80 @@ impl IddPushCapturer {
return Ok(());
}
if Instant::now() >= next_kick {
// Reaching a kick at all means the driver did NOT republish a retained frame
// (pre-stash driver, or a never-composed display) — worth a line in the field log.
tracing::debug!(
target_id = self.target_id,
driver_status = st,
"IDD push: no first frame after attach delivery — falling back to a synthetic \
compose kick (stash-capable drivers republish instantly; old driver?)"
);
kick_dwm_compose(self.target_id);
next_kick = Instant::now() + Duration::from_millis(800);
}
if Instant::now() > deadline {
bail!(
"IDD-push: driver_status={st} but no frame published within 4s (despite compose \
kicks) the virtual display is likely in a format/size the ring can't match \
(fullscreen game?); falling back"
"IDD-push: no frame published within 4s (despite compose kicks) — {}; \
falling back",
self.no_first_frame_diagnosis(st)
);
}
std::thread::sleep(Duration::from_millis(20));
// Event-driven wait (latency plan P0.6): the driver signals the frame-ready event on
// every publish, so wake on it instead of a blind sleep — the 20 ms timeout keeps the
// driver_status polls above live (status writes don't signal the event). Consuming a
// signal here is fine: `next_frame` re-checks the atomic `latest` token, never the
// event, for truth.
// SAFETY: `self.event` is this capturer's owned, live auto-reset event handle;
// `WaitForSingleObject` only reads the handle and the 20 ms timeout bounds the wait.
let _ = unsafe { WaitForSingleObject(HANDLE(self.event.as_raw_handle()), 20) };
}
}
/// Name a first-frame timeout from the driver's own evidence — `driver_status` plus the live
/// OPENED detail word (proto `pack_opened_detail`) — instead of guessing. The three no-frames
/// states look identical from the host side but have disjoint causes and fixes; the lid-closed
/// field report burned days for lack of exactly this line. Appends a console-session hint when
/// the host itself is in the wrong session (display writes + input kicks can't work from there).
fn no_first_frame_diagnosis(&self, st: u32) -> String {
let what = match st {
// The delivery was never consumed: no swap-chain worker ran for this monitor at all.
DRV_STATUS_NONE => "the driver never attached — the channel delivery was never \
consumed, so the OS ran no swap-chain worker for this monitor (display not \
composed at all: console display-off / modern standby, or the mode commit \
never reached the adapter)"
.to_string(),
DRV_STATUS_OPENED => {
// SAFETY: in-bounds, aligned u32 read of the live, owned shared-header mapping
// (same best-effort diagnostic access as the `driver_status` read in the caller);
// no reference into the shared region is formed.
let detail = unsafe { (*self.header).driver_status_detail };
match unpack_opened_detail(detail) {
Some((0, _)) => "driver attached with a live swap-chain, but DWM composed \
ZERO frames an undamaged or powered-off desktop, and the compose \
kicks didn't bite (synthetic input is blocked on the secure desktop)"
.to_string(),
Some((offered, mismatched)) => format!(
"driver attached and DWM composed {offered} frame(s), but none matched \
the ring {mismatched} dropped for a size/format mismatch (the \
display's actual mode differs from what the host sized the ring to: \
a mid-open mode-set, a fullscreen game, or a stale GDI view)"
),
// A pre-detail driver never stamps the live bit — say so rather than guess.
None => "driver attached but published nothing; this pf-vdisplay build \
predates attach diagnostics, so the cause can't be named update the \
driver for a precise line here"
.to_string(),
}
}
other => format!("driver_status={other} (unexpected at this point)"),
};
match pf_win_display::console_session_mismatch() {
Some((own, console)) => format!(
"{what} [host is in session {own} but the console is session {console} — display \
writes and input kicks cannot work from a non-console session; reconnect the \
console or run via the installed service]"
),
None => what,
}
}
@@ -1546,12 +1343,19 @@ impl IddPushCapturer {
}
// Same idle-desktop stall as the open-time attach gate: after a mid-session ring
// recreate (HDR flip / mode change) an idle desktop composes nothing, so the fresh ring
// never sees a frame and the 3 s recover-or-drop above kills a healthy session. Nudge
// DWM (rate-limited) once the natural post-recreate compose has had its chance.
// never sees a frame and the 3 s recover-or-drop above kills a healthy session. A
// stash-capable driver republishes its retained frame at the re-attach, so this kick
// is the legacy-driver fallback here too. Nudge DWM (rate-limited) once the natural
// post-recreate compose (and the stash republish) has had its chance.
if since.elapsed() > Duration::from_millis(600)
&& self.last_kick.elapsed() > Duration::from_millis(800)
{
self.last_kick = Instant::now();
tracing::debug!(
target_id = self.target_id,
"IDD push: no frame after ring recreate — falling back to a synthetic compose \
kick (stash-capable drivers republish at re-attach; old driver?)"
);
kick_dwm_compose(self.target_id);
}
}
@@ -1646,24 +1450,70 @@ impl IddPushCapturer {
// doesn't read as a DWM stall.
self.stall_watch.reset();
} else if let Some(stall) = self.stall_watch.note_fresh(now) {
// OS display events inside the gap (plus a lead-in margin: the event that CAUSED the
// hole lands just before DWM stops delivering) — the attribution that turns "DWM
// stopped composing" into "…because Windows re-enumerated SAMSUNG on HDMI".
let window = stall.gap + Duration::from_millis(300);
let events = now
.checked_sub(window)
.map(|from| pf_win_display::display_events::events_between(from, now))
.unwrap_or_default();
self.stalls_seen = self.stalls_seen.saturating_add(1);
if !events.is_empty() {
self.stalls_with_os_events = self.stalls_with_os_events.saturating_add(1);
}
// debug (not warn): a single hole also happens when content legitimately pauses;
// the reportable signal is the metronomic cycle below. Mounjay-class triage runs
// at debug level, and the web-console debug ring captures these.
tracing::debug!(
gap_ms = stall.gap.as_millis() as u64,
os_display_events = %pf_win_display::display_events::summarize(&events),
"IDD-push capture stall — the desktop was composing at speed, then DWM \
delivered no frame for the gap; the present path stalled below capture"
);
if let Some(period) = stall.metronomic {
tracing::warn!(
period_s = format!("{:.2}", period.as_secs_f64()),
"capture stalls are METRONOMIC — DWM stops composing the virtual display \
on a stable period, i.e. a periodic display-path disturbance BELOW \
capture (DWM present clock / GPU driver / display-poller software). \
Correlate with 'slow display-descriptor poll'; if that never fires, the \
disturbance is outside punktfunk try display topology=primary or \
extend (keep a physical output active), or a different refresh rate"
);
let suspects = pf_win_display::display_events::connected_inactive_externals();
let suspects = if suspects.is_empty() {
"none".to_string()
} else {
suspects.join(", ")
};
let correlated = format!("{}/{}", self.stalls_with_os_events, self.stalls_seen);
// Half-or-more of the stalls carrying a coinciding OS event = the reaction
// cascade is OS-visible; otherwise the disturbance never surfaces above the
// driver. Different classes, different cures — say which one this box has.
if self.stalls_with_os_events * 2 >= self.stalls_seen {
tracing::warn!(
period_s = format!("{:.2}", period.as_secs_f64()),
os_correlated = correlated,
connected_inactive = %suspects,
"capture stalls are METRONOMIC and coincide with Windows monitor \
hot-plug/re-enumeration events a connected display (or its \
cable/switch/AVR) re-probes the link on a timer and Windows re-reacts \
each time. Cures, best-first: that display's OSD 'auto input \
scan/detect' OFF (and on TVs: instant-on/quick-start + CEC off), \
unplug its cable at the GPU, an HPD-holding adapter/dummy plug, or \
keep it active while streaming; the pnp_disable_monitors policy axis \
suppresses the Windows-side reaction (see connected_inactive for the \
suspects)"
);
} else {
tracing::warn!(
period_s = format!("{:.2}", period.as_secs_f64()),
os_correlated = correlated,
connected_inactive = %suspects,
"capture stalls are METRONOMIC with NO coinciding OS display event — \
the disturbance is BELOW Windows: the GPU driver servicing a \
connected-but-asleep sink (standby HPD/DDC/link probing), \
display-poller software (the SteelSeries-GG/SignalRGB class \
correlate 'slow display-descriptor poll' lines), or the DWM present \
clock (try a different refresh rate). If connected_inactive lists a \
display, its standby probing is the prime suspect: unplug it at the \
GPU, disable its OSD auto input scan (TVs: instant-on/quick-start + \
CEC off), use an HPD-holding adapter/dummy, or keep it active while \
streaming"
);
}
}
}
self.last_fresh = now; // feeds the driver-death watch
@@ -1780,7 +1630,7 @@ impl Capturer for IddPushCapturer {
// PQ VUI; pair that with a mastering-display SEI so any decoder tone-maps from a real grade. The
// driver doesn't (yet) forward the OS's IDDCX_HDR10_METADATA, so use the generic HDR10 baseline
// (the same metadata the native HDR path sends on the 0xCE datagram).
self.display_hdr.then(crate::hdr::generic_hdr10)
self.display_hdr.then(pf_frame::hdr::generic_hdr10)
}
fn pipeline_depth(&self) -> usize {
@@ -1788,7 +1638,40 @@ impl Capturer for IddPushCapturer {
// NVENC encodes N on the ASIC. We hand a rotating `OUT_RING` of output textures, so this is safe.
// `PUNKTFUNK_IDD_DEPTH` overrides (1 disables pipelining; clamp to ≤ OUT_RING so a frame in flight
// always has its own texture).
crate::config::config().idd_depth.clamp(1, OUT_RING)
pf_host_config::config().idd_depth.clamp(1, OUT_RING)
}
fn capture_target_id(&self) -> Option<u32> {
Some(self.target_id)
}
fn resize_output(&mut self, width: u32, height: u32) -> bool {
// Host-initiated resize (latency plan P2.3): the session's resize handler has already
// committed the display's new mode (the manager's in-place mode set), so recreate the ring
// at the new size NOW — no DescriptorPoller two-strike debounce (that stays, unchanged,
// for EXTERNAL changes: HDR flips, game mode-sets). The driver re-attaches to the fresh
// ring and republishes; on an in-place mode set the OS's mode-set full redraw gives the
// stash/first frame within the recover window. Same recover-or-drop arming as the
// poller-driven recreate, so a ring that can't re-attach still fails the session cleanly
// instead of freezing.
if (width, height) == (self.width, self.height) {
return true; // already at the requested size (refresh-only change) — nothing to do
}
tracing::info!(
target_id = self.target_id,
from = format!("{}x{}", self.width, self.height),
to = format!("{width}x{height}"),
"IDD push: host-initiated resize — recreating the ring at the new mode"
);
self.recovering_since.get_or_insert_with(Instant::now);
if let Err(e) = self.recreate_ring(self.display_hdr, width, height) {
tracing::warn!(
error = %format!("{e:#}"),
"IDD push: host-initiated ring recreate failed — falling back to a full rebuild"
);
return false;
}
true
}
}
@@ -1820,6 +1703,7 @@ impl Drop for IddPushCapturer {
#[cfg(test)]
mod tests {
use super::stall::Stall;
use super::*;
/// Feed a [`StallWatch`] fresh frames at the given offsets (ms from a common origin) and
@@ -0,0 +1,198 @@
//! The sealed frame channel's handle-duplication broker (plan §W4, carved out of the IDD-push
//! capturer): duplicates the unnamed shared header / ring / event handles into the driver's WUDFHost
//! and delivers them as bare handle values over the SYSTEM-only control device.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
/// The sealed channel's handle-duplication broker (`design/idd-push-security.md`): the frame objects
/// are unnamed, so the ONLY way the driver can reach them is handles this broker duplicates into its
/// WUDFHost process and delivers — as bare handle VALUES — over the SYSTEM-only control device
/// (`IOCTL_SET_FRAME_CHANNEL`). Ownership is a strict hand-off: on IOCTL success the DRIVER owns the
/// duplicates (it closes them); on any failure [`Self::send`] reaps every duplicate it already made
/// (`DUPLICATE_CLOSE_SOURCE`), so a half-delivered channel never leaks handles in WUDFHost.
pub(super) struct ChannelBroker {
/// `PROCESS_DUP_HANDLE | SYNCHRONIZE` handle to the driver's WUDFHost (pid from the ADD reply;
/// `ProcessSharingDisabled` makes that process exclusively pf-vdisplay's). `SYNCHRONIZE` lets the
/// handle double as the driver-death probe ([`Self::driver_alive`]).
process: OwnedHandle,
/// The WUDFHost pid `process` refers to (diagnostics for the driver-death bail).
pub(super) wudf_pid: u32,
/// Delivers a filled `SetFrameChannelRequest` to the pf-vdisplay driver
/// (`IOCTL_SET_FRAME_CHANNEL`). The host facade builds this from the vdisplay control device +
/// `send_frame_channel` IOCTL wrapper, so this crate delivers the channel without reaching into
/// the orchestrator's `vdisplay` module (plan §W6). Called once per generation, never per-frame.
sender: crate::FrameChannelSender,
}
impl ChannelBroker {
/// Open the duplication target. Fails when the driver predates the sealed channel (`wudf_pid == 0`
/// can't survive the v2 version handshake, but guard anyway) or the WUDFHost is gone (device
/// restart mid-open) — either way the caller fails the capture open cleanly.
///
/// `wudf_pid` comes from the driver's ADD reply, so before we duplicate whole-desktop frame handles
/// INTO it we VERIFY it is a genuine system WUDFHost ([`verify_is_wudfhost`]). Without that check a
/// spoofed devnode (same interface GUID) could name an arbitrary process and receive the frames; a
/// fully-compromised REAL pf_vdisplay driver is already a frame endpoint, so this specifically closes
/// the reachable-without-owning-the-driver case (`design/idd-push-security.md` §hardening).
pub(super) fn open(wudf_pid: u32, sender: crate::FrameChannelSender) -> Result<Self> {
if wudf_pid == 0 {
bail!("driver reported no WUDFHost pid for the frame channel");
}
// SAFETY: plain FFI; `wudf_pid` is a copy. The handle (checked by `?`) is owned solely here and
// moved into the `OwnedHandle` (single owner, closes on drop); `verify_is_wudfhost` borrows it
// for the duration of the synchronous check and forms no lasting alias.
let process = unsafe {
let h = OpenProcess(
PROCESS_DUP_HANDLE | PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_SYNCHRONIZE,
false,
wudf_pid,
)
.context("OpenProcess(PROCESS_DUP_HANDLE) on the driver's WUDFHost")?;
let process = OwnedHandle::from_raw_handle(h.0 as _);
verify_is_wudfhost(HANDLE(process.as_raw_handle()), wudf_pid, "frame-channel")?;
process
};
Ok(Self {
process,
wudf_pid,
sender,
})
}
/// Whether the driver's WUDFHost is still alive. The pinned process handle doubles as the
/// liveness probe (`SYNCHRONIZE` requested at open): signaled ⇔ the process exited. This is the
/// definitive "driver died mid-session" signal — at the ring, a dead driver and an idle desktop
/// are indistinguishable (both simply stop publishing).
pub(super) fn driver_alive(&self) -> bool {
// SAFETY: `process` is the live `OwnedHandle` this broker owns (borrowed for this synchronous
// call); a 0 ms wait only reads the handle's signaled state.
unsafe { WaitForSingleObject(HANDLE(self.process.as_raw_handle()), 0) != WAIT_OBJECT_0 }
}
/// Duplicate `h` into the WUDFHost handle table, returning the handle VALUE valid there (and only
/// there — the value is meaningless in any other process). `access = Some(rights)` grants the
/// driver's handle exactly those rights (least privilege — see [`SECTION_MAP_RW`]);
/// `access = None` copies the source handle's access (`DUPLICATE_SAME_ACCESS`), used only where the
/// source is already scoped (the DXGI shared-texture handles, minted by `CreateSharedHandle` with
/// just `DXGI_SHARED_RESOURCE_READ|WRITE`).
///
/// # Safety
/// `h` must be a live handle of the current process.
unsafe fn dup_into(&self, h: HANDLE, access: Option<u32>) -> Result<u64> {
let mut out = HANDLE::default();
let (desired, options) = match access {
Some(rights) => (rights, DUPLICATE_HANDLE_OPTIONS(0)),
None => (0, DUPLICATE_SAME_ACCESS),
};
// SAFETY: `h` is live per the contract; `self.process` is the live PROCESS_DUP_HANDLE target;
// `&mut out` is a valid out-param. Either an explicit least-privilege access mask (options == 0)
// or `DUPLICATE_SAME_ACCESS` (desired ignored) — never both.
unsafe {
DuplicateHandle(
GetCurrentProcess(),
h,
HANDLE(self.process.as_raw_handle()),
&mut out,
desired,
false,
options,
)
}
.context("DuplicateHandle into the driver's WUDFHost")?;
Ok(out.0 as usize as u64)
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) {
if value == 0 {
return;
}
// SAFETY: `self.process` is the live duplication target and `value` is a handle value THIS
// broker just created in that process's table (callers only pass back `dup_into` results the
// driver never received); closing it there cannot touch any other process's handles.
unsafe {
let _ = DuplicateHandle(
HANDLE(self.process.as_raw_handle()),
HANDLE(value as usize as *mut core::ffi::c_void),
HANDLE::default(),
std::ptr::null_mut(),
0,
false,
DUPLICATE_CLOSE_SOURCE,
);
}
}
/// Duplicate the whole ring (header + event + every slot texture) into WUDFHost and deliver the
/// values via `IOCTL_SET_FRAME_CHANNEL`. All-or-nothing: on any failure every duplicate already
/// made is reaped remotely and an error returns (the caller fails the open / logs the recreate).
/// The ownership contract with the driver is adopt-on-success only — it closes the handles iff the
/// IOCTL succeeded, we reap them iff it didn't, so no value is ever closed twice.
///
/// # Safety
/// `header` and `event` must be live handles of the current process (the capturer's own section +
/// event, borrowed for this synchronous call).
pub(super) unsafe fn send(
&self,
target_id: u32,
generation: u32,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
debug_assert!(slots.len() <= control::RING_LEN_USIZE);
let mut req = control::SetFrameChannelRequest {
target_id,
generation,
ring_len: slots.len() as u32,
_pad: 0,
header_handle: 0,
event_handle: 0,
texture_handles: [0; control::RING_LEN_USIZE],
};
// SAFETY: `header`/`event` are live per this fn's contract; each slot's `shared` is the live
// `OwnedHandle` the slot keeps for exactly this purpose.
let result = unsafe { self.duplicate_and_deliver(&mut req, header, event, slots) };
if result.is_err() {
// The driver never adopted the delivery — reap every remote duplicate so nothing lingers.
self.close_remote(req.header_handle);
self.close_remote(req.event_handle);
for v in req.texture_handles {
self.close_remote(v);
}
}
result
}
/// The fallible middle of [`Self::send`]: fill `req` with fresh duplicates, then issue the IOCTL.
/// Split out so `send` can reap whatever landed in `req` when any step errors.
///
/// # Safety
/// As [`Self::send`].
unsafe fn duplicate_and_deliver(
&self,
req: &mut control::SetFrameChannelRequest,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
// SAFETY: forwarded from the caller's contract — `header`/`event`/each `slot.shared` are live
// handles of this process. The `sender` closure encapsulates the manager's control handle +
// the `send_frame_channel` IOCTL (its precondition — a live control handle — is upheld by the
// host facade that built it).
unsafe {
// Least privilege per handle: the header maps read/write, the event is only signalled, and
// the textures keep their already-scoped `CreateSharedHandle` access (see `dup_into`).
req.header_handle = self.dup_into(header, Some(SECTION_MAP_RW))?;
req.event_handle = self.dup_into(event, Some(EVENT_MODIFY_STATE))?;
for (k, s) in slots.iter().enumerate() {
req.texture_handles[k] = self.dup_into(HANDLE(s.shared.as_raw_handle()), None)?;
}
(self.sender)(req)
}
}
}
@@ -0,0 +1,121 @@
//! Off-thread display-descriptor polling (plan §W4, carved out of the IDD-push capturer): the
//! live HDR state + active resolution of the virtual target, sampled off the capture loop via CCD.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
/// Creates + owns the shared ring; yields the driver's frames as [`FramePayload::D3d11`].
/// The display descriptor the capture loop follows: live HDR state + active resolution of the
/// virtual target.
#[derive(Clone, Copy, PartialEq, Eq)]
pub(super) struct DisplayDescriptor {
pub(super) hdr: bool,
pub(super) width: u32,
pub(super) height: u32,
}
/// Off-thread poller for [`DisplayDescriptor`]. The CCD queries behind it (`QueryDisplayConfig`,
/// twice per sample) serialize on the session-global display-configuration lock, which display-
/// topology events and third-party display-poller software (the SteelSeries-GG class) can hold
/// for tens-to-hundreds of milliseconds at a time. Polled inline — the old design — that stall
/// landed ON the capture/encode thread: a periodic frame hitch on an otherwise healthy host, and
/// invisible in any log. Now a dedicated thread samples every [`Self::INTERVAL`] and publishes a
/// snapshot; the capture thread's per-frame cost is one uncontended mutex read, and a slow CCD
/// sample is *measured and logged* instead of silently stalling the stream.
///
/// Failure policy is last-known-good, per field: a transient CCD failure — including the target
/// briefly missing from the active-path list during a topology re-probe — keeps the previous
/// value instead of reading as `hdr = false` (the old behavior, which on an HDR session turned
/// every blip into TWO ring recreates: false, then true again a poll later). `seq` bumps only
/// when at least one query succeeded, so the consumer's debounce counts real observations, never
/// failures.
pub(super) struct DescriptorPoller {
/// Latest merged sample + its sequence number; the poller holds the lock only to copy it.
snap: Arc<Mutex<(DisplayDescriptor, u64)>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl DescriptorPoller {
/// Poll cadence — the old inline throttle. With the consumer's two-strikes debounce on top, a
/// real "Use HDR" flip or mode-set is acted on within ~2 samples (≈ ½ s).
const INTERVAL: Duration = Duration::from_millis(250);
/// A sample slower than this means something is sitting on the display-config lock (topology
/// churn / display-poller software) — the disturbance class behind periodic virtual-display
/// stream hitches. Logged (rate-limited) so an affected host self-diagnoses.
const SLOW: Duration = Duration::from_millis(50);
pub(super) fn spawn(target_id: u32, initial: DisplayDescriptor) -> Self {
let snap = Arc::new(Mutex::new((initial, 0u64)));
let stop = Arc::new(AtomicBool::new(false));
let (snap_t, stop_t) = (snap.clone(), stop.clone());
let thread = std::thread::Builder::new()
.name("pf-idd-desc-poll".into())
.spawn(move || {
let mut last = initial;
let mut seq = 0u64;
let mut last_slow_log: Option<Instant> = None;
while !stop_t.load(Ordering::Relaxed) {
let t = Instant::now();
// SAFETY: both are read-only CCD queries taking only a copy of the plain `u32`
// target id (see their own SAFETY docs); nothing is borrowed across the calls.
let (hdr, res) = unsafe {
(
pf_win_display::win_display::advanced_color_enabled(target_id),
pf_win_display::win_display::active_resolution(target_id),
)
};
let took = t.elapsed();
if took >= Self::SLOW
&& last_slow_log.is_none_or(|t| t.elapsed() >= Duration::from_secs(10))
{
last_slow_log = Some(Instant::now());
tracing::warn!(
took_ms = took.as_millis() as u64,
target_id,
"slow display-descriptor poll — something is holding the Windows \
display-config lock (topology churn / display-poller software); on \
a host with periodic stream hitches, correlate this cadence"
);
}
if hdr.is_some() || res.is_some() {
if let Some(hdr) = hdr {
last.hdr = hdr;
}
if let Some((width, height)) = res {
last.width = width;
last.height = height;
}
seq += 1;
*snap_t.lock().unwrap() = (last, seq);
}
// Park (not sleep) so `drop` wakes the thread immediately via `unpark`.
std::thread::park_timeout(Self::INTERVAL);
}
})
.map_err(|e| {
// Degraded, not fatal: the session streams, it just never follows a mid-session
// HDR flip / mode-set (seq stays 0 → the consumer sees no changes).
tracing::warn!(error = %e, "IDD push: descriptor-poller thread failed to spawn — mid-session HDR/mode changes won't be followed");
})
.ok();
Self { snap, stop, thread }
}
/// The latest sample (lock held only for the copy — the poller writes at 4 Hz).
pub(super) fn snapshot(&self) -> (DisplayDescriptor, u64) {
*self.snap.lock().unwrap()
}
}
impl Drop for DescriptorPoller {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(t) = self.thread.take() {
t.thread().unpark();
let _ = t.join();
}
}
}
@@ -0,0 +1,82 @@
//! Capture-stall detection (plan §W4, carved out of the IDD-push capturer): flags multi-hundred-ms
//! holes in DWM frame delivery that open while the desktop was actively composing.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
/// A detected capture stall: a multi-hundred-ms hole in DWM's frame delivery that opened while the
/// desktop was actively composing right beforehand (see [`StallWatch`]).
pub(super) struct Stall {
/// How long the hole lasted (last fresh frame → the frame that ended it).
pub(super) gap: Duration,
/// `Some(mean period)` when this stall completes a metronomic cycle (see
/// [`pf_frame::metronome::Metronome`]).
pub(super) metronomic: Option<Duration>,
}
/// Capture-stall watch — the "sole virtual display" stutter diagnostic (field reports: Exclusive
/// topology = periodic double-jolt, Extend = smooth, i.e. the disturbance lives in the display/present
/// path BELOW capture and only while no physical output is active).
///
/// On a damage-driven capture an idle desktop legitimately goes quiet (no damage → no frames), so a
/// gap only counts as a stall when the [`Self::RECENT`] frames before it all arrived within
/// [`Self::ACTIVE_SPAN`] — sustained ≥ ~20 fps flow (a game or video), not a blinking caret or a
/// mouse twitch. Each stall feeds a [`pf_frame::metronome::Metronome`], so periodic stalls self-diagnose
/// in the log WITHOUT needing any client keyframe request — discriminating "DWM stopped composing"
/// from encode/network causes that the recovery-cadence detector covers. Pure logic — unit-tested
/// below; the caller does the logging.
pub(super) struct StallWatch {
/// The last [`Self::RECENT`] fresh-frame instants (pre-gap history for the activity gate).
recent: std::collections::VecDeque<Instant>,
cadence: pf_frame::metronome::Metronome,
}
impl StallWatch {
/// Frames of pre-gap history that must be tight for flow to count as active. Stalls are thus
/// naturally spaced ≥ RECENT frame times apart — no extra log rate limit needed.
const RECENT: usize = 8;
/// The RECENT pre-gap frames must all fit in this span (8 frames in 400 ms ≈ ≥ 20 fps flow —
/// loose enough for a 30 fps-capped game, tight enough to reject idle-desktop damage).
const ACTIVE_SPAN: Duration = Duration::from_millis(400);
/// The smallest hole that counts as a stall (~9 missed frames at 60 Hz) — well below the
/// reported 300700 ms freezes, above encode/present jitter.
const STALL_MIN: Duration = Duration::from_millis(150);
pub(super) fn new() -> Self {
Self {
recent: std::collections::VecDeque::with_capacity(Self::RECENT + 1),
cadence: pf_frame::metronome::Metronome::new(),
}
}
/// Forget the flow history (a ring recreate's gap is self-inflicted, not a DWM stall — without
/// the reset the first post-recreate frame would read as one).
pub(super) fn reset(&mut self) {
self.recent.clear();
}
/// Record a fresh driver frame at `now`; `Some` exactly when it ended a stall.
pub(super) fn note_fresh(&mut self, now: Instant) -> Option<Stall> {
let was_active = self.recent.len() == Self::RECENT
&& self
.recent
.back()
.zip(self.recent.front())
.is_some_and(|(b, f)| b.duration_since(*f) <= Self::ACTIVE_SPAN);
let gap = self.recent.back().map(|last| now.duration_since(*last));
self.recent.push_back(now);
if self.recent.len() > Self::RECENT {
self.recent.pop_front();
}
let gap = gap?;
if !was_active || gap < Self::STALL_MIN {
return None;
}
Some(Stall {
gap,
metronomic: self.cadence.note(now),
})
}
}
@@ -2,15 +2,15 @@
//! without a real capture session.
//!
//! The native AMF path (and the D3D11 zero-copy NVENC/QSV paths) require an NV12 texture that lives
//! on the GPU — the CPU-Bgrx [`SyntheticCapturer`](crate::capture::SyntheticCapturer) can't provide
//! on the GPU — the CPU-Bgrx [`SyntheticCapturer`](crate::SyntheticCapturer) can't provide
//! one, and DXGI Desktop Duplication can't create one under an ssh session-0 (E_ACCESSDENIED). This
//! source builds an NV12 texture on the selected render adapter and fills it with a **moving** luma
//! ramp each frame, so the encoder sees genuine motion (P-frame residuals + the intra-refresh wave
//! under content change) — exactly what an intra-refresh recovery validation needs. Driven by
//! `spike --source synthetic-nv12`.
use crate::capture::dxgi::{make_device, D3d11Frame};
use crate::capture::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use crate::dxgi::{make_device, D3d11Frame};
use crate::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{Context, Result};
use windows::Win32::Graphics::Direct3D11::{
ID3D11Device, ID3D11DeviceContext, ID3D11Texture2D, D3D11_BIND_SHADER_RESOURCE,
@@ -140,7 +140,7 @@ impl Capturer for SyntheticNv12Capturer {
/// Calls DXGI factory/adapter enumeration; returns owned COM objects or an error.
unsafe fn resolve_render_adapter() -> Result<IDXGIAdapter1> {
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("CreateDXGIFactory1")?;
if let Some(luid) = crate::win_adapter::resolve_render_adapter_luid() {
if let Some(luid) = pf_gpu::resolve_render_adapter_luid() {
if let Ok(a) = factory.EnumAdapterByLuid::<IDXGIAdapter1>(luid) {
return Ok(a);
}
+2 -2
View File
@@ -60,7 +60,7 @@ impl AudioPlayer {
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, recycle_tx, stop_t, ready_tx, channels as u8)
{
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
tracing::warn!(error = %format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
@@ -232,7 +232,7 @@ impl MicStreamer {
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
tracing::warn!(error = format!("{e:#}"), "mic uplink thread ended");
tracing::warn!(error = %format!("{e:#}"), "mic uplink thread ended");
}
})
.context("spawn mic thread")?;
+51 -131
View File
@@ -31,7 +31,7 @@
//!
//! This thread is also the single consumer of the rumble and HID-output pull planes.
use punktfunk_core::client::NativeClient;
use punktfunk_core::client::{ActuatorQuirks, NativeClient};
use punktfunk_core::config::GamepadPref;
use punktfunk_core::input::{gamepad as wire, InputEvent, InputKind};
use punktfunk_core::quic::{HidOutput, RichInput};
@@ -61,24 +61,14 @@ const ESCAPE_CHORD: [u32; 4] = [wire::BTN_LB, wire::BTN_RB, wire::BTN_START, wir
/// Hold the [`ESCAPE_CHORD`] at least this long to disconnect (escalates the leave-fullscreen press).
const DISCONNECT_HOLD: Duration = Duration::from_millis(1500);
/// Steam Deck built-in haptic keep-alive interval. The Deck's actuator decays inside SDL's
/// ~2 s internal rumble resend (`SDL_RUMBLE_RESEND_MS`), and SDL short-circuits a repeated
/// identical `set_rumble` value to a no-op device write — so a STEADY host value (which the
/// host delivers only as unchanging 500 ms refreshes) never re-kicks the motor and is felt as
/// a periodic pulse. We re-issue below the decay so the bursts fuse into a continuous buzz;
/// 40 ms mirrors SDL's sibling Steam-Controller driver keep-alive. Deck-only (see
/// [`Worker::issue_rumble`]); every other pad sustains rumble at the hardware level and is
/// left untouched.
const DECK_RUMBLE_KEEPALIVE_MS: u64 = 40;
/// Ceiling on a *legacy* (no-TTL) host's Steam Deck rumble: silence the actuator once a real host
/// update has been absent this long. A legacy host re-sends the held level as a flat 500 ms refresh,
/// so a genuinely-held rumble refreshes the per-slot update clock (`RumbleState::updated_at`) every
/// 500 ms and never approaches this — only a lost *stop* datagram (the host went quiet entirely)
/// lets the 40 ms keep-alive drone on. 2× the 500 ms refresh bounds that lost stop to ~1 s,
/// mirroring the Windows host's `RUMBLE_IDLE_TIMEOUT` residual cutoff. The v2 path is bounded by its
/// lease `deadline` instead and never trips this (see [`Worker::render_feedback`]).
const LEGACY_RUMBLE_CEILING_MS: u64 = 1_000;
/// Steam Deck actuator-decay keepalive cadence, declared to the core's rumble policy engine as an
/// [`ActuatorQuirks`] at slot open. The Deck's built-in actuator decays inside SDL's ~2 s internal
/// rumble resend (`SDL_RUMBLE_RESEND_MS`) and SDL short-circuits an identical `set_rumble` value
/// to a no-op device write — so a steady level is felt as a periodic pulse without sub-decay
/// re-kicks; 40 ms mirrors SDL's sibling Steam-Controller driver keep-alive. The engine owns the
/// re-kick timing, the 1-LSB dedupe-defeat jitter, and every staleness/lease bound — this worker
/// only applies the commands it emits (`design/rumble-root-fix.md` §D).
const DECK_RUMBLE_KEEPALIVE_MS: u16 = 40;
/// Stick deflection below this is ignored for menu navigation (0.5 of full scale — Apple
/// `GamepadMenuInput` parity; menus want deliberate flicks, not drift).
@@ -626,32 +616,6 @@ impl Ds5Feedback {
}
}
/// Per-controller rumble render state (the Steam Deck keep-alive + the host's v2 lease). Held
/// per [`Slot`] so a rumble the host addressed to pad N drives only pad N's actuator.
#[derive(Default)]
struct RumbleState {
/// Last rumble value handed to this pad (the logical host value, pre-jitter) and when —
/// drives the Steam Deck haptic keep-alive in [`Worker::render_feedback`].
last: (u16, u16),
last_at: Option<Instant>,
/// When the last *real* host rumble datagram landed on this slot — set only in the feedback
/// drain, never bumped by the Deck keep-alive re-kick (unlike `last_at`, which the keep-alive
/// refreshes every ~40 ms). A legacy host carries no lease, so this per-slot clock is what
/// bounds a lost stop-frame: once it is stale past `LEGACY_RUMBLE_CEILING_MS` the keep-alive
/// stops and issues one (0, 0). See [`Worker::render_feedback`].
updated_at: Option<Instant>,
/// Toggles the 1-LSB low-motor nudge that forces SDL past its identical-value dedupe on a
/// Deck keep-alive re-issue (see [`Worker::issue_rumble`]).
jitter: bool,
/// The host lease from a v2 rumble envelope: last non-zero level expires at this instant
/// unless the host renews it. `None` outside a live rumble or against a legacy host (which
/// sends no lease — the pad then relies on SDL's own duration expiry as before).
deadline: Option<Instant>,
/// The host-supplied TTL (ms) of the current envelope, handed to SDL as the `set_rumble`
/// duration; `0` = legacy host (fall back to the proven 1.5 s duration).
ttl_ms: u16,
}
/// One forwarded controller during an attached session: the open SDL handle, its stable wire
/// pad index (0..[`MAX_PADS`](punktfunk_core::input::MAX_PADS)), and the per-pad wire/feedback
/// state that used to be single-scalar on the Worker. Opening the device is what grabs the
@@ -683,7 +647,6 @@ struct Slot {
/// close lift a click held across detach/unplug.
held_clicks: [bool; 2],
last_accel: [i16; 3],
rumble: RumbleState,
}
impl Slot {
@@ -699,7 +662,6 @@ impl Slot {
surface_last: [(0, 0, false); 2],
held_clicks: [false; 2],
last_accel: [0; 3],
rumble: RumbleState::default(),
}
}
@@ -928,6 +890,20 @@ impl Worker {
// uses the session-default kind.
if let Some(c) = &self.attached {
send(c, InputKind::GamepadArrival, pref.to_u8() as u32, 0, index);
// Declare the actuator's quirks to the shared rumble policy engine. ALWAYS
// set (defaults for a well-behaved pad): wire indices are reused within a
// connection, so a Deck slot that closes must not leave its keepalive quirk
// behind for the next pad on the same index.
let quirks = if pref == GamepadPref::SteamDeck {
ActuatorQuirks {
keepalive_ms: DECK_RUMBLE_KEEPALIVE_MS,
min_pulse_ms: 0,
dedup_jitter: true,
}
} else {
ActuatorQuirks::default()
};
c.set_rumble_quirks(index as u16, quirks);
}
tracing::info!(id, index, pref = ?pref, "gamepad forwarding (slot opened)");
self.slots.push(slot);
@@ -940,6 +916,10 @@ impl Worker {
/// the SDL handle. The flush only emits wire events, so it is safe even when the device is
/// already gone (unplug).
fn close_slot_at(&mut self, i: usize) {
// Best-effort physical silence before the handle drops: a slot closed mid-buzz (detach /
// unplug) must not depend on what SDL does to a rumbling device at close. Errors are
// expected for an already-unplugged pad.
let _ = self.slots[i].pad.set_rumble(0, 0, 100);
if let Some(c) = self.attached.clone() {
Self::flush_slot(&c, &mut self.slots[i]);
// Signal the host to tear down this pad's virtual device (native hot-unplug). Sent
@@ -1464,107 +1444,47 @@ impl Worker {
}
}
/// Hand a rumble value to SDL on one slot's pad, remembering it for the Deck keep-alive.
/// SDL short-circuits an identical `(low, high)` with NO device write (it only re-arms its
/// expiration), so on a Deck keep-alive re-issue of the same non-zero value we flip a single
/// low-motor LSB — an imperceptible amplitude nudge — to force the write through and keep the
/// actuator physically fed. The SDL duration is the host's envelope TTL (a lease continuously
/// refreshed by renewals, so a sustained rumble never dies mid-effect and an abandoned one
/// self-silences at the TTL); against a legacy host (`ttl_ms == 0`) it stays the proven 1.5 s.
fn issue_rumble(slot: &mut Slot, low: u16, high: u16, deck: bool) {
let dur_ms: u32 = if slot.rumble.ttl_ms == 0 {
1_500 // legacy host: no lease — keep the proven duration
/// Hand one policy-engine command to SDL on a slot's pad, verbatim. The core engine owns all
/// rumble policy — leases, legacy-host staleness, the Deck keepalive + its dedupe-defeat
/// jitter (declared as quirks at slot open) — so this worker keeps no rumble state at all.
/// `backstop_ms` becomes the SDL duration: the hardware-level net under a stalled worker
/// thread (the engine emits explicit zeros at every policy stop, so it is never the stop
/// mechanism).
fn issue_rumble(slot: &mut Slot, low: u16, high: u16, backstop_ms: u32) {
let dur_ms: u32 = if (low, high) == (0, 0) {
100 // a stop takes effect immediately; the duration is irrelevant
} else {
// Floor the lease so a jittered renewal (or the ~40 ms Deck re-kick) can never gap the
// actuator between SDL writes.
(slot.rumble.ttl_ms as u32).max(DECK_RUMBLE_KEEPALIVE_MS as u32 * 4)
backstop_ms.max(160) // floor: a jittered renewal can never gap the actuator
};
let (out_low, out_high) =
if deck && (low, high) == slot.rumble.last && (low, high) != (0, 0) {
slot.rumble.jitter = !slot.rumble.jitter;
(low ^ slot.rumble.jitter as u16, high)
} else {
(low, high)
};
// Surface a failed SDL rumble write: a swallowed error here (DualSense not in the right
// HIDAPI mode, etc.) reads exactly like "rumble doesn't work". The host logs the send side
// on 0xCA with the pad index, so the two together pinpoint host-game vs client-render.
match slot.pad.set_rumble(out_low, out_high, dur_ms) {
match slot.pad.set_rumble(low, high, dur_ms) {
Err(e) => {
tracing::warn!(pad = slot.index, low, high, error = %e, "rumble: SDL set_rumble failed")
}
Ok(()) => tracing::debug!(pad = slot.index, low, high, "rumble: rendered"),
Ok(()) => tracing::trace!(pad = slot.index, low, high, "rumble: rendered"),
}
slot.rumble.last = (low, high);
slot.rumble.last_at = Some(Instant::now());
}
/// Drain and render the feedback planes — rumble plus HID output (lightbar / player LEDs /
/// adaptive triggers) — routing each update to the forwarded slot on its wire pad index; this
/// thread is their single consumer. Rumble arrives as self-terminating v2 envelopes: each
/// carries a TTL the host renews while the level holds and lets expire when it stops, so the
/// actuator's divergence from the host's intent is bounded by the wire, not by a client guess.
/// A legacy host (`ttl == None`) has no lease — the pad falls back to SDL's own 1.5 s duration
/// expiry as before.
/// thread is their single consumer. Rumble arrives as EFFECTIVE commands from the core's
/// shared policy engine, which already applied every policy — v2 lease expiry, legacy-host
/// staleness, the Deck actuator keepalive + jitter (via the quirks declared at slot open),
/// and connection-close drain zeros — so this worker applies commands verbatim and keeps no
/// rumble state of its own (`design/rumble-root-fix.md` §D).
fn render_feedback(&mut self) {
let Some(connector) = self.attached.clone() else {
return;
};
// Rumble envelopes (0xCA) → the slot holding that wire pad index. An update for an index
// with no live slot (a pad that just unplugged) is dropped.
while let Ok((pad, low, high, ttl)) = connector.next_rumble_ttl(Duration::ZERO) {
if let Some(slot) = self.slots.iter_mut().find(|s| s.index as u16 == pad) {
let deck = slot.pref == GamepadPref::SteamDeck;
slot.rumble.ttl_ms = ttl.unwrap_or(0);
// A v2 lease sets an explicit client-side deadline; a legacy update clears it and
// leans on SDL's own duration expiry (unchanged behaviour).
slot.rumble.deadline = match ttl {
Some(ms) if (low, high) != (0, 0) => {
Some(Instant::now() + Duration::from_millis(ms as u64))
}
_ => None,
};
// Mark this as a real host update. Unlike `last_at` (which the Deck keep-alive
// re-kick refreshes every ~40 ms), this clock advances only here, so a legacy
// lost-stop can be bounded by `LEGACY_RUMBLE_CEILING_MS` in the keep-alive below.
slot.rumble.updated_at = Some(Instant::now());
Self::issue_rumble(slot, low, high, deck);
}
}
// Steam Deck keep-alive, per slot: the built-in actuator decays inside SDL's ~2 s internal
// rumble resend, and SDL dedupes an unchanged `set_rumble` to a no-op device write — so a
// steady host value is felt as a periodic pulse. Re-kick a Deck slot below the decay
// (`DECK_RUMBLE_KEEPALIVE_MS`) so its discrete bursts fuse into a continuous buzz, but
// silence it once the host's lease expires (the host stopped renewing — a lost stop, or the
// host died). The per-slot timing guards make this idempotent with a fresh datagram this
// tick (a just-set `last_at`/`deadline` fails both checks). Non-Deck slots sustain/expire at
// the SDL/hardware level and never enter here.
for slot in self.slots.iter_mut() {
if slot.pref != GamepadPref::SteamDeck || slot.rumble.last == (0, 0) {
continue;
}
if slot.rumble.deadline.is_some_and(|d| Instant::now() >= d) {
slot.rumble.deadline = None;
slot.rumble.ttl_ms = 0;
Self::issue_rumble(slot, 0, 0, true);
} else if slot.rumble.ttl_ms == 0
&& slot
.rumble
.updated_at
.is_some_and(|t| t.elapsed() >= Duration::from_millis(LEGACY_RUMBLE_CEILING_MS))
{
// Legacy host (no v2 lease): a held rumble refreshes `updated_at` every ~500 ms, so
// this only trips on a lost stop-frame the host never followed up — silence the
// actuator once instead of letting the 40 ms keep-alive drone forever. `issue_rumble`
// sets `last` to (0, 0), so the top-of-loop guard skips this slot on later ticks.
Self::issue_rumble(slot, 0, 0, true);
} else if slot
.rumble
.last_at
.is_none_or(|t| t.elapsed() >= Duration::from_millis(DECK_RUMBLE_KEEPALIVE_MS))
{
let (low, high) = slot.rumble.last;
Self::issue_rumble(slot, low, high, true);
// Engine commands → the slot holding that wire pad index. A command for an index with no
// live slot (a pad that just unplugged) is dropped. The loop ends on NoFrame (drained
// dry this tick) or Closed (session over — the engine delivered its close-drain zeros
// first; the physical silence backstop is in `close_slot_at`).
while let Ok(cmd) = connector.next_rumble_command(Duration::ZERO) {
if let Some(slot) = self.slots.iter_mut().find(|s| s.index as u16 == cmd.pad) {
Self::issue_rumble(slot, cmd.low, cmd.high, cmd.backstop_ms);
}
}
// HID output (lightbar / player LEDs / adaptive triggers) → the slot on that wire index.
+8
View File
@@ -33,6 +33,14 @@ pub mod session;
pub mod trust;
#[cfg(any(target_os = "linux", windows))]
pub mod video;
#[cfg(any(target_os = "linux", windows))]
mod video_color;
#[cfg(any(target_os = "linux", windows))]
mod video_software;
#[cfg(target_os = "linux")]
mod video_vaapi;
#[cfg(any(target_os = "linux", windows))]
mod video_vulkan;
// PyroWave decode — Linux + `pyrowave` feature only (plan §4.5; the Windows client's
// present-path decision and the Apple Metal port are their own phases).
#[cfg(windows)]
+1 -66
View File
@@ -125,7 +125,7 @@ pub fn agent(
.with_safe_default_protocol_versions()
.map_err(|e| bad("tls config", &e))?
.dangerous()
.with_custom_certificate_verifier(Arc::new(PinVerify { pin }));
.with_custom_certificate_verifier(Arc::new(punktfunk_core::tls::PinVerify::new(pin)));
let cert = rustls::pki_types::CertificateDer::from_pem_slice(identity.0.as_bytes())
.map_err(|e| bad("client cert pem", &e))?;
let key = rustls::pki_types::PrivateKeyDer::from_pem_slice(identity.1.as_bytes())
@@ -251,71 +251,6 @@ fn classify(e: ureq::Error) -> LibraryError {
}
}
/// Fingerprint-pinning verifier — the client-HTTP twin of core's (private) QUIC
/// `PinVerify`: trust is the SHA-256 of the host's self-signed leaf cert. The handshake
/// signatures MUST still be verified for real: CertificateVerify is what proves the peer
/// *holds the pinned cert's private key* — skip it and an active MITM can replay the
/// host's (public) certificate, match the pin, and complete the handshake with its own key.
#[derive(Debug)]
struct PinVerify {
pin: Option<[u8; 32]>,
}
impl rustls::client::danger::ServerCertVerifier for PinVerify {
fn verify_server_cert(
&self,
end_entity: &rustls::pki_types::CertificateDer<'_>,
_intermediates: &[rustls::pki_types::CertificateDer<'_>],
_server_name: &rustls::pki_types::ServerName<'_>,
_ocsp: &[u8],
_now: rustls::pki_types::UnixTime,
) -> Result<rustls::client::danger::ServerCertVerified, rustls::Error> {
if let Some(expected) = self.pin {
let fp = punktfunk_core::quic::endpoint::cert_fingerprint(end_entity.as_ref());
if fp != expected {
return Err(rustls::Error::InvalidCertificate(
rustls::CertificateError::ApplicationVerificationFailure,
));
}
}
Ok(rustls::client::danger::ServerCertVerified::assertion())
}
fn verify_tls12_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls12_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls13_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
rustls::crypto::ring::default_provider()
.signature_verification_algorithms
.supported_schemes()
}
}
#[cfg(test)]
mod tests {
use super::*;
+2 -2
View File
@@ -222,7 +222,7 @@ fn pump(
preferred = punktfunk_core::quic::CODEC_PYROWAVE;
} else {
tracing::warn!(
"PUNKTFUNK_PREFER_PYROWAVE=1 but the presenter device failed the pyrowave probe — keeping the normal codec preference"
"PUNKTFUNK_PREFER_PYROWAVE=1 but the presenter device failed the pyrowave probe — keeping the normal codec preference"
);
}
}
@@ -791,7 +791,7 @@ fn spawn_audio(
buf.extend_from_slice(&pcm[..n]);
player.push(buf);
}
Err(e) => tracing::debug!(error = %e, "opus decode"),
Err(e) => tracing::debug!(error = %e, "opus decode failed"),
}
}
Err(PunktfunkError::NoFrame) => {}
File diff suppressed because it is too large Load Diff
+210
View File
@@ -0,0 +1,210 @@
//! The stream's per-frame colour signalling (`ColorDesc`) + the YCbCr→RGB CSC matrix (`csc_rows`).
#![allow(clippy::unnecessary_cast)]
use ffmpeg_next as ffmpeg;
/// The stream's colour signaling, read PER-FRAME from the decoder (HEVC VUI → the
/// `AVFrame` CICP fields). The Windows host switches an HDR desktop to Main10 BT.2020 PQ
/// **in-band** (the Welcome still says SDR — clients are expected to follow the VUI, as
/// the Windows/Apple/Android clients do), so rendering must follow the frames, not the
/// handshake — else PQ content drawn as BT.709 comes out washed out and desaturated.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct ColorDesc {
/// H.273 code points as signaled (2 = unspecified → the renderer picks the SDR default).
pub primaries: u8,
pub transfer: u8,
pub matrix: u8,
pub full_range: bool,
}
impl ColorDesc {
/// Read the CICP fields off a raw decoded frame. Public: the Windows client's raw-FFI
/// D3D11VA/software decoders build their per-frame `ColorDesc` with it too (same
/// `ffmpeg-next` major, so the `AVFrame` type unifies across the workspace).
///
/// # Safety
/// `frame` must point to a valid `AVFrame` (alive for the duration of the call).
pub unsafe fn from_raw(frame: *const ffmpeg::ffi::AVFrame) -> ColorDesc {
// SAFETY: caller guarantees a live AVFrame; these are plain enum field reads.
unsafe {
ColorDesc {
primaries: (*frame).color_primaries as u32 as u8,
transfer: (*frame).color_trc as u32 as u8,
matrix: (*frame).colorspace as u32 as u8,
full_range: (*frame).color_range == ffmpeg::ffi::AVColorRange::AVCOL_RANGE_JPEG,
}
}
}
/// PQ (SMPTE ST.2084) transfer — the HDR10 signal.
pub fn is_pq(&self) -> bool {
self.transfer == 16
}
}
/// The YCbCr→RGB conversion as three vec4 rows for a shader constant buffer / push-constant
/// block: `rgb[i] = dot(r[i].xyz, yuv) + r[i].w` — bit-depth exact. The ONE coefficient
/// implementation every presenter derives its CSC from (Vulkan push constants, the Windows
/// client's D3D11 constant buffer), so a stream's signaled matrix/range is honored identically
/// everywhere; the Apple client ports this function (and its tests) to Swift.
///
/// `depth` picks the limited-range code points (8-bit: 16/235/240 over 255; 10-bit:
/// 64/940/960 over 1023 — NOT the same normalized values, the difference is ~half a
/// code). `msb_packed` folds in the P010/X6 packing factor: 10 significant bits live in
/// the MSBs of 16, so a UNORM16 sample reads `code·64/65535` — multiplying by
/// `65535/65472` recovers exact `code/1023`.
pub fn csc_rows(desc: ColorDesc, depth: u8, msb_packed: bool) -> [[f32; 4]; 3] {
// BT.601 (5/6), BT.2020 (9/10); everything else — incl. unspecified — is the host's
// BT.709 SDR default (mirrors the software path's swscale coefficient choice).
let (kr, kb) = match desc.matrix {
5 | 6 => (0.299, 0.114),
9 | 10 => (0.2627, 0.0593),
_ => (0.2126, 0.0722),
};
let kg = 1.0 - kr - kb;
let max = f64::from((1u32 << depth) - 1); // 255 / 1023
let step = f64::from(1u32 << (depth - 8)); // code points per 8-bit step: 1 / 4
let pack = if msb_packed { 65535.0 / 65472.0 } else { 1.0 };
let (sy, oy, sc) = if desc.full_range {
(pack, 0.0f64, pack)
} else {
(
pack * max / (219.0 * step),
-(16.0 * step) / max,
pack * max / (224.0 * step),
)
};
// rgb = M * (yuv + off) = M*yuv + M*off — rows of M with the offset dot folded into
// w. `yuv` is the SAMPLED (packed) value, so the offsets divide by the packing
// factor to land on the same scale.
let off = [oy / pack, -0.5 / pack, -0.5 / pack];
let m = [
[sy, 0.0, 2.0 * (1.0 - kr) * sc],
[
sy,
-2.0 * (1.0 - kb) * kb / kg * sc,
-2.0 * (1.0 - kr) * kr / kg * sc,
],
[sy, 2.0 * (1.0 - kb) * sc, 0.0],
];
core::array::from_fn(|r| {
let w: f64 = (0..3).map(|c| m[r][c] * off[c]).sum();
[m[r][0] as f32, m[r][1] as f32, m[r][2] as f32, w as f32]
})
}
#[cfg(test)]
mod tests {
use super::*;
fn desc(matrix: u8, full_range: bool) -> ColorDesc {
ColorDesc {
primaries: 1,
transfer: 1,
matrix,
full_range,
}
}
fn apply(rows: &[[f32; 4]; 3], yuv: [f32; 3]) -> [f32; 3] {
core::array::from_fn(|r| {
rows[r][0] * yuv[0] + rows[r][1] * yuv[1] + rows[r][2] * yuv[2] + rows[r][3]
})
}
/// 10-bit limited MSB-packed (P010/X6): reference white Y=940, black Y=64, neutral
/// chroma 512 — sampled as UNORM16 of `code << 6`.
#[test]
fn bt2020_10bit_limited_white_black() {
let rows = csc_rows(desc(9, false), 10, true);
let s = |code: u32| ((code << 6) as f32) / 65535.0;
let white = apply(&rows, [s(940), s(512), s(512)]);
let black = apply(&rows, [s(64), s(512), s(512)]);
for (w, b) in white.iter().zip(black) {
assert!((w - 1.0).abs() < 0.002, "white {white:?}");
assert!(b.abs() < 0.002, "black {black:?}");
}
}
/// Reference white (Y=235, U=V=128 limited) → RGB 1.0; reference black (Y=16) → 0.0
/// — the GL presenter's test, in row form.
#[test]
fn bt709_limited_white_black() {
let rows = csc_rows(desc(1, false), 8, false);
let white = apply(&rows, [235.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0]);
let black = apply(&rows, [16.0 / 255.0, 128.0 / 255.0, 128.0 / 255.0]);
for (w, b) in white.iter().zip(black) {
assert!((w - 1.0).abs() < 0.005, "white {white:?}");
assert!(b.abs() < 0.005, "black {black:?}");
}
}
/// Full-range identity points + the 601-vs-709 red excursion (guards the
/// matrix-code dispatch), same as the GL presenter's test.
#[test]
fn full_range_and_red_excursion() {
let rows = csc_rows(desc(5, true), 8, false);
let white = apply(&rows, [1.0, 0.5, 0.5]);
assert!(white.iter().all(|v| (v - 1.0).abs() < 1e-5), "{white:?}");
let red = apply(&rows, [0.0, 0.5, 1.0]);
assert!((red[0] - 2.0 * (1.0 - 0.299) * 0.5).abs() < 1e-4, "{red:?}");
let rows709 = csc_rows(desc(1, true), 8, false);
let red709 = apply(&rows709, [0.0, 0.5, 1.0]);
assert!(
(red709[0] - 2.0 * (1.0 - 0.2126) * 0.5).abs() < 1e-4,
"{red709:?}"
);
assert!((red[0] - red709[0]).abs() > 0.05);
}
/// The row form must agree with the GL presenter's column-major `yuv_to_rgb` on a
/// grid of inputs — same math, different packing.
#[test]
fn rows_match_the_gl_matrix_form() {
for (matrix, full) in [(1u8, false), (1, true), (5, false), (9, false), (9, true)] {
let d = desc(matrix, full);
let rows = csc_rows(d, 8, false);
// Reimplementation of video_gl::yuv_to_rgb's application for comparison.
let (kr, kb) = match matrix {
5 | 6 => (0.299f32, 0.114f32),
9 | 10 => (0.2627, 0.0593),
_ => (0.2126, 0.0722),
};
let kg = 1.0 - kr - kb;
let (sy, oy, sc) = if full {
(1.0f32, 0.0f32, 1.0f32)
} else {
(255.0 / 219.0, -16.0 / 255.0, 255.0 / 224.0)
};
let mat = [
sy,
sy,
sy,
0.0,
-2.0 * (1.0 - kb) * kb / kg * sc,
2.0 * (1.0 - kb) * sc,
2.0 * (1.0 - kr) * sc,
-2.0 * (1.0 - kr) * kr / kg * sc,
0.0,
];
let off = [oy, -0.5, -0.5];
for yuv in [
[0.1f32, 0.3, 0.7],
[0.9, 0.5, 0.5],
[0.5, 0.2, 0.8],
[16.0 / 255.0, 0.5, 0.5],
] {
let v = [yuv[0] + off[0], yuv[1] + off[1], yuv[2] + off[2]];
let gl: [f32; 3] =
core::array::from_fn(|r| (0..3).map(|c| mat[c * 3 + r] * v[c]).sum());
let ours = apply(&rows, yuv);
for (a, b) in gl.iter().zip(ours) {
assert!(
(a - b).abs() < 1e-5,
"{matrix}/{full}: gl {gl:?} rows {ours:?}"
);
}
}
}
}
}
+4 -3
View File
@@ -537,7 +537,8 @@ impl PyroWaveDecoder {
let fence = device.create_fence(&vk::FenceCreateInfo::default(), None)?;
tracing::info!(
mode = %format!("{width}x{height}"),
width,
height,
"PyroWave decoder open on the presenter's device (compute iDWT, BT.709 limited)"
);
Ok(PyroWaveDecoder {
@@ -602,8 +603,8 @@ impl PyroWaveDecoder {
});
self.next = 0;
tracing::info!(
from = %format!("{}x{}", self.width, self.height),
to = %format!("{width}x{height}"),
from = %format_args!("{}x{}", self.width, self.height),
to = %format_args!("{width}x{height}"),
"PyroWave decoder rebuilt for mid-stream resize"
);
self.width = width;
+264
View File
@@ -0,0 +1,264 @@
//! CPU/libavcodec software decode backend (swscale → RGBA).
use crate::video::{averr, CpuFrame};
use crate::video_color::ColorDesc;
use anyhow::{anyhow, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
use std::ptr;
// --- software backend ---------------------------------------------------------------
pub(crate) struct SoftwareDecoder {
decoder: ffmpeg::decoder::Video,
/// Rebuilt whenever the decoded format/size — or the colour signaling (a mid-stream
/// SDR↔HDR flip) — changes.
sws: Option<(scaling::Context, Pixel, u32, u32, ColorDesc)>,
}
impl SoftwareDecoder {
pub(crate) fn new(codec_id: ffmpeg::codec::Id) -> Result<SoftwareDecoder> {
let codec = ffmpeg::decoder::find(codec_id)
.ok_or_else(|| anyhow!("no {codec_id:?} decoder in libavcodec"))?;
let mut ctx = ffmpeg::codec::Context::new_with_codec(codec);
unsafe {
let raw = ctx.as_mut_ptr();
(*raw).flags |= ffmpeg::ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
// Slice threading adds no frame delay (frame threading adds thread_count-1).
(*raw).thread_type = ffmpeg::ffi::FF_THREAD_SLICE;
(*raw).thread_count = 0; // auto
}
let decoder = ctx.decoder().video().context("open video decoder")?;
Ok(SoftwareDecoder { decoder, sws: None })
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<CpuFrame>> {
let packet = ffmpeg::Packet::copy(au);
self.decoder
.send_packet(&packet)
.map_err(|e| anyhow!("send_packet: {e}"))?;
let mut frame = AvFrame::empty();
let mut out = None;
while self.decoder.receive_frame(&mut frame).is_ok() {
out = Some(self.convert_rgba(&frame)?);
}
Ok(out)
}
fn convert_rgba(&mut self, frame: &AvFrame) -> Result<CpuFrame> {
let (fmt, w, h) = (frame.format(), frame.width(), frame.height());
// SAFETY: `frame.as_ptr()` is the decoder-owned live AVFrame for this call.
let color = unsafe { ColorDesc::from_raw(frame.as_ptr()) };
let rebuild = !matches!(&self.sws,
Some((_, f, sw, sh, c)) if *f == fmt && *sw == w && *sh == h && *c == color);
if rebuild {
let mut ctx =
scaling::Context::get(fmt, w, h, Pixel::RGBA, w, h, scaling::Flags::POINT)
.context("swscale context")?;
// swscale defaults to BT.601 coefficients — set them from the FRAME's signaling
// (unspecified → BT.709 limited, the host's SDR default; a Windows HDR desktop
// streams BT.2020 in-band). Without this, YUV→RGB decodes with the wrong matrix
// and colours shift. Destination = full-range RGB; the transfer function stays
// baked in (the presenter tags PQ textures so GTK applies the EOTF).
const SWS_CS_ITU709: i32 = 1;
const SWS_CS_ITU601: i32 = 5;
const SWS_CS_BT2020: i32 = 9;
let cs = match color.matrix {
9 | 10 => SWS_CS_BT2020,
5 | 6 => SWS_CS_ITU601,
_ => SWS_CS_ITU709,
};
unsafe {
let coeffs = ffmpeg::ffi::sws_getCoefficients(cs);
ffmpeg::ffi::sws_setColorspaceDetails(
ctx.as_mut_ptr(),
coeffs, // inv_table: source (YUV) coefficients per the VUI
color.full_range as i32, // srcRange: 0 = limited/studio (MPEG)
coeffs, // table: destination coefficients (ignored for RGB output)
1, // dstRange: 1 = full-range RGB
0,
1 << 16,
1 << 16, // brightness, contrast, saturation (defaults)
);
}
self.sws = Some((ctx, fmt, w, h, color));
}
let (sws, ..) = self.sws.as_mut().unwrap();
// Single-pass conversion: swscale writes straight into the Vec the texture will
// wrap. (The old path scaled into a scratch AVFrame and then copied `data(0)` out
// — a second full-frame pass per frame.) 64-byte row alignment keeps swscale on
// aligned SIMD stores; `GdkMemoryTexture` takes the resulting stride explicitly.
const ALIGN: i32 = 64;
use ffmpeg::ffi;
let dst_fmt = ffi::AVPixelFormat::AV_PIX_FMT_RGBA;
// SAFETY: pure size computation from format/dimensions; no pointers involved.
let size = unsafe { ffi::av_image_get_buffer_size(dst_fmt, w as i32, h as i32, ALIGN) };
if size < 0 {
return Err(averr("av_image_get_buffer_size", size));
}
let rgba = vec![0u8; size as usize];
let mut dst_data: [*mut u8; 4] = [ptr::null_mut(); 4];
let mut dst_linesize: [i32; 4] = [0; 4];
// SAFETY: fill_arrays only derives plane pointers/strides into `rgba` (sized by
// av_image_get_buffer_size above, same format/align) — no allocation, no
// ownership transfer; `rgba` outlives the scale below.
let r = unsafe {
ffi::av_image_fill_arrays(
dst_data.as_mut_ptr(),
dst_linesize.as_mut_ptr(),
rgba.as_ptr(),
dst_fmt,
w as i32,
h as i32,
ALIGN,
)
};
if r < 0 {
return Err(averr("av_image_fill_arrays", r));
}
// SAFETY: src pointers/strides belong to the decoder-owned `frame` (alive for the
// call); dst pointers were just filled over `rgba`, and sws_scale writes rows
// [0, h) only — exactly the buffer fill_arrays sized.
let r = unsafe {
ffi::sws_scale(
sws.as_mut_ptr(),
(*frame.as_ptr()).data.as_ptr() as *const *const u8,
(*frame.as_ptr()).linesize.as_ptr(),
0,
h as i32,
dst_data.as_ptr(),
dst_linesize.as_ptr(),
)
};
if r < 0 {
return Err(averr("sws_scale", r));
}
Ok(CpuFrame {
width: w,
height: h,
stride: dst_linesize[0] as usize,
rgba,
color,
// `is_key()` reads the same intra flag `frame_is_keyframe` derives from pict_type
// for the hardware paths; ffmpeg-next handles the FFmpeg-version binding split.
keyframe: frame.is_key(),
})
}
}
#[cfg(test)]
mod tests {
use super::*;
/// The wire → `ColorDesc` plumbing: an HDR10 stream's VUI (BT.2020 primaries, PQ
/// transfer, BT.2020-NCL matrix, limited range) must arrive on the decoded frame —
/// this is what the Windows host emits in-band for an HDR desktop, and mis-rendering
/// it as BT.709 is the washed-out-colors bug. Fixture: one 64×64 Main10 IDR
/// (`tests/pq-frame.h265`, x265 with explicit VUI).
#[test]
fn software_decode_carries_pq_signaling() {
let au = include_bytes!("../tests/pq-frame.h265");
let mut dec = SoftwareDecoder::new(ffmpeg::codec::Id::HEVC).expect("hevc decoder");
let mut got = dec.decode(au).expect("decode");
if got.is_none() {
// Low-delay decoders may still hold the frame until a flush — send EOF.
dec.decoder.send_eof().ok();
let mut frame = AvFrame::empty();
if dec.decoder.receive_frame(&mut frame).is_ok() {
got = Some(dec.convert_rgba(&frame).expect("convert"));
}
}
let f = got.expect("no frame decoded from the PQ fixture");
assert_eq!(
f.color,
ColorDesc {
primaries: 9,
transfer: 16,
matrix: 9,
full_range: false
}
);
assert!(f.color.is_pq());
assert_eq!((f.width, f.height), (64, 64));
}
/// Golden colour fixtures: one 256×64 LOSSLESS x265 IDR of 8 fully-saturated colour bars per
/// signaling variant (generated offline with ffmpeg/libx265; the RGB→YUV conversion matched
/// to the VUI each fixture declares, so the original RGB is recoverable ±1 code). Decoding
/// through the real CPU path (`SoftwareDecoder` → per-frame `ColorDesc` → swscale with the
/// signaled matrix/range) must reproduce the bars — the end-to-end guard for the
/// signaling-driven CSC across BT.601/709 × limited/full. A hardcoded-709 regression fails
/// the 601 fixture by tens of code points; a range mix-up fails the full-range one.
#[test]
fn software_decode_reproduces_golden_bars() {
const BARS: [(u8, u8, u8); 8] = [
(255, 255, 255),
(255, 255, 0),
(0, 255, 255),
(0, 255, 0),
(255, 0, 255),
(255, 0, 0),
(0, 0, 255),
(0, 0, 0),
];
let fixtures: [(&str, &[u8], ColorDesc); 3] = [
(
"601-limited",
include_bytes!("../tests/bars-601-limited.h265"),
ColorDesc {
primaries: 1,
transfer: 1,
matrix: 5, // BT.470BG — what a Linux host's RGB-input NVENC signals
full_range: false,
},
),
(
"709-limited",
include_bytes!("../tests/bars-709-limited.h265"),
ColorDesc {
primaries: 1,
transfer: 1,
matrix: 1,
full_range: false,
},
),
(
"709-full",
include_bytes!("../tests/bars-709-full.h265"),
ColorDesc {
primaries: 1,
transfer: 1,
matrix: 1,
full_range: true, // the PUNKTFUNK_444_FULLRANGE experiment's signaling
},
),
];
for (name, au, want_color) in fixtures {
let mut dec = SoftwareDecoder::new(ffmpeg::codec::Id::HEVC).expect("hevc decoder");
let mut got = dec.decode(au).expect("decode");
if got.is_none() {
dec.decoder.send_eof().ok();
let mut frame = AvFrame::empty();
if dec.decoder.receive_frame(&mut frame).is_ok() {
got = Some(dec.convert_rgba(&frame).expect("convert"));
}
}
let f = got.unwrap_or_else(|| panic!("{name}: no frame decoded"));
assert_eq!(f.color, want_color, "{name}: signaling");
assert_eq!((f.width, f.height), (256, 64), "{name}: dims");
for (i, (r, g, b)) in BARS.iter().enumerate() {
let (cx, cy) = (i * 32 + 16, 32usize);
let o = cy * f.stride + cx * 4;
let px = &f.rgba[o..o + 3];
for (got, want) in px.iter().zip([r, g, b]) {
assert!(
got.abs_diff(*want) <= 3,
"{name} bar {i}: got {px:?}, want ({r},{g},{b})"
);
}
}
}
}
}
+243
View File
@@ -0,0 +1,243 @@
//! VAAPI (libavcodec hwaccel) decode backend → DRM-PRIME dmabuf for the presenter. Linux-only.
use crate::video::{
averr, drm_fourcc_for, frame_is_keyframe, DmabufFrame, DmabufPlane, DrmFrameGuard,
AVERROR_EAGAIN,
};
use crate::video_color::ColorDesc;
use anyhow::{anyhow, bail, Result};
use ffmpeg_next as ffmpeg;
use std::ptr;
/// libavcodec offers the formats it can decode into; pick the VAAPI hw surface. Falling
/// back to the first (software) entry would silently decode on the CPU *and* break our
/// dmabuf mapping — return NONE instead so the error surfaces and the session demotes
/// to the software backend explicitly.
#[cfg(target_os = "linux")]
unsafe extern "C" fn pick_vaapi(
_ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
) -> ffmpeg::ffi::AVPixelFormat {
unsafe {
while *list != ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE {
if *list == ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_VAAPI {
return ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_VAAPI;
}
list = list.add(1);
}
}
ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE
}
#[cfg(target_os = "linux")]
pub(crate) struct VaapiDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
}
// Single-owner pointers, only touched from the session pump thread.
#[cfg(target_os = "linux")]
unsafe impl Send for VaapiDecoder {}
#[cfg(target_os = "linux")]
impl VaapiDecoder {
pub(crate) fn new(codec_id: ffmpeg::codec::Id) -> Result<VaapiDecoder> {
use ffmpeg::ffi;
unsafe {
let mut hw_device: *mut ffi::AVBufferRef = ptr::null_mut();
let r = ffi::av_hwdevice_ctx_create(
&mut hw_device,
ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VAAPI,
ptr::null(),
ptr::null_mut(),
0,
);
if r < 0 {
bail!("no VAAPI device ({})", ffmpeg::Error::from(r));
}
// The negotiated codec's decoder id (av_codec_id maps 1:1 from ffmpeg::codec::Id).
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
ffi::av_buffer_unref(&mut hw_device);
bail!("no {codec_id:?} decoder");
}
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device);
(*ctx).get_format = Some(pick_vaapi);
(*ctx).flags |= ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
(*ctx).thread_count = 1; // hwaccel: threads only add latency
// The presenter holds mapped surfaces PAST receive_frame (the paintable's
// current texture + the newest frame in flight each pin one until GDK's
// release func) — surfaces libavcodec doesn't know are missing from its
// fixed-size VAAPI pool. Without headroom the decoder can recycle a surface
// the renderer is still sampling (intermittent block corruption) or fail
// allocation under scheduling jitter.
(*ctx).extra_hw_frames = 4;
let r = ffi::avcodec_open2(ctx, codec, ptr::null_mut());
if r < 0 {
let mut ctx = ctx;
ffi::avcodec_free_context(&mut ctx);
let mut hw_device = hw_device;
ffi::av_buffer_unref(&mut hw_device);
bail!("avcodec_open2: {}", ffmpeg::Error::from(r));
}
Ok(VaapiDecoder {
ctx,
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
})
}
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<DmabufFrame>> {
use ffmpeg::ffi;
unsafe {
let r = ffi::av_new_packet(self.packet, au.len() as i32);
if r < 0 {
return Err(averr("av_new_packet", r));
}
ptr::copy_nonoverlapping(au.as_ptr(), (*self.packet).data, au.len());
let r = ffi::avcodec_send_packet(self.ctx, self.packet);
ffi::av_packet_unref(self.packet);
if r < 0 {
return Err(averr("send_packet", r));
}
let mut out = None;
loop {
let r = ffi::avcodec_receive_frame(self.ctx, self.frame);
if r == AVERROR_EAGAIN {
break;
}
if r < 0 {
return Err(averr("receive_frame", r));
}
out = Some(self.map_dmabuf()?); // newest wins; older guards drop here
ffi::av_frame_unref(self.frame);
}
Ok(out)
}
}
/// Map the VAAPI surface to DRM PRIME (zero copy) and lift the descriptor into a
/// `DmabufFrame`. The mapped frame keeps the surface alive via its buffer refs.
///
/// FFmpeg's VAAPI export uses `VA_EXPORT_SURFACE_SEPARATE_LAYERS`, so an NV12 surface
/// comes back as TWO layers (`R8` luma + `GR88` chroma), each one plane — NOT a single
/// `NV12` layer. The previous code took `layers[0]` only: GTK then saw an `R8`
/// single-plane texture with the chroma dropped, painting the screen green. The fix:
/// derive the COMBINED fourcc from the decoder's software pixel format (NV12 →
/// `DRM_FORMAT_NV12`) and flatten every plane across every layer in order (Y then UV).
unsafe fn map_dmabuf(&mut self) -> Result<DmabufFrame> {
use ffmpeg::ffi;
unsafe {
if (*self.frame).format != ffi::AVPixelFormat::AV_PIX_FMT_VAAPI as i32 {
bail!("decoder returned a software frame (no VAAPI surface)");
}
// The real pixel layout lives on the hardware frames context, not the
// DRM-PRIME layer formats (those are the per-plane R8/GR88 component formats).
let sw_format = {
let hwfc = (*self.frame).hw_frames_ctx;
if hwfc.is_null() {
bail!("VAAPI frame without a hardware frames context");
}
(*((*hwfc).data as *const ffi::AVHWFramesContext)).sw_format
};
let fourcc = drm_fourcc_for(sw_format)
.ok_or_else(|| anyhow!("unsupported VAAPI output format {sw_format:?}"))?;
let drm = ffi::av_frame_alloc();
(*drm).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME as i32;
let r = ffi::av_hwframe_map(drm, self.frame, ffi::AV_HWFRAME_MAP_READ as i32);
if r < 0 {
let mut drm = drm;
ffi::av_frame_free(&mut drm);
return Err(averr("av_hwframe_map", r));
}
let desc = (*drm).data[0] as *const ffi::AVDRMFrameDescriptor;
let guard = DrmFrameGuard(drm);
let d = &*desc;
if d.nb_layers < 1 || d.nb_objects < 1 {
bail!("DRM descriptor without layers/objects");
}
// Flatten planes across ALL layers, in declared order — the combined fourcc's
// plane order (Y, then UV for NV12) matches the layer order FFmpeg emits.
let mut planes = Vec::new();
for layer in &d.layers[..d.nb_layers as usize] {
for p in &layer.planes[..layer.nb_planes as usize] {
let obj = &d.objects[p.object_index as usize];
planes.push(DmabufPlane {
fd: obj.fd,
offset: p.offset as u32,
stride: p.pitch as u32,
});
}
}
// The whole surface shares one tiling modifier (one BO on radeonsi); GTK takes
// a single modifier for the texture.
let modifier = d.objects[0].format_modifier;
log_descriptor_once(d, sw_format, fourcc, modifier);
Ok(DmabufFrame {
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
fourcc,
modifier,
planes,
// SAFETY: `self.frame` is the live decoded AVFrame (unref'd only after
// this returns); plain CICP field reads.
color: ColorDesc::from_raw(self.frame),
keyframe: frame_is_keyframe(self.frame),
guard,
})
}
}
}
/// One-time dump of the DRM descriptor layout (objects, layers, planes, modifier) — so a
/// new client/driver combination's real layout is visible in the logs without a debugger.
#[cfg(target_os = "linux")]
fn log_descriptor_once(
d: &ffmpeg_next::ffi::AVDRMFrameDescriptor,
sw: ffmpeg_next::ffi::AVPixelFormat,
fourcc: u32,
modifier: u64,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if !ONCE.swap(false, Ordering::Relaxed) {
return;
}
let layers: Vec<(u32, i32)> = d.layers[..d.nb_layers.max(0) as usize]
.iter()
.map(|l| (l.format, l.nb_planes))
.collect();
tracing::info!(
sw_format = ?sw,
chosen_fourcc = format_args!("{:#010x}", fourcc),
nb_objects = d.nb_objects,
nb_layers = d.nb_layers,
?layers,
modifier = format_args!("{:#018x}", modifier),
"VAAPI dmabuf descriptor layout (first frame)"
);
}
#[cfg(target_os = "linux")]
impl Drop for VaapiDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
unsafe {
ffi::av_packet_free(&mut self.packet);
ffi::av_frame_free(&mut self.frame);
ffi::avcodec_free_context(&mut self.ctx);
ffi::av_buffer_unref(&mut self.hw_device);
}
}
}
+419
View File
@@ -0,0 +1,419 @@
//! FFmpeg Vulkan Video decode over the presenter's own VkDevice (zero-copy VkImage).
#![allow(clippy::unnecessary_cast)]
use crate::video::{
averr, frame_is_keyframe, DrmFrameGuard, QueueLock, VkVideoFrame, VulkanDecodeDevice,
AVERROR_EAGAIN,
};
use crate::video_color::ColorDesc;
use anyhow::{bail, Result};
use ffmpeg_next as ffmpeg;
use std::ptr;
// --- Vulkan Video backend -------------------------------------------------------------
/// FFmpeg's Vulkan Video decoder over the PRESENTER's device: the hwdevice context is
/// built from [`VulkanDecodeDevice`]'s handles (not `av_hwdevice_ctx_create`, which
/// would make FFmpeg create its own device the presenter can't sample from). Output
/// frames are `AVVkFrame`s whose VkImage the presenter feeds straight to its CSC pass.
pub(crate) struct VulkanDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
/// `vkWaitSemaphores` on the shared device — the decode-complete measurement
/// (resolved through the same get_proc_addr chain FFmpeg uses).
wait_semaphores: pf_ffvk::PFN_vkWaitSemaphores,
vk_device: pf_ffvk::VkDevice,
/// Storage `AVVulkanDeviceContext` points into (extension string arrays + the
/// feature chain) — FFmpeg reads the extension lists past init (frames-context
/// setup keys code paths off them), so this lives exactly as long as `hw_device`.
_ctx_storage: Box<VkCtxStorage>,
}
// Single-owner pointers, only touched from the session pump thread.
unsafe impl Send for VulkanDecoder {}
struct VkCtxStorage {
_inst: Vec<std::ffi::CString>,
inst_ptrs: Vec<*const std::os::raw::c_char>,
_dev: Vec<std::ffi::CString>,
dev_ptrs: Vec<*const std::os::raw::c_char>,
f11: pf_ffvk::VkPhysicalDeviceVulkan11Features,
f12: pf_ffvk::VkPhysicalDeviceVulkan12Features,
f13: pf_ffvk::VkPhysicalDeviceVulkan13Features,
/// Keeps the shared queue lock alive for `AVHWDeviceContext.user_opaque` — the
/// `lock_queue`/`unlock_queue` trampolines below dereference it for as long as the
/// hw device context can fire them.
_queue_lock: std::sync::Arc<QueueLock>,
}
/// FFmpeg `AVVulkanDeviceContext.lock_queue` trampoline: take the device's shared
/// [`QueueLock`] (stashed in `AVHWDeviceContext.user_opaque`; owned by
/// [`VkCtxStorage`], which outlives the context). Replaces FFmpeg's internal default,
/// which only serializes FFmpeg against itself — the presenter submits to the same
/// graphics queue from another thread and holds this same lock around its calls.
unsafe extern "C" fn ffvk_lock_queue(
ctx: *mut pf_ffvk::AVHWDeviceContext,
_queue_family: u32,
_index: u32,
) {
let dev = ctx as *mut ffmpeg::ffi::AVHWDeviceContext;
let lock = (*dev).user_opaque as *const QueueLock;
(*lock).lock();
}
/// The matching `unlock_queue` trampoline — see [`ffvk_lock_queue`].
unsafe extern "C" fn ffvk_unlock_queue(
ctx: *mut pf_ffvk::AVHWDeviceContext,
_queue_family: u32,
_index: u32,
) {
let dev = ctx as *mut ffmpeg::ffi::AVHWDeviceContext;
let lock = (*dev).user_opaque as *const QueueLock;
(*lock).unlock();
}
impl VulkanDecoder {
pub(crate) fn new(
codec_id: ffmpeg::codec::Id,
vk: &VulkanDecodeDevice,
) -> Result<VulkanDecoder> {
use ffmpeg::ffi;
unsafe {
let mut hw_device =
ffi::av_hwdevice_ctx_alloc(ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VULKAN);
if hw_device.is_null() {
bail!("av_hwdevice_ctx_alloc(VULKAN) failed (FFmpeg built without Vulkan?)");
}
let devctx = (*hw_device).data as *mut ffi::AVHWDeviceContext;
let hwctx = (*devctx).hwctx as *mut pf_ffvk::AVVulkanDeviceContext;
// Pinned storage for everything the context points into.
let mut store = Box::new(VkCtxStorage {
_inst: vk.instance_extensions.clone(),
inst_ptrs: Vec::new(),
_dev: vk.device_extensions.clone(),
dev_ptrs: Vec::new(),
f11: std::mem::zeroed(),
f12: std::mem::zeroed(),
f13: std::mem::zeroed(),
_queue_lock: vk.queue_lock.clone(),
});
store.inst_ptrs = store._inst.iter().map(|c| c.as_ptr()).collect();
store.dev_ptrs = store._dev.iter().map(|c| c.as_ptr()).collect();
// The features enabled at device creation, as the 1.1/1.2/1.3 chain FFmpeg
// walks to learn what it may use (sType values are vulkan.h constants).
store.f11.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_1_FEATURES;
store.f11.samplerYcbcrConversion = vk.f_sampler_ycbcr as u32;
store.f12.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_2_FEATURES;
store.f12.timelineSemaphore = vk.f_timeline_semaphore as u32;
store.f13.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_FEATURES;
store.f13.synchronization2 = vk.f_synchronization2 as u32;
store.f11.pNext = &mut store.f12 as *mut _ as *mut std::ffi::c_void;
store.f12.pNext = &mut store.f13 as *mut _ as *mut std::ffi::c_void;
(*hwctx).get_proc_addr = std::mem::transmute::<usize, pf_ffvk::PFN_vkGetInstanceProcAddr>(
vk.get_instance_proc_addr,
);
(*hwctx).inst = vk.instance as pf_ffvk::VkInstance;
(*hwctx).phys_dev = vk.physical_device as pf_ffvk::VkPhysicalDevice;
(*hwctx).act_dev = vk.device as pf_ffvk::VkDevice;
(*hwctx).device_features.sType =
pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2;
(*hwctx).device_features.pNext = &mut store.f11 as *mut _ as *mut std::ffi::c_void;
(*hwctx).enabled_inst_extensions = store.inst_ptrs.as_ptr();
(*hwctx).nb_enabled_inst_extensions = store.inst_ptrs.len() as i32;
(*hwctx).enabled_dev_extensions = store.dev_ptrs.as_ptr();
(*hwctx).nb_enabled_dev_extensions = store.dev_ptrs.len() as i32;
// Queue map: the deprecated per-role indices (tx/comp are "Required") plus
// the qf[] list, which per the header must also carry every family named
// above. One merged entry when decode shares the graphics family.
let g = vk.graphics_qf as i32;
let d = vk.decode_qf as i32;
(*hwctx).queue_family_index = g;
(*hwctx).nb_graphics_queues = 1;
(*hwctx).queue_family_tx_index = g;
(*hwctx).nb_tx_queues = 1;
(*hwctx).queue_family_comp_index = g;
(*hwctx).nb_comp_queues = 1;
(*hwctx).queue_family_encode_index = -1;
(*hwctx).nb_encode_queues = 0;
(*hwctx).queue_family_decode_index = d;
(*hwctx).nb_decode_queues = 1;
const VIDEO_DECODE_BIT: u32 = 0x20; // VK_QUEUE_VIDEO_DECODE_BIT_KHR
// `flags`/`video_caps` are bindgen enum types: i32 under MSVC, u32 under
// Linux clang — the `as _` casts absorb the difference.
if g == d {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: (vk.graphics_queue_flags | VIDEO_DECODE_BIT) as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 1;
} else {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags as _,
video_caps: 0,
};
(*hwctx).qf[1] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: d,
num: 1,
flags: VIDEO_DECODE_BIT as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 2;
}
// Shared-queue external sync (see [`QueueLock`]): FFmpeg must take the
// same lock the presenter holds around its own submits/presents — set
// BEFORE init so FFmpeg never installs its internal defaults (which only
// serialize FFmpeg against itself; the cross-thread race with the
// presenter's queue was an intermittent VK_ERROR_DEVICE_LOST).
(*devctx).user_opaque =
std::sync::Arc::as_ptr(&store._queue_lock) as *mut std::ffi::c_void;
(*hwctx).lock_queue = Some(ffvk_lock_queue);
(*hwctx).unlock_queue = Some(ffvk_unlock_queue);
let r = ffi::av_hwdevice_ctx_init(hw_device);
if r < 0 {
ffi::av_buffer_unref(&mut hw_device);
return Err(averr("av_hwdevice_ctx_init(VULKAN)", r));
}
// vkWaitSemaphores for the pump's decode-complete stat: loader →
// vkGetDeviceProcAddr → device fn (core 1.2, guaranteed by our gate).
let gipa = (*hwctx)
.get_proc_addr
.expect("get_proc_addr was just set above");
let gdpa: pf_ffvk::PFN_vkGetDeviceProcAddr =
std::mem::transmute(gipa((*hwctx).inst, c"vkGetDeviceProcAddr".as_ptr()));
let wait_semaphores: pf_ffvk::PFN_vkWaitSemaphores = std::mem::transmute(gdpa
.expect("vkGetDeviceProcAddr resolvable")(
(*hwctx).act_dev,
c"vkWaitSemaphores".as_ptr(),
));
if wait_semaphores.is_none() {
ffi::av_buffer_unref(&mut hw_device);
bail!("vkWaitSemaphores unresolvable on this device");
}
let vk_device = (*hwctx).act_dev;
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
ffi::av_buffer_unref(&mut hw_device);
bail!("no {codec_id:?} decoder");
}
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device);
(*ctx).get_format = Some(pick_vulkan);
(*ctx).flags |= ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
(*ctx).thread_count = 1; // hwaccel: threads only add latency
// Same pool headroom rationale as VAAPI: the presenter pins the on-screen
// frame + the newest in flight past receive_frame.
(*ctx).extra_hw_frames = 4;
let r = ffi::avcodec_open2(ctx, codec, ptr::null_mut());
if r < 0 {
let mut ctx = ctx;
ffi::avcodec_free_context(&mut ctx);
ffi::av_buffer_unref(&mut hw_device);
return Err(averr("avcodec_open2 (vulkan)", r));
}
Ok(VulkanDecoder {
ctx,
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
wait_semaphores,
vk_device,
_ctx_storage: store,
})
}
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<VkVideoFrame>> {
use ffmpeg::ffi;
unsafe {
let r = ffi::av_new_packet(self.packet, au.len() as i32);
if r < 0 {
return Err(averr("av_new_packet", r));
}
ptr::copy_nonoverlapping(au.as_ptr(), (*self.packet).data, au.len());
let r = ffi::avcodec_send_packet(self.ctx, self.packet);
ffi::av_packet_unref(self.packet);
if r < 0 {
return Err(averr("send_packet", r));
}
let mut out = None;
loop {
let r = ffi::avcodec_receive_frame(self.ctx, self.frame);
if r == AVERROR_EAGAIN {
break;
}
if r < 0 {
return Err(averr("receive_frame", r));
}
out = Some(self.extract()?); // newest wins; older guards drop here
ffi::av_frame_unref(self.frame);
}
Ok(out)
}
}
/// Block until the timeline semaphore reaches `value` (GPU decode complete) or the
/// timeout passes. Pure measurement — the presenter's own GPU wait is what gates
/// sampling, so a timeout here only degrades the stat, never the picture.
pub(crate) fn wait_timeline(&self, sem: u64, value: u64, timeout_ns: u64) -> bool {
let sems = [sem as pf_ffvk::VkSemaphore];
let values = [value];
let info = pf_ffvk::VkSemaphoreWaitInfo {
sType: pf_ffvk::VkStructureType_VK_STRUCTURE_TYPE_SEMAPHORE_WAIT_INFO,
pNext: std::ptr::null(),
flags: 0,
semaphoreCount: 1,
pSemaphores: sems.as_ptr(),
pValues: values.as_ptr(),
};
// SAFETY: resolved from this device at init; handles outlive the decoder.
let r = unsafe {
self.wait_semaphores.expect("checked at init")(self.vk_device, &info, timeout_ns)
};
r == 0 // VK_SUCCESS (VK_TIMEOUT = 2)
}
/// Lift the decoded `AVVkFrame` into a [`VkVideoFrame`]: clone the AVFrame (the
/// guard — keeps the image + frames context alive through present) and ship the
/// POINTERS; the presenter reads the live sync state under the frames-context lock
/// at its own submit time.
unsafe fn extract(&mut self) -> Result<VkVideoFrame> {
use ffmpeg::ffi;
unsafe {
if (*self.frame).format != ffi::AVPixelFormat::AV_PIX_FMT_VULKAN as i32 {
bail!("decoder returned a non-Vulkan frame");
}
let hwfc_ref = (*self.frame).hw_frames_ctx;
if hwfc_ref.is_null() {
bail!("Vulkan frame without a hardware frames context");
}
let fc = (*hwfc_ref).data as *mut ffi::AVHWFramesContext;
let sw = (*fc).sw_format;
if sw != ffi::AVPixelFormat::AV_PIX_FMT_NV12
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P010LE
{
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010 only)");
}
let vkfc = (*fc).hwctx as *const pf_ffvk::AVVulkanFramesContext;
let vk_format = (*vkfc).format[0] as i32;
let lock_frame = (*vkfc).lock_frame.map_or(0, |f| f as usize);
let unlock_frame = (*vkfc).unlock_frame.map_or(0, |f| f as usize);
if lock_frame == 0 || unlock_frame == 0 {
bail!("Vulkan frames context without lock functions");
}
let clone = ffi::av_frame_clone(self.frame);
if clone.is_null() {
bail!("av_frame_clone failed");
}
let vkf = (*clone).data[0] as *mut pf_ffvk::AVVkFrame;
// v1 handles the (default) single multiplanar image; a disjoint/multi-image
// pool would need per-plane images — bail so the session demotes cleanly.
if !(*vkf).img[1].is_null() {
let mut clone = clone;
ffi::av_frame_free(&mut clone);
bail!("multi-image Vulkan frames unsupported (disjoint pool)");
}
// Safe without the frames lock: the handle is creation-constant and
// sem_value was last written by the decode submission on THIS thread.
let timeline_sem = (*vkf).sem[0] as u64;
let decode_done_value = (*vkf).sem_value[0];
Ok(VkVideoFrame {
vkframe: vkf as usize,
frames_ctx: fc as usize,
lock_frame,
unlock_frame,
vk_format,
timeline_sem,
decode_done_value,
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
color: ColorDesc::from_raw(self.frame),
keyframe: frame_is_keyframe(self.frame),
guard: DrmFrameGuard(clone),
})
}
}
}
impl Drop for VulkanDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
unsafe {
ffi::av_packet_free(&mut self.packet);
ffi::av_frame_free(&mut self.frame);
ffi::avcodec_free_context(&mut self.ctx);
ffi::av_buffer_unref(&mut self.hw_device);
}
}
}
/// libavcodec offers the formats it can decode into; pick the Vulkan hw surface and
/// hand the decoder OUR frames context — the default one lacks
/// `VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT`, without which the presenter can't create the
/// per-plane views its CSC pass samples. Returning NONE (over the software entry) keeps
/// failures loud: the session demotes explicitly instead of silently CPU-decoding.
unsafe extern "C" fn pick_vulkan(
ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
) -> ffmpeg::ffi::AVPixelFormat {
use ffmpeg::ffi;
unsafe {
let mut offered = false;
while *list != ffi::AVPixelFormat::AV_PIX_FMT_NONE {
if *list == ffi::AVPixelFormat::AV_PIX_FMT_VULKAN {
offered = true;
break;
}
list = list.add(1);
}
if !offered {
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
let mut fr: *mut ffi::AVBufferRef = ptr::null_mut();
let r = ffi::avcodec_get_hw_frames_parameters(
ctx,
(*ctx).hw_device_ctx,
ffi::AVPixelFormat::AV_PIX_FMT_VULKAN,
&mut fr,
);
if r < 0 || fr.is_null() {
tracing::warn!(code = r, "avcodec_get_hw_frames_parameters(VULKAN) failed");
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
let fc = (*fr).data as *mut ffi::AVHWFramesContext;
let vkfc = (*fc).hwctx as *mut pf_ffvk::AVVulkanFramesContext;
// MUTABLE_FORMAT: per-plane views (spec requirement); ALIAS is FFmpeg's default.
// (`as _`: the FlagBits constants are i32 under MSVC, the img_flags field u32.)
(*vkfc).img_flags = (pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT)
as _;
let r = ffi::av_hwframe_ctx_init(fr);
if r < 0 {
tracing::warn!(code = r, "av_hwframe_ctx_init(VULKAN) failed");
let mut fr = fr;
ffi::av_buffer_unref(&mut fr);
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
if !(*ctx).hw_frames_ctx.is_null() {
ffi::av_buffer_unref(&mut (*ctx).hw_frames_ctx);
}
(*ctx).hw_frames_ctx = fr; // the codec owns our ref now
ffi::AVPixelFormat::AV_PIX_FMT_VULKAN
}
}
+47
View File
@@ -0,0 +1,47 @@
# Shared clipboard (plan §W6 shape, design/clipboard-and-file-transfer.md §4): the host-side
# session-clipboard backends — `ext-data-control-v1` (KWin/wlroots/Sway/Hyprland) and Mutter's
# *direct* `org.gnome.Mutter.RemoteDesktop.Session` clipboard on Linux, the Win32 clipboard
# (delayed rendering) on Windows — behind one `HostClipboard`, plus the backend-agnostic session
# coordinator bridging it to the QUIC clipboard plane. The wire protocol and the client half live
# in `punktfunk-core`; the orchestrator consumes only the portable facade (policy / `ClipCoordCmd` /
# `start`), so it stays free of platform cfg.
[package]
name = "pf-clipboard"
version = "0.12.0"
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
description = "punktfunk host shared clipboard: per-OS session-clipboard backends behind one HostClipboard + the QUIC clipboard-plane coordinator."
publish = false
[dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
anyhow = "1"
tracing = "0.1"
quinn = "0.11"
tokio = { version = "1", features = ["rt", "rt-multi-thread", "sync", "time", "macros"] }
[target.'cfg(target_os = "linux")'.dependencies]
# Mutter's direct RemoteDesktop clipboard is raw D-Bus via `ashpd::zbus` — NOT the xdg
# `org.freedesktop.portal.Clipboard`, which needs an interactive grant a headless host can't
# answer. Reusing ashpd's zbus re-export keeps one zbus version across the workspace.
ashpd = "0.13"
futures-util = "0.3"
# Raw fd plumbing on the paste pipes: `pipe2(O_CLOEXEC)` + `poll` on the data-control receive
# side, `fcntl` un-nonblocking on Mutter's transfer fd.
libc = "0.2"
wayland-client = "0.31"
# `staging`: `ext_data_control_v1` (the session-clipboard protocol) ships in the staging set.
wayland-protocols = { version = "0.32", features = ["client", "staging"] }
[target.'cfg(target_os = "windows")'.dependencies]
# The Win32 clipboard on a hidden message-loop window: `WM_CLIPBOARDUPDATE` listener + OLE
# delayed rendering (`WM_RENDERFORMAT`) for text / CF_HTML / RTF / PNG.
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_System_DataExchange",
"Win32_System_LibraryLoader",
"Win32_System_Memory",
"Win32_System_Ole",
"Win32_UI_WindowsAndMessaging",
] }
+409
View File
@@ -0,0 +1,409 @@
//! Host-side shared-clipboard backend.
//!
//! The wire protocol and the client half live in `punktfunk-core`
//! (`punktfunk_core::quic` + `punktfunk_core::clipboard`); this module drives the **host's** real
//! session clipboard so it can offer what a host app copied and paste what the remote client
//! offered (`design/clipboard-and-file-transfer.md` §4).
//!
//! Concrete backends, selected at session start ([`HostClipboard::open`]) and presented as one
//! [`HostClipboard`] to the [`session`] coordinator:
//! * [`wayland`] (Linux) — `ext-data-control-v1` (KWin, wlroots / Sway, Hyprland). Preferred when present.
//! * [`mutter`] (Linux) — GNOME. Mutter implements **no** wlr/ext data-control, but its *direct*
//! `org.gnome.Mutter.RemoteDesktop.Session` D-Bus API carries the same clipboard operations (the
//! xdg `org.freedesktop.portal.Clipboard` would need an interactive grant a headless host can't
//! answer — so we skip it and talk to Mutter directly, as the input injector already does).
//! * [`windows`] — the Win32 clipboard: a hidden message-only window watches `WM_CLIPBOARDUPDATE`
//! and serves client content via OLE delayed rendering (`WM_RENDERFORMAT`).
//!
//! The `zwlr-data-control-unstable-v1` fallback (older wlroots/KWin) is a follow-up. The module
//! compiles on Linux and Windows; the [`session`] coordinator is backend-agnostic.
#[cfg(target_os = "linux")]
mod mutter;
#[cfg(target_os = "linux")]
mod wayland;
#[cfg(target_os = "windows")]
mod windows;
/// Pure Win32-clipboard ↔ wire byte conversions (CF_HTML offset math, UTF-16 text, RTF NUL
/// trimming). Free of any Win32 dependency, so it compiles — and its unit tests run — on any host
/// (`cfg(test)`); the Windows backend is the only production consumer.
#[cfg(any(target_os = "windows", test))]
mod winfmt;
pub mod session;
#[cfg(target_os = "linux")]
use std::io::Write as _;
#[cfg(target_os = "linux")]
use std::os::fd::OwnedFd;
use std::sync::Arc;
/// A clipboard event surfaced by a host backend to the [`session`] coordinator. Both the
/// data-control and Mutter backends emit this identical shape.
pub enum ClipEvent {
/// The host selection changed (a host app copied). `mimes` are the **wire** MIMEs offered (empty
/// = the clipboard was cleared). The coordinator forwards these as a `ClipOffer` to the client;
/// bytes cross only if the client later fetches.
Selection { mimes: Vec<String> },
/// A host app is pasting content the client offered. The coordinator fetches the wire-`mime`
/// bytes from the client and hands them to `responder`.
Paste {
mime: String,
responder: PasteResponder,
},
/// The backend ended (compositor / session gone).
Closed,
}
/// How a backend receives the bytes answering a [`ClipEvent::Paste`]. The two host clipboard
/// mechanisms complete a paste differently, so the coordinator stays agnostic by handing bytes to
/// whichever responder the backend attached.
pub enum PasteResponder {
/// data-control: the compositor handed us the destination pipe on the `send` event — write the
/// bytes and close it (EOF completes the paste).
#[cfg(target_os = "linux")]
Fd(OwnedFd),
/// Mutter: hand the bytes back to the backend actor, which owns the `SelectionWrite` fd and the
/// trailing `SelectionWriteDone` call that Mutter's transfer requires.
#[cfg(target_os = "linux")]
Channel(tokio::sync::oneshot::Sender<Vec<u8>>),
/// Windows: hand the bytes to the `WM_RENDERFORMAT` handler blocking the clipboard message-loop
/// thread, which then `SetClipboardData`s them for the pasting app (`std::sync::mpsc`, since that
/// thread waits synchronously — see [`windows`]).
#[cfg(target_os = "windows")]
Sync(std::sync::mpsc::Sender<Vec<u8>>),
}
impl PasteResponder {
/// Deliver the fetched bytes (empty on a failed fetch → an empty paste, never a hang).
pub async fn respond(self, bytes: Vec<u8>) {
match self {
#[cfg(target_os = "linux")]
PasteResponder::Fd(fd) => {
let _ = tokio::task::spawn_blocking(move || fulfill_paste(fd, &bytes)).await;
}
#[cfg(target_os = "linux")]
PasteResponder::Channel(tx) => {
let _ = tx.send(bytes);
}
#[cfg(target_os = "windows")]
PasteResponder::Sync(tx) => {
let _ = tx.send(bytes);
}
}
}
}
/// Write `bytes` into a paste pipe `fd` and close it (EOF signals the reader). Blocking — run off the
/// reactor for large payloads.
#[cfg(target_os = "linux")]
fn fulfill_paste(fd: OwnedFd, bytes: &[u8]) -> std::io::Result<()> {
let mut file = std::fs::File::from(fd);
file.write_all(bytes)?;
Ok(())
}
/// The active host clipboard backend, chosen per session: `ext-data-control`
/// (KWin/wlroots/Hyprland/Sway) or Mutter's direct RemoteDesktop clipboard (GNOME) on Linux, or the
/// Win32 clipboard on Windows. Presented as one type so the [`session`] coordinator is
/// backend-agnostic.
pub enum HostClipboard {
#[cfg(target_os = "linux")]
DataControl(wayland::ClipboardBackend),
#[cfg(target_os = "linux")]
Mutter(mutter::MutterClipboard),
#[cfg(target_os = "windows")]
Windows(windows::WindowsClipboard),
}
impl HostClipboard {
/// Open whichever backend this session supports. Linux tries data-control first
/// (KWin/wlroots/Hyprland/Sway) then Mutter's direct clipboard (GNOME); Windows opens the Win32
/// clipboard. Errors when none is available (gamescope, no live compositor) — the caller then
/// reports `BACKEND_UNAVAILABLE`.
pub async fn open() -> anyhow::Result<(
HostClipboard,
tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
)> {
#[cfg(target_os = "linux")]
{
// data-control's bind does blocking Wayland roundtrips — keep them off the reactor.
let dc = tokio::task::spawn_blocking(wayland::ClipboardBackend::open)
.await
.map_err(|e| anyhow::anyhow!("data-control open join: {e}"))?;
match dc {
Ok((b, rx)) => return Ok((HostClipboard::DataControl(b), rx)),
Err(e) => tracing::debug!(
error = format!("{e:#}"),
"no ext-data-control — trying Mutter direct clipboard"
),
}
let (m, rx) = mutter::MutterClipboard::open().await.map_err(|e| {
e.context("no clipboard backend (neither ext-data-control nor Mutter)")
})?;
Ok((HostClipboard::Mutter(m), rx))
}
#[cfg(target_os = "windows")]
{
let (b, rx) = windows::WindowsClipboard::open().await?;
Ok((HostClipboard::Windows(b), rx))
}
}
/// The current host selection's wire MIMEs (empty = nothing to offer).
pub fn current_wire_mimes(&self) -> Vec<String> {
match self {
#[cfg(target_os = "linux")]
HostClipboard::DataControl(b) => b.current_wire_mimes(),
#[cfg(target_os = "linux")]
HostClipboard::Mutter(m) => m.current_wire_mimes(),
#[cfg(target_os = "windows")]
HostClipboard::Windows(w) => w.current_wire_mimes(),
}
}
/// Install a client's offered formats as the host selection.
pub fn set_offer(&self, wire_mimes: &[String]) -> anyhow::Result<()> {
match self {
#[cfg(target_os = "linux")]
HostClipboard::DataControl(b) => b.set_offer(wire_mimes),
#[cfg(target_os = "linux")]
HostClipboard::Mutter(m) => {
m.set_offer(wire_mimes);
Ok(())
}
#[cfg(target_os = "windows")]
HostClipboard::Windows(w) => {
w.set_offer(wire_mimes);
Ok(())
}
}
}
/// Drop the host selection we own.
pub fn clear_offer(&self) -> anyhow::Result<()> {
match self {
#[cfg(target_os = "linux")]
HostClipboard::DataControl(b) => b.clear_offer(),
#[cfg(target_os = "linux")]
HostClipboard::Mutter(m) => {
m.clear_offer();
Ok(())
}
#[cfg(target_os = "windows")]
HostClipboard::Windows(w) => {
w.clear_offer();
Ok(())
}
}
}
/// Read one wire format of the current host selection (a client's fetch). Async: data-control
/// blocks on a pipe (offloaded), Mutter round-trips D-Bus + reads a pipe, Windows reads the
/// clipboard on a blocking thread.
pub async fn read_current(self: &Arc<Self>, wire_mime: &str) -> anyhow::Result<Vec<u8>> {
match &**self {
#[cfg(target_os = "linux")]
HostClipboard::DataControl(_) => {
let me = Arc::clone(self);
let wire = wire_mime.to_string();
tokio::task::spawn_blocking(move || match &*me {
HostClipboard::DataControl(b) => b.read_current(&wire),
_ => unreachable!("variant checked above"),
})
.await
.map_err(|e| anyhow::anyhow!("data-control read join: {e}"))?
}
#[cfg(target_os = "linux")]
HostClipboard::Mutter(m) => m.read_current(wire_mime).await,
#[cfg(target_os = "windows")]
HostClipboard::Windows(w) => w.read_current(wire_mime).await,
}
}
}
// ---- Format normalization (design/clipboard-and-file-transfer.md §3.5) ------------------------
//
// One portable vocabulary crosses the wire; each end maps to platform types at fetch time. Phase 1
// covers text / RTF / HTML / PNG (files are Phase 2). The wire MIMEs match the core's table.
/// Wire MIME for UTF-8 plain text.
pub const WIRE_TEXT: &str = "text/plain;charset=utf-8";
/// Wire MIME for HTML.
pub const WIRE_HTML: &str = "text/html";
/// Wire MIME for rich text.
pub const WIRE_RTF: &str = "text/rtf";
/// Wire MIME for a PNG image.
pub const WIRE_PNG: &str = "image/png";
/// Map a Wayland selection MIME to its canonical wire MIME, or `None` to drop it (internal targets
/// like `TARGETS`/`TIMESTAMP`/`SAVE_TARGETS`, and formats we don't sync in Phase 1). Aliases
/// collapse onto one canonical wire name so the offered list dedups cleanly.
#[cfg(target_os = "linux")]
pub fn wayland_to_wire(wl: &str) -> Option<&'static str> {
// Strip any parameter noise for the plain-text aliases (some apps send `text/plain;charset=...`
// with odd charsets, or bare `text/plain`).
let base = wl.split(';').next().unwrap_or(wl).trim();
match wl {
"text/html" => Some(WIRE_HTML),
"text/rtf" | "application/rtf" | "text/richtext" => Some(WIRE_RTF),
"image/png" => Some(WIRE_PNG),
_ => match base {
"text/plain" | "UTF8_STRING" | "STRING" | "TEXT" => Some(WIRE_TEXT),
_ => None,
},
}
}
/// The Wayland MIME candidates to request, in preference order, when a client fetches `wire` from
/// the host clipboard. The first one present in the current offer is used.
#[cfg(target_os = "linux")]
pub fn wayland_candidates(wire: &str) -> &'static [&'static str] {
match wire {
WIRE_TEXT => &[
"text/plain;charset=utf-8",
"text/plain",
"UTF8_STRING",
"STRING",
"TEXT",
],
WIRE_HTML => &["text/html"],
WIRE_RTF => &["text/rtf", "application/rtf", "text/richtext"],
WIRE_PNG => &["image/png"],
_ => &[],
}
}
/// Pick the Wayland MIME to `receive()` for a wire fetch: the first [`wayland_candidates`] entry the
/// current selection actually advertises.
#[cfg(target_os = "linux")]
pub fn pick_wayland_mime(wire: &str, available: &[String]) -> Option<String> {
wayland_candidates(wire)
.iter()
.find(|c| available.iter().any(|a| a == *c))
.map(|c| c.to_string())
}
/// Normalize a raw Wayland offer's MIME list into the deduplicated wire MIME list announced to the
/// client (drops internal targets; collapses aliases; preserves a stable order).
#[cfg(target_os = "linux")]
pub fn offer_wire_mimes(raw: &[String]) -> Vec<&'static str> {
let mut out: Vec<&'static str> = Vec::new();
for m in raw {
if let Some(wire) = wayland_to_wire(m) {
if !out.contains(&wire) {
out.push(wire);
}
}
}
out
}
/// The Wayland MIMEs to advertise when installing a source for a client's offer. Each wire MIME
/// expands to its canonical Wayland name(s); a rich-text-only offer also advertises `text/plain`
/// so plain-text targets always paste (§3.5 synthesis — destination-side, one direction only).
#[cfg(target_os = "linux")]
pub fn wayland_offers_for(wire_mimes: &[String]) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let mut push = |s: &str| {
if !out.iter().any(|o| o == s) {
out.push(s.to_string());
}
};
let mut has_plain = false;
let mut has_rich = false;
for w in wire_mimes {
match w.as_str() {
WIRE_TEXT => {
has_plain = true;
push("text/plain;charset=utf-8");
push("text/plain");
push("UTF8_STRING");
push("STRING");
}
WIRE_HTML => {
has_rich = true;
push("text/html");
}
WIRE_RTF => {
has_rich = true;
push("text/rtf");
}
WIRE_PNG => push("image/png"),
other => push(other),
}
}
// Synthesis: rich text without plain text → also advertise plain (the source derives it lazily).
if has_rich && !has_plain {
push("text/plain;charset=utf-8");
push("text/plain");
push("UTF8_STRING");
push("STRING");
}
out
}
#[cfg(all(test, target_os = "linux"))]
mod tests {
use super::*;
#[test]
fn wayland_to_wire_canonicalizes_and_drops_targets() {
assert_eq!(wayland_to_wire("text/plain"), Some(WIRE_TEXT));
assert_eq!(wayland_to_wire("UTF8_STRING"), Some(WIRE_TEXT));
assert_eq!(wayland_to_wire("text/plain;charset=utf-8"), Some(WIRE_TEXT));
assert_eq!(wayland_to_wire("text/html"), Some(WIRE_HTML));
assert_eq!(wayland_to_wire("application/rtf"), Some(WIRE_RTF));
assert_eq!(wayland_to_wire("image/png"), Some(WIRE_PNG));
// Internal targets and unsupported formats are dropped.
assert_eq!(wayland_to_wire("TARGETS"), None);
assert_eq!(wayland_to_wire("TIMESTAMP"), None);
assert_eq!(wayland_to_wire("image/jpeg"), None);
}
#[test]
fn offer_wire_mimes_dedups_aliases() {
let raw = vec![
"TARGETS".to_string(),
"UTF8_STRING".to_string(),
"text/plain;charset=utf-8".to_string(),
"text/plain".to_string(),
"text/html".to_string(),
];
// text aliases collapse to one WIRE_TEXT; TARGETS dropped; html kept.
assert_eq!(offer_wire_mimes(&raw), vec![WIRE_TEXT, WIRE_HTML]);
}
#[test]
fn pick_wayland_mime_prefers_canonical() {
let avail = vec!["text/plain".to_string(), "UTF8_STRING".to_string()];
// Canonical charset form isn't present, so it falls to the next candidate.
assert_eq!(
pick_wayland_mime(WIRE_TEXT, &avail),
Some("text/plain".to_string())
);
let avail2 = vec![
"text/plain;charset=utf-8".to_string(),
"text/plain".to_string(),
];
assert_eq!(
pick_wayland_mime(WIRE_TEXT, &avail2),
Some("text/plain;charset=utf-8".to_string())
);
assert_eq!(pick_wayland_mime(WIRE_PNG, &avail2), None);
}
#[test]
fn wayland_offers_synthesizes_plain_for_rich_only() {
let offers = wayland_offers_for(&[WIRE_HTML.to_string()]);
assert!(offers.iter().any(|m| m == "text/html"));
assert!(
offers.iter().any(|m| m == "text/plain;charset=utf-8"),
"rich-only offer must synthesize plain text: {offers:?}"
);
// Plain already present → no duplicate synthesis, and text aliases included.
let offers2 = wayland_offers_for(&[WIRE_TEXT.to_string()]);
assert!(offers2.iter().any(|m| m == "UTF8_STRING"));
assert_eq!(offers2.iter().filter(|m| *m == "text/plain").count(), 1);
}
}
+524
View File
@@ -0,0 +1,524 @@
//! GNOME clipboard backend via Mutter's **direct** `org.gnome.Mutter.RemoteDesktop.Session` D-Bus
//! API (`design/clipboard-and-file-transfer.md` §4.1).
//!
//! Mutter implements no wlr/ext `data-control` (a deliberate privacy stance), so [`super::wayland`]
//! can't bind on GNOME. But Mutter's own RemoteDesktop session — the same interface the input
//! injector drives directly to dodge the xdg-portal approval dialog (`inject/linux/libei.rs`
//! `connect_mutter`) — carries the full clipboard surface: `EnableClipboard`, `SetSelection`,
//! `SelectionRead`/`SelectionWrite`/`SelectionWriteDone`, and the `SelectionOwnerChanged` /
//! `SelectionTransfer` signals. We open our **own** standalone session for it (it coexists with the
//! injector's input session; validated on GNOME 50), so this backend is self-contained just like the
//! data-control one.
//!
//! One actor task owns the zbus connection + session and multiplexes the two signals with a command
//! channel; the fds Mutter hands out are **non-blocking**, so reads/writes flip them to blocking and
//! run on a blocking thread. Option/signal dict keys are hyphenated: `mime-types`, `session-is-owner`.
use std::collections::HashMap;
use std::io::{Read as _, Write as _};
use std::os::fd::{AsRawFd, OwnedFd};
use std::sync::{Arc, Mutex};
use anyhow::{anyhow, Context, Result};
use ashpd::zbus::{
self,
zvariant::{OwnedObjectPath, OwnedValue, Value},
};
use futures_util::StreamExt;
use tokio::sync::{mpsc, oneshot};
use super::{ClipEvent, PasteResponder};
const RD_BUS: &str = "org.gnome.Mutter.RemoteDesktop";
const RD_PATH: &str = "/org/gnome/Mutter/RemoteDesktop";
const RD_IFACE: &str = "org.gnome.Mutter.RemoteDesktop";
const SESSION_IFACE: &str = "org.gnome.Mutter.RemoteDesktop.Session";
/// Upper bound on one `SelectionRead` (matches the wire clipboard cap, §7).
const CLIP_READ_CAP: u64 = 64 << 20;
/// Handle to the Mutter clipboard actor, held (inside a [`super::HostClipboard`]) by the session
/// coordinator.
pub struct MutterClipboard {
cmd_tx: mpsc::UnboundedSender<Cmd>,
/// Raw MIMEs the current host selection advertises (empty when we own it, or nothing is copied).
/// Written by the actor on `SelectionOwnerChanged`; read for `current_wire_mimes` / fetches.
current_raw: Arc<Mutex<Vec<String>>>,
}
enum Cmd {
SetOffer(Vec<String>),
ClearOffer,
ReadCurrent {
wire: String,
reply: oneshot::Sender<Result<Vec<u8>>>,
},
}
impl MutterClipboard {
/// Create a standalone Mutter RemoteDesktop session, `Start` + `EnableClipboard` it, and spawn
/// the actor. Errors when Mutter isn't running (not GNOME) — the caller falls through to
/// `BACKEND_UNAVAILABLE`.
pub async fn open() -> Result<(MutterClipboard, mpsc::UnboundedReceiver<ClipEvent>)> {
let conn = zbus::Connection::session()
.await
.context("session D-Bus (Mutter clipboard)")?;
let rd = zbus::Proxy::new(&conn, RD_BUS, RD_PATH, RD_IFACE)
.await
.context("Mutter RemoteDesktop proxy (is gnome-shell running?)")?;
let session_path: OwnedObjectPath = rd
.call("CreateSession", &())
.await
.context("Mutter RemoteDesktop.CreateSession (clipboard)")?;
let session = zbus::Proxy::new(&conn, RD_BUS, session_path, SESSION_IFACE)
.await
.context("Mutter RemoteDesktop.Session proxy")?;
session
.call_method("Start", &())
.await
.context("Mutter RemoteDesktop.Session.Start (clipboard)")?;
let empty: HashMap<&str, Value> = HashMap::new();
session
.call_method("EnableClipboard", &(empty,))
.await
.context("Mutter EnableClipboard")?;
let (event_tx, event_rx) = mpsc::unbounded_channel();
let (cmd_tx, cmd_rx) = mpsc::unbounded_channel();
let current_raw = Arc::new(Mutex::new(Vec::new()));
tokio::spawn(actor(conn, session, cmd_rx, event_tx, current_raw.clone()));
tracing::info!("clipboard backend bound (Mutter RemoteDesktop direct)");
Ok((
MutterClipboard {
cmd_tx,
current_raw,
},
event_rx,
))
}
/// Install a client's offered formats as the host selection (fire-and-forget on the actor).
pub fn set_offer(&self, wire_mimes: &[String]) {
let _ = self.cmd_tx.send(Cmd::SetOffer(wire_mimes.to_vec()));
}
/// Relinquish the selection we own.
pub fn clear_offer(&self) {
let _ = self.cmd_tx.send(Cmd::ClearOffer);
}
/// The current host selection's wire MIMEs (empty = nothing / we own it).
pub fn current_wire_mimes(&self) -> Vec<String> {
super::offer_wire_mimes(&self.current_raw.lock().unwrap())
.into_iter()
.map(str::to_string)
.collect()
}
/// Read one wire format of the current host selection (a client's fetch). Round-trips the actor
/// (SelectionRead + a blocking fd read).
pub async fn read_current(&self, wire: &str) -> Result<Vec<u8>> {
let (reply, rx) = oneshot::channel();
self.cmd_tx
.send(Cmd::ReadCurrent {
wire: wire.to_string(),
reply,
})
.map_err(|_| anyhow!("Mutter clipboard actor gone"))?;
rx.await
.map_err(|_| anyhow!("Mutter clipboard read dropped"))?
}
}
/// The actor: owns the connection + session, subscribes to the two clipboard signals, and serves
/// commands. Exits when the command channel closes (session ending) or a signal stream ends.
async fn actor(
conn: zbus::Connection,
session: zbus::Proxy<'static>,
mut cmd_rx: mpsc::UnboundedReceiver<Cmd>,
event_tx: mpsc::UnboundedSender<ClipEvent>,
current_raw: Arc<Mutex<Vec<String>>>,
) {
let (mut owner, mut transfer) = match (
session.receive_signal("SelectionOwnerChanged").await,
session.receive_signal("SelectionTransfer").await,
) {
(Ok(o), Ok(t)) => (o, t),
_ => {
tracing::warn!("Mutter clipboard: could not subscribe to selection signals");
let _ = event_tx.send(ClipEvent::Closed);
return;
}
};
loop {
tokio::select! {
sig = owner.next() => {
let Some(msg) = sig else { break };
let Ok((opts,)) = msg.body().deserialize::<(HashMap<String, OwnedValue>,)>() else {
continue;
};
let is_owner = dict_bool(&opts, "session-is-owner").unwrap_or(false);
let raw = dict_mimes(&opts, "mime-types");
if is_owner {
// Our own offer (the client's content) — not host clipboard; don't announce it,
// and clear `current_raw` so a fetch never reads our own source back.
current_raw.lock().unwrap().clear();
} else {
*current_raw.lock().unwrap() = raw.clone();
let wire = super::offer_wire_mimes(&raw)
.into_iter()
.map(str::to_string)
.collect();
if event_tx.send(ClipEvent::Selection { mimes: wire }).is_err() {
break;
}
}
}
sig = transfer.next() => {
let Some(msg) = sig else { break };
let Ok((mime, serial)) = msg.body().deserialize::<(String, u32)>() else {
continue;
};
match super::wayland_to_wire(&mime) {
Some(wire) => {
// A host app pastes our offer: hand the fetch to the coordinator, then serve
// the returned bytes into the SelectionWrite fd off-task. NB Mutter issues
// *two* transfers per read (a size probe + the real read), so the coordinator
// fetches from the client twice per paste — correct, just not deduplicated.
let (tx, rx) = oneshot::channel();
if event_tx
.send(ClipEvent::Paste {
mime: wire.to_string(),
responder: PasteResponder::Channel(tx),
})
.is_err()
{
break;
}
let session = session.clone();
tokio::spawn(async move {
let bytes = rx.await.unwrap_or_default();
serve_write(&session, serial, bytes).await;
});
}
// Format we don't sync — fail the transfer cleanly.
None => serve_write(&session, serial, Vec::new()).await,
}
}
cmd = cmd_rx.recv() => {
let Some(cmd) = cmd else { break }; // coordinator gone → session ending
match cmd {
Cmd::SetOffer(wire) => {
let wl = super::wayland_offers_for(&wire);
if let Err(e) = set_selection(&session, &wl).await {
tracing::debug!(error = %e, "Mutter SetSelection failed");
}
}
Cmd::ClearOffer => {
if let Err(e) = set_selection(&session, &[]).await {
tracing::debug!(error = %e, "Mutter clear selection failed");
}
}
Cmd::ReadCurrent { wire, reply } => {
let raw = current_raw.lock().unwrap().clone();
let _ = reply.send(read_selection(&session, &wire, &raw).await);
}
}
}
}
}
// Keep the connection owned for the actor's whole life (Mutter ties the session to it).
drop(conn);
let _ = event_tx.send(ClipEvent::Closed);
}
/// Offer `wl_mimes` as the host selection; an empty list relinquishes ownership.
async fn set_selection(session: &zbus::Proxy<'_>, wl_mimes: &[String]) -> Result<()> {
let mut opts: HashMap<&str, Value> = HashMap::new();
if !wl_mimes.is_empty() {
let refs: Vec<&str> = wl_mimes.iter().map(String::as_str).collect();
opts.insert("mime-types", Value::from(refs));
}
session
.call_method("SetSelection", &(opts,))
.await
.context("Mutter SetSelection")?;
Ok(())
}
/// Read the current selection's `wire` format: pick a concrete offered MIME, `SelectionRead` it, and
/// read the (non-blocking) fd to EOF on a blocking thread.
async fn read_selection(session: &zbus::Proxy<'_>, wire: &str, raw: &[String]) -> Result<Vec<u8>> {
let mime =
super::pick_wayland_mime(wire, raw).context("format not offered by the host clipboard")?;
let fd: zbus::zvariant::OwnedFd = session
.call("SelectionRead", &(mime.as_str(),))
.await
.context("Mutter SelectionRead")?;
let fd = OwnedFd::from(fd);
tokio::task::spawn_blocking(move || read_fd_to_end(fd))
.await
.map_err(|e| anyhow!("SelectionRead join: {e}"))?
}
/// Serve one `SelectionTransfer`: `SelectionWrite` → write `bytes` → `SelectionWriteDone`. Any write
/// failure still reports done (success=false) so Mutter completes the transfer.
async fn serve_write(session: &zbus::Proxy<'_>, serial: u32, bytes: Vec<u8>) {
let ok = match write_selection(session, serial, bytes).await {
Ok(()) => true,
Err(e) => {
tracing::debug!(error = %e, "Mutter SelectionWrite failed");
false
}
};
let _ = session
.call_method("SelectionWriteDone", &(serial, ok))
.await;
}
async fn write_selection(session: &zbus::Proxy<'_>, serial: u32, bytes: Vec<u8>) -> Result<()> {
let fd: zbus::zvariant::OwnedFd = session
.call("SelectionWrite", &(serial,))
.await
.context("Mutter SelectionWrite")?;
let fd = OwnedFd::from(fd);
tokio::task::spawn_blocking(move || write_fd(fd, &bytes))
.await
.map_err(|e| anyhow!("SelectionWrite join: {e}"))?
}
/// Read a Mutter clipboard fd to EOF (capped). The fd is `O_NONBLOCK`; flip it to blocking first.
fn read_fd_to_end(fd: OwnedFd) -> Result<Vec<u8>> {
set_blocking(&fd)?;
let file = std::fs::File::from(fd);
let mut buf = Vec::new();
file.take(CLIP_READ_CAP)
.read_to_end(&mut buf)
.context("read SelectionRead fd")?;
Ok(buf)
}
/// Write `bytes` into a Mutter clipboard fd and close it (EOF). Flip the `O_NONBLOCK` fd to blocking.
fn write_fd(fd: OwnedFd, bytes: &[u8]) -> Result<()> {
set_blocking(&fd)?;
let mut file = std::fs::File::from(fd);
file.write_all(bytes).context("write SelectionWrite fd")?;
Ok(())
}
/// Peel any `Value::Value` (variant) wrappers to the concrete value. The `a{sv}` dict values Mutter
/// sends arrive as variants, so a plain `TryFrom<OwnedValue>` (which matches the concrete type) never
/// sees through them — this strips the layer first.
fn peel<'a>(v: &'a Value<'a>) -> &'a Value<'a> {
let mut cur = v;
while let Value::Value(inner) = cur {
cur = inner;
}
cur
}
/// Extract a boolean dict entry (e.g. `session-is-owner`), unwrapping the variant.
fn dict_bool(opts: &HashMap<String, OwnedValue>, key: &str) -> Option<bool> {
match peel(opts.get(key)?) {
Value::Bool(b) => Some(*b),
_ => None,
}
}
/// Extract a string-array dict entry (e.g. `mime-types`), unwrapping the variant. Mutter wraps the
/// array in a single-field struct (`(as)`, seen on `SelectionOwnerChanged`), so unwrap that too.
fn dict_mimes(opts: &HashMap<String, OwnedValue>, key: &str) -> Vec<String> {
let Some(v) = opts.get(key) else {
return Vec::new();
};
let mut val = peel(v);
if let Value::Structure(s) = val {
match s.fields().first() {
Some(first) => val = peel(first),
None => return Vec::new(),
}
}
let Value::Array(arr) = val else {
return Vec::new();
};
arr.inner()
.iter()
.filter_map(|e| match peel(e) {
Value::Str(s) => Some(s.to_string()),
_ => None,
})
.collect()
}
/// Clear `O_NONBLOCK` on a Mutter clipboard fd so a blocking `spawn_blocking` read/write works.
fn set_blocking(fd: &OwnedFd) -> Result<()> {
let raw = fd.as_raw_fd();
// SAFETY: `raw` is a valid fd owned by `fd` for the duration of these fcntl calls.
let flags = unsafe { libc::fcntl(raw, libc::F_GETFL) };
if flags < 0 {
return Err(anyhow!(
"fcntl F_GETFL: {}",
std::io::Error::last_os_error()
));
}
// SAFETY: as above; clearing O_NONBLOCK on our own fd.
let rc = unsafe { libc::fcntl(raw, libc::F_SETFL, flags & !libc::O_NONBLOCK) };
if rc < 0 {
return Err(anyhow!(
"fcntl F_SETFL: {}",
std::io::Error::last_os_error()
));
}
Ok(())
}
/// On-glass tests against a **live GNOME/Mutter** session (`WAYLAND_DISPLAY=wayland-0`). `#[ignore]`d
/// — run explicitly under GNOME (Mutter has no `wl-clipboard`, so a second Mutter session plays the
/// "host app"):
///
/// ```text
/// cargo test -p punktfunk-host --bin punktfunk-host -- --ignored --nocapture clipboard::mutter::live
/// ```
///
/// Skips (does not fail) when Mutter isn't running, so `--ignored` off-GNOME is a clean no-op.
#[cfg(test)]
mod live {
use super::*;
use std::time::Duration;
/// A second Mutter session standing in for a host clipboard app.
struct Helper {
session: zbus::Proxy<'static>,
_conn: zbus::Connection,
}
impl Helper {
async fn open() -> Result<Helper> {
let conn = zbus::Connection::session().await?;
let rd = zbus::Proxy::new(&conn, RD_BUS, RD_PATH, RD_IFACE).await?;
let path: OwnedObjectPath = rd.call("CreateSession", &()).await?;
let session = zbus::Proxy::new(&conn, RD_BUS, path, SESSION_IFACE).await?;
session.call_method("Start", &()).await?;
let empty: HashMap<&str, Value> = HashMap::new();
session.call_method("EnableClipboard", &(empty,)).await?;
Ok(Helper {
session,
_conn: conn,
})
}
/// Own the selection offering plain text, serving `payload` on every transfer request.
async fn offer_text(&self, payload: &'static [u8]) {
let mut transfer = self
.session
.receive_signal("SelectionTransfer")
.await
.unwrap();
let session = self.session.clone();
tokio::spawn(async move {
while let Some(msg) = transfer.next().await {
if let Ok((_mime, serial)) = msg.body().deserialize::<(String, u32)>() {
serve_write(&session, serial, payload.to_vec()).await;
}
}
});
set_selection(
&self.session,
&[
"text/plain;charset=utf-8".to_string(),
"text/plain".to_string(),
],
)
.await
.unwrap();
}
/// Paste the current selection's plain text.
async fn read_text(&self) -> Vec<u8> {
let fd: zbus::zvariant::OwnedFd = self
.session
.call("SelectionRead", &("text/plain;charset=utf-8",))
.await
.unwrap();
let fd = OwnedFd::from(fd);
tokio::task::spawn_blocking(move || read_fd_to_end(fd))
.await
.unwrap()
.unwrap()
}
}
async fn next_selection(
rx: &mut mpsc::UnboundedReceiver<ClipEvent>,
timeout: Duration,
) -> Option<Vec<String>> {
tokio::time::timeout(timeout, async {
loop {
match rx.recv().await {
Some(ClipEvent::Selection { mimes }) if !mimes.is_empty() => {
return Some(mimes)
}
Some(_) => continue,
None => return None,
}
}
})
.await
.ok()
.flatten()
}
#[test]
#[ignore = "needs a live GNOME/Mutter session (WAYLAND_DISPLAY=wayland-0)"]
fn live_mutter_roundtrip() {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build()
.unwrap();
rt.block_on(async {
let (backend, mut rx) = match MutterClipboard::open().await {
Ok(v) => v,
Err(e) => {
eprintln!("SKIP: no Mutter clipboard (not GNOME?): {e:#}");
return;
}
};
let helper = Helper::open().await.expect("helper Mutter session");
// --- host copy → backend observes Selection + read_current returns the bytes ---
helper.offer_text(b"gnome-host-copied").await;
let mimes = next_selection(&mut rx, Duration::from_secs(3))
.await
.expect("Selection after the helper offered text");
assert!(
mimes.iter().any(|m| m == super::super::WIRE_TEXT),
"offer carries wire text: {mimes:?}"
);
let got = backend
.read_current(super::super::WIRE_TEXT)
.await
.expect("read_current text");
assert_eq!(got, b"gnome-host-copied");
// --- backend offers client content → the host app (helper) pastes it ---
backend.set_offer(&[super::super::WIRE_TEXT.to_string()]);
tokio::time::sleep(Duration::from_millis(500)).await; // let SetSelection take effect
// Answer every Paste request the host app (helper) triggers, until the read completes.
let paste_side = async {
while let Some(ev) = rx.recv().await {
if let ClipEvent::Paste { responder, .. } = ev {
responder.respond(b"punktfunk-served".to_vec()).await;
}
}
};
let read = tokio::select! {
r = helper.read_text() => r,
_ = paste_side => Vec::new(),
};
assert_eq!(read, b"punktfunk-served");
});
}
}
+254
View File
@@ -0,0 +1,254 @@
//! Host clipboard coordinator (`design/clipboard-and-file-transfer.md` §4.2).
//!
//! One async task per streaming session that bridges the real session clipboard (whichever
//! [`super::HostClipboard`] backend this platform opened) to the QUIC clipboard plane. It owns all
//! four data paths:
//!
//! * **host copy → client** — a backend [`ClipEvent::Selection`] becomes a [`ClipOffer`] pushed to the
//! control loop (`offer_tx`), which forwards it to the client.
//! * **client fetch of the host clipboard** — the fetch-stream `accept_bi` loop lives here; each
//! stream is answered by reading the current host selection ([`ClipboardBackend::read_current`]).
//! * **client copy → host** — a [`ClipCoordCmd::RemoteOffer`] installs the client's formats as a lazy
//! host selection ([`HostClipboard::set_offer`]).
//! * **host paste of client content** — a backend [`ClipEvent::Paste`] triggers an *outbound* fetch to
//! the client, whose bytes are handed to the backend's [`PasteResponder`].
//!
//! The coordinator is backend-agnostic (Linux data-control / Mutter, Windows Win32); the control loop
//! reaches it through the portable [`ClipCoordCmd`] channel so the host's native control loop
//! compiles on every host platform.
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::mpsc::{UnboundedReceiver, UnboundedSender};
use punktfunk_core::clipboard::CLIP_FETCH_CAP;
use punktfunk_core::quic::{
clipstream, ClipFetch, ClipFetchHdr, ClipKind, ClipOffer, CLIP_FETCH_OK, CLIP_FETCH_STALE,
CLIP_FETCH_UNAVAILABLE, CLIP_FILE_INDEX_NONE,
};
use super::{ClipEvent, HostClipboard, PasteResponder};
use crate::ClipCoordCmd;
/// Upper bound on one outbound fetch (host pasting client content). A client that never answers must
/// not hang the pasting host app's pipe read (§3.4) — the paste falls back to empty instead.
const FETCH_TIMEOUT: Duration = Duration::from_secs(60);
/// Open whichever host clipboard backend this session supports (data-control, else Mutter direct) and
/// spawn the coordinator. Returns `true` when a live backend was bound (the caller's control loop
/// then serves real clipboard data); `false` when none is available (gamescope, no live compositor),
/// in which case the channels are dropped so the control loop reports `CLIP_REASON_BACKEND_UNAVAILABLE`
/// and declines fetches defensively.
pub async fn start(
conn: quinn::Connection,
clip_enabled: Arc<AtomicBool>,
cmd_rx: UnboundedReceiver<ClipCoordCmd>,
offer_tx: UnboundedSender<ClipOffer>,
) -> bool {
match HostClipboard::open().await {
Ok((backend, clip_rx)) => {
tokio::spawn(run(
conn,
Arc::new(backend),
clip_rx,
clip_enabled,
cmd_rx,
offer_tx,
));
true
}
Err(e) => {
tracing::info!(error = %format!("{e:#}"), "clipboard backend unavailable — fetches will be declined");
false
}
}
}
/// The coordinator loop. Multiplexes control-loop commands, backend clipboard events, and inbound
/// fetch streams; exits when any of the three peers goes away (session ending).
async fn run(
conn: quinn::Connection,
backend: Arc<HostClipboard>,
mut clip_rx: UnboundedReceiver<ClipEvent>,
clip_enabled: Arc<AtomicBool>,
mut cmd_rx: UnboundedReceiver<ClipCoordCmd>,
offer_tx: UnboundedSender<ClipOffer>,
) {
// Seq of the offer the host most recently announced; a client fetch naming a different seq is
// stale (the host clipboard moved on) and is declined.
let host_seq = Arc::new(AtomicU32::new(0));
let mut next_seq: u32 = 1;
// Seq of the client's most recent offer, echoed on the outbound fetch we open when a host app
// pastes client content (informational for the client's serve side).
let mut client_seq: u32 = 0;
loop {
tokio::select! {
cmd = cmd_rx.recv() => {
let Some(cmd) = cmd else { break }; // control loop gone → session ending
match cmd {
ClipCoordCmd::SetEnabled(true) => {
// A just-enabled client should see whatever the host already has copied.
let mimes = backend.current_wire_mimes();
if !mimes.is_empty() {
let _ = offer_tx.send(build_offer(&mut next_seq, &host_seq, mimes));
}
}
ClipCoordCmd::SetEnabled(false) => {
if let Err(e) = backend.clear_offer() {
tracing::debug!(error = %e, "clipboard clear_offer failed");
}
}
ClipCoordCmd::RemoteOffer { seq, mimes } => {
client_seq = seq;
let res = if mimes.is_empty() {
backend.clear_offer()
} else {
backend.set_offer(&mimes)
};
if let Err(e) = res {
tracing::debug!(error = %e, "clipboard apply remote offer failed");
}
}
}
}
ev = clip_rx.recv() => {
let Some(ev) = ev else { break }; // backend dispatch thread ended
match ev {
ClipEvent::Selection { mimes } => {
// Forward host copies (empty `mimes` = the clipboard was cleared) only while
// the client has sync on — the offer is metadata; bytes still cross lazily.
if clip_enabled.load(Ordering::SeqCst) {
let _ = offer_tx.send(build_offer(&mut next_seq, &host_seq, mimes));
}
}
ClipEvent::Paste { mime, responder } => {
// A host app is pasting the client's offered content: pull that format from
// the client and hand it to the backend's responder. Off-task so the loop
// keeps serving.
tokio::spawn(fetch_into_pipe(conn.clone(), client_seq, mime, responder));
}
ClipEvent::Closed => break,
}
}
accepted = conn.accept_bi() => {
let Ok((send, recv)) = accepted else { break }; // connection gone
// The control stream is already accepted at the handshake, so every stream here is a
// clipboard fetch. Serve it off-task (the read blocks on the source app's pipe).
tokio::spawn(serve_fetch(
send,
recv,
Arc::clone(&backend),
Arc::clone(&host_seq),
clip_enabled.load(Ordering::SeqCst),
));
}
}
}
// Session ending: don't leave our lazy source as the compositor's active selection.
let _ = backend.clear_offer();
}
/// Mint a [`ClipOffer`] for `mimes`, advancing the host offer seq (skipping 0, the "never offered"
/// sentinel) and publishing it as the current one for staleness checks.
fn build_offer(next_seq: &mut u32, host_seq: &AtomicU32, mimes: Vec<String>) -> ClipOffer {
let seq = *next_seq;
*next_seq = next_seq.wrapping_add(1);
if *next_seq == 0 {
*next_seq = 1;
}
host_seq.store(seq, Ordering::SeqCst);
let kinds = mimes
.into_iter()
.map(|mime| ClipKind { mime, size_hint: 0 })
.collect();
ClipOffer { seq, kinds }
}
/// Serve one inbound fetch stream (a client pulling the host clipboard): validate the header +
/// request, then answer with the current host selection's bytes for the requested wire MIME.
async fn serve_fetch(
mut send: quinn::SendStream,
mut recv: quinn::RecvStream,
backend: Arc<HostClipboard>,
host_seq: Arc<AtomicU32>,
enabled: bool,
) {
let _ = send.set_priority(-1);
match clipstream::read_stream_header(&mut recv).await {
Ok(k) if k == clipstream::CLIP_STREAM_KIND_FETCH => {}
_ => {
let _ = send.reset(clipstream::cancelled_code());
return;
}
}
let req = match clipstream::read_fetch(&mut recv).await {
Ok(r) => r,
Err(_) => return,
};
let decline = |status: u8| ClipFetchHdr {
status,
total_size: 0,
};
if !enabled {
let _ = clipstream::write_fetch_hdr(&mut send, &decline(CLIP_FETCH_UNAVAILABLE)).await;
return;
}
if req.seq != host_seq.load(Ordering::SeqCst) {
let _ = clipstream::write_fetch_hdr(&mut send, &decline(CLIP_FETCH_STALE)).await;
return;
}
// `read_current` reads the host selection (a blocking pipe read, offloaded by the backend).
match backend.read_current(&req.mime).await {
Ok(data) => {
let hdr = ClipFetchHdr {
status: CLIP_FETCH_OK,
total_size: data.len() as u64,
};
if clipstream::write_fetch_hdr(&mut send, &hdr).await.is_ok() {
let _ = clipstream::write_data(&mut send, &data).await;
}
}
// The format vanished (clipboard changed mid-fetch) or the read failed → nothing to send.
Err(_) => {
let _ = clipstream::write_fetch_hdr(&mut send, &decline(CLIP_FETCH_UNAVAILABLE)).await;
}
}
}
/// Pull `mime` of the client's current offer (`seq`) over an outbound fetch stream and hand the bytes
/// to the backend's paste `responder`. Any failure (timeout, decline, I/O) responds with empty bytes
/// so the pasting host app gets an empty paste instead of hanging.
async fn fetch_into_pipe(
conn: quinn::Connection,
seq: u32,
mime: String,
responder: PasteResponder,
) {
let req = ClipFetch {
seq,
file_index: CLIP_FILE_INDEX_NONE,
mime,
};
let fetched = tokio::time::timeout(FETCH_TIMEOUT, async {
let (send, mut recv) = clipstream::open_fetch(&conn, &req).await.ok()?;
let hdr = clipstream::read_fetch_hdr(&mut recv).await.ok()?;
if hdr.status != CLIP_FETCH_OK {
return None;
}
let data = clipstream::read_data(&mut recv, CLIP_FETCH_CAP)
.await
.ok()?;
drop(send); // clean close of our half
Some(data)
})
.await
.ok()
.flatten();
responder.respond(fetched.unwrap_or_default()).await;
}
+599
View File
@@ -0,0 +1,599 @@
//! `ext-data-control-v1` clipboard backend (`design/clipboard-and-file-transfer.md` §4.1).
//!
//! A dedicated thread owns the `wayland-client` [`EventQueue`] and runs a poll loop that dispatches
//! selection + paste events, emitting them over a channel. Everything else — installing a lazy
//! source (a client's offer) and `receive()`-ing the host selection (a client's fetch) — is issued
//! from the session thread on the shared, `Send + Sync` proxy handles; only *dispatch* is
//! single-threaded (per the wayland-client contract). Templated on `inject/linux/wlr.rs`.
//!
//! The `zwlr-data-control-unstable-v1` fallback for older wlroots/KWin is a mechanical parallel of
//! this file (the protocols are 1:1) — a follow-up.
use std::collections::HashMap;
use std::io::Read;
use std::os::fd::{AsFd, AsRawFd, FromRawFd, OwnedFd};
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
use std::sync::{Arc, Mutex};
use anyhow::{anyhow, Context, Result};
use wayland_client::backend::ObjectId;
use wayland_client::protocol::wl_registry;
use wayland_client::protocol::wl_seat::WlSeat;
use wayland_client::{event_created_child, Connection, Dispatch, Proxy, QueueHandle};
use wayland_protocols::ext::data_control::v1::client::{
ext_data_control_device_v1::{self, ExtDataControlDeviceV1},
ext_data_control_manager_v1::ExtDataControlManagerV1,
ext_data_control_offer_v1::{self, ExtDataControlOfferV1},
ext_data_control_source_v1::{self, ExtDataControlSourceV1},
};
use super::{ClipEvent, PasteResponder};
/// Upper bound on bytes read from one `receive()` transfer (matches the wire clipboard cap, §7) so a
/// hostile host app can't stream unboundedly into our buffer.
const CLIP_READ_CAP: u64 = 64 << 20;
/// The current host selection, shared between the dispatch thread (writer) and the session thread
/// (reader, for `receive()`).
struct CurrentSelection {
offer: ExtDataControlOfferV1,
/// Raw Wayland MIMEs the offer advertises (what `receive()` accepts).
mimes: Vec<String>,
}
/// Dispatch-thread state. Also collects the manager + seat during the bind roundtrip.
struct State {
mgr: Option<ExtDataControlManagerV1>,
seat: Option<WlSeat>,
/// Offers accumulating their MIME list before the `selection` event promotes one.
pending: HashMap<ObjectId, Vec<String>>,
current: Arc<Mutex<Option<CurrentSelection>>>,
/// Pending count of our own `set_selection`s whose `selection` echo must be dropped rather than
/// announced back to the client (loop prevention, §3.4). Bumped by the session before each set;
/// each of our sets produces exactly one echo on wlroots/KWin, so one decrement per echo pairs
/// them up — a counter (not a bool) keeps rapid back-to-back offers from leaking a self-echo.
suppress_echoes: Arc<AtomicU32>,
tx: tokio::sync::mpsc::UnboundedSender<ClipEvent>,
}
impl Dispatch<wl_registry::WlRegistry, ()> for State {
fn event(
state: &mut Self,
registry: &wl_registry::WlRegistry,
event: wl_registry::Event,
_: &(),
_: &Connection,
qh: &QueueHandle<Self>,
) {
if let wl_registry::Event::Global {
name,
interface,
version,
} = event
{
match interface.as_str() {
"ext_data_control_manager_v1" => {
state.mgr = Some(registry.bind(name, version.min(1), qh, ()));
}
"wl_seat" => {
state.seat = Some(registry.bind(name, version.min(7), qh, ()));
}
_ => {}
}
}
}
}
// Manager + seat emit nothing we consume.
impl Dispatch<ExtDataControlManagerV1, ()> for State {
fn event(
_: &mut Self,
_: &ExtDataControlManagerV1,
_: <ExtDataControlManagerV1 as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
impl Dispatch<WlSeat, ()> for State {
fn event(
_: &mut Self,
_: &WlSeat,
_: <WlSeat as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
impl Dispatch<ExtDataControlDeviceV1, ()> for State {
fn event(
state: &mut Self,
_dev: &ExtDataControlDeviceV1,
event: ext_data_control_device_v1::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
use ext_data_control_device_v1::Event;
match event {
// A new offer is being introduced; its `offer` events follow before `selection`.
Event::DataOffer { id } => {
state.pending.insert(id.id(), Vec::new());
}
// The active selection changed. `Some` = a new clipboard; `None` = cleared.
Event::Selection { id } => {
// Consume one pending self-echo if any (atomic vs. the session thread's bumps; the
// dispatch thread is the only decrementer). `Ok` = there was one → suppress.
let suppressed = state
.suppress_echoes
.fetch_update(Ordering::SeqCst, Ordering::SeqCst, |c| c.checked_sub(1))
.is_ok();
match id {
Some(offer) => {
let mimes = state.pending.remove(&offer.id()).unwrap_or_default();
if suppressed {
// Our own source's echo — don't store it as the host clipboard and
// don't announce it back to the client.
return;
}
let wire = super::offer_wire_mimes(&mimes)
.into_iter()
.map(str::to_string)
.collect::<Vec<_>>();
*state.current.lock().unwrap() = Some(CurrentSelection { offer, mimes });
let _ = state.tx.send(ClipEvent::Selection { mimes: wire });
}
None => {
*state.current.lock().unwrap() = None;
if !suppressed {
let _ = state.tx.send(ClipEvent::Selection { mimes: Vec::new() });
}
}
}
}
Event::Finished => {
let _ = state.tx.send(ClipEvent::Closed);
}
// Primary selection is out of scope for the shared clipboard.
_ => {}
}
}
event_created_child!(State, ExtDataControlDeviceV1, [
ext_data_control_device_v1::EVT_DATA_OFFER_OPCODE => (ExtDataControlOfferV1, ()),
]);
}
impl Dispatch<ExtDataControlOfferV1, ()> for State {
fn event(
state: &mut Self,
offer: &ExtDataControlOfferV1,
event: ext_data_control_offer_v1::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
if let ext_data_control_offer_v1::Event::Offer { mime_type } = event {
if let Some(list) = state.pending.get_mut(&offer.id()) {
list.push(mime_type);
}
}
}
}
impl Dispatch<ExtDataControlSourceV1, ()> for State {
fn event(
state: &mut Self,
_src: &ExtDataControlSourceV1,
event: ext_data_control_source_v1::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
use ext_data_control_source_v1::Event;
match event {
// A host app pasted our (the client's) offered data.
Event::Send { mime_type, fd } => match super::wayland_to_wire(&mime_type) {
Some(wire) => {
let _ = state.tx.send(ClipEvent::Paste {
mime: wire.to_string(),
responder: PasteResponder::Fd(fd),
});
}
// We can't satisfy this format — closing the fd yields an empty paste.
None => drop(fd),
},
// Our source was superseded (a host app or another client set a new selection).
Event::Cancelled => {}
_ => {}
}
}
}
/// The host clipboard backend handle used by the session thread.
pub struct ClipboardBackend {
conn: Connection,
mgr: ExtDataControlManagerV1,
device: ExtDataControlDeviceV1,
qh: QueueHandle<State>,
current: Arc<Mutex<Option<CurrentSelection>>>,
suppress_echoes: Arc<AtomicU32>,
active_source: Mutex<Option<ExtDataControlSourceV1>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl ClipboardBackend {
/// Connect to the active session's Wayland display (env already applied by
/// `vdisplay::apply_session_env`), bind `ext_data_control`, and start the dispatch thread.
/// Returns the handle plus the event stream. Errors if the compositor lacks the protocol
/// (caller reports `BackendUnavailable`).
pub fn open() -> Result<(
ClipboardBackend,
tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
)> {
let conn = Connection::connect_to_env()
.context("connect to Wayland for clipboard (WAYLAND_DISPLAY/XDG_RUNTIME_DIR set?)")?;
let mut queue = conn.new_event_queue();
let qh = queue.handle();
let _registry = conn.display().get_registry(&qh, ());
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
let current = Arc::new(Mutex::new(None));
let suppress_echoes = Arc::new(AtomicU32::new(0));
let mut state = State {
mgr: None,
seat: None,
pending: HashMap::new(),
current: current.clone(),
suppress_echoes: suppress_echoes.clone(),
tx,
};
queue
.roundtrip(&mut state)
.context("Wayland registry roundtrip")?;
let mgr = state
.mgr
.clone()
.context("compositor lacks ext_data_control_manager_v1")?;
let seat = state
.seat
.clone()
.context("compositor advertised no wl_seat")?;
let device = mgr.get_data_device(&seat, &qh, ());
// Second roundtrip: the compositor sends the initial selection for the freshly-bound device
// (the current host clipboard), which the session announces to the client.
queue
.roundtrip(&mut state)
.context("Wayland get_data_device roundtrip")?;
let stop = Arc::new(AtomicBool::new(false));
let thread = {
let conn = conn.clone();
let stop = stop.clone();
std::thread::Builder::new()
.name("punktfunk-clipboard".into())
.spawn(move || dispatch_loop(conn, queue, state, stop))
.context("spawn clipboard dispatch thread")?
};
Ok((
ClipboardBackend {
conn,
mgr,
device,
qh,
current,
suppress_echoes,
active_source: Mutex::new(None),
stop,
thread: Some(thread),
},
rx,
))
}
/// Install a lazy source advertising a client's offered formats (wire MIMEs) as the host
/// selection. A later host-app paste fires a [`ClipEvent::Paste`]. Replaces any previous offer.
pub fn set_offer(&self, wire_mimes: &[String]) -> Result<()> {
let wl_mimes = super::wayland_offers_for(wire_mimes);
if wl_mimes.is_empty() {
return self.clear_offer();
}
let src = self.mgr.create_data_source(&self.qh, ());
for m in &wl_mimes {
src.offer(m.clone());
}
// Suppress the selection echo our own set triggers (loop prevention).
self.suppress_echoes.fetch_add(1, Ordering::SeqCst);
self.device.set_selection(Some(&src));
self.conn.flush().context("flush set_selection")?;
let mut slot = self.active_source.lock().unwrap();
if let Some(old) = slot.take() {
old.destroy();
}
*slot = Some(src);
Ok(())
}
/// Drop the host selection we own (client disabled sync / offered nothing).
pub fn clear_offer(&self) -> Result<()> {
let mut slot = self.active_source.lock().unwrap();
if let Some(old) = slot.take() {
self.suppress_echoes.fetch_add(1, Ordering::SeqCst);
self.device.set_selection(None);
old.destroy();
self.conn.flush().context("flush clear selection")?;
}
Ok(())
}
/// The current host selection's wire MIMEs (what a client offer announcement would carry), or
/// empty if the clipboard is empty. Used to answer an immediate query.
pub fn current_wire_mimes(&self) -> Vec<String> {
match self.current.lock().unwrap().as_ref() {
Some(sel) => super::offer_wire_mimes(&sel.mimes)
.into_iter()
.map(str::to_string)
.collect(),
None => Vec::new(),
}
}
/// Read one format (`wire_mime`) of the current host selection into a byte vector — a client's
/// lazy fetch. BLOCKS on the pipe until the source app finishes, so call from a blocking
/// context (e.g. `spawn_blocking`). Errors if there is no selection or the format isn't offered.
pub fn read_current(&self, wire_mime: &str) -> Result<Vec<u8>> {
let (offer, wl_mime) = {
let cur = self.current.lock().unwrap();
let sel = cur.as_ref().context("no current host selection")?;
let wl = super::pick_wayland_mime(wire_mime, &sel.mimes)
.context("format not offered by the host clipboard")?;
(sel.offer.clone(), wl)
};
let (read_fd, write_fd) = make_pipe()?;
offer.receive(wl_mime, write_fd.as_fd());
self.conn.flush().context("flush receive")?;
// Close our write end so the pipe reaches EOF once the source app closes its dup.
drop(write_fd);
let mut buf = Vec::new();
// `read_fd` is a fresh, uniquely-owned pipe read end; `File` takes sole ownership and closes
// it on drop.
let file = std::fs::File::from(read_fd);
file.take(CLIP_READ_CAP)
.read_to_end(&mut buf)
.context("read clipboard transfer")?;
Ok(buf)
}
}
impl Drop for ClipboardBackend {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
/// The dispatch thread: poll the Wayland socket with a short timeout so `stop` is honored promptly,
/// dispatching selection/paste events into `state`.
fn dispatch_loop(
conn: Connection,
mut queue: wayland_client::EventQueue<State>,
mut state: State,
stop: Arc<AtomicBool>,
) {
while !stop.load(Ordering::SeqCst) {
if queue.dispatch_pending(&mut state).is_err() {
break;
}
if conn.flush().is_err() {
break;
}
let Some(guard) = conn.prepare_read() else {
// Events are already queued; loop to dispatch them.
continue;
};
let raw_fd = guard.connection_fd().as_raw_fd();
let mut pfd = libc::pollfd {
fd: raw_fd,
events: libc::POLLIN,
revents: 0,
};
// SAFETY: `pfd` is a single valid pollfd; `poll` reads/writes exactly it for 200 ms.
let rc = unsafe { libc::poll(&mut pfd, 1, 200) };
if rc < 0 {
let err = std::io::Error::last_os_error();
drop(guard);
if err.kind() == std::io::ErrorKind::Interrupted {
continue; // EINTR — recheck stop, retry
}
break;
}
if rc == 0 {
drop(guard); // timeout — recheck stop
continue;
}
if pfd.revents & libc::POLLIN != 0 {
if guard.read().is_err() {
break;
}
} else {
drop(guard); // POLLHUP / POLLERR — connection gone
break;
}
}
let _ = state.tx.send(ClipEvent::Closed);
}
/// Create a `pipe2(O_CLOEXEC)`, returning `(read_end, write_end)` as owned fds.
fn make_pipe() -> Result<(OwnedFd, OwnedFd)> {
let mut fds = [0 as libc::c_int; 2];
// SAFETY: `pipe2` fully initializes the 2-element `fds` on success (returns 0); on failure (-1)
// we bail before reading it. Each returned fd is fresh and owned by exactly one `OwnedFd`.
let rc = unsafe { libc::pipe2(fds.as_mut_ptr(), libc::O_CLOEXEC) };
if rc < 0 {
return Err(anyhow!("pipe2 failed: {}", std::io::Error::last_os_error()));
}
// SAFETY: `fds[0]`/`fds[1]` are the fresh, uniquely-owned pipe ends from the checked `pipe2`.
let read_fd = unsafe { OwnedFd::from_raw_fd(fds[0]) };
// SAFETY: as above for the write end.
let write_fd = unsafe { OwnedFd::from_raw_fd(fds[1]) };
Ok((read_fd, write_fd))
}
/// On-glass tests against a **live** `data-control` compositor (Hyprland / Sway / KWin). `#[ignore]`d
/// — run explicitly under such a session with `wl-clipboard` present:
///
/// ```text
/// WAYLAND_DISPLAY=wayland-1 cargo test -p punktfunk-host --bin punktfunk-host \
/// -- --ignored --nocapture clipboard::wayland::live
/// ```
///
/// Each test skips (does not fail) when `open()` finds no backend — so `--ignored` on GNOME (no
/// data-control) or a headless CI runner is a clean no-op instead of a false failure.
#[cfg(test)]
mod live {
use super::*;
use std::io::Write as _;
use std::process::{Command, Stdio};
use std::time::{Duration, Instant};
/// Poll the event channel (sync `try_recv`, no runtime) until `pred` matches or `timeout`.
fn wait_event(
rx: &mut tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
timeout: Duration,
mut pred: impl FnMut(&ClipEvent) -> bool,
) -> Option<ClipEvent> {
let deadline = Instant::now() + timeout;
loop {
match rx.try_recv() {
Ok(ev) if pred(&ev) => return Some(ev),
Ok(_) => {}
Err(tokio::sync::mpsc::error::TryRecvError::Empty) => {
if Instant::now() >= deadline {
return None;
}
std::thread::sleep(Duration::from_millis(20));
}
Err(tokio::sync::mpsc::error::TryRecvError::Disconnected) => return None,
}
}
}
/// Set the compositor selection from a "host app" (`wl-copy`, which forks a server that holds it).
fn wl_copy(bytes: &[u8], mime: &str) {
let mut child = Command::new("wl-copy")
.arg("--type")
.arg(mime)
.stdin(Stdio::piped())
.spawn()
.expect("spawn wl-copy");
child
.stdin
.take()
.unwrap()
.write_all(bytes)
.expect("write to wl-copy");
let _ = child.wait(); // foreground exits; the fork keeps serving
std::thread::sleep(Duration::from_millis(150));
}
fn open_or_skip() -> Option<(
ClipboardBackend,
tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
)> {
if Command::new("wl-copy").arg("--version").output().is_err() {
eprintln!("SKIP: wl-clipboard not installed");
return None;
}
match ClipboardBackend::open() {
Ok(v) => Some(v),
Err(e) => {
eprintln!("SKIP: no data-control backend on this compositor: {e:#}");
None
}
}
}
/// Host copy → we observe a `Selection` and can `read_current` the exact bytes back — both text
/// and PNG (§3.5 format normalization end to end).
#[test]
#[ignore = "needs a live data-control compositor (WAYLAND_DISPLAY)"]
fn live_host_copy_is_readable() {
let Some((backend, mut rx)) = open_or_skip() else {
return;
};
// Text.
wl_copy(b"hello-from-host-app", "text/plain;charset=utf-8");
let ev = wait_event(&mut rx, Duration::from_secs(3), |e| {
matches!(e, ClipEvent::Selection { mimes } if mimes.iter().any(|m| m == super::super::WIRE_TEXT))
})
.expect("Selection event carrying text after wl-copy");
assert!(matches!(ev, ClipEvent::Selection { .. }));
assert_eq!(
backend.read_current(super::super::WIRE_TEXT).unwrap(),
b"hello-from-host-app"
);
// PNG (arbitrary bytes tagged image/png — data-control is format-agnostic).
let png = b"\x89PNG\r\n\x1a\n-fake-but-tagged-image/png";
wl_copy(png, "image/png");
wait_event(&mut rx, Duration::from_secs(3), |e| {
matches!(e, ClipEvent::Selection { mimes } if mimes.iter().any(|m| m == super::super::WIRE_PNG))
})
.expect("Selection event carrying image/png");
assert_eq!(backend.read_current(super::super::WIRE_PNG).unwrap(), png);
}
/// We install a client's offer as the host selection; a host app (`wl-paste`) pasting it fires a
/// `Paste` event that we fulfill with bytes, and the host app receives exactly those bytes.
#[test]
#[ignore = "needs a live data-control compositor (WAYLAND_DISPLAY)"]
fn live_set_offer_is_pasteable() {
let Some((backend, mut rx)) = open_or_skip() else {
return;
};
backend
.set_offer(&[super::super::WIRE_TEXT.to_string()])
.expect("install offer");
// A host app pastes our offered selection.
let child = Command::new("wl-paste")
.arg("-n")
.stdout(Stdio::piped())
.spawn()
.expect("spawn wl-paste");
let paste = wait_event(&mut rx, Duration::from_secs(3), |e| {
matches!(e, ClipEvent::Paste { .. })
})
.expect("Paste event after wl-paste reads our offer");
match paste {
ClipEvent::Paste { mime, responder } => {
assert_eq!(
mime,
super::super::WIRE_TEXT,
"paste requested the text format"
);
match responder {
PasteResponder::Fd(fd) => {
super::super::fulfill_paste(fd, b"served-by-punktfunk").expect("fulfill");
}
PasteResponder::Channel(_) => panic!("data-control paste must carry an fd"),
}
}
_ => unreachable!(),
}
let out = child.wait_with_output().expect("wl-paste output");
assert_eq!(out.stdout, b"served-by-punktfunk");
}
}
+664
View File
@@ -0,0 +1,664 @@
//! Host-side shared-clipboard backend for Windows (`design/clipboard-and-file-transfer.md` §4, Phase
//! 3). The Win32 clipboard is thread-affine and message-driven, so the whole backend lives on one
//! dedicated **message-loop thread** owning a hidden message-only window:
//!
//! * **host copy → client** — `AddClipboardFormatListener` delivers `WM_CLIPBOARDUPDATE`; we map the
//! available formats to wire MIMEs, cache them, and emit [`ClipEvent::Selection`]. Our own offers
//! are suppressed by the owner-check (we never forward what we ourselves put on the clipboard).
//! * **client fetch of the host clipboard** — a `Cmd::Read` reads the requested format's HGLOBAL and
//! converts it to wire bytes ([`super::winfmt`]).
//! * **client copy → host** — a `Cmd::SetOffer` installs the client's formats via OLE **delayed
//! rendering**: `SetClipboardData(fmt, NULL)`, so no bytes cross until a host app actually pastes.
//! * **host paste of client content** — the paste triggers `WM_RENDERFORMAT`; the message-loop thread
//! blocks (bounded) while the coordinator fetches the bytes from the client, then `SetClipboardData`s
//! them for the pasting app.
//!
//! The async coordinator drives the thread through a [`Cmd`] channel woken by `PostMessage(WM_APP_CMD)`
//! (`PostMessage` is the documented thread-safe way to poke a message loop). Per-window state hangs
//! off `GWLP_USERDATA`, so multiple concurrent sessions each get their own window + state.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use std::cell::RefCell;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use anyhow::Context as _;
use ::windows::core::{w, PCWSTR};
use ::windows::Win32::Foundation::{
GetLastError, GlobalFree, HANDLE, HGLOBAL, HINSTANCE, HWND, LPARAM, LRESULT, WPARAM,
};
use ::windows::Win32::System::DataExchange::{
AddClipboardFormatListener, CloseClipboard, EmptyClipboard, GetClipboardData,
GetClipboardOwner, IsClipboardFormatAvailable, OpenClipboard, RegisterClipboardFormatW,
SetClipboardData,
};
use ::windows::Win32::System::LibraryLoader::GetModuleHandleW;
use ::windows::Win32::System::Memory::{
GlobalAlloc, GlobalLock, GlobalSize, GlobalUnlock, GMEM_MOVEABLE, GMEM_ZEROINIT,
};
use ::windows::Win32::System::Ole::CF_UNICODETEXT;
use ::windows::Win32::UI::WindowsAndMessaging::{
CreateWindowExW, DefWindowProcW, DestroyWindow, DispatchMessageW, GetMessageW,
GetWindowLongPtrW, PostMessageW, PostQuitMessage, RegisterClassW, SetWindowLongPtrW,
TranslateMessage, GWLP_USERDATA, HWND_MESSAGE, MSG, WINDOW_EX_STYLE, WINDOW_STYLE, WM_APP,
WM_CLIPBOARDUPDATE, WM_DESTROY, WM_RENDERFORMAT, WNDCLASSW,
};
use super::winfmt;
use super::{ClipEvent, PasteResponder, WIRE_HTML, WIRE_PNG, WIRE_RTF, WIRE_TEXT};
/// Custom app message that wakes the pump to drain the [`Cmd`] channel.
const WM_APP_CMD: u32 = WM_APP + 1;
/// Upper bound the message-loop thread waits for the client's bytes during a `WM_RENDERFORMAT` paste.
/// The pasting app is frozen until we answer, so this caps how long a paste can hang; on expiry the
/// format is left unrendered (an empty paste) rather than blocking indefinitely.
const RENDER_TIMEOUT: Duration = Duration::from_secs(10);
/// `OpenClipboard` fails while another process transiently holds the clipboard (clipboard managers do
/// this constantly); retry briefly before giving up.
const OPEN_RETRIES: u32 = 20;
const OPEN_RETRY_DELAY: Duration = Duration::from_millis(5);
/// `RegisterClassW` returns this when the (process-global) class already exists — expected on the 2nd+
/// concurrent session, and not an error (we never unregister the class).
const ERROR_CLASS_ALREADY_EXISTS: u32 = 1410;
type ClipTx = tokio::sync::mpsc::UnboundedSender<ClipEvent>;
/// A command from the async coordinator into the message-loop thread. Delivered over a tokio channel
/// and drained on `WM_APP_CMD`.
enum Cmd {
/// Install the client's wire MIMEs as a delayed-render host selection (empty ⇒ clear).
SetOffer(Vec<String>),
/// Drop the selection we own.
Clear,
/// Read one wire format of the current host selection for a client fetch.
Read {
wire: String,
resp: tokio::sync::oneshot::Sender<anyhow::Result<Vec<u8>>>,
},
/// Tear the window + thread down.
Shutdown,
}
/// Message-loop-thread-owned state, reached from the `WndProc` via `GWLP_USERDATA`. Only the message
/// thread ever dereferences it, so the `RefCell`s are sound (no cross-thread sharing); the fields the
/// async handle also touches (`current_wire`) are behind their own `Arc<Mutex>`.
struct WinClip {
/// Backend → coordinator events.
clip_tx: ClipTx,
/// The current host selection's wire MIMEs, shared with the [`WindowsClipboard`] handle.
current_wire: Arc<Mutex<Vec<String>>>,
/// Coordinator → backend commands, drained on `WM_APP_CMD`.
cmd_rx: RefCell<tokio::sync::mpsc::UnboundedReceiver<Cmd>>,
/// Clipboard format ids we currently promise via delayed rendering (for `WM_RENDERFORMAT`).
offered: RefCell<Vec<u32>>,
fmt_html: u32,
fmt_rtf: u32,
fmt_png: u32,
/// Our own message window — used for the owner-check and clipboard opens.
own_hwnd: HWND,
}
impl WinClip {
/// `WM_CLIPBOARDUPDATE`: a host app copied (or the clipboard was cleared). Suppress our own
/// delayed-render echoes via the owner-check, else announce the new wire MIMEs.
fn on_clipboard_update(&self, hwnd: HWND) {
// SAFETY: GetClipboardOwner has no preconditions and needs no open clipboard.
let owner = unsafe { GetClipboardOwner() }.unwrap_or_default();
if owner.0 == hwnd.0 {
// Our own offer's echo (we own the clipboard) — not a host copy.
return;
}
let mimes = self.available_wire_mimes();
*self.current_wire.lock().unwrap() = mimes.clone();
let _ = self.clip_tx.send(ClipEvent::Selection { mimes });
}
/// The wire MIMEs the current clipboard advertises, in a stable order.
fn available_wire_mimes(&self) -> Vec<String> {
// SAFETY: IsClipboardFormatAvailable has no preconditions and needs no open clipboard.
let avail = |fmt: u32| unsafe { IsClipboardFormatAvailable(fmt) }.is_ok();
let mut out = Vec::new();
if avail(CF_UNICODETEXT.0 as u32) {
out.push(WIRE_TEXT.to_string());
}
if avail(self.fmt_html) {
out.push(WIRE_HTML.to_string());
}
if avail(self.fmt_rtf) {
out.push(WIRE_RTF.to_string());
}
if avail(self.fmt_png) {
out.push(WIRE_PNG.to_string());
}
out
}
/// `WM_APP_CMD`: run every queued coordinator command on this thread. Drained into a `Vec` first so
/// the `cmd_rx` borrow is released before any command runs (defensive against re-entry).
fn drain_commands(&self, hwnd: HWND) {
let mut cmds = Vec::new();
{
let mut rx = self.cmd_rx.borrow_mut();
while let Ok(c) = rx.try_recv() {
cmds.push(c);
}
}
for c in cmds {
match c {
Cmd::SetOffer(wire) => self.apply_offer(hwnd, &wire),
Cmd::Clear => self.clear(hwnd),
Cmd::Read { wire, resp } => {
let _ = resp.send(self.read(&wire));
}
Cmd::Shutdown => {
// Drop our offer first so no WM_RENDERALLFORMATS fires as the window dies (we do
// NOT want the client's content to outlive the session on the host clipboard).
self.clear(hwnd);
// SAFETY: our own live window; triggers WM_DESTROY → PostQuitMessage → pump exit.
unsafe {
let _ = DestroyWindow(hwnd);
}
return;
}
}
}
}
/// Install the client's offer as a delayed-render host selection.
fn apply_offer(&self, hwnd: HWND, wire: &[String]) {
let fmts = self.formats_for_offer(wire);
if fmts.is_empty() {
self.clear(hwnd);
return;
}
if open_clipboard_retry(hwnd).is_err() {
tracing::debug!("clipboard: OpenClipboard for set_offer failed");
return;
}
let _guard = ClipboardGuard;
// SAFETY: the clipboard is open (ClipboardGuard closes it); EmptyClipboard makes us the owner,
// then each SetClipboardData(_, None) registers a delayed-render promise for that format.
unsafe {
let _ = EmptyClipboard();
for &f in &fmts {
let _ = SetClipboardData(f, None);
}
}
*self.offered.borrow_mut() = fmts;
}
/// Drop the selection we own (empty the clipboard iff we're still its owner).
fn clear(&self, hwnd: HWND) {
let had = {
let mut o = self.offered.borrow_mut();
let was = !o.is_empty();
o.clear();
was
};
if !had {
return;
}
// SAFETY: GetClipboardOwner has no preconditions.
let owner = unsafe { GetClipboardOwner() }.unwrap_or_default();
if owner.0 != hwnd.0 {
return; // someone else took the clipboard already
}
if open_clipboard_retry(hwnd).is_err() {
return;
}
let _guard = ClipboardGuard;
// SAFETY: the clipboard is open (ClipboardGuard closes it); empty it to drop our promises.
unsafe {
let _ = EmptyClipboard();
}
}
/// Read one wire format of the current host selection (a client fetch).
fn read(&self, wire: &str) -> anyhow::Result<Vec<u8>> {
let fmt = self
.format_for_wire(wire)
.context("unsupported wire MIME")?;
// If we own the clipboard, its content is our own delayed-render offer (the client's copy),
// not a host selection — declining avoids GetClipboardData re-entering our own WM_RENDERFORMAT.
// SAFETY: GetClipboardOwner has no preconditions.
if unsafe { GetClipboardOwner() }.unwrap_or_default().0 == self.own_hwnd.0 {
anyhow::bail!("clipboard currently held by our own offer");
}
open_clipboard_retry(self.own_hwnd)?;
let _guard = ClipboardGuard;
// SAFETY: the clipboard is open (ClipboardGuard closes it). GetClipboardData hands back a
// clipboard-owned HGLOBAL (we must NOT free it); GlobalLock/Size/Unlock are balanced and we
// copy exactly GlobalSize bytes out before the lock is released.
let raw = unsafe {
let handle = GetClipboardData(fmt).context("GetClipboardData")?;
let hg = HGLOBAL(handle.0);
let p = GlobalLock(hg);
if p.is_null() {
anyhow::bail!("GlobalLock failed");
}
let n = GlobalSize(hg);
let mut buf = vec![0u8; n];
std::ptr::copy_nonoverlapping(p as *const u8, buf.as_mut_ptr(), n);
let _ = GlobalUnlock(hg);
buf
};
Ok(convert_from_win(wire, &raw))
}
/// `WM_RENDERFORMAT`: a host app is pasting a format we promised. Fetch the bytes from the client
/// (blocking this thread, bounded) and `SetClipboardData` them for the paster.
fn on_render_format(&self, fmt: u32) {
let Some(wire) = self.wire_for_format(fmt) else {
return;
};
let (tx, rx) = std::sync::mpsc::channel::<Vec<u8>>();
let ev = ClipEvent::Paste {
mime: wire.to_string(),
responder: PasteResponder::Sync(tx),
};
if self.clip_tx.send(ev).is_err() {
return; // coordinator gone
}
let bytes = match rx.recv_timeout(RENDER_TIMEOUT) {
Ok(b) => b,
Err(_) => return, // timeout / dropped → leave the format unrendered (empty paste)
};
let win_bytes = convert_to_win(wire, &bytes);
let Ok(hg) = alloc_hglobal(&win_bytes) else {
return;
};
// Do NOT OpenClipboard here — the pasting app already holds it open across WM_RENDERFORMAT.
// SAFETY: `hg` is a freshly-filled moveable HGLOBAL. On success the clipboard takes ownership
// (we must not free it); on failure ownership stays with us, so we free it.
unsafe {
if SetClipboardData(fmt, Some(HANDLE(hg.0))).is_err() {
let _ = GlobalFree(Some(hg));
}
}
}
/// The Win32 clipboard format id for a wire MIME (`None` = unsupported).
fn format_for_wire(&self, wire: &str) -> Option<u32> {
match wire {
WIRE_TEXT => Some(CF_UNICODETEXT.0 as u32),
WIRE_HTML => Some(self.fmt_html),
WIRE_RTF => Some(self.fmt_rtf),
WIRE_PNG => Some(self.fmt_png),
_ => None,
}
}
/// The wire MIME for a Win32 clipboard format id (`None` = one we don't offer).
fn wire_for_format(&self, fmt: u32) -> Option<&'static str> {
if fmt == CF_UNICODETEXT.0 as u32 {
Some(WIRE_TEXT)
} else if fmt == self.fmt_html {
Some(WIRE_HTML)
} else if fmt == self.fmt_rtf {
Some(WIRE_RTF)
} else if fmt == self.fmt_png {
Some(WIRE_PNG)
} else {
None
}
}
/// The clipboard format ids to promise for a client offer (dedup, 1:1 with the wire MIMEs — the OS
/// auto-synthesizes CF_TEXT/CF_OEMTEXT from CF_UNICODETEXT, so no manual text fan-out is needed).
fn formats_for_offer(&self, wire: &[String]) -> Vec<u32> {
let mut out = Vec::new();
for w in wire {
if let Some(f) = self.format_for_wire(w) {
if !out.contains(&f) {
out.push(f);
}
}
}
out
}
}
/// The active Windows clipboard backend handle held by [`super::HostClipboard`]. All Win32 work runs
/// on the message-loop thread; this is just the async-side control surface.
pub struct WindowsClipboard {
cmd_tx: tokio::sync::mpsc::UnboundedSender<Cmd>,
/// The message window's `HWND` as an `isize` (so the handle stays `Send`/`Sync`); rebuilt for the
/// `PostMessage` wakeups, which are documented thread-safe.
hwnd: isize,
current_wire: Arc<Mutex<Vec<String>>>,
join: Option<std::thread::JoinHandle<()>>,
}
impl WindowsClipboard {
/// Spin up the message-loop thread + hidden window and return once it has bound (or failed).
pub async fn open() -> anyhow::Result<(
WindowsClipboard,
tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
)> {
let (clip_tx, clip_rx) = tokio::sync::mpsc::unbounded_channel::<ClipEvent>();
let (cmd_tx, cmd_rx) = tokio::sync::mpsc::unbounded_channel::<Cmd>();
let current_wire = Arc::new(Mutex::new(Vec::new()));
// Register the three custom formats up front — process-global and thread-agnostic, so this is
// fine off the message thread and lets bring-up fail cleanly if the atoms can't be created.
let fmt_html = register_format(w!("HTML Format"))?;
let fmt_rtf = register_format(w!("Rich Text Format"))?;
let fmt_png = register_format(w!("PNG"))?;
let (ready_tx, ready_rx) = tokio::sync::oneshot::channel::<anyhow::Result<isize>>();
let cw = Arc::clone(&current_wire);
let join = std::thread::Builder::new()
.name("punktfunk-clipboard-win".into())
.spawn(move || pump_thread(clip_tx, cmd_rx, cw, fmt_html, fmt_rtf, fmt_png, ready_tx))
.context("spawn windows clipboard thread")?;
let hwnd = match tokio::time::timeout(Duration::from_secs(3), ready_rx).await {
Ok(Ok(Ok(h))) => h,
Ok(Ok(Err(e))) => return Err(e),
Ok(Err(_)) => anyhow::bail!("windows clipboard thread exited during bring-up"),
Err(_) => anyhow::bail!("windows clipboard bring-up timed out"),
};
Ok((
WindowsClipboard {
cmd_tx,
hwnd,
current_wire,
join: Some(join),
},
clip_rx,
))
}
/// The current host selection's wire MIMEs (empty = nothing to offer).
pub fn current_wire_mimes(&self) -> Vec<String> {
self.current_wire.lock().unwrap().clone()
}
/// Install a client's offered formats as the host selection (fire-and-forget onto the thread).
pub fn set_offer(&self, wire_mimes: &[String]) {
let _ = self.cmd_tx.send(Cmd::SetOffer(wire_mimes.to_vec()));
self.wake();
}
/// Drop the host selection we own (fire-and-forget onto the thread).
pub fn clear_offer(&self) {
let _ = self.cmd_tx.send(Cmd::Clear);
self.wake();
}
/// Read one wire format of the current host selection (a client's fetch).
pub async fn read_current(&self, wire_mime: &str) -> anyhow::Result<Vec<u8>> {
let (tx, rx) = tokio::sync::oneshot::channel();
self.cmd_tx
.send(Cmd::Read {
wire: wire_mime.to_string(),
resp: tx,
})
.map_err(|_| anyhow::anyhow!("clipboard thread gone"))?;
self.wake();
rx.await
.map_err(|_| anyhow::anyhow!("clipboard read dropped"))?
}
/// Poke the message loop so it drains the command channel.
fn wake(&self) {
// SAFETY: PostMessageW is documented thread-safe; `hwnd` is our message window (or already
// destroyed, in which case the post harmlessly fails and is ignored).
let _ = unsafe {
PostMessageW(
Some(HWND(self.hwnd as *mut core::ffi::c_void)),
WM_APP_CMD,
WPARAM(0),
LPARAM(0),
)
};
}
}
impl Drop for WindowsClipboard {
fn drop(&mut self) {
let _ = self.cmd_tx.send(Cmd::Shutdown);
self.wake();
if let Some(j) = self.join.take() {
let _ = j.join();
}
}
}
/// RAII `CloseClipboard` guard — pairs with a successful `open_clipboard_retry`, closing on scope exit
/// (including early `?`/`bail!` returns).
struct ClipboardGuard;
impl Drop for ClipboardGuard {
fn drop(&mut self) {
// SAFETY: constructed only after a successful OpenClipboard on this thread.
unsafe {
let _ = CloseClipboard();
}
}
}
/// Register (or resolve the existing id of) a custom clipboard format.
fn register_format(name: PCWSTR) -> anyhow::Result<u32> {
// SAFETY: RegisterClipboardFormatW is thread-agnostic and process-global; `name` is a static
// NUL-terminated wide literal.
let id = unsafe { RegisterClipboardFormatW(name) };
if id == 0 {
anyhow::bail!("RegisterClipboardFormatW failed");
}
Ok(id)
}
/// Allocate a moveable HGLOBAL holding `bytes` (zero-init so an empty payload is still a valid, locked
/// buffer). Ownership is transferred to the clipboard by a following `SetClipboardData`.
fn alloc_hglobal(bytes: &[u8]) -> anyhow::Result<HGLOBAL> {
// SAFETY: allocate at least one byte (GlobalLock of a 0-size block is unreliable), lock it, copy
// the payload in, unlock. Alloc/lock/unlock are balanced; on lock failure we free before erroring.
unsafe {
let hg = GlobalAlloc(GMEM_MOVEABLE | GMEM_ZEROINIT, bytes.len().max(1))
.context("GlobalAlloc")?;
let p = GlobalLock(hg);
if p.is_null() {
let _ = GlobalFree(Some(hg));
anyhow::bail!("GlobalLock failed");
}
std::ptr::copy_nonoverlapping(bytes.as_ptr(), p as *mut u8, bytes.len());
let _ = GlobalUnlock(hg);
Ok(hg)
}
}
/// `OpenClipboard(hwnd)` with a brief retry loop (another process often holds it transiently).
fn open_clipboard_retry(hwnd: HWND) -> anyhow::Result<()> {
for _ in 0..OPEN_RETRIES {
// SAFETY: OpenClipboard with our window as owner; balanced by ClipboardGuard/CloseClipboard.
if unsafe { OpenClipboard(Some(hwnd)) }.is_ok() {
return Ok(());
}
std::thread::sleep(OPEN_RETRY_DELAY);
}
anyhow::bail!("OpenClipboard failed after retries")
}
/// Convert a Win32 clipboard payload to wire bytes.
fn convert_from_win(wire: &str, raw: &[u8]) -> Vec<u8> {
match wire {
WIRE_TEXT => winfmt::text_from_utf16(raw),
WIRE_HTML => winfmt::html_from_cf(raw),
WIRE_RTF => winfmt::rtf_from_cf(raw),
_ => raw.to_vec(), // PNG + anything else: verbatim
}
}
/// Convert wire bytes to a Win32 clipboard payload.
fn convert_to_win(wire: &str, wire_bytes: &[u8]) -> Vec<u8> {
match wire {
WIRE_TEXT => winfmt::text_to_utf16(wire_bytes),
WIRE_HTML => winfmt::html_to_cf(wire_bytes),
_ => wire_bytes.to_vec(), // RTF + PNG + anything else: verbatim
}
}
/// Create the hidden message-only window (registering the class once, process-wide).
fn create_window() -> anyhow::Result<HWND> {
// SAFETY: standard window-class registration + message-only window creation; every argument is a
// valid handle / static literal, and `wndproc` matches the WNDPROC ABI.
unsafe {
let hinstance: HINSTANCE = GetModuleHandleW(PCWSTR::null())
.context("GetModuleHandleW")?
.into();
let class_name = w!("PunktfunkClipboardWindow");
let wc = WNDCLASSW {
lpfnWndProc: Some(wndproc),
hInstance: hinstance,
lpszClassName: class_name,
..Default::default()
};
if RegisterClassW(&wc) == 0 {
let code = GetLastError();
if code.0 != ERROR_CLASS_ALREADY_EXISTS {
anyhow::bail!("RegisterClassW failed: {code:?}");
}
}
let hwnd = CreateWindowExW(
WINDOW_EX_STYLE(0),
class_name,
w!(""),
WINDOW_STYLE(0),
0,
0,
0,
0,
Some(HWND_MESSAGE),
None,
Some(hinstance),
None,
)
.context("CreateWindowExW")?;
Ok(hwnd)
}
}
/// The message-loop thread body: build the window, wire up state, then pump until `WM_QUIT`.
fn pump_thread(
clip_tx: ClipTx,
cmd_rx: tokio::sync::mpsc::UnboundedReceiver<Cmd>,
current_wire: Arc<Mutex<Vec<String>>>,
fmt_html: u32,
fmt_rtf: u32,
fmt_png: u32,
ready_tx: tokio::sync::oneshot::Sender<anyhow::Result<isize>>,
) {
let hwnd = match create_window() {
Ok(h) => h,
Err(e) => {
let _ = ready_tx.send(Err(e));
return;
}
};
// A clone that outlives the boxed state, so we can announce Closed after the pump ends.
let closed_tx = clip_tx.clone();
let state = Box::new(WinClip {
clip_tx,
current_wire,
cmd_rx: RefCell::new(cmd_rx),
offered: RefCell::new(Vec::new()),
fmt_html,
fmt_rtf,
fmt_png,
own_hwnd: hwnd,
});
let ptr = Box::into_raw(state);
// SAFETY: stash the state pointer for the WndProc; the window was created on this thread and the
// pointer stays valid until we reclaim the Box after the pump exits.
unsafe {
SetWindowLongPtrW(hwnd, GWLP_USERDATA, ptr as isize);
}
// Snapshot whatever is already on the host clipboard, so the first client `enable` announces it
// (AddClipboardFormatListener only delivers *subsequent* changes).
{
// SAFETY: `ptr` is the live state we just stored; only this thread dereferences it.
let st = unsafe { &*ptr };
*st.current_wire.lock().unwrap() = st.available_wire_mimes();
}
// SAFETY: `hwnd` is our live window; start receiving WM_CLIPBOARDUPDATE.
if let Err(e) = unsafe { AddClipboardFormatListener(hwnd) } {
// SAFETY: tear down the half-built window and reclaim the leaked state box.
unsafe {
let _ = DestroyWindow(hwnd);
drop(Box::from_raw(ptr));
}
let _ = ready_tx.send(Err(
anyhow::Error::new(e).context("AddClipboardFormatListener")
));
return;
}
let _ = ready_tx.send(Ok(hwnd.0 as isize));
// SAFETY: the standard Win32 message pump. GetMessageW returns >0 for a message, 0 for WM_QUIT,
// and -1 on error — `.0 > 0` exits on both 0 and -1.
unsafe {
let mut msg = MSG::default();
while GetMessageW(&mut msg, None, 0, 0).0 > 0 {
let _ = TranslateMessage(&msg);
DispatchMessageW(&msg);
}
}
// Pump exited (window destroyed): reclaim the leaked state box. No WndProc runs after this point.
// SAFETY: `ptr` came from Box::into_raw above, is dereferenced only on this thread, and the
// message loop has ended so no further access occurs.
unsafe {
drop(Box::from_raw(ptr));
}
let _ = closed_tx.send(ClipEvent::Closed);
}
/// The window procedure. Reaches per-window state through `GWLP_USERDATA`; runs only on the message
/// thread. Registered as the class `WNDPROC` (a safe fn coerces to the `unsafe extern "system"` ABI).
extern "system" fn wndproc(hwnd: HWND, msg: u32, wparam: WPARAM, lparam: LPARAM) -> LRESULT {
// SAFETY: GWLP_USERDATA holds the `*const WinClip` stored right after window creation (0/null for
// the WM_(NC)CREATE messages that fire before that — handled by the null check below).
let ptr = unsafe { GetWindowLongPtrW(hwnd, GWLP_USERDATA) } as *const WinClip;
if ptr.is_null() {
// SAFETY: default processing before our state pointer is attached.
return unsafe { DefWindowProcW(hwnd, msg, wparam, lparam) };
}
// SAFETY: `ptr` is the live Box<WinClip> leaked in pump_thread, owned by this (the only) message
// thread and freed only after the pump exits; the WndProc is not re-entered for this window, so
// `&*ptr` is a valid shared borrow.
let st = unsafe { &*ptr };
match msg {
WM_CLIPBOARDUPDATE => {
st.on_clipboard_update(hwnd);
LRESULT(0)
}
WM_RENDERFORMAT => {
st.on_render_format(wparam.0 as u32);
LRESULT(0)
}
WM_APP_CMD => {
st.drain_commands(hwnd);
LRESULT(0)
}
WM_DESTROY => {
// SAFETY: ends the GetMessageW pump by posting WM_QUIT to this thread's queue.
unsafe {
PostQuitMessage(0);
}
LRESULT(0)
}
// SAFETY: default handling for every other message.
_ => unsafe { DefWindowProcW(hwnd, msg, wparam, lparam) },
}
}
+257
View File
@@ -0,0 +1,257 @@
//! Pure byte conversions between the Win32 clipboard formats and the portable wire MIMEs
//! (`design/clipboard-and-file-transfer.md` §3.5). Kept free of any `windows`-crate dependency so it
//! compiles on every host and its unit tests exercise the fiddly bits (CF_HTML offset math, UTF-16
//! (de)serialization) without a Windows box. The [`super::windows`] backend is the only production
//! consumer; it wraps these with the actual `GetClipboardData`/`SetClipboardData` calls.
//!
//! Format map (Win32 ↔ wire):
//! * `CF_UNICODETEXT` (UTF-16LE + NUL) ↔ `text/plain;charset=utf-8`
//! * `"HTML Format"` (CF_HTML, UTF-8 + ASCII header) ↔ `text/html`
//! * `"Rich Text Format"` (raw RTF) ↔ `text/rtf`
//! * `"PNG"` (raw PNG) ↔ `image/png` — identity, handled inline by the backend.
// ---- CF_UNICODETEXT ↔ text/plain;charset=utf-8 -----------------------------------------------
/// `CF_UNICODETEXT` HGLOBAL bytes → UTF-8 wire bytes. `raw` is the exact `GlobalSize`-length buffer;
/// it holds little-endian UTF-16 code units terminated by a single `0x0000`.
pub fn text_from_utf16(raw: &[u8]) -> Vec<u8> {
// Reinterpret each LE 2-byte pair as a UTF-16 code unit; a stray odd trailing byte (never present
// in valid data) is dropped by `chunks_exact`.
let mut units: Vec<u16> = raw
.chunks_exact(2)
.map(|c| u16::from_le_bytes([c[0], c[1]]))
.collect();
// Strip exactly one trailing NUL terminator if present (guard against eating a real code unit).
if units.last() == Some(&0) {
units.pop();
}
String::from_utf16_lossy(&units).into_bytes()
}
/// UTF-8 wire bytes → `CF_UNICODETEXT` HGLOBAL bytes (UTF-16LE + a required `0x0000` terminator).
pub fn text_to_utf16(wire: &[u8]) -> Vec<u8> {
let s = String::from_utf8_lossy(wire);
let mut out = Vec::with_capacity(wire.len() * 2 + 2);
for u in s.encode_utf16() {
out.extend_from_slice(&u.to_le_bytes());
}
out.extend_from_slice(&0u16.to_le_bytes()); // REQUIRED NUL terminator for CF_UNICODETEXT
out
}
// ---- "HTML Format" (CF_HTML) ↔ text/html -----------------------------------------------------
//
// CF_HTML is UTF-8: an ASCII `Key:Value\r\n` header carrying byte offsets, then the HTML with
// `<!--StartFragment-->`/`<!--EndFragment-->` markers. Offsets are byte counts from buffer start;
// the offsets live *inside* the header, so their digit-width feeds back into the header length. The
// spec-blessed fix (Chromium/Firefox/LibreOffice) is fixed-width 10-digit zero-padded offsets, which
// makes the header a compile-time constant and every offset a one-pass computation.
const CF_HTML_HEADER: &str = "Version:0.9\r\n\
StartHTML:0000000000\r\n\
EndHTML:0000000000\r\n\
StartFragment:0000000000\r\n\
EndFragment:0000000000\r\n";
const CF_HTML_PREFIX: &str = "<html><body>\r\n<!--StartFragment-->";
const CF_HTML_SUFFIX: &str = "<!--EndFragment-->\r\n</body></html>";
/// UTF-8 HTML fragment (wire bytes) → a `CF_HTML` buffer, NUL-terminated. The trailing NUL is the
/// conventional CF_HTML expectation (§4); `EndHTML` still points at content end, before the NUL.
pub fn html_to_cf(wire: &[u8]) -> Vec<u8> {
let fragment = String::from_utf8_lossy(wire);
let start_html = CF_HTML_HEADER.len(); // 105
let start_fragment = start_html + CF_HTML_PREFIX.len(); // 139
let end_fragment = start_fragment + fragment.len(); // byte length — fragment may be multibyte
let end_html = end_fragment + CF_HTML_SUFFIX.len();
let mut buf = Vec::with_capacity(end_html + 1);
buf.extend_from_slice(CF_HTML_HEADER.as_bytes());
buf.extend_from_slice(CF_HTML_PREFIX.as_bytes());
buf.extend_from_slice(fragment.as_bytes());
buf.extend_from_slice(CF_HTML_SUFFIX.as_bytes());
// Overwrite the four zero-padded fields in place, restricting the search to the header region so a
// fragment that happens to contain "StartHTML:" can't fool the patcher.
patch_offset(&mut buf[..start_html], b"StartHTML:", start_html);
patch_offset(&mut buf[..start_html], b"EndHTML:", end_html);
patch_offset(&mut buf[..start_html], b"StartFragment:", start_fragment);
patch_offset(&mut buf[..start_html], b"EndFragment:", end_fragment);
buf.push(0); // conventional NUL terminator
buf
}
/// A `CF_HTML` buffer → the UTF-8 HTML fragment (wire bytes). Uses the `StartFragment`/`EndFragment`
/// offsets; falls back to `StartHTML`/`EndHTML`, then to the whole buffer, if the markers are absent.
pub fn html_from_cf(raw: &[u8]) -> Vec<u8> {
let range = header_range(raw, b"StartFragment:", b"EndFragment:")
.or_else(|| header_range(raw, b"StartHTML:", b"EndHTML:"));
match range {
// Content is UTF-8 per spec; return the exact slice (drop any trailing NUL for cleanliness).
Some((start, end)) => {
let slice = &raw[start..end];
strip_trailing_nul(slice).to_vec()
}
None => strip_trailing_nul(raw).to_vec(),
}
}
/// Resolve `[start_label .. end_label]` into a validated byte range within `raw`.
fn header_range(raw: &[u8], start_label: &[u8], end_label: &[u8]) -> Option<(usize, usize)> {
let start = read_header_offset(raw, start_label)?;
let end = read_header_offset(raw, end_label)?;
if start <= end && end <= raw.len() {
Some((start, end))
} else {
None
}
}
/// Overwrite the 10 ASCII digits following `label` in `header` with `value`, zero-padded.
fn patch_offset(header: &mut [u8], label: &[u8], value: usize) {
if let Some(pos) = find(header, label) {
let at = pos + label.len();
if at + 10 <= header.len() {
let digits = format!("{value:010}");
header[at..at + 10].copy_from_slice(digits.as_bytes());
}
}
}
/// Read the decimal integer following `label:` in the ASCII header. The colon-suffixed labels only
/// match in the header, never the marker comments (`<!--StartFragment-->`) or fragment text.
fn read_header_offset(raw: &[u8], label: &[u8]) -> Option<usize> {
let mut at = find(raw, label)? + label.len();
let mut n: usize = 0;
let mut any = false;
while let Some(&b) = raw.get(at) {
if b.is_ascii_digit() {
n = n.checked_mul(10)?.checked_add((b - b'0') as usize)?;
any = true;
at += 1;
} else {
break; // stops at '\r'
}
}
any.then_some(n)
}
fn find(hay: &[u8], needle: &[u8]) -> Option<usize> {
if needle.is_empty() || needle.len() > hay.len() {
return None;
}
hay.windows(needle.len()).position(|w| w == needle)
}
// ---- "Rich Text Format" ↔ text/rtf -----------------------------------------------------------
/// `"Rich Text Format"` HGLOBAL bytes → RTF wire bytes. RTF is `{ }`-delimited; some producers append
/// a NUL past the final `}`, so strip a single trailing NUL to keep the wire payload byte-clean.
pub fn rtf_from_cf(raw: &[u8]) -> Vec<u8> {
strip_trailing_nul(raw).to_vec()
}
fn strip_trailing_nul(b: &[u8]) -> &[u8] {
match b.last() {
Some(0) => &b[..b.len() - 1],
_ => b,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn text_round_trips_and_handles_terminator() {
// UTF-8 → UTF-16LE+NUL → UTF-8.
let wire = "héllo 🌍".as_bytes();
let cf = text_to_utf16(wire);
// Ends with a single 0x0000 terminator.
assert_eq!(&cf[cf.len() - 2..], &[0, 0]);
assert_eq!(text_from_utf16(&cf), wire);
// A CF buffer *without* a terminator still decodes (no code unit eaten).
let no_term: Vec<u8> = "hi".encode_utf16().flat_map(u16::to_le_bytes).collect();
assert_eq!(text_from_utf16(&no_term), b"hi");
// Empty text → just the terminator → empty wire.
assert_eq!(text_to_utf16(b""), vec![0, 0]);
assert_eq!(text_from_utf16(&[0, 0]), b"");
}
#[test]
fn cf_html_matches_the_spec_offsets() {
// The worked example from the format reference: fragment "Hello".
let cf = html_to_cf(b"Hello");
let s = String::from_utf8(cf.clone()).unwrap();
assert!(s.contains("StartHTML:0000000105"), "{s}");
assert!(s.contains("EndHTML:0000000178"), "{s}");
assert!(s.contains("StartFragment:0000000139"), "{s}");
assert!(s.contains("EndFragment:0000000144"), "{s}");
// The declared fragment range must slice back to exactly "Hello".
let start = read_header_offset(&cf, b"StartFragment:").unwrap();
let end = read_header_offset(&cf, b"EndFragment:").unwrap();
assert_eq!(&cf[start..end], b"Hello");
// Trailing NUL present, and EndHTML points *before* it.
assert_eq!(*cf.last().unwrap(), 0);
assert_eq!(read_header_offset(&cf, b"EndHTML:").unwrap(), cf.len() - 1);
}
#[test]
fn cf_html_round_trips_including_multibyte() {
for frag in [
"Hello",
"<b>bold</b> & <i>ital</i>",
"café ☕ <span>x</span>",
"",
] {
let cf = html_to_cf(frag.as_bytes());
assert_eq!(html_from_cf(&cf), frag.as_bytes(), "fragment {frag:?}");
}
}
#[test]
fn cf_html_extract_tolerates_foreign_producers() {
// A producer that adds SourceURL and uses Version 1.0 — offsets must still drive extraction,
// never a hardcoded 105-byte header.
let fragment = "picked";
let prefix = "<html><body><!--StartFragment-->";
let header_body = format!(
"Version:1.0\r\nStartHTML:{sh:010}\r\nEndHTML:{eh:010}\r\n\
StartFragment:{sf:010}\r\nEndFragment:{ef:010}\r\nSourceURL:https://x/\r\n",
sh = 0,
eh = 0,
sf = 0,
ef = 0,
);
// Compute real offsets against this ad-hoc layout.
let start_html = header_body.len();
let start_fragment = start_html + prefix.len();
let end_fragment = start_fragment + fragment.len();
let end_html = end_fragment + "<!--EndFragment--></body></html>".len();
let full = format!(
"Version:1.0\r\nStartHTML:{start_html:010}\r\nEndHTML:{end_html:010}\r\n\
StartFragment:{start_fragment:010}\r\nEndFragment:{end_fragment:010}\r\nSourceURL:https://x/\r\n\
{prefix}{fragment}<!--EndFragment--></body></html>"
);
assert_eq!(html_from_cf(full.as_bytes()), fragment.as_bytes());
}
#[test]
fn cf_html_extract_falls_back_without_markers() {
// No fragment markers at all → whole buffer (minus any NUL).
let mut b = b"<p>no markers</p>".to_vec();
assert_eq!(html_from_cf(&b), b"<p>no markers</p>");
b.push(0);
assert_eq!(html_from_cf(&b), b"<p>no markers</p>");
}
#[test]
fn rtf_strips_one_trailing_nul() {
assert_eq!(rtf_from_cf(br"{\rtf1 hi}"), br"{\rtf1 hi}");
assert_eq!(rtf_from_cf(b"{\\rtf1 hi}\0"), br"{\rtf1 hi}");
// Only one NUL is stripped.
assert_eq!(rtf_from_cf(b"x\0\0"), b"x\0");
}
}
+147
View File
@@ -0,0 +1,147 @@
//! Shared clipboard, host side (plan §W6 shape; `design/clipboard-and-file-transfer.md` §4).
//!
//! The wire protocol and the client half live in `punktfunk-core` (`punktfunk_core::quic` +
//! `punktfunk_core::clipboard`); this crate drives the **host's** real session clipboard through
//! the per-OS backends in [`host`] and bridges it to the QUIC clipboard plane through the
//! [`host::session`] coordinator.
//!
//! The orchestrator consumes only this portable facade — [`policy`] / [`enabled`] /
//! [`cap_advertised`], the [`ClipCoordCmd`] channel vocabulary, [`start`], and
//! [`spawn_decline_loop`] — so its control loop compiles unchanged on every host platform; the
//! platform split lives entirely behind [`start`].
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use punktfunk_core::quic::ClipOffer;
/// The per-OS backends (`ext-data-control-v1` / Mutter direct / Win32) behind one
/// `HostClipboard`, plus the backend-agnostic [`host::session`] coordinator.
#[cfg(any(target_os = "linux", target_os = "windows"))]
pub mod host;
/// Operator clipboard policy from `PUNKTFUNK_CLIPBOARD` (`design/clipboard-and-file-transfer.md`
/// §4.2): `off` (default — the whole feature is dark), `on` / `1` (text + files), `text-only` /
/// `no-files` (text/RTF/HTML/image only). Returns `None` when clipboard is off (the host neither
/// advertises the cap nor accepts fetch streams); otherwise the permitted-format
/// [`punktfunk_core::quic::CLIP_POLICY_TEXT`] / `CLIP_POLICY_FILES` bitfield.
///
/// The policy gates the advertised capability and whether the [`host::session`] coordinator
/// starts. `off` keeps the whole feature dark.
pub fn policy() -> Option<u8> {
use punktfunk_core::quic::{CLIP_POLICY_FILES, CLIP_POLICY_TEXT};
match std::env::var("PUNKTFUNK_CLIPBOARD")
.unwrap_or_default()
.trim()
.to_ascii_lowercase()
.as_str()
{
"" | "0" | "off" | "false" => None,
"text-only" | "no-files" | "text" => Some(CLIP_POLICY_TEXT),
_ => Some(CLIP_POLICY_TEXT | CLIP_POLICY_FILES), // "on" / "1" / anything truthy
}
}
/// Whether the shared clipboard is enabled at all for this host (policy not `off`).
pub fn enabled() -> bool {
policy().is_some()
}
/// Whether the host should advertise `HOST_CAP_CLIPBOARD` in the `Welcome`: the operator policy
/// enables it AND this platform has a backend (Linux data-control / Mutter, or the Win32
/// clipboard) — the client greys the toggle out otherwise. A Linux host whose compositor lacks
/// data-control still advertises it and answers a later enable with `BACKEND_UNAVAILABLE`, so the
/// client can surface *why* it's unavailable.
pub fn cap_advertised() -> bool {
enabled() && cfg!(any(target_os = "linux", target_os = "windows"))
}
/// A command from the session control loop into the host clipboard coordinator
/// ([`host::session`]). Defined here — portable — so the control loop compiles on every host
/// platform; the coordinator that consumes it exists only where a backend does.
pub enum ClipCoordCmd {
/// The client toggled sync. When enabled, the coordinator (re)announces the current host
/// clipboard; when disabled, it drops any selection it owns and stops forwarding host copies.
SetEnabled(bool),
/// The client copied: install its offered wire MIMEs as a lazy host selection (empty = clear).
RemoteOffer { seq: u32, mimes: Vec<String> },
}
/// Handle to the host clipboard coordinator, held by the session control loop.
pub struct ClipCoord {
/// Whether a real backend is live. `false` on gamescope / older GNOME / an unsupported
/// platform; the control loop then answers an enable request with
/// `CLIP_REASON_BACKEND_UNAVAILABLE` and [`spawn_decline_loop`] handles any stray fetch stream.
pub available: bool,
pub cmd_tx: tokio::sync::mpsc::UnboundedSender<ClipCoordCmd>,
/// Host-copy announcements from the coordinator → control loop → client.
pub offer_rx: tokio::sync::mpsc::UnboundedReceiver<ClipOffer>,
}
/// Open the host clipboard backend (when the operator policy allows it, this session mirrors a
/// real compositor, and the platform has a backend) and spawn its coordinator, returning a handle.
/// Otherwise the handle is inert (`available = false`, channels dropped) so the caller's control
/// loop stays platform-agnostic. `has_compositor` is false for the synthetic protocol-test source,
/// which has no display/clipboard to share — keeping it out of the real session clipboard.
pub async fn start(
conn: quinn::Connection,
clip_enabled: Arc<AtomicBool>,
has_compositor: bool,
) -> ClipCoord {
let (cmd_tx, cmd_rx) = tokio::sync::mpsc::unbounded_channel();
let (offer_tx, offer_rx) = tokio::sync::mpsc::unbounded_channel();
#[cfg(any(target_os = "linux", target_os = "windows"))]
let available = if has_compositor && enabled() {
host::session::start(conn, clip_enabled, cmd_rx, offer_tx).await
} else {
drop((conn, clip_enabled, cmd_rx, offer_tx));
false
};
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
let available = {
let _ = (conn, clip_enabled, cmd_rx, offer_tx, has_compositor);
false
};
ClipCoord {
available,
cmd_tx,
offer_rx,
}
}
/// Clipboard fetch-stream accept loop, fallback flavor (`design/clipboard-and-file-transfer.md`
/// §3.3, §4.2). When a backend is live the coordinator (spawned by [`start`]) owns `accept_bi` and
/// serves real host clipboard bytes. This is for the other case: the operator allowed the cap but
/// no backend bound (gamescope / older GNOME / a not-yet-implemented platform), so a stray or
/// hostile fetch stream is answered `CLIP_FETCH_UNAVAILABLE` instead of hanging. Exactly one
/// `accept_bi` consumer runs (this OR the coordinator). The control stream is the FIRST bi-stream
/// (already accepted at the handshake), so this loop only ever sees clipboard fetch streams; it
/// dies with the connection.
pub fn spawn_decline_loop(conn: quinn::Connection) {
tokio::spawn(async move {
use punktfunk_core::quic::{clipstream, ClipFetchHdr, CLIP_FETCH_UNAVAILABLE};
while let Ok((mut send, mut recv)) = conn.accept_bi().await {
tokio::spawn(async move {
// Validate the stream header + request; a malformed/unknown stream is dropped.
match clipstream::read_stream_header(&mut recv).await {
Ok(k) if k == clipstream::CLIP_STREAM_KIND_FETCH => {}
_ => {
let _ = send.reset(clipstream::cancelled_code());
return;
}
}
if clipstream::read_fetch(&mut recv).await.is_err() {
return;
}
let _ = clipstream::write_fetch_hdr(
&mut send,
&ClipFetchHdr {
status: CLIP_FETCH_UNAVAILABLE,
total_size: 0,
},
)
.await;
});
}
});
}
+7 -774
View File
@@ -9,22 +9,22 @@
//! `save_layer_alpha` so a screen fades as a unit, never element by element. The
//! backdrop crossfades in parallel when the screens disagree (aurora ↔ form).
use crate::anim::{approach, ease_out_cubic, Progress};
use crate::glyphs::{hint_bar, GlyphStyle, Hint, HintKey};
use crate::anim::Progress;
use crate::glyphs::GlyphStyle;
use crate::library::{mesh_sksl, LibraryShared};
use crate::model::{ConsoleBus, ConsoleCmd, ConsoleShared, HostRow, PairPhase, WakeStatus};
use crate::screens::{Bg, ConnectIntent, Ctx, Nav, Outbox, Screen};
use crate::theme::{white, Fonts, PanelStroke, DIM, W, WHITE};
use anyhow::{anyhow, Result};
use pf_client_core::gamepad::{MenuDir, MenuEvent, MenuPulse, PadInfo};
use pf_client_core::trust;
use pf_presenter::overlay::OverlayAction;
use skia_safe::{
gradient_shader, Canvas, Color4f, Data, Paint, Point, Rect, RuntimeEffect, TileMode,
};
use skia_safe::{Canvas, Color4f, Data, Paint, Rect, RuntimeEffect};
use std::collections::VecDeque;
use std::time::Instant;
mod overlays;
mod render;
const TRANSITION_S: f64 = 0.26;
/// Chrome bands (design units): the pinned title above, hints below.
const TOP_BAND: f64 = 64.0;
@@ -425,176 +425,6 @@ impl Shell {
}
}
// --- Rendering -------------------------------------------------------------------------
#[allow(clippy::too_many_arguments)]
pub(crate) fn render(
&mut self,
canvas: &Canvas,
width: u32,
height: u32,
fonts: &Fonts,
pad: Option<&str>,
pad_pref: Option<punktfunk_core::config::GamepadPref>,
pads: &[PadInfo],
) {
let now = Instant::now();
let dt = self
.last_frame
.replace(now)
.map_or(1.0 / 60.0, |t| (now - t).as_secs_f64().clamp(0.0, 0.05));
self.sync();
self.pads = pads.to_vec();
self.glyphs = GlyphStyle::from_pref(pad_pref);
self.chip = Some(pad.map_or_else(
|| "No controller — keyboard works too".to_string(),
str::to_owned,
));
let (w, h) = (f64::from(width), f64::from(height));
let k = (h / 800.0).clamp(0.75, 3.0);
let t = self.t();
// Advance the transition; a finished pop finally drops its leaving screen.
let motion_p = match &mut self.motion {
Motion::None => None,
Motion::Push(p) => {
p.advance(dt);
let v = p.value();
if p.done() {
self.motion = Motion::None;
None
} else {
Some(v)
}
}
Motion::Pop { t, .. } => {
t.advance(dt);
let v = t.value();
if t.done() {
self.motion = Motion::None;
None
} else {
Some(v)
}
}
};
// Backdrop crossfade follows the top screen.
let bg_target = match self.stack.last().expect("non-empty").background() {
Bg::Aurora => 0.0,
Bg::Form => 1.0,
};
self.bg_mix = approach(self.bg_mix, bg_target, dt, 0.12);
if (self.bg_mix - bg_target).abs() < 0.005 {
self.bg_mix = bg_target;
}
if self.bg_mix < 1.0 {
self.draw_aurora(canvas, w, h, t);
} else {
canvas.clear(Color4f::new(0.0, 0.0, 0.0, 1.0));
}
if self.bg_mix > 0.0 {
canvas.save_layer_alpha_f(None, self.bg_mix as f32);
crate::theme::draw_form_background(canvas, w, h);
canvas.restore();
}
// The screens, through the transition choreography.
let content = Rect::from_ltrb(
0.0,
(TOP_BAND * k) as f32,
w as f32,
(h - BOTTOM_BAND * k) as f32,
);
// One paint recipe per layer: (alpha, slide, scale). Everything below borrows
// disjoint fields of `self` per call, so the borrow checker stays happy.
let mut env = LayerEnv {
canvas,
w,
h,
content,
k,
dt,
fonts,
hosts: &self.hosts,
library: &self.library,
settings: &mut self.settings,
pads: &self.pads,
deck: self.deck,
device_name: &self.device_name,
t,
glyphs: self.glyphs,
// A modal card owns B/A while it's up — the screen's legend would lie.
show_hints: self.connecting.is_none() && self.wake.is_none(),
};
match (&mut self.motion, motion_p) {
(Motion::Push(_), Some(raw)) => {
let p = ease_out_cubic(raw);
let n = self.stack.len();
// Outgoing recedes underneath…
if n >= 2 {
let (below, top) = self.stack.split_at_mut(n - 1);
env.paint(&mut below[n - 2], 1.0 - p, 0.0, 1.0 - 0.04 * p);
// …while the incoming slides up out of a fade.
env.paint(&mut top[0], p, 36.0 * k * (1.0 - p), 0.985 + 0.015 * p);
} else {
env.paint(
&mut self.stack[0],
p,
36.0 * k * (1.0 - p),
0.985 + 0.015 * p,
);
}
}
(Motion::Pop { leaving, .. }, Some(raw)) => {
let p = ease_out_cubic(raw);
// The revealed screen grows back in…
let n = self.stack.len();
env.paint(&mut self.stack[n - 1], 0.4 + 0.6 * p, 0.0, 0.96 + 0.04 * p);
// …while the leaving one slides down into a fade.
env.paint(leaving.as_mut(), 1.0 - p, 36.0 * k * p, 1.0);
}
_ => {
let n = self.stack.len();
env.paint(&mut self.stack[n - 1], 1.0, 0.0, 1.0);
}
}
// Persistent chrome: the controller chip (top-right, above every layer).
if let Some(chip) = &self.chip {
let size = 12.0 * k;
let tw = f64::from(fonts.measure(chip, W::Medium, size));
let (bh, pad_x) = (24.0 * k, 12.0 * k);
let bx = w - 24.0 * k - tw - 2.0 * pad_x;
let rect = Rect::from_xywh(
bx as f32,
(18.0 * k) as f32,
(tw + 2.0 * pad_x) as f32,
bh as f32,
);
crate::theme::panel(
canvas,
rect,
(bh / 2.0 / k) as f32,
None,
PanelStroke::Plain(0.12),
k as f32,
);
fonts.draw(
canvas,
chip,
bx + pad_x,
18.0 * k + 16.0 * k,
W::Medium,
size,
white(0.7),
);
}
self.draw_overlays(canvas, w, h, k, dt, t, fonts);
}
fn draw_aurora(&self, canvas: &Canvas, w: f64, h: f64, t: f64) {
let uniforms: [f32; 3] = [w as f32, h as f32, t as f32];
let bytes = unsafe { std::slice::from_raw_parts(uniforms.as_ptr().cast::<u8>(), 12) };
@@ -609,604 +439,7 @@ impl Shell {
}
}
}
// --- Overlays (connecting / waking / toast) --------------------------------------------
#[allow(clippy::too_many_arguments)]
fn draw_overlays(
&mut self,
canvas: &Canvas,
w: f64,
h: f64,
k: f64,
dt: f64,
t: f64,
fonts: &Fonts,
) {
// Resolve the connect/wake takeover — the two phases of reaching a host — into one
// full-screen shape (spinner, title, one detail line, its own hints). Connecting flows
// straight out of a wake (see `sync`) so they share the same backdrop and never blink
// between them. Mirrors the Android client's unified `ConnectOverlay`.
let takeover: Option<(f64, bool, String, String, Vec<Hint>)> =
if let Some(c) = &mut self.connecting {
c.appear = approach(c.appear, 1.0, dt, 0.07);
if c.canceling {
Some((
c.appear,
true,
"Canceling…".to_string(),
String::new(),
vec![],
))
} else if c.request_access {
Some((
c.appear,
true,
"Waiting for approval…".to_string(),
format!(
"Approve this device in {}'s console or web UI — no PIN needed.",
c.title
),
vec![Hint::new(HintKey::Back, "Cancel")],
))
} else {
Some((
c.appear,
true,
format!("Connecting to {}", c.title),
"Starting the stream in this window.".to_string(),
vec![Hint::new(HintKey::Back, "Cancel")],
))
}
} else if let Some(wk) = &self.wake {
// Service-driven, so it appears settled (no fade-in).
if wk.timed_out {
Some((
1.0,
false,
format!("{} didn't wake", wk.name),
"Check its power settings, or wake it manually and try again.".to_string(),
vec![
Hint::new(HintKey::Confirm, "Try Again"),
Hint::new(HintKey::Back, "Cancel"),
],
))
} else {
Some((
1.0,
true,
format!("Waking {}", wk.name),
format!("Waiting for it to come online · {} s", wk.seconds),
// A wake-only wait (no dial after) offers "Stop Waiting"; a wake-&-connect
// is a plain "Cancel".
vec![Hint::new(
HintKey::Back,
if wk.then_connect {
"Cancel"
} else {
"Stop Waiting"
},
)],
))
}
} else {
None
};
if let Some((appear, spinner, title, body, hints)) = takeover {
self.draw_takeover(
canvas, w, h, k, appear, t, fonts, spinner, &title, &body, &hints,
);
}
// The toast: a transient pill above the hint bar; slides in, fades out.
if self.toast.as_ref().is_some_and(|toast| t - toast.at > 4.0) {
self.toast = None;
}
if let Some(toast) = &self.toast {
let age = t - toast.at;
{
let slide = ease_out_cubic((age / 0.25).min(1.0));
let fade = if age > 3.4 {
(1.0 - (age - 3.4) / 0.6).max(0.0)
} else {
1.0
};
let alpha = (slide * fade) as f32;
let size = 13.0 * k;
let tw = f64::from(fonts.measure(&toast.text, W::Medium, size));
let (pad_x, bh) = (16.0 * k, 34.0 * k);
let bw = tw + 2.0 * pad_x;
let bx = (w - bw) / 2.0;
let by = h - BOTTOM_BAND * k - bh - 8.0 * k + (1.0 - slide) * 12.0 * k;
canvas.save_layer_alpha_f(None, alpha);
let rect = Rect::from_xywh(bx as f32, by as f32, bw as f32, bh as f32);
canvas.draw_rrect(
skia_safe::RRect::new_rect_xy(rect, (bh / 2.0) as f32, (bh / 2.0) as f32),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.6), None),
);
crate::theme::panel(
canvas,
rect,
(bh / 2.0 / k) as f32,
None,
PanelStroke::Plain(0.14),
k as f32,
);
fonts.draw(
canvas,
&toast.text,
bx + pad_x,
by + bh / 2.0 + size * 0.36,
W::Medium,
size,
white(0.92),
);
canvas.restore();
}
}
}
}
/// Everything one screen layer needs to paint — bundled so the transition arms stay
/// readable and each `paint` call borrows `Shell` fields disjointly.
struct LayerEnv<'a> {
canvas: &'a Canvas,
w: f64,
h: f64,
content: Rect,
k: f64,
dt: f64,
fonts: &'a Fonts,
hosts: &'a [HostRow],
library: &'a LibraryShared,
settings: &'a mut trust::Settings,
pads: &'a [PadInfo],
deck: bool,
device_name: &'a str,
t: f64,
glyphs: GlyphStyle,
show_hints: bool,
}
impl LayerEnv<'_> {
/// One screen composited as a unit: `alpha` fade, `dy` vertical slide, `scale`
/// about the screen center — its pinned title and hint bar ride inside the layer,
/// so chrome travels with content through a transition.
fn paint(&mut self, screen: &mut Screen, alpha: f64, dy: f64, scale: f64) {
let canvas = self.canvas;
canvas.save_layer_alpha_f(None, alpha.clamp(0.0, 1.0) as f32);
canvas.translate((0.0, dy as f32));
let (cx, cy) = ((self.w / 2.0) as f32, (self.h / 2.0) as f32);
canvas.translate((cx, cy));
canvas.scale((scale as f32, scale as f32));
canvas.translate((-cx, -cy));
let mut ctx = Ctx {
hosts: self.hosts,
library: self.library,
settings: self.settings,
pads: self.pads,
deck: self.deck,
device_name: self.device_name,
t: self.t,
};
self.fonts.centered(
canvas,
&screen.title(&ctx),
W::Bold,
30.0 * self.k,
WHITE,
self.w / 2.0,
18.0 * self.k,
self.w * 0.7,
);
screen.render(canvas, self.content, self.k, self.dt, self.fonts, &mut ctx);
if self.show_hints {
let hints = screen.hints(&ctx);
hint_bar(
canvas,
self.fonts,
&hints,
self.glyphs,
18.0 * self.k,
self.h - 18.0 * self.k,
self.k,
);
}
canvas.restore();
}
}
impl Shell {
/// A full-screen connect/wake takeover: a fresh aurora over everything (so the carousel and
/// chrome fall away), a centered spinner (or none, when a wake has timed out), a title, one
/// detail line, and its own bottom hint row. `appear` fades the whole thing in over the home;
/// a wake that hands off to a connect passes 1.0 so the two never blink between them. The
/// console counterpart of the Android/Apple `ConnectOverlay` — one full-screen shape, not a
/// centered modal card.
#[allow(clippy::too_many_arguments)]
fn draw_takeover(
&self,
canvas: &Canvas,
w: f64,
h: f64,
k: f64,
appear: f64,
t: f64,
fonts: &Fonts,
spinner: bool,
title: &str,
body: &str,
hints: &[Hint],
) {
let cx = w / 2.0;
canvas.save_layer_alpha_f(None, appear as f32);
// Opaque aurora — the same living backdrop the home wears, so the takeover reads as the
// console taking over rather than a card popping up.
self.draw_aurora(canvas, w, h, t);
// A soft pool of shade under the centre seats the white text against a bright aurora.
let mut vignette = Paint::default();
vignette.set_shader(gradient_shader::radial(
Point::new(cx as f32, (h / 2.0) as f32),
(w.max(h) * 0.42) as f32,
gradient_shader::GradientShaderColors::Colors(&[
Color4f::new(0.0, 0.0, 0.0, 0.5).to_color(),
Color4f::new(0.0, 0.0, 0.0, 0.0).to_color(),
]),
None,
TileMode::Clamp,
None,
None,
));
canvas.draw_rect(Rect::from_wh(w as f32, h as f32), &vignette);
// Centre the spinner + title + detail as a group around the middle of the screen.
let title_y = h / 2.0 + if spinner { 14.0 * k } else { 0.0 };
if spinner {
crate::theme::spinner(canvas, cx, title_y - 52.0 * k, 22.0 * k, t);
}
fonts.centered(
canvas,
title,
W::SemiBold,
23.0 * k,
WHITE,
cx,
title_y,
w * 0.82,
);
if !body.is_empty() {
fonts.centered(
canvas,
body,
W::Regular,
14.0 * k,
DIM,
cx,
title_y + 32.0 * k,
w * 0.66,
);
}
if !hints.is_empty() {
// Centered near the bottom, where every console screen's legend sits.
let probe = hint_bar(canvas, fonts, hints, self.glyphs, -10_000.0, -10_000.0, k);
hint_bar(
canvas,
fonts,
hints,
self.glyphs,
cx - probe.0 / 2.0,
h - 34.0 * k,
k,
);
}
canvas.restore();
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::model::WakeStatus;
use crate::screens::home::HomeScreen;
use crate::screens::library::LibraryScreen;
use punktfunk_core::config::GamepadPref;
/// Point the settings/known-hosts stores at a throwaway HOME — the settings screen
/// SAVES on adjust, and a test must never write the developer's real config.
fn fake_home() {
use std::sync::OnceLock;
static HOME: OnceLock<std::path::PathBuf> = OnceLock::new();
let dir = HOME.get_or_init(|| {
let dir = std::env::temp_dir().join(format!("pf-console-test-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
std::env::set_var("HOME", &dir);
dir.clone()
});
std::env::set_var("HOME", dir);
}
fn hosts() -> Vec<HostRow> {
let base = HostRow {
key: String::new(),
name: String::new(),
addr: "10.0.0.20".into(),
port: 9777,
fp_hex: String::new(),
paired: false,
saved: true,
online: false,
mgmt_port: 47990,
can_wake: false,
last_used: None,
};
vec![
HostRow {
key: "aa11".into(),
name: "Living Room PC".into(),
fp_hex: "aa11".into(),
paired: true,
online: true,
last_used: Some(1),
..base.clone()
},
HostRow {
key: "bb22".into(),
name: "Office Tower".into(),
addr: "10.0.0.21".into(),
fp_hex: "bb22".into(),
paired: true,
can_wake: true,
..base.clone()
},
HostRow {
key: "10.0.0.30:9777".into(),
name: "steambox".into(),
addr: "10.0.0.30".into(),
saved: false,
online: true,
..base
},
]
}
fn shell(stack: Vec<Screen>) -> (Shell, ConsoleShared, LibraryShared) {
fake_home();
let console = ConsoleShared::default();
console.set_hosts(hosts());
let library = LibraryShared::default();
let bus = ConsoleBus::default();
let shell = Shell::new(
console.clone(),
library.clone(),
bus,
ConsoleOptions {
device_name: "deck".into(),
deck: false,
},
stack,
)
.unwrap();
(shell, console, library)
}
/// The shell survives a full navigation lap (a smoke test over every screen's
/// input handling — no rendering, no GPU).
#[test]
fn navigation_lap() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
// Home → Settings (X), adjust something, back out.
s.handle_menu(MenuEvent::Tertiary);
assert_eq!(s.stack.len(), 2);
finish_motion(&mut s);
s.handle_menu(MenuEvent::Move(MenuDir::Down));
s.handle_menu(MenuEvent::Move(MenuDir::Right));
s.handle_menu(MenuEvent::Back);
finish_motion(&mut s);
assert_eq!(s.stack.len(), 1);
// Home → Library on the paired host (Y), then back.
s.handle_menu(MenuEvent::Secondary);
assert_eq!(s.stack.len(), 2);
finish_motion(&mut s);
s.handle_menu(MenuEvent::Back);
finish_motion(&mut s);
assert_eq!(s.stack.len(), 1);
// B at the root quits.
s.handle_menu(MenuEvent::Back);
assert!(matches!(s.take_action(), Some(OverlayAction::Quit)));
}
#[test]
fn connect_flow_raises_launch_and_cancel() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
s.handle_menu(MenuEvent::Confirm); // paired+online host focused first
assert!(matches!(
s.take_action(),
Some(OverlayAction::Launch { launch: None, .. })
));
assert!(s.connecting.is_some());
// While connecting: B cancels exactly once.
s.handle_menu(MenuEvent::Back);
assert!(matches!(
s.take_action(),
Some(OverlayAction::CancelConnect)
));
s.handle_menu(MenuEvent::Back);
assert!(s.take_action().is_none(), "cancel is idempotent");
// The canceled dial ends silently.
s.session_ended(None);
assert!(s.connecting.is_none());
}
fn finish_motion(s: &mut Shell) {
// Transitions block input; tests fast-forward them.
s.motion = Motion::None;
}
#[test]
fn wake_gates_input_in_the_same_press() {
let (mut s, _console, _library) = shell(vec![Screen::Home(HomeScreen::new())]);
s.sync();
// Focus "Office Tower" (offline + wakeable), then A: the wake starts.
s.handle_menu(MenuEvent::Move(MenuDir::Right));
s.handle_menu(MenuEvent::Confirm);
let w = s
.wake
.as_ref()
.expect("Waking card raised in the SAME call as the A press");
assert_eq!(w.name, "Office Tower");
assert!(!w.online);
// The very next input is modal-gated — the cursor can't drift onto Add Host —
// and sync (which runs first in handle_menu) must not clear the placeholder
// before the service thread reports its first real status.
assert!(s.handle_menu(MenuEvent::Move(MenuDir::Right)).is_none());
assert!(
s.wake.is_some(),
"optimistic card survived a sync with no service status"
);
// B cancels: the gate releases and navigation works again.
s.handle_menu(MenuEvent::Back);
assert!(s.wake.is_none());
assert!(s.handle_menu(MenuEvent::Move(MenuDir::Left)).is_some());
}
/// Render every console scene to PNGs for the eyeball pass (ignored; run with
/// `PF_CONSOLE_DUMP=<dir> cargo test -p pf-console-ui --release -- --ignored dump`).
/// CPU raster — the SkSL aurora, layers and text all run without a GPU.
#[test]
#[ignore]
fn dump_console_screens() {
let dir = std::env::var("PF_CONSOLE_DUMP").expect("set PF_CONSOLE_DUMP to an output dir");
let fonts = crate::theme::build_fonts().unwrap();
let (w, h) = (1280, 800);
let pads: Vec<PadInfo> = Vec::new();
let dump = |shell: &mut Shell, frames: usize, sleep_ms: u64, name: &str, pad: bool| {
let mut surface = skia_safe::surfaces::raster_n32_premul((w, h)).unwrap();
for _ in 0..frames {
shell.render(
surface.canvas(),
w as u32,
h as u32,
&fonts,
pad.then_some("Xbox Wireless Controller"),
pad.then_some(GamepadPref::Xbox360),
&pads,
);
std::thread::sleep(std::time::Duration::from_millis(sleep_ms));
}
let png = surface
.image_snapshot()
.encode(None, skia_safe::EncodedImageFormat::PNG, 100)
.unwrap();
std::fs::write(format!("{dir}/{name}.png"), png.as_bytes()).unwrap();
};
// Home, settled, with a pad (Letters glyphs).
let (mut s, console, library) = shell(vec![Screen::Home(HomeScreen::new())]);
dump(&mut s, 40, 8, "01-home", true);
// Mid-push into Settings (the transition still): a couple of fast frames land
// the capture around p ≈ 0.4 — both layers visible.
s.handle_menu(MenuEvent::Tertiary);
dump(&mut s, 3, 25, "02-transition", true);
dump(&mut s, 40, 8, "03-settings", true);
// Add Host with the keyboard tray up (keyboard glyph style: no pad).
s.handle_menu(MenuEvent::Back);
dump(&mut s, 40, 8, "_back", true);
for _ in 0..3 {
s.handle_menu(MenuEvent::Move(MenuDir::Right));
}
s.handle_menu(MenuEvent::Confirm); // Add Host screen
dump(&mut s, 40, 8, "04-addhost", false);
s.handle_menu(MenuEvent::Confirm); // open the Name keyboard
for ev in [
MenuEvent::Move(MenuDir::Down),
MenuEvent::Confirm,
MenuEvent::Confirm,
] {
s.handle_menu(ev);
}
dump(&mut s, 40, 8, "05-addhost-keyboard", false);
// Pair (focused on the unpaired discovered host).
s.handle_menu(MenuEvent::Back); // close keyboard
s.handle_menu(MenuEvent::Back); // leave add-host
dump(&mut s, 40, 8, "_back2", true);
s.handle_menu(MenuEvent::Move(MenuDir::Left)); // onto "steambox"
s.handle_menu(MenuEvent::Confirm);
dump(&mut s, 40, 8, "06-pair", true);
// Library with placeholder posters.
library.set_games(
[
"Hades II",
"Elden Ring",
"Hollow Knight",
"Baldur's Gate 3",
"Celeste",
"Deep Rock Galactic",
"Portal 2",
]
.iter()
.enumerate()
.map(|(i, t)| crate::library::LibraryGame {
id: format!("steam:{i}"),
title: (*t).to_string(),
store: "steam".into(),
})
.collect(),
);
let (mut s2, _c2, _l2) = {
let console2 = ConsoleShared::default();
console2.set_hosts(hosts());
let bus = ConsoleBus::default();
let sh = Shell::new(
console2.clone(),
library.clone(),
bus,
ConsoleOptions {
device_name: "deck".into(),
deck: false,
},
vec![
Screen::Home(HomeScreen::new()),
Screen::Library(LibraryScreen::new(&hosts()[0])),
],
)
.unwrap();
(sh, console2, library.clone())
};
s2.handle_menu(MenuEvent::Move(MenuDir::Right));
s2.handle_menu(MenuEvent::Move(MenuDir::Right));
dump(&mut s2, 40, 8, "07-library", true);
// The wake and connecting overlays + a toast.
console.set_wake(Some(WakeStatus {
key: "bb22".into(),
name: "Office Tower".into(),
seconds: 12,
timed_out: false,
online: false,
then_connect: true,
}));
dump(&mut s, 10, 8, "08-waking", true);
console.set_wake(Some(WakeStatus {
key: "bb22".into(),
name: "Office Tower".into(),
seconds: 90,
timed_out: true,
online: false,
then_connect: true,
}));
dump(&mut s, 10, 8, "08b-wake-timed-out", true);
console.set_wake(None);
s.set_connecting(Some("Elden Ring".into()));
dump(&mut s, 10, 8, "09-connecting", true);
s.set_connecting(None);
s.session_failed("Connection timed out");
dump(&mut s, 10, 8, "10-toast", true);
}
}
mod tests;

Some files were not shown because too many files have changed in this diff Show More